Python Full Stack Development Foundation "Supplement" to resolve TCP sticky packets

First, what is sticky bag

Note: Only TCP has a sticky packet phenomenon, UDP will never stick packets

Sticky bags do not necessarily occur

If this happens: 1. may have been glued on the client

2. The client is not sticky and may be stuck on the service side

First you need to master the principle of a Socket transceiver message

The data that the application sees is a whole, or a stream, and how many bytes of a message are not visible to the application, so the TCP protocol is a stream-oriented protocol, which is also the cause of the sticky-packet problem. (because TCP is a streaming protocol, don't know when to start and when to end). And UDP is a message-oriented protocol, each UDP segment is a message, the application must be in the message to extract data, not one time to extract arbitrary bytes of data, which is very different from TCP. How do you define a message? Can think of the other one-time Write/send data for a message, it is necessary to understand that when the other side send a message, regardless of the underlying how fragmented shards, the TCP protocol layer will constitute the entire message of the data segment is completed before rendering in the kernel buffer.

The so-called sticky packet problem is mainly because the receiver does not know the boundary between the message, do not know how many bytes of data extracted at once.

Two cases of adhesive packet occurrence

The sending side need to wait for the buffer full to send out, resulting in sticky packets (send data time interval is very short, the data is very small, as a package issued to produce sticky packets)

#服务端from Socket Import *phone = socket (af_inet,sock_stream) phone.setsockopt (sol_socket,sock_stream,1) phone.bind (' 127.0.0.1 ', 8080)) Phone.listen (5) Print (' Start running ... ') coon,addr = phone.accept () #等待连接data1 = Coon.recv (ten) data2 = COON.RECV print ('------------> ', Data1.decode (' Utf-8 ') print ('------------> ', Data2.decode (' Utf-8 ')) Coon.close () phone.close () #客户端from socket import *import timephone = socket (af_inet,sock_stream) phone.connect (' 127.0.0.1 ', 8080)) phone.send (' Hello '. Encode (' Utf-8 ')) phone.send (' HelloWorld '. Encode (' Utf-8 ')) Phone.close ()

The receiver does not receive the buffer in time packets, resulting in multiple packets received (the client sent a piece of data, the server only received a small portion of the service end of the next time to collect the last data from the buffer, resulting in sticky packets)

#服务端from Socket Import *phone = socket (af_inet,sock_stream) phone.setsockopt (sol_socket,sock_stream,1) phone.bind (' 127.0.0.1 ', 8080)) Phone.listen (5) Print (' Start running ... ') coon,addr = phone.accept () #等待连接data1 = Coon.recv (2) # The Complete data2 = COON.RECV (Ten) #下一次接收的时候会先取旧的数据 is not received at one time  , then the new # data3 = Coon.recv (1024x768)  #接收等5秒后的信息print ('------------ > ', Data1.decode (' Utf-8 ')) print ('------------> ', Data2.decode (' Utf-8 ')) # print ('------------> ', Data3.decode (' Utf-8 ')) Coon.close () phone.close () #客户端from socket import *import timephone = socket (af_inet,sock_stream Phone.connect (' 127.0.0.1 ', 8080)) phone.send (' Hello '. Encode (' Utf-8 ')) Time.sleep (5) phone.send (' Haiyan '. Encode (' Utf-8 ')) Phone.close ()

Third, the method of solving the sticky bag

The root of the problem is that the receiver does not know the length of the stream to be transmitted by the sender, so the way to solve the sticky packet is around, how to let the sending side before sending the data, the total size of the bytes will be sent to the receiving end to know, and then receive the end of a dead loop to receive all the data

#服务端import socketimport Subprocessimport structphone = Socket.socket (socket.af_inet,socket. SOCK_STREAM) #买手机phone. Bind ((' 127.0.0.1 ', 8080)) #绑定手机卡phone. Listen (5) #阻塞的最大数print (' Start runing ... ') while True: # Link Loop coon,addr = phone.accept () # Waiting to receive phone print (COON,ADDR) while True: #通信循环 # Send and receive message cmd = COON.RECV (10 #接收的最大数 print (' Received:%s '%cmd.decode (' Utf-8 ')) #处理过程 res = subprocess. Popen (Cmd.decode (' utf-8 '), Shell = True, stdout=subprocess. PIPE, #标准输出 stderr=subprocess. PIPE #标准错误) stdout = Res.stdout.read () stderr = Res.stderr.read () #先发 Header (turn to a fixed-length bytes type, then how do you turn it?)        The struct module is used) #len (stdout) + len (stderr) #统计数据的长度 header = Struct.pack (' i ', Len (stdout) +len (stderr)) #制作报头 Coon.send (header) #再发命令的结果 coon.send (stdout) coon.send (stderr) coon.close () phone.close () #客户端imp ORT SocketimportStructphone = Socket.socket (socket.af_inet,socket. Sock_stream) Phone.connect ((' 127.0.0.1 ', 8080)) #连接服while True: # send Message cmd = input (' Please enter command >>: '). Strip () if no T cmd:continue phone.send (Cmd.encode (' Utf-8 ')) #发送 #先收报头 header_struct = phone.recv (4) #收四个 unpack_res = Struc T.unpack (' i ', header_struct) total_size = unpack_res[0] #总长度 #后收数据 recv_size = 0 total_data=b "while Recv_ Size<total_size: #循环的收 recv_data = Phone.recv (1024x768) #1024只是一个最大的限制 Recv_size+=len (recv_data) # tot Al_data+=recv_data # print (' message returned: '%s '%total_data.decode (' GBK ')) Phone.close ()

　　

Iv. solve the problem of sticky package upgrade: Complete solution

#服务端import socketimport subprocessimport Structimport jsonphone = Socket.socket (socket.af_inet,socket. SOCK_STREAM) #买手机phone. setsockopt (socket. Sol_socket,socket. so_reuseaddr,1) Phone.bind ((' 127.0.0.1 ', 8080)) #绑定手机卡phone. Listen (5) #阻塞的最大数print (' Start runing ... ') while True: # Link Loop coon,addr = phone.accept () # Waiting to receive phone print (COON,ADDR) while True: #通信循环 # Send and receive message cmd = COON.RECV (10 #接收的最大数 print (' Received:%s '%cmd.decode (' Utf-8 ')) #处理过程 res = subprocess. Popen (Cmd.decode (' utf-8 '), Shell = True, stdout=subprocess. PIPE, #标准输出 stderr=subprocess. PIPE #标准错误) stdout = Res.stdout.read () stderr = Res.stderr.read () # System            Make the header header_dic = {' Total_size ': Len (stdout) +len (stderr), # Total size ' filename ': None, ' MD5 ': None} Header_json = Json.dumps (header_dic) #字符串类型 header_bytes = The Header_json.encode (' Utf-8 ') #转成bytes类型 (but the length is variable) #先发报头的长度 coon.send (struct.pack (' I ', Len (header_bytes))) #发    Send fixed-Length header #再发报头 coon.send (header_bytes) #最后发命令的结果 coon.send (stdout) coon.send (stderr) Coon.close () Phone.close ()

#客户端import socketimport Structimport jsonphone = Socket.socket (socket.af_inet,socket. Sock_stream) Phone.connect ((' 127.0.0.1 ', 8080)) #连接服务器while True:    # send message    cmd = input (' Please enter command >>: '). Strip ()    if not cmd:continue    phone.send (Cmd.encode (' Utf-8 ')) #发送    #先收报头的长度    header_len = struct.unpack (' i ', PHONE.RECV (4)) [0]  #吧bytes类型的反解    #在收报头    header_bytes = phone.recv (header_len) #收过来的也是bytes类型    header_ JSON = Header_bytes.decode (' utf-8 ')   #拿到json格式的字典    header_dic = json.loads (Header_json)  #反序列化拿到字典了    total_size = header_dic[' total_size ']  #就拿到数据的总长度了    #最后收数据    recv_size = 0    total_data=b " While    recv_size<total_size: #循环的收        recv_data = phone.recv (1024x768) #1024只是一个最大的限制        Recv_size+=len ( Recv_data) #有可能接收的不是1024个字节, perhaps more than 1024,        # then receive when the reception is not complete, so also add the received that length        total_data+=recv_data #最终的结果    Print (' message returned: '%s '%total_data.decode (' GBK ') ') Phone.close ()

　　

Five, struct module

#该模块可以把一个类型, such as numbers, to a fixed-length bytes type import structres = Struct.pack (' i ', 12345) print (Res,len (res), type (res))  # Length is 4res2 = Struct.pack (' i ', 12345111) print (Res,len (res), type (res2))  #长度也是4unpack_res =struct.unpack (' i ', res2) Print (Unpack_res)  # (12345111,) print (unpack_res[0]) #12345111

　　

Python Full Stack Development Foundation "Supplement" to resolve TCP sticky packets