Python learning _day30_ the sticky packet phenomenon based on TCP protocol

1. Programs based on remote execution commands

The subprocess module needs to be

Service side:

#1, execute the instructions sent by the clientImportSocketImportSubprocessphone=Socket.socket (Socket.af_inet,socket. Sock_stream) Phone.bind (('127.0.0.1', 8090)) Phone.listen (5) whiletrue:conn,addr=phone.accept ()Print('ip:%s port:%s'% (addr[0],addr[1]))     whileTrue:Try: cmd=CONN.RECV (1024)            if  notCmd: Break            #Execute CommandObj=subprocess. Popen (Cmd.decode ('Utf-8'), shell=True, stdout=subprocess. PIPE, stderr=subprocess. PIPE) stdout=obj.stdout.read () stderr=Obj.stderr.read () conn.send (stdout+stderr)exceptException: Breakconn.close () phone.close ( )

Client:

ImportSocketphone=Socket.socket (Socket.af_inet,socket. Sock_stream) Phone.connect (('127.0.0.1', 8090)) whileTrue:cmd=input ('>>>'). Strip ()if  notCmd:Continuephone.send (Cmd.encode ('Utf-8')) Res=PHONE.RECV (1024)    Print(Res.decode ('GBK') ) Phone.close ()

Note Note:

Res=subprocess. Popen (Cmd.decode (' Utf-8 '),
Shell=true,
Stderr=subprocess. PIPE,
Stdout=subprocess. PIPE)

The result of the encoding is based on the current system, if it is windows, then Res.stdout.read () read out is GBK encoded, at the receiving end needs to use GBK decoding, and can only read the results from the pipeline.

2, sticky bag phenomenon

Only TCP has sticky packet phenomenon, TCP protocol is a flow-oriented protocol, which is prone to sticky packet problems. For example, the TCP-based socket client uploads a file to the server, sending the file content is sent in accordance with a paragraph of the stream of bytes, in the receiving party, at all, do not know where the file's stream of bytes started, where to end. The so-called sticky packet problem is mainly because the receiver does not know the boundary between the message, do not know how many bytes of data extracted at once.

In addition, the packet caused by the sender is caused by the TCP protocol itself, TCP to improve transmission efficiency, the sender often to collect enough data before sending a TCP segment. If there are few data to send in a few consecutive times, TCP will usually send the data to a TCP segment based on the optimization algorithm, and the receiver receives the sticky packet data.

In both cases, a sticky bag will occur.

The sending side need to wait for the buffer full to send out, resulting in sticky packets (send data time interval is very short, the data is very small, to join together to produce sticky packets)

The receiver does not receive the buffer in time packets, resulting in multiple packets received (the client sent a piece of data, the server only received a small portion of the service end of the next time to collect the last data from the buffer, resulting in sticky packets)

Workaround 1:

Service side:

ImportSocketImportsubprocessImportStructphone=Socket.socket (Socket.af_inet,socket. Sock_stream) Phone.bind (('127.0.0.1', 8090)) Phone.listen (5)Print('starting ...') whiletrue:conn,addr=phone.accept ()Print('ip:%s,port:%s'% (addr[0],addr[1]))     whileTrue:Try: cmd=CONN.RECV (1024)            #Execute command cmdObj=subprocess. Popen (Cmd.decode ('Utf-8'), shell=True, stdout=subprocess. PIPE, stderr=subprocess. PIPE) stdout=obj.stdout.read () stderr=Obj.stderr.read ()#send description of real data: lengthHeader=struct.pack ('I', Len (stdout) +len (stderr))#I mode for integer data, the result of the output is 4conn.send (header)#Send real DataConn.send (stdout) conn.send (stderr)exceptException: Breakconn.close () phone.close ( )

Client:

ImportSocketImportStructphone=Socket.socket (Socket.af_inet,socket. Sock_stream) Phone.connect (('127.0.0.1', 8090)) whileTrue:cmd=input ('>>>'). Strip () Phone.send (Cmd.encode ('Utf-8')) Header=PHONE.RECV (4)#Specify to receive 4 bytesTotal_size=struct.unpack ('I', header) [0]#to unpack the 4 bytes of data received to receive the data size tuple: (data size,)    #Loop receive DataTotal_data=b"'recv_size=0 whilerecv_size<Total_size:recv_data=PHONE.RECV (1024) Total_data+=Recv_data recv_size+=Len (recv_data)Print(Total_data.decode ('GBK') ) Phone.close ()

Due to the existence of an infinite integer for the type I used in the pack method of the module struct, there is a drawback of error, so the following solution is proposed:

Workaround 2:

Service side:

ImportSocketImportsubprocessImportstructImportJsonphone=Socket.socket (Socket.af_inet,socket. Sock_stream) Phone.bind (('127.0.0.1', 8090)) Phone.listen (5)Print('starting ...') whiletrue:conn,addr=phone.accept ()Print('ip:%s,port:%s'% (addr[0],addr[1]))     whileTrue:Try: cmd=CONN.RECV (1024)            if  notCmd: Breakobj=subprocess. Popen (Cmd.decode ('Utf-8'), shell=True, stdout=subprocess. PIPE, stderr=subprocess. PIPE) stdout=obj.stdout.read () stderr=Obj.stderr.read ()#Making Headersheader_dic={'filename':'a.txt',                          'total_size': Len (stdout) +Len (stderr),'MD5':'asdfa123xvc123'} Header_json=json.dumps (header_dic) header_bytes=header_json.encode ('Utf-8')            #Send Header LengthConn.send (Struct.pack ('I', Len (header_bytes))) #Send Headerconn.send (header_bytes)#Send DataConn.send (stdout) conn.send (stderr)exceptException: Breakconn.close () phone.close ( )

Client:

ImportSocketImportstructImportJsonphone=Socket.socket (Socket.af_inet,socket. Sock_stream) Phone.connect (('127.0.0.1', 8090)) whileTrue:cmd=input ('>>>'). Strip ()if  notCmd:Continuephone.send (Cmd.encode ('Utf-8'))    #Receive header content lengthOBJ=PHONE.RECV (4) Header_size=struct.unpack ('I', obj) [0]#Receive header dictionaryheader_bytes=phone.recv (header_size) Header_json=header_bytes.decode ('Utf-8') Header_dic=json.loads (Header_json) total_size=header_dic['total_size'] Total_data=b"'recv_size=0 whilerecv_size<Total_size:recv_data=PHONE.RECV (1024) Total_data+=Recv_data recv_size+=Len (recv_data)Print(Total_data.decode ('GBK') ) Phone.close ()

Python learning _day30_ the sticky packet phenomenon based on TCP protocol