Python Day30 Sticky Bag

First, what is sticky bag

Only TCP has sticky packet phenomenon, UDP will never sticky, why, and listen to my word

First, you need to master the principle of a socket transceiver message:

The so-called sticky packet problem is mainly because the receiver does not know the boundary between the message, do not know how many bytes of data extracted at once caused by

In both cases, the sticky pack will occur:

1. The sending side need to wait for the buffer full to send out, resulting in sticky packets (send data time interval is very short, the data is very small, to join together to produce sticky bag)

2. The receiver does not receive the buffer in time packets, resulting in multiple packets received (the client sent a piece of data, the server only received a small portion of the service end of the next time, or from the buffer to take the last remaining data, resulting in sticky packets)

Second, the solution of the adhesive package

1. Introduction of the module used:

　　

1. Subprocess module: Execution System module

Ubprocess. Popen (): Invokes the current system to execute the command.

STDOUT: Standard correct output

STDERR: Incorrect output

stdin: Input

Shell=true: System Interpreter

Subprocess. Pipe: It's a pipe.

2. struct MODULE

　　　　　Struct.pack: Packaged into a Bates type

Struct.unpack: Unpacking, unpacking a Bates type

Importjson,struct#Imagine uploading 1t:1073741824000 files through a client a.txt#to avoid sticky packets, you must customize the headerheader={'file_size': 1073741824000,'file_name':'/a/b/c/d/e/a.txt','MD5':'8f6fbf8347faa4924a76856701edb0f3'}#1T data, file path and MD5 values#for this header to be transmitted, it needs to be serialized and converted to bytesHead_bytes=bytes (Json.dumps (header), encoding='Utf-8')#serialized and turned into bytes for transmission#to let the client know the length of the header, use struck to convert the number of the header length to a fixed length: 4 bytesHead_len_bytes=struct.pack ('I', Len (head_bytes))#These 4 bytes contain only a number, which is the length of the header#The client starts sendingConn.send (Head_len_bytes)#first Outgoing header length, 4 x bytesConn.send (Head_bytes)#re-send the byte format of the headerConn.sendall (File contents)#then send real content in byte format#Server start receivingHEAD_LEN_BYTES=S.RECV (4)#Header 4 bytes to get the byte format of the header lengthX=struct.unpack ('I', head_len_bytes) [0]#extract the length of the headerhead_bytes=S.RECV (x)#the bytes format of the header is charged according to the header length xHeader=json.loads (Json.dumps (header))#Extract Header#Finally, the actual data is extracted according to the content of the header, such asREAL_DATA_LEN=S.RECV (header['file_size']) s.recv (Real_data_len)

We can make a dictionary of the header, the dictionary contains the details of the real data to be sent, then JSON serialization, and then use struck to package the serialized length of the data into 4 bytes (4 of them enough)

When sending:

First outgoing header length

Re-encode the header contents and send

Finally send the real content

When receiving:

Initiator header length, removed with struct

The header content is charged according to the length taken, then decoded and deserialized

Extract the details of the data to be fetched from the deserialized results and then fetch the actual data content

1 takes the form of a custom header:

#Service SideImportSocket#Import ModuleImportstruct#Import ModuleImportSubprocess#Import ModuleDin=socket.socket (Socket.af_inet,socket. SOCK_STREAM)#transmission in the form of data streams based on the network familyIp= ('127.0.0.1', 8080)#identifies the server-side unique addressDin.bind (IP)#Binding AddressDin.listen (5)#set maximum number of link buffer pools whiletrue:conn,addr=din.accept ()#wait for link to enter    Print('------>', addr) whileTrue:Try:#start catching exceptions, using in WindowsCMD=CONN.RECV (1024)#assigns the received content to the variable cmdRes=subprocess. Popen (Cmd.decode ('Utf-8'), shell=true,stdout=subprocess. Pipe,stderr=subprocess. PIPE)#set up a pipeline, write it as a shell, and judge if the correct command is placed in a pipe of standard output, or put it in the wrong output pipeDui=res.stdout.read ()#reads the contents of the standard output pipeline to DUICuo=res.stderr.read ()#reads the contents of the error output pipeline to CuOData_bat=len (DUI) +len (CuO)#Add the content length of DUI and CuO to Data_batConn.send (Struct.pack ('I', Data_bat))#send to client via module emulation headerConn.send (DUI)#Send the DUI to the clientConn.send (CuO)#send the CuO to the client        exceptException:#End Catch Exception             Break             #JumpConn.close ()#Break LinkDin.close ()#Turn off links that are based on network family transfers

#ClientImportSocketImportStructdin=Socket.socket (Socket.af_inet,socket. SOCK_STREAM) Ip_port=('127.0.0.1', 8080) Din.connect (ip_port) whileTrue:cmd=input ('Please enter a command:'). Strip ()if  notCmd:Continuedin.send (bytes (cmd,encoding='Utf-8')) Baotou=DIN.RECV (4) Data_size=struct.unpack ('I', Baotou) [0] Rec_size=0 Rec_data=b"'     whileRec_size <Data_size:data=DIN.RECV (1024) Rec_size+=len (data) Rec_data+=DataPrint(Rec_data.decode ('GBK') ) Din.close ()

2, the form of the awesome header:

#Service SideImportstruct#Import ModuleImportSocket#Import ModuleImportJson#Import ModuleImportSubprocess#Import ModuleDin=socket.socket (Socket.af_inet,socket. SOCK_STREAM)#Create a link path based on the network family streamIp= ('127.0.0.1', 8080)#set the server-side unique addressDin.bind (IP)#Binding AddressDin.listen (5)#Activate startup, set the maximum number of slow link pools whiletrue:conn,addr=din.accept ()#wait for the data link to be accepted    Print('------->>>')     whileTrue:Try:#start catching exceptions, using in WindowsDR=CONN.RECV (1024)#receive content and assign a value to DrRes=subprocess. Popen (Dr.decode ('Utf-8'), shell=true,stdout=subprocess. Pipe,stderr=subprocess. PIPE)#set up a pipeline, put the right content in a pair of pipes, and put the wrong into a wrong pipe. Dui=res.stdout.read ()#reads the contents of the standard output and assigns the value to DUICuo=res.stderr.read ()#read the contents of the error output and assign a value to CuOData_lik=len (DUI) +len (CuO)#calculates the total length of standard output and error output and assigns a value to Data_likhead_dic={'Data_lik':d Ata_lik}#top A dictionary and assign it to Head_dicHead_json=json.dumps (Head_dic)#to serialize a dictionary of your own definitionHead_bytes=head_json.encode ('Utf-8')#Transcode the serialized content because the network transmission is transmitted through bytesHead_len=len (Head_bytes)#take and assign to head_len the length of the converted byte code.            #Send Header LengthConn.send (Struct.pack ('I', Head_len))#Send yourself a defined header length            #Send HeaderConn.send (Head_bytes)#send a header of its own definition            #Send real DataConn.send (DUI)#Send real-world dataConn.send (CuO)#Send real-world data        exceptException:#End Catch Exception             Break                       #JumpConn.close ()#Break LinkDin.close ()#Close Connection

#ClientImportstruct#Import ModuleImportSocket#Import ModuleImportJson#Import ModuleDin=socket.socket (Socket.af_inet,socket. SOCK_STREAM)#Create a connection based on the network family and the way the stream is transferredIp= ('127.0.0.1', 8080)#set a unique addressDin.connect (IP)#Binding Address whileTrue:run=input ('Please enter a command:')#User Interaction Interface    if  notRunContinue           #Judging if run is empty, jump outDin.send (Bytes (run,encoding='Utf-8'))#send user-entered content in byte formDATA=DIN.RECV (4)#receive content length of 4, and assign value to dataHead_len=struct.unpack ('I', data) [0]#extract the header and index the value assigned to Head_lenHEAD_BYTES=DIN.RECV (Head_len)#The receive length is the value in the Head_len variable and is assigned to Head_bytesHead_json=head_bytes.decode ('Utf-8')#Convert the content that was just received into Utf-8Head_dic=json.loads (Head_json)#In the form of deserializing the content converted to utf-8 into a dictionarydata_lik=head_dic['Data_lik']#assigning the value of a dictionary lookup to Data_likRecv_size=0#set a variable and assign a value of 0Recv_data=b"'                      #set the variable and create an empty byte     whileRecv_size < Data_lik:#If the variable is smaller than the value found by the dictionary, the loop is true. DATA=DIN.RECV (1024)#receive content assignment to dataRecv_size+=len (data)#Add the resulting content length to the variable recv_size, so that you can achieve the effect if you don't finish the contentRecv_data+=data#add the received value to the newly created byte        Print(Recv_data.decode ('GBK'))#Print the contents of GBK transcodingDin.close ()#Close Connection

Python Day30 Sticky Bag