Python socket module

Sends the message to send the data to the sending buffer of its own side, and the receiving message is received from the buffer of its own side.

1. Tcp:send send Message, recv receive message

2. Udp:sendto send Message, recvfrom receive message

TCP is based on data flow, and UDP is datagram-based:

Send (Bytes_data): sends the data stream, the data stream Bytes_data if is empty, own this part of the buffer also is empty, the operating system does not control TCP protocol to send the empty packet, cannot the client, the server cannot enter the empty.

SendTo (Bytes_data,ip_port): Send a datagram, Bytes_data is empty, and ip_port, all even send empty bytes_data, the datagram is not empty, their own buffer to receive the contents of the end, The operating system will control the UDP protocol packet. (because there are ip_port, I also think that the UDP itself is the protocol of the Relationship)

1.TCP Protocol:

(1) If the data in the message buffer is empty, then the recv will be blocked (blocking is simple, is waiting to receive)

(2) only the TCP protocol client send an empty data is really empty data, the client even if there are infinite send empty, also with no one.

(3) TCP link-based communication

Based on the link, the listen is required to specify the size of the half-connection pool.

Based on the link, the server must run first, and then the client initiates the link request

For Mac Systems: If one end of the link is broken, the other end of the link is also finished recv will not block, received is empty (the workaround is: the service side after receiving the message plus if judgment, the empty message will break off the communication loop).

2.UDP protocol

(1) If the data in the message buffer is "empty", Recvfrom will also block

(2) Only the UDP protocol client sendinto an empty data is not really empty data (including: empty data + address information, the resulting report will still not be empty), so as long as the client has a sendinto (whether or not to send empty data, is not really empty data), The server can recvfrom to the data.

(3) UDP no link

• No links, so no listen, no more connection pool
• No link, UDP sendinto do not have a running server, you can kept the message, but the data is lost
• Recvfrom the data received is less than the data sent by Sendinto, the data is lost directly on Mac and Linux system, and it is sent on the Windows system more direct error than received
• Only sendinto send data without recvfrom data, data loss.

Res=subprocess. Popen (Cmd.decode (' Utf-8 '),
Shell=true,
Stderr=subprocess. PIPE,
Stdout=subprocess. PIPE)

The result of the encoding is based on the current system, if it is windows, then res.stdout.read () read out is GBK encoded , at the receiving end need to use GBK decoding

And can only read one result from the pipe.

Sticky bag phenomenon

Only TCP has a sticky packet phenomenon, UDP never sticky packets.

The sender can be a K-K to send the data, and the receiving side of the application can be two K two k to take the data, of course, it is possible to take 3 K or 6K data at a time, or only a few bytes of data at a time, that is, the application sees the data is a whole, or a stream (stream), The number of bytes of a message is not visible to the application, so the TCP protocol is a stream-oriented protocol, which is also the cause of the sticky packet problem. And UDP is a message-oriented protocol, each UDP segment is a message, the application must be in the message to extract data, not one time to extract arbitrary bytes of data, which is very different from TCP. How do you define a message? Can think of the other one-time Write/send data for a message, it is necessary to understand that when the other side send a message, regardless of the underlying how fragmented shards, the TCP protocol layer will constitute the entire message of the data segment is completed before rendering in the kernel buffer.

For example, the TCP-based socket client to the server to upload files, the content of the file is sent in accordance with a paragraph of the stream of bytes sent, in the receiver looked at, do not know where the file's byte stream from where to start, where to end

The so-called sticky packet problem is mainly because the receiver does not know the boundary between the message, do not know how many bytes of data extracted at once.

In addition, the packet caused by the sender is caused by the TCP protocol itself, TCP to improve transmission efficiency, the sender often to collect enough data before sending a TCP segment. If there are few data to send in a few consecutive times, TCP will usually send the data to a TCP segment based on the optimization algorithm, and the receiver receives the sticky packet data.

1 TCP (Transport Control Protocol, transmission Protocol) is connection-oriented, stream-oriented and provides high reliability services. Both ends of the transceiver (client and server side) have one by one pairs of sockets, so the sending side in order to send multiple packets to the receiver, more efficient to the other side, the use of the optimization method (Nagle algorithm), the multiple interval small and small data volume data, combined into a large block of data, and then to the packet. In this way, the receiving end, it is difficult to distinguish out, must provide a scientific unpacking mechanism. That is, stream-oriented communication is a non-message-protected boundary.

2 UDP (User Datagram Protocol, Subscriber Datagram Protocol) is non-connected, message-oriented, providing efficient service. The Block merging optimization algorithm is not used, because UDP supports a one-to-many pattern, so the receiver Skbuff (socket buffer) uses a chain structure to record each incoming UDP packet, in each UDP packet there is a message header (message source address, port and other information), so for the receiving end , it is easy to distinguish between the processing. that is, message-oriented communication is a message-protected boundary.

3 TCP is based on data flow, so send and receive messages can not be empty, which requires the client and the server to add a null message processing mechanism to prevent the program stuck, and UDP is based on the datagram, even if you enter the empty content (direct carriage), it is not an empty message, the UDP protocol will help you encapsulate the message header.

UDP Recvfrom is blocked, a recvfrom (x) must be to a sendto (y), after the X-byte data is completed, if the y>x data is lost, which means that UDP is not sticky packets, but will lose data, unreliable

TCP protocol data is not lost, no packets are received, the next time it is received, it continues to receive the last time, and the buffer content is always cleared when the ACK is received by the client. The data is reliable, but it will stick to the package.

The 1024 specified in recv means that 1024 bytes of data are taken out of the cache at a time.

The byte stream of send is put into the cache first, then the cache content is sent to the peer by the Protocol control, if the byte stream to be sent is larger than the buffer space, then the data is lost, and the data will be called by Sendall.

Low method to solve sticky pack

The program runs much faster than the network transfer speed, so before sending a byte, send the word throttle length with send, this way will magnify the performance loss caused by network delay. So it's irrelevant to say more code. It is important to reduce the interaction between the server segment and the client segment.

A better way

Add a custom fixed-length header to the byte stream, the header contains the byte stream length, and then send to the peer at a time, and the peer receives the fixed-length header from the cache before fetching the real data.

We can make a dictionary of the header, the dictionary contains the details of the real data to be sent, then JSON serialization, and then use struck to package the serialized length of the data into 4 bytes (4 of them enough)

When sending:

First outgoing header length

Re-encode the header contents and send

Finally send the real content

When receiving:

Initiator header length, removed with struct

The header content is charged according to the length taken, then decoded and deserialized

Extract the details of the data to be fetched from the deserialized results and then fetch the actual data content.

Python socket module