TCP and Udpsocket in So_sndbuf and So_rcvbuf

1.BackgroundWinsock Kernel buffer

To optimize performance on the application layer, Winsock copies data buffers from application send calls to a Winsock Kernel buffer. Then, the stack uses it own heuristics (such as Nagle algorithm) to determine when to actually put the packet in the wire .
You can change the amount of Winsock kernel buffer allocated to the socket using the SO_SNDBUF option (it's 8K by def Ault). If necessary, Winsock can buffer significantly more than the SO_SNDBUF buffer size.

Send completion in most cases

In most cases, the send completion in the application only indicates the data buffer in a application send call is Co Pied to the Winsock kernel buffer and does not indicate that the data have hit the network medium.
The only exception are when you disable the Winsock buffering by setting So_sndbuf to 0.

Rules to indicate a send completion

Winsock uses the following rules to indicate a send completion to the application (depending on how the send is invoked, T He completion notification could be the function returning from a blocking call, signaling an event or calling a Notificat Ion function, and so forth):

• If the socket is still within so_sndbuf quota, Winsock copies the data from the application send and indicate s the send completionto the application.
• If the socket is beyond so_sndbuf quota and there are only one previously buffered send still in the Stac K kernel buffer, Winsock copies the data from the application send and indicates the send completion to the application.
• If the socket is beyond so_sndbuf quota and there are more than one previously buffered send in the stack Kernel buffer, Winsock copies the data from the application send. Winsock does not indicate the send completion to the application until the stack completes enough sends to put the SOC Ket back within SO_SNDBUF quota or only one outstanding send condition.

https://support.microsoft.com/en-us/kb/214397

2.so_sndbuf & so_rcvbuf2.1 Basic Instructions

So_sndbuf
Sets send buffer size. This option is takes an int value. (It is 8K by default).
So_rcvbuf
Sets receive buffer size. This option is takes an int value.

Note: so stands for Socket Option

Each set of interfaces has a send buffer and a receive buffer, and the default buffer size can be changed using SO_SNDBUF & So_rcvbuf.

For the client, the SO_RCVBUF option must be set before connect.
For servers, the SO_RCVBUF option must be set before listen.

2.2 Using in C + +

int setsockopt (SOCKET s,int level,int optname,const char* optval,int optlen);

SOCKET socket = ... int nRcvBufferLen = 64*1024; int nSndBufferLen = 4*1024*1024; int nLen          = sizeof(int);

setsockopt(socket, SOL_SOCKET, SO_SNDBUF, (char*)&nSndBufferLen, nLen);
setsockopt(socket, SOL_SOCKET, SO_RCVBUF, (char*)&nRcvBufferLen, nLen);

The reliability of TCP

The outstanding characteristic of TCP is that the reliability is better, how to realize it mainly?
Reliability means no error, good reliability implies a strong fault tolerance.
A strong fault tolerance requires a backup, that is, to have a cache, so that the ability to support retransmission and so on.
Each socket has its own send buffer and receive buffer.
When the send and recv operations, immediately return, in fact, the data is not sent out, but stored in the corresponding send buffer and receive buffer immediately return to success.

A point in the documentation for send buffer indicates the UDP send buffer

"we show the socket send buffer as a dashed box because it doesn ' t really exist.
A UDP socket has a send buffer size (which we can change with the SO_SNDBUF socket option, section 7.5) imply a upper limit on the maximum-sized UDP datagram that can is writtento the socket.
If An application writes a datagram larger than the socket send buffer size, Emsgsize is returned.
Since UDP is unreliable, it does does need to keep a copy of the application's data and does not need an actual send buffer .
(The application data is normally copied to a kernel buffer
of some form as it passes down the protocol stack, but this copy was discarded by the datalink layer after the data is Tran smitted.) "
(UNIX?) Network Programming Volume 1, third edition:the Sockets Networking api,pub date:november 21, 2003)

According to the above "UNIX network programming First Volume" (This version is published in 2003, but did not query to other valid literature) description, for UDP, using the value set by the SO_SNDBUF, is the maximum value of the UDP message that can be written to the socket If the current program receives a message larger than the send buffer size, Emsgsize is returned.

function and meaning how the receive buffer is used

The receive buffer caches the data into the kernel, and the application process does not call read for reading, and the data is cached in the corresponding socket's receive buffer .
Again, regardless of whether the process reads the socket, the data sent to the end is received through the kernel and cached in the socket's kernel receive buffer.
The job of read is to copy the data from the kernel buffer into the buffer of the application-level user, and that's it.

Processing policy for receive buffers after buffer full

The receive buffer is used by TCP and UDP to cache the data on the network until the application process is read away.

• Tcp
  For TCP, if the application process has not been read and the buffer is full, the action that occurs is to notify the window in the peer TCP protocol to close. This is the implementation of the sliding window.
  Ensure that the TCP socket receive buffer does not overflow, thus ensuring that TCP is reliable transmission. Because the other party is not allowed to emit more than the advertised window size data. This is the traffic control of TCP, and if the other party ignores the window size and emits more data than the window size, the receiver TCP discards it.
• Udp
  When the socket receive buffer is full, the new datagram fails to enter the receive buffer, and the datagram is discarded . UDP is no traffic control, the fast sender can easily drown the slow receiver, causing the receiver's UDP drop datagram.

How the send buffer uses the send buffer

When the process invokes the data sent by send, the simplest case (and the general case) is to copy the data into the socket's kernel send buffer, and send will return on the upper level.
In other words,when send returns, the data is not necessarily sent to the peer (and the write file is a bit similar), and send simply copies the data from theapplication layer buffer into the socket's kernel and sends the buffer .
Each UDP socket has a receive buffer, no send buffer, conceptually, as long as there is data to send, regardless of whether the other party can receive correctly, so do not buffer, do not need to send buffer.

Size of the SO_SNDBUF

in order to achieve maximum network throughput, the socket send buffer size (SO_SNDBUF) should not be less than the product of bandwidth and latency .
I've had 2 performance problems before, all of which are related to the SO_SNDBUF setup too small.
However, when writing a program may not know how to set SO_SNDBUF, and SO_SNDBUF is not appropriate to set too big, wasting memory ah (yes?? )。

Operating system dynamic Adjustment So_sndbuf

As a result, the OS provides the ability to dynamically adjust the buffer size so that the application is no longer tuned to SO_SNDBUF. (Accepting buffer so_rcvbuf is a similar problem and should not be less than the product of bandwidth and latency).

Dynamic send buffering for TCP is added on Windows 7 and Windows Server R2. By default, the dynamic send buffering for TCP is enabled unless a application sets the so_sndbuf socket option on t He stream socket.

The newer OS supports the automatic adjustment of socket buffer without the need for application tuning . But for Windows 2012 (and Win8) before Windows, in order to achieve maximum network throughput, it is necessary for the application to worry about SO_SNDBUF settings.

Other than that

It is important to note that if the app is set so_sndbuf,dynamic send buffering will fail. https://msdn.microsoft.com/enus/library/windows/desktop/bb736549 (v=vs.85). aspx

Setting So_rcvbuf so_sndbuf to 0 is no good.

Let's look at what the system handles a typical send call when the send buffer size is Non-zero.
When an application makes a send call, if there is sufficient buffer space, the data was copied into the socket ' s send buff ERS, the call completes immediately with success, and the completion is posted.
on the other hand, if the socket ' s send buffer was full, then the application's send buffer is locked and the s End call fails with wsa_io_pending . after the data in the "Send buffer is processed" (for example, handed-to-TCP for processing), then Winsock would Process the locked buffer directly. That was, the data is handed directly to TCP from the application ' s buffer and the socket's send buffer is completely by PA Ssed .

As you can see when sending data, if the socket's send buffer (kernel layer) is full, the application's send buffer (application layer) is locked and the send call returns WSA_IO_PENDING.
When the data in send buffer has been processed, Winsock handles the locked send buffer (application layer) directly. In other words, the program skips the socket's send buffer and directly handles the program's buffer (application layer).

The opposite is true for receiving data. When a overlapped receive call is performed, if data have already been received on the connection, it'll be buffered in The socket ' s receive buffer. This data would be copied directly into the application ' s buffer (as much as would fit), the receive call returns success, a nd a completion is posted. However, if the socket ' s receive buffer is empty, when the overlapped receive call is made, the application ' s buffer i s locked and the call fails with wsa_io_pending. Once data arrives on the connection, it'll be a copied directly into the application ' s buffer, bypassing the socket ' s rece Ive buffer altogether.

The processing of the receive buffer is also true.

Setting the per-socket buffers to zero generally would not increase performanceBecause the extra memory copy can be avoided as long as there is always enough overlapped send and receive operations POS Ted. Disabling the socket ' s send buffer had less of a performance impact than disabling the receive buffer because the Applicat Ion's send buffer would always be locked until it can is passed down to the TCP for processing. However,if the receive buffer is set to zero and there Yes no outstanding overlapped receive calls, any incoming data can B Uffered on the TCP level. The TCP driver would buffer only up-to-the-receive window size, which is kb-tcp would increase these buffers as needed to this limit; Normally the buffers is much smaller.
These TCP buffers (one per connection) is allocated out of non-paged pool, which means if the server has connections And no receives posted at all, MB of the non-paged pool would be consumed!
The non-paged pool is a limited resource, and unless the server can guarantee there be always receives posted for a Conne Ction, the Per-socket receive buffer should is left intact.
Only in a few specific cases would leaving the receive buffer intact leads to decreased performance. Consider the situation in which a server handles many thousands of connections and cannot has a receive posted on each co Nnection (this can become very expensive, as you'll see on the next section). In addition, the clients send data sporadically. Incoming data buffered in the Per-socket receive buffer and when the server does issue an overlapped receive, it I s performing unnecessary work. The overlapped operation issues an I/O request packet (IRP) that completes, immediately after which notification is sent T o The completion port. In this case, the server cannot keep enough receives posted, so it's better off performing simple non-blocking receive CA Lls.

References:
Http://pubs.opengroup.org/onlinepubs/009695399/functions/setsockopt.html
Unix? Network Programming Volume 1, third edition:the Sockets Networking api,pub date:november 21, 2003
http://blog.csdn.net/xiaokaige198747/article/details/75388458
Http://www.cnblogs.com/kex1n/p/7801343.html
http://blog.csdn.net/summerhust/article/details/6726337

Flyingpenguin
Links: https://www.jianshu.com/p/755da54807cd
Source: Pinterest
The copyright of the book is owned by the author, and any form of reprint should be contacted by the author for authorization and attribution.

TCP and Udpsocket in So_sndbuf and So_rcvbuf