Details of the differences between So_reuseport and SO_REUSEADDR in sockets

Source: Internet
Author: User
Below for you to share an elaborate socket in the so_reuseport and so_reuseaddr differences, with a very good reference value, I hope to be helpful to everyone. Come and see it together.

Basic background of socket

In discussing the differences between these two options, we need to know that the BSD implementation is the origin of all socket implementations. Basically all of the other systems refer to the BSD socket implementations (or at least their interfaces) to some extent, and then start their own independent evolution. It is clear that BSD itself is constantly evolving over time. So the late reference BSD system more early reference BSD system more features. So understanding the BSD socket implementation is the cornerstone of understanding other sockets implementations. Let's analyze the BSD socket implementation.

Before that, we first need to understand how to uniquely identify the TCP/UDP connection. TCP/UDP is uniquely identified by the following five tuples:


{<protocol>, <src addr>, <src port>, <dest addr>, <dest Port>}


Any unique combination of these values can be used to make a single connection. Then, for any connection, none of the five values will be identical. Otherwise, the operating system cannot differentiate these connections.

The protocol for a socket is set when the socket () is initialized. The source address and source port are set when the bind () is called. The destination (destination address) and destination port (destination port) are set when calling Connect (). Where UDP is not connected, the UDP socket can be used without connection to the destination port. However, UDP can also be used in some cases after establishing a connection with the destination address and port. In the case where data is sent using no-connection UDP, if bind () is not explicitly called, the system will automatically bind the UDP socket to the local address and a port when the data is first sent (otherwise the program cannot accept any data that the remote host replied to). Similarly, a TCP socket without a bound address is automatically bound to a native address and port when the connection is established.

If we bind a port manually, we can bind the socket to port 0 and bind to port 0, which means letting the system decide which port to use (typically from a set of operating system-specific, pre-determined number of ports), so it is the meaning of any port. Similarly, we can use a wildcard character to let the system decide which source address to bind to (the IPv4 wildcard is the 0.0.0.0,ipv6 wildcard character::). Unlike a port, a socket can be bound to any of the addresses of all the interfaces on the host. Based on the corresponding information in the destination address and routing table of this socket, the operating system will select the appropriate address to bind the socket and replace the previous wildcard IP address with this address.

By default, any two sockets cannot be bound to the same source address and source port combination. For example, we bind the Socketa to the a:x address, bind the SOCKETB to the B:y address, where A and B are the IP addresses, and x and Y are the ports. Then in the case of a==b x!=y must be satisfied, in the case of x==y a!=b must be satisfied. It is important to note that if a socket is bound to a wildcard IP address, then in fact all IP in this machine is considered to be bound by the system. For example, a socket is bound to a 0.0.0.0:21, in which case any other socket can no longer bind to port 21 regardless of which specific IP address is selected. Because the wildcard IP0.0.0.0 conflicts with all local IPs.

All of the above content is essentially the same in the main operating system. And each of the SO_REUSEADDR will have different meanings. First, let's discuss the BSD implementation. Because BSD tries the source of all other socket implementations.

Bsd

So_reuseaddr

If the So_reuseaddr property of a socket is set before it is bound to an address and port, then unless this socket is in conflict with the combination of the source address and the source port that caused the attempt to bind to the other socket exactly the same. Otherwise, the socket can be successfully bound to this address port pair. It sounds as if it were the same as before. But the keywords are completely. So_reuseaddr mainly changes the way the system treats wildcard IP address conflicts.

Without so_reuseaddr, if we bind Socketa to 0.0.0.0:21, any move that binds the other socket of the native to Port 21, such as binding to 192.168.1.1:21, will result in a eaddrinuse error. Because 0.0.0.0 is a wildcard IP address, it means any IP address, so any other IP address on this computer is considered to be occupied by the system. If the SO_REUSEADDR option is set, because 0.0.0.0:21 and 192.168.1.1:21 are not identical address port pairs (one is a wildcard IP address and the other is a native IP address), such a binding can be successful. It is important to note that, regardless of the order in which the Socketa and SOCKETB are initialized, the bindings will succeed as long as the SO_REUSEADDR is set, and the bindings will not succeed as long as the SO_REUSEADDR is not set.

The following table lists some of the possible scenarios and their results.


so_reuseaddr Socketa Socketb Result
On/Off 192.168.1.1:21 192.168.1.1:21 ERROR (Eaddrinuse)
On/Off 192.168.1.1:21 10.0.1.1:21 Ok
On/Off 10.0.1.1:21 192.168.1.1:21 Ok
OFF 192.168.1.1:21 0.0.0.0:21 ERROR (Eaddrinuse)
OFF 0.0.0.0:21 192.168.1.1:21 ERROR (Eaddrinuse)
On 192.168.1.1:21 0.0.0.0:21 Ok
On 0.0.0.0:21 192.168.1.1:21 Ok
On/Off 0.0.0.0:21 0.0.0.0:21 Ok

This table assumes that Socketa has successfully bound the corresponding address in the table, and then SOCKETB is initialized with the SO_REUSEADDR setting as shown in the first column of the table, and then socketb trying to bind the corresponding address in the table. The result column is the outcome of its binding. If the value in the first column is on/off, then the SO_REUSEADDR setting is independent of the result.

The effect of SO_REUSEADDR on the wildcard IP address is discussed above, but it is not the only function. Another function is why the SO_REUSEADDR option is used when you are doing server-side programming. To understand its other role and its important applications, we need to discuss in more depth how the TCP protocol works.

Each socket has its corresponding send buffer (buffer). When the Send () method is successfully called, the data we request is not necessarily sent immediately, but is added to the send buffer. For UDP sockets, even if they are not sent immediately, the data is usually sent out quickly. However, for a TCP socket, after adding data to the send buffer, it may be necessary to wait for a relatively long time before the data is actually sent. Therefore, when we close a TCP socket, it may actually still have data waiting to be sent in its send buffer. But at this point, because Send () returns success, our code thinks the data has actually been sent successfully. If the TCP socket shuts down directly after we call Close (), all of this data will be lost and our code will never know. However, TCP is a reliable transport layer protocol, and it is obviously undesirable to discard the data to be transmitted directly. In fact, if its close () method is called in the socket's send buffer, it will enter a so-called time_wait state if it has yet to send the data. In this state, the socket will continue to attempt to send the buffer's data until all data has been successfully sent, or until the timeout is triggered, the socket will be forced to close.

The maximum wait time for an operating system's kernel before forcing a socket to close is called a delay time (Linger times). In most systems, the delay time has been set globally and is relatively long (most systems set it to 2 minutes). We can also use the So_linger option when initializing a socket to specifically set the delay time for each socket. We can even turn off latency altogether. However, it is important to note that setting the delay time to 0 (full shutdown delay wait) is not a good programming practice. Because gracefully shutting down a TCP socket is a complex process that involves exchanging several packets with a remote host (including lost retransmissions in case of packet loss), and the time required for this packet exchange is also included in the delay time. If we deactivate the delay wait, the socket will not only discard all data to be sent when it is closed, but will always be forced to shut down (because TCP is a connection-oriented protocol, not switching off the packet with the remote port will cause the remote port to be in a long wait state). So usually we don't recommend doing this in real programming. The TCP disconnection process is beyond the scope of this article, and if you are interested, you can refer to this page. And in fact, if we disable delay waiting, and our program exits without explicitly shutting down the socket, BSD (which may include other systems) ignores our settings for deferred wait. For example, if our program calls the exit () method, or its process is terminated with a signal (including a process that crashes because of an illegal memory access). So we can't guarantee that a socket will terminate in all cases ignoring the delay wait time.

The problem here is how the operating system treats sockets in the time_wait phase. If the SO_REUSEADDR option is not set, the socket in the time_wait phase is still considered to be bound to the original address and port. Until the socket is completely closed (ending the time_wait phase), any other attempt to bind a new socket to that address port pair is unsuccessful. This waiting process may be as long as the delay is waiting. So we can't immediately bind a new socket to the address port pair of the socket that was just closed. In most cases, this operation will fail.

However, if we set the SO_REUSEADDR option on the new socket, if there is another socket bound at the current address port pair and is in the time_wait phase, then this existing binding relationship will be ignored. In fact, the socket in the time_wait phase is already semi-closed, and binding a new socket to this address port pair will not have any problem. In this case, the socket that was originally bound to this port would not have an effect on the new socket. However, it is important to note that at some point, binding a new socket to an address port corresponding to a socket in the time_wait phase but still working will produce some undesirable, unanticipated negative effects that we do not want. But this problem is beyond the scope of this article. And fortunately, these negative effects are seldom seen in practice.

Finally, one thing we should note about SO_REUSEADDR is that all of the above content is set up as soon as we have so_reuseaddr on the new socket. There is no effect on whether or not the socket in the time_wait phase is set SO_REUSEADDR if the original is already bound to the current address port pair. The code that determines whether the bind operation succeeds only checks the new SO_REUSEADDR option of the socket that is passed to the bind () method. Other SO_REUSEADDR options related to the socket are not checked.

So_reuseport

Many people regard so_reuseaddr as a so_reuseport. Basically, So_reuseport allows us to bind any number of sockets to the exact same source address port pair, as long as all the previously bound sockets have the So_reuseport option set. If the first socket on the port pair of the address is not set so_reuseport, it cannot be bound to an address that is identical to the address port, regardless of whether the socket is set So_reuseport. This binding relationship is freed unless the first socket on the port pair of this address is bound. Unlike SO_REUSEADDR, the code that handles So_reuseport not only checks the so_reuseport of the socket currently attempting to bind, but also checks the socket that was previously bound to the address port pair that is currently attempting to bind So_ Reuseport option.

So_reuseport is not equal to SO_REUSEADDR. The implication is that if a socket that already has an address is not set so_reuseport, and the other new socket is set to so_reuseport and attempts to bind to the same port address pair as the current socket, the bind attempt will fail. Also, if the current socket is already in the time_wait phase and the new socket with the So_reuseport option is attempting to bind to the current address, the bind operation will fail. In order to be able to bind the new socket to an address port pair that is currently in the time_wait phase of the socket, we either need to set the SO_REUSEADDR option for this new socket before binding. Either you need to set the So_reuseport option to two sockets before binding. Of course, it is also possible to set the SO_REUSEADDR and So_reuseport options for the socket at the same time.

The So_reuseport is added to the BSD system after so_reuseaddr. This is why there are currently no so_reuseport options in the socket implementations of some systems. Because they refer to the BSD socket implementation before this option is added to the BSD system. Before this option was added, there was no way for the BSD system to bind two sockets to the exact same address port pair.

Connect () return eaddrinuse?

There are times when the bind () operation returns a Eaddrinuse error. But strangely enough, it is also possible to get a eaddrinuse error when we call the Connect () operation. What is this for? Why is a remote address that we try to make a connection to the current port also be occupied? Does the operation of connecting multiple sockets to the same remote address cause problems?

As stated earlier in this article, a connection relationship is determined by a five-tuple. For any connection relationship, this five-tuple must be unique. Otherwise, the system will not be able to distinguish between two connections. Now when we use address multiplexing, we can bind two sockets with the same protocol to the same address port pair. This means that for these two sockets, {<protocol>, <src addr>, <src Port>} are already the same for the five tuples. In this case, if we try to connect them all to the same remote address port, the five-tuple of the two connection relationships will be exactly the same. In other words, two identical connections are generated. This is not allowed in the TCP protocol (UDP is not connected). If one of the two identically connected species receives the data, the system will not be able to tell which connection the data belongs to. So in this case, at least the address and port of the remote host to which the two sockets are trying to connect cannot be the same. Only then can the system continue to differentiate between the two connection relationships.

So when we bind two sockets with the same protocol to the same local address port pair, if we try to make them connect to the same destination address port pair, the second socket that tries to call the Connect () method will report Eaddrinuse errors. This means that a socket with exactly the same five-tuple already exists.

Multicast Address

The multicast address is used for one-to-many communications relative to the unicast address used for one-to-one communication. Both IPV4 and IPV6 have multicast addresses. But the multicast in IPv4 is actually rarely used on public networks.

The meaning of so_reuseaddr is different in the case of multicast address. In this case, SO_REUSEADDR allows us to bind multiple sockets to the exact same source broadcast address port pair. In other words, for multicast addresses, SO_REUSEADDR is equivalent to So_reuseport in unicast communication. In fact, in the case of multicast, so_reuseaddr and So_reuseport function exactly the same.

Freebsd/openbsd/netbsd

All of these systems refer to the newer native BSD system code. So these three systems offer exactly the same socket options as BSD, which have the same meaning as the original BSD.

MacOS X

MacOS X's core code implementation is based on a newer version of the original BSD BSD-style UNIX, so MacOS X provides the same socket options as BSD, and they also have the same meaning as BSD systems.

Ios

iOS is actually a slightly modified MacOS X, so iOS is also available for MacOS X.

Linux

Before Linux3.9, only the SO_REUSEADDR option exists. This option is essentially the same as the BSD system. But there are still two important differences.

The first difference is if a TCP socket in the listening (server) state is already bound to a wildcard IP address and a specific port, then no matter whether these two sockets have the SO_REUSEADDR option set, any other TCP The socket can no longer be bound to the same port. Even if another socket uses a specific IP address (as allowed in the BSD system). The non-listener (client) TCP socket does not have this limitation.

The second difference is that for UDP sockets, the SO_REUSEADDR function is exactly the same as in BSD So_reuseport. So if all two UDP sockets are set SO_REUSEADDR, they can be bound to a set of identical address port pairs.

Linux3.9 added the So_reuseport option. As long as all sockets (including the first one) have this option set before the binding address, two or more, TCP or UDP, a listener (server) or a non-listener (client) socket can be bound to the exact same address port combination. Also, in order to prevent port hijacking, there is a special restriction: All sockets attempting to bind to the same address port combination must belong to a process with the same user ID (hijacking). So one user cannot "steal" ports from another user.

In addition to this, Socket,linux kernel, which has the So_reuseport option set, also performs special operations that are not performed by other systems: for UDP sockets that are bound to the same address port combination, Kernel attempts to evenly distribute the packets received between them, and for TCP listeners bound to the same address port combination Socket,kernel Try to distribute the received connection requests evenly between them (calls to the Accept () method request). This means that Linux attempts to optimize traffic allocation compared to other systems that allow address multiplexing but randomly receive packets or connection requests to a socket connected to the same address port combination. For example, a few different instances of a simple server process can easily use So_reuseport to achieve a simple load balancing, and this load balancer is kernel responsible, completely free for the program!

Android

The core part of Android is a slightly modified Linux kernel, so all Linux-suited operations also apply to Android.

Windows

Windows has only the SO_REUSEADDR option. Setting SO_REUSEADDR on a socket in Windows is the same as setting So_reuseport and SO_REUSEADDR for a socket at the same time in BSD. But the difference is that even if another socket with a bound address is not set SO_REUSEADDR, a socket with SO_REUSEADDR can always be bound to the same address port combination as the other bound socket. This behavior can be said to be somewhat dangerous. Because it allows one app to steal data from another reference connected port. Microsoft is aware of this issue, so another socket option has been added: So_exclusiveaddruse. Setting the So_exclusiveaddruse on a socket ensures that once the socket is bound to an address port combination, any other socket, regardless of the setting so_reuseaddr, can no longer bind the current address port combination.

Solaris

Solaris is the successor of SunOS. SunOS is also a branch of an earlier version of BSD in some way. Solaris therefore only provides SO_REUSEADDR, and its performance is essentially the same as in BSD systems. As far as I know, the same functionality as So_reuseport cannot be implemented in a Solaris system. This means that two sockets cannot be bound to the exact same address port combination in Solaris.

Similar to Windows, Solaris also provides an exclusive binding option for the socket--so_exclbind. If a socket sets this option before the binding address, even if the other sockets are set SO_REUSEADDR will not be able to bind to the same address. For example, if Socketa is bound to a wildcard IP address and SOCKETB is set to SO_REUSEADDR and is bound to a combination of a specific IP address and the same port as Socketa, this operation is not set in Socketa So_ The exclbind will succeed, or it will fail.

Reference:

http://stackoverflow.com/a/14388707/6037083


Related Article

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.