Similarities and differences between SO_REUSEADDR and So_reuseport

The article is from StackOverflow on the answer, written very detailed http://stackoverflow.com/questions/14388706/ Socket-options-so-reuseaddr-and-so-reuseport-how-do-they-differ-do-they-mean-t
Although there are some differences in how sockets are implemented on different systems, they all originate from the implementation of BSD sockets, so it is useful to understand the implementation of BSD sockets before discussing other systems. First we need to understand some basic knowledge, a TCP/UDP connection is determined by a five-tuple:
{, , , , }
Therefore, it is not possible for any two connections to have the same five-tuple, or the system will not differentiate between the two connections.
When creating a socket using the socket () function, we specify the Protocol (protocol) used by the socket, the bind () function sets the source address and the source port number, and the destination address and destination port number are set by the Connect () function. Although UDP is allowed to be "connected" (which in some cases is very helpful for the design of the application), because UDP is a non-connectivity protocol, UDP sockets can still be used without connection. A UDP socket that is not connected is not bound until the data is first sent, and is automatically bound by the system when it is sent, so an unbound UDP socket cannot receive (reply) data. The same is true for unbound TCP, which is automatically bound at the time of the connection.
If you explicitly bind a socket, it is possible to bind it to Port 0, which means "any port" ("arbitrary ports"). Since a socket cannot really be bound to all the ports on the system, in this case the system will have to select a specific port number (referring to "any port"). The source address uses a similar wildcard character, which is the "any address" (IPv4 in 0.0.0.0 and IPv6::). Unlike ports, a socket can be bound to any address, which refers to all addresses of the local network interface. Since the socket cannot bind to all source IP addresses at the same time, the system will have to pick a source IP address when the next one is connected. Taking into account the destination address and routing information in the routing table, the system will select an appropriate source address and replace any address with a selected address as the source address.
By default, any two sockets cannot be bound to the same source IP address and source port (that is, the source and source ports are the same). As long as the source port number is not the same, then the source address actually doesn't matter. Binding Socketa to address A and Port X (a:x), Socketb bound to address B and Port y (b:y), so long as X! = Y, then this binding is possible. However, as long as the x==y is a! = B, this binding is still possible, for example: an FTP server Socketa bound to 192.168.0.1:21 and a SOCKETB bound to another FTP server is 10.0.0.1:21, both bindings will succeed. Remember: A socket may be bound to the local "any address". For example, if a socket is bound to 0.0.0.0:21, then it binds all local addresses at the same time, in which case, no matter what particular IP address the other sockets choose, they cannot bind to port 21 because both 0.0.0.0 and all local addresses conflict.
That's the same for all the major operating systems. When it comes to address reuse, the differences between the OS appear, as previously stated, other implementations are derived from BSD implementations, so we start with BSD.

Bsd
So_reuseaddr

If SO_REUSEADDR is set before binding a socket, both bindings are feasible unless the source address and port number of the two socket bindings are the same. Maybe you'll wonder if it's different from what it was before? The key is that SO_REUSEADDR changes the way the wildcard address ("Any IP address") is handled when the source address conflict is processed.
When SO_REUSEADDR is not set, Socketa is bound to 0.0.0.0:21 and then Socketb bound to 192.168.0.1:21 will fail (eaddrinuse error) because 0.0.0.0 means " Any local IP address ", which is" all local IP addresses, "so all IP addresses, including 192.168.0.1, are considered to be in use. However, SOCKETB bindings will succeed after setting SO_REUSEADDR because 0.0.0.0 and 192.168.0.1 are not actually the same IP address, one is a wildcard address representing all addresses, and the other is a specific address. Note that the above statement is irrelevant for the binding order of Socketa and SOCKETB, without setting the SO_REUSEADDR, they will fail, set the SO_REUSEADDR, and it will succeed.
Here is a table listing all possible combinations:

SO_REUSEADDR       Socketa        socketb       Result---------------------------------------------------------------------on/off       192.168.0.1:21   192.168.0.1:21    Error (eaddrinuse) on/off       192.168.0.1:21      10.0.0.1:21    OK          on/off 10.0.0.1:21   192.168.0.1:21    OK   OFF             0.0.0.0:21   192.168.1.0:21    Error (eaddrinuse)   OFF         192.168.1.0:21       0.0.0.0:21    Error (eaddrinuse)   on              0.0.0.0:21   192.168.1.0:21    OK   on          192.168.1.0:21       0.0.0.0:21    OK       on/off 0.0.0.0:21 0.0.0.0:21    Error (eaddrinuse)

The table above assumes that Socketa has been successfully bound, and then creates a SOCKETB binding the result of the given address in the case of setting so_reuseaddr. Result indicates whether the binding behavior of SOCKETB will succeed. If the first column is on/off, then the value of SO_REUSEADDR will be irrelevant.

Now we know that SO_REUSEADDR has an impact on the distribution address, but this is not the only aspect that it affects. There is also a well-known impact that is also the primary reason most people use SO_REUSEADDR on server programs. To understand how other so_reuseaddr are important to use, we need to know more about how the TCP protocol works.
A socket has a send buffer, and when the call to the Send () function succeeds, this does not mean that all data is actually sent out, it only means that the data is sent to the send buffer. For UDP sockets, the data is usually sent out quickly if not sent immediately, but for a TCP socket, the delay between the data being added to the buffer and actually being sent is quite long. This leads to the possibility that when we close a TCP socket, the data waiting to be sent is saved in the send buffer (due to the successful return of Send (), so you may think that the data has been sent). If the TCP implementation is to shut down the socket immediately, then all of this data will be lost and your program is simply impossible to know. TCP is known as a reliable protocol, and the way that data is lost is less reliable. This is why when we close a TCP socket, if it still has data waiting to be sent, then the socket will enter the TIME_WAIT state. This state will persist until the data is all sent or a timeout occurs.
The total time to wait before the kernel shuts down the socket completely (regardless of whether there is data waiting to be sent in the send buffer) is called linger. Linger time is a global configuration item on most systems and is quite long by default (two minutes on most systems). Of course, for each socket we can also use the socket option So_linger to configure, you can set the wait time a little longer or a little bit or even disable it. Disabling linger time is definitely a bad idea, although gracefully closing the socket is a slightly more complex process and involves sending packets back and forth (and resending them after the packet is lost), and the process is constrained by the linger time. If disabling linger Time,socket may be missing more than just the data to be sent, but it will also be rude to close the socket, and in most cases it should not be used. The details of how to gracefully close the TCP connection are not discussed here, and if you want to learn more, I suggest you read: Http://www.freesoft.org/CIE/Course/Section4/11.htm. And if you disable linger time with So_linger, and your program terminates before you explicitly close the socket, BSD (and possibly other systems) will still wait, regardless of whether it has been disabled. An example of this is when your program calls exit (which is common in small server programs) or when a process is signaled (or it can be terminated by a process accessing illegal memory). In this case, you cannot disable linger for a particular socket, no matter what the circumstances.
The question is, how does the system view the TIME_WAIT state? If SO_REUSEADDR is not set yet, a socket in time_wait is still considered bound to the source address and port, and any other attempt to bind a socket on the same address and port will fail until the original socket is actually closed. This usually takes a long time to wait for linger. So don't expect to bind the source address and port to a new socket immediately after a socket is closed, and in most cases this behavior will fail. However, attempting to bind (bind the same address and port) after setting up SO_REUSEADDR will only be ignored, and you can bind the same address to a different socket. Note that when a socket is in the TIME_WAIT state and you try to bind it to the same address and port, this can lead to unexpected results because the socket in the TIME_WAIT state is still "working", and fortunately this rarely happens.
The last thing you need to know about SO_REUSEADDR is that only the socket you want to bind has the address reuse enabled, but it doesn't have to be checked before it's been bound or time_ Wait sockets also set this option when they are bound. In other words, the success of the binding will only check that the current BIND socket has this flag turned on, and will not look at any other sockets.

So_reuseport

The meaning of So_reuseport is the same as most people's understanding of SO_REUSEADDR. Basically, So_reuseport allows you to bind multiple sockets to the same address and port as long as they are set so_reuseport before binding. If the first socket that binds an address and port is not set So_reuseport, then the other socket cannot bind to the address and port regardless of the setting so_reuseport until the first socket releases the binding.
So_reuseport does not mean SO_REUSEADDR. This means that if a socket is not set so_reuseport at bind time, as expected, the other sockets will fail to bind to the same address and port, but if the socket that binds the same address and port is in the TIME_WAIT state, The new binding will also fail. When a socket is in the TIME_WAIT state (when released), in order for the other sockets to bind the same address and port to succeed, you need to set the SO_REUSEADDR or set So_reuseport on the two sockets. Of course, it is also possible to set both So_reuseport and SO_REUSEADDR on the socket.
About So_reuseport In addition to the time it was added to the system than so_reuseport late there is no other need to say, which is why in some system socket implementation you can not find this option, Because the code for these systems is to fork the BSD before this option is added to BSD, it is not possible to bind two sockets to the really identical "address" (Address+port).

Connect () returning eaddrinuse?

Most people know that bind () may fail to return eaddrinuse, but when you start using address reuse, you may encounter strange situations: Connect ()
Failure returns the same error eaddrinuse. How did this happen? A remote address, after all, is a connect added to the socket, how is it already used? Connecting multiple sockets to the same remote address never happened, why is that?
As I said at the beginning, a connection is defined by a five-tuple. I also said that the five-tuple of any two connections could not be exactly the same, because then the kernel would not be able to differentiate between the two connections. However, in the case of address reuse, you can bind two sockets of the same protocol to the exact same source address and source port, which means that there are already three elements in the five tuple (protocol, source address, source port). If you try to connect these sockets to the same destination and destination ports, you create two identical connections. This is not possible, at least not for TCP (UDP does not actually have a real connection). If the data reaches either of these two connections, then the system will not be able to differentiate who the data belongs to. Therefore, when the source and source ports are the same, the destination address or destination port must be different, otherwise the kernel cannot differentiate, in which case the Connect () will return eaddrinuse when the second socket tries to connect.

multicast address (multicast addresses)

Most people will ignore the existence of multicast addresses, but they do exist. Unicast addresses (unicast address) are used for single-to-single communication, and multicast addresses are used for single-to-multiple communication. Most people notice the existence of multicast addresses after they have studied IPv6, but there are multicast addresses in IPv4, although they do not use much on the public Internet.
For multicast addresses, the meaning of so_reuseaddr has changed because it allows multiple sockets to bind to exactly the same multicast address and port, that is, the behavior of SO_REUSEADDR to the multicast address is exactly the same as So_reuseport for unicast addresses. In fact, for multicast addresses, the processing of SO_REUSEADDR and So_reuseport is exactly the same, and for all multicast addresses, SO_REUSEADDR also means so_reuseport.

Freebsd/openbsd/netbsd

They are all very late in the system derived from the native BSD, and they are the same as the native BSD options and behavior.

MacOS X

MacOS X's kernel is a BSD-type UNIX, based on a very new BSD code, and even Mac OS 10.3 release and FreeBSD 5 are synchronous, so MacOS and BSD provide the same options, processing behavior is the same.

Ios

iOS just slightly modifies MacOS on the kernel, so the options and processing behavior are the same as MacOS.

Linux

Prior to Linux 3.9, only the option so_reuseaddr existed. Apart from the two important differences, it is generally the same as BSD. The first difference: when a listening (listening) TCP socket is bound to a wildcard address and a specific port, regardless of the other socket or all sockets (including the listening socket) are set SO_REUSEADDR, the other TCP Sockets cannot be bound to the same port (BSD), not to mention the use of a specific address. This restriction is not used on non-listening TCP sockets, when a listener socket is bound to a specific address and port combination, and then the other socket is bound to the wildcard address and the same port, which is possible. The second difference: when the so_reuseaddr is used on a UDP socket, it behaves exactly the same as BSD so_reuseport, so two UDP sockets can be bound to the same address and port as long as they have so_reuseaddr set.
Linux 3.9 joins the So_reuseport. This option allows multiple sockets (TCP or UDP) to be bound to the exact same address and port as long as it is set before binding, whether it is a listening socket or a non-listener socket. There is a special limitation in order to prevent Port hijacking: All sockets wishing to share the source address and port must have the same valid user ID (effective user ID). Therefore, one user cannot "steal" the port from another user. In addition, the kernel handles the so_reuseport socket using "special magic" that is not used on other systems: for UDP sockets, the kernel tries to transmit the average forwarding datagram, and for TCP listener sockets, the kernel attempts to request a new client connection ( Returned by accept), on average, to the socket (listening socket) that shares the same address and port. This means that on other systems the socket receives a datagram or connection request that is more or less random, but Linux tries to optimize the allocation. For example, multiple instances of a simple server program can use the So_reuseport socket to achieve a simple load balancer because the kernel has already done the copy assignment.

Android

Although the entire Android system is different from most Linux distributions, its kernel is a slightly modified Linux kernel, so its so_reuseaddr and So_reuseport are the same as Linux.

There is only so_reuseaddr option on Windows Windows, no so_reuseport. The SO_REUSEADDR socket on Windows has the same behavior as the BSD set So_reuseport and SO_REUSEADDRD, with only one difference: a set of So_ The REUSEADDR socket can always be bound to the source address and source port that have been bound, regardless of whether the socket that was previously tied to this address and port is set to SO_REUSEADDR. This behavior is somewhat dangerous because it allows an application to "steal" a connected port from another application. Needless to say, this has a great impact on security, and Microsoft is aware of this issue and joins another socket option: So_execlusiveaddruse. The so_execlusiveaddruse socket is set to ensure that once the binding is successful, the source port and address are bound to this socket only, the other sockets can not be bound, even if they use the so_reuseaddr is useless. Solaris Solaris is SunOS, SunOS originated in Bsd,sunos 5 and later is based on SVR4, but SVR4 is Bsd,system V and Xenix, so in a way, Solaris is also a BSD branch , and it's quite an early branch. This resulted in Solaris being only so_reuseaddr and not so_reuseport. The behavior of SO_REUSEADDR on Solaris is very similar to that of BSD. From what I know, there is no way to implement So_reuseport on Solaris, which means that it is not possible to bind two sockets to the same source address and port.
Like Windows, Solaris has an option to provide mutually exclusive bindings, which is called So_exclbind. If you set this option before a socket is bound, setting so_reuseaddr on the other sockets will have no effect. For example, Socketa binds a wildcard address, SOCKETB sets the so_reuseaddr and binds to a non-wildcard address and the same port, then this binding succeeds unless Socketa is set to So_exclbind, in which case The SOCKETB binding will fail regardless of whether it is set to SO_REUSEADDR.

Similarities and differences between SO_REUSEADDR and So_reuseport