Detailed description of the maximum number of TCP connections for Linux servers

Network programming
In a TCP application, the server listens to a fixed port beforehand, the client initiates the connection, and a TCP connection is established after a three-way handshake. So what is the maximum number of concurrent TCP connections for a single machine?

How to identify a TCP connection
Before determining the maximum number of connections, let's look at how the system identifies a TCP connection. The system uniquely identifies a TCP connection using a 44-tuple: {Local IP, local port,remote ip,remote port}.

Client maximum number of TCP connections
Each time the client initiates a TCP connection request, unless the port is bound, the system usually picks up an idle local port, which is exclusive and cannot be shared with other TCP connections. The data type of the TCP port is unsigned short, so the maximum number of local ports is only 65536, Port 0 has a special meaning, can not be used, so that the available ports up to 65535, so in the case of all as the client side, the maximum number of TCP connections is 65535, These connections can be connected to different server IPs.

Server maximum number of TCP connections
The server is usually fixed on a local port for listening, waiting for a client connection request. Regardless of address reuse (Unix's so_reuseaddr option), the local listener port is exclusive even if there are multiple IPs on the server side, so only the remote IP (that is, the client IP) and the remote in the server-side TCP connection 4-tuple Port (client port) is mutable, so the maximum TCP connection is the client IP number x client port number, to IPV4, regardless of the IP address classification and other factors, the maximum number of TCP connections is about 2 32 (IP number) x2 16 (port number), That is, the server-side stand-alone maximum number of TCP connections is about 2 48-time side.

So how many concurrent TCP connections can be on a single server

The above is given is the theoretical maximum number of single-machine connections, in the actual environment, by the computer resources, operating system, and so on, especially the sever end, the maximum number of concurrent TCP connections can not reach the theoretical limit. The main factors that limit the number of connections under Unix/linux are memory and the number of file descriptors allowed (each TCP connection consumes a certain amount of memory, each socket is a file descriptor), and a port below 1024 is usually a reserved port. Under the default 2.6 kernel configuration, the 15~20k between each socket occupies within the test.
The parameters that affect the memory of a socket include:
Rmem_max
Wmem_max
Tcp_rmem
Tcp_wmem
Tcp_mem
grep skbuff/proc/slabinfo
On the server side, by increasing the memory, modify the maximum number of file descriptors and other parameters, the maximum number of concurrent TCP connections more than 100,000 is no problem, foreign Urban Airship Company in the product environment has done 500,000 concurrency. In the practical application, the c10k problem should be considered in the large-scale network application.

The following is a detailed explanation of file handle restrictions and port restrictions two common sense

Common sense One: file handle restrictions

A friend who writes a Web server program under Linux must know that every TCP connection takes up a file descriptor, and once this file descriptor is used up, the error that the new connection comes back to us is "Socket/file:can ' t open so many files".

At this point you need to understand the operating system's limit on the maximum number of files that can be opened.

Process limits

Perform ulimit-n output 1024, which means that you can open up to 1024 files for a process, so you will be able to use this default configuration up to thousands of TCP connections.

Temporary modification: ulimit-n 1000000, but this temporary modification is only valid for the current logged-on user's current usage environment, the system restarts, or the user exits and expires.

Modified after reboot (but I tested under CentOS 6.5, no expiration was found after reboot): Edit the/etc/security/limits.conf file and modify the content to

* Soft Nofile 1000000

* Hard Nofile 1000000

Permanent modification: Edit/etc/rc.local and add the following after

Ulimit-shn 1000000

Global limits

Perform CAT/PROC/SYS/FS/FILE-NR output 9344 0 592026, respectively: 1. Number of file handles already allocated, 2. Number of file handles that have been allocated but not used, 3. Maximum number of file handles. But the value of the second item in kernel 2.6 is always 0, which is not an error, it actually means that the file descriptor that has been allocated has been used without a waste.

We can change this to a larger number, with root permission to modify the/etc/sysctl.conf file:

Fs.file-max = 1000000

Net.ipv4.ip_conntrack_max = 1000000

Net.ipv4.netfilter.ip_conntrack_max = 1000000

Common sense Two: port number range limit?

Operating system top number 1024 The following is reserved for the system, and from 1024-65535 is used by the user. Because each TCP connection takes up a port number, we can have up to 60,000 concurrent connections. I think there are a few friends who have this wrong idea? (which I have been thinking of in the past)

Let's analyze that.

How to identify a TCP connection: The system uniquely identifies a TCP connection using a 44-tuple: {Local IP, local port,remote ip,remote port}. Well, we take Out "UNIX Network programming: Volume One" in the fourth chapter of the explanation of accept to see the concept of things, the second parameter cliaddr represents the client's IP address and port number. And we as the server actually only use the bind when this port, the port number 65535 is not the limit of concurrency.

Server maximum number of TCP connections: server is usually fixed on a local port for listening, waiting for client connection requests. Regardless of address reuse (Unix's so_reuseaddr option), the local listener port is exclusive even if there are multiple IPs on the server side, so only the remote IP (that is, the client IP) and the remote in the server-side TCP connection 4-tuple Port (client port) is mutable, so the maximum TCP connection is the client IP number x client port number, to IPV4, regardless of the IP address classification and other factors, the maximum number of TCP connections is about 2 32 (IP number) x2 16 (port number), That is, the server-side stand-alone maximum number of TCP connections is about 2 48-time side.

Detailed description of the maximum number of TCP connections for Linux servers