Linux (centos) network Kernel Parameter Optimization to improve server concurrent processing capability

Introduction

There are many ways to improve server performance, such as dividing image servers, Master/Slave database servers, and website servers on servers. However, when the hardware resources are rated limited, many O & M technicians think about how to maximize the performance of the server and improve the concurrent processing capability of the server. To improve the load capacity in Linux, you can use nginx and other Web servers with strong native concurrent processing capabilities. If Apache is used, you can enable its worker mode to improve its concurrent processing capabilities. In addition, you can modify the TCP Parameters related to the Linux kernel to maximize server performance when saving costs. Of course, upgrading the server hardware is the most fundamental problem to improve the load.

Time_wait

In Linux, after a TCP connection is disconnected, the port is released only after a certain period of time is retained in the time_wait status. When there are too many concurrent requests, a large number of connections in the time_wait status will be generated. If the connection cannot be closed in time, a large amount of port resources and server resources will be occupied. At this time, we can optimize the TCP kernel parameters to promptly clear the port in the time_wait status.

The method described in this article is only effective for system resource consumption caused by connections with a large number of time_wait States. Otherwise, the effect may not be obvious. You can run the netstat command to check the connection status of time_wait. Enter the combined command below to check the status of the current TCP connection and the number of corresponding connections:

Netstat-N | awk '/^ TCP/{++ s [$ NF]} end {for (a in S) print a, s [a]}'

This command will output results similar to the following:
Last_ack 16
Syn_recv 348
Established 70
Fin_wait1 229
Fin_wait2 30
Closing 33
Time_wait 18098
We only care about the number of time_wait instances. here we can see that there are more than 18000 time_wait instances, occupying more than 18000 ports. You must know that the number of ports is only 65535, and the number of ports is one less, which seriously affects subsequent new connections. In this case, it is necessary to adjust the Linux TCP kernel parameters so that the system can release the time_wait connection more quickly.

Use Vim to open the configuration file: # Vim/etc/sysctl. conf

Add the following lines to this file:
Net. ipv4.tcp _ syncookies = 1
Net. ipv4.tcp _ tw_reuse = 1
Net. ipv4.tcp _ tw_recycle = 1
Net. ipv4.tcp _ fin_timeout = 30

Enter the following command to make the kernel parameters take effect: # sysctl-P

Briefly describe the meaning of the above parameters:

Net. ipv4.tcp _ syncookies = 1
# Enable syn cookies. When a SYN wait queue overflows, cookies are enabled to prevent a small number of SYN attacks. The default value is 0, indicating that the process is disabled;
Net. ipv4.tcp _ tw_reuse = 1
# Indicates enabling reuse. Allow time-Wait sockets to be re-used for a New TCP connection. The default value is 0, indicating that the TCP connection is disabled;
Net. ipv4.tcp _ tw_recycle = 1
# Indicates enabling quick time-Wait sockets recovery in TCP connections. The default value is 0, indicating that the time-Wait sockets is disabled;
Net. ipv4.tcp _ fin_timeout
# Modify the default system timeout time.

After such adjustments, in addition to further improving the server load capabilities, it can also defend against DoS, CC, and SYN attacks with low traffic.

In addition, if you have many connections, we can optimize the range of available TCP ports to further improve the server's concurrency. Add the following configurations to the above parameter file:
Net. ipv4.tcp _ keepalive_time = 1200
Net. ipv4.ip _ local_port_range = 10000 65000
Net. ipv4.tcp _ max_syn_backlog = 8192
Net. ipv4.tcp _ max_tw_buckets = 5000
# We recommend that you enable these parameters only on servers with extremely high traffic, which will produce remarkable results. In general, there is no need to set these parameters on servers with low traffic.

Net. ipv4.tcp _ keepalive_time = 1200
# Indicates the frequency of keepalive messages sent by TCP when keepalive is in use. The default value is 2 hours, which is changed to 20 minutes.
Net. ipv4.ip _ local_port_range = 10000 65000
# Indicates the port range used for external connection. The default value is small: 32768 to 61000, Which is changed to 10000 to 65000. (Note: Do not set the minimum value too low here, otherwise it may occupy the normal port !)
Net. ipv4.tcp _ max_syn_backlog = 8192
# Indicates the length of the SYN queue. The default value is 1024. The length of the queue is 8192, which can accommodate more network connections waiting for connection.
Net. ipv4.tcp _ max_tw_buckets = 6000
# Indicates that the system maintains the maximum number of time_wait instances at the same time. If this number is exceeded, time_wait is immediately cleared and warning information is printed. Mo believes that 180000 is changed to 6000. For servers such as Apache and nginx, the number of time_wait sockets can be greatly reduced by parameters in the previous lines, but the effect on squid is not great. This parameter can control the maximum number of time_wait instances to prevent the squid server from being dragged to death by a large number of time_wait instances.

Other kernel TCP Parameters:
Net. ipv4.tcp _ max_syn_backlog = 65536
# The maximum number of connection requests that have not received confirmation from the client. For systems with 1024 MB of memory, the default value is 128, while for systems with small memory, the value is.
Net. Core. netdev_max_backlog = 32768
# The maximum number of packets that can be sent to the queue when each network interface receives packets faster than the kernel processes these packets.
Net. Core. somaxconn = 32768
# The backlog of the listen function in the Web application will limit the net. Core. somaxconn of the kernel parameter to 128 by default, while the ngx_listen_backlog defined by nginx is 511 by default, so it is necessary to adjust this value.

net. core. wmem_default = 8388608
net. core. rmem_default = 8388608
net. core. rmem_max = 16777216 # maximum socket read buffer, which can be referenced in the following optimization values: 873200
net. core. wmem_max = 16777216 # maximum socket write buffer, which can be referenced in the following optimization values: 873200
net. ipv4.tcp _ timestsmps = 0
# The timestamp can avoid serial number winding. A 1 Gbit/s link must have a previously used serial number. The timestamp allows the kernel to accept such "abnormal" packets. Disable it here.
net. ipv4.tcp _ synack_retries = 2
# To enable the peer connection, the kernel needs to send a SYN with an ACK that responds to the previous syn. That is, the second handshake in the three-way handshake. This setting determines the number of SYN + ACK packets sent before the kernel disconnects.
net. ipv4.tcp _ syn_retries = 2
# Number of SYN packets sent before the kernel abandons the connection.
# net. ipv4.tcp _ tw_len = 1
net. ipv4.tcp _ tw_reuse = 1
# Enable reuse. Allow time-Wait sockets to be re-used for a New TCP connection.

Net. ipv4.tcp _ WMEM = 8192 436600 873200
# TCP write buffer, which can be referenced in the optimization value: 8192 436600 873200
Net. ipv4.tcp _ rmem = 32768 436600 873200
# TCP read buffer, available optimization values: 32768 436600 873200
Net. ipv4.tcp _ mem = 94500000 91500000 92700000
# There are also three values, meaning:
Net. ipv4.tcp _ mem [0]: lower than this value, TCP has no memory pressure.
Net. ipv4.tcp _ mem [1]: Under this value, it enters the memory pressure stage.
Net. ipv4.tcp _ mem [2]: higher than this value, TCP rejects socket allocation.
The above memory unit is page, not byte. The available optimization values are: 786432 1048576 1572864

Net. ipv4.tcp _ max_orphans = 3276800
# The maximum number of TCP sockets in the system is not associated with any user file handle.
If this number is exceeded, the connection is immediately reset and a warning is printed.
This restriction is only intended to prevent simple DoS attacks. It cannot be excessively relied on or artificially reduced,
This value should be increased (if the memory is increased ).
Net. ipv4.tcp _ fin_timeout = 30
# If the socket is disabled by the local end, this parameter determines the time it remains in the fin-wait-2 state. The peer can make an error and never close the connection, or even become an unexpected machine. The default value is 60 seconds. 2.2 The kernel value is usually 180 seconds. You can follow this setting, but remember that even if your machine is a lightweight web server, there is also a risk of memory overflow due to a large number of dead sockets. The risk of fin-wait-2 is smaller than that of fin-wait-1, because it can only eat K of memory at most, however, they have a longer lifetime.

After such Optimization Configuration, your server's TCP concurrency processing capability will be significantly improved. The above configuration is for reference only. Please use it in the production environment based on your actual situation.