Linux OS Network kernel optimization

Server do Web application (http/nginx) Linux kernel parameter optimization

Under Linux systems, when a TCP connection disconnects, it retains a certain amount of time in the TIME_WAIT state before the port is released. When there are too many concurrent requests,

Will generate a large number of time_wait state connections, can not be broken in time, it will occupy a large number of port resources and server resources. At this point we can optimize the kernel parameters of TCP,

To clean up the ports of the TIME_WAIT state in a timely manner.

The method described in this article only causes system resource consumption to be valid for connections that have a large number of time_wait states, and if not, the effect may not be obvious.

You can use the netstat command to check the connection status of the Time_wait state, enter the following combination command to see the status of the current TCP connection and the corresponding number of connections:

#Netstat-n | awk '/^tcp/{++s[$NF]} END {for (a in S) print A, S[a]} '

This command will output a result similar to the following:

Last_ack 16

SYN_RECV 348

Established 70

Fin_wait1 229

Fin_wait2 30

CLOSING 33

Time_wait 18098

We only care about the number of time_wait , here we can see that there are more than 18,000 time_wait, so it occupies more than 18,000 ports. To know that the number of ports is only 65,535,

Taking one less one will seriously affect the subsequent new connection. In this case, it is necessary to adjust the TCP kernel parameters under Linux, so that the system can release the TIME_WAIT connection faster.

Optimization of kernel parameters:

#表示系统同时保持TIME_WAIT的最大数量, if this number is exceeded, time_wait is immediately cleared and the warning message is printed.

The default is 180000, which changes to 6000. For Apache, Nginx and other servers, the parameters of the last few lines can be a good way to reduce the number of time_wait sockets,

But for Squid, the effect is small. This parameter can control the maximum number of time_wait and avoid the squid server being dragged to death by a large number of time_wait.

Net.ipv4.tcp_max_tw_buckets = 6000

#表示用于向外连接的随机端口范围. Small by default: 32768 to 61000, 10000 to 65000

(Note: Do not set the minimum value too low, otherwise it may take off the normal port!) ）

Net.ipv4.ip_local_port_range = 10000 65536

#表示开启TCP连接中TIME-wait Sockets Fast Recovery, default is 0, indicating off;

Net.ipv4.tcp_tw_recycle = 1

#开启重用. Allows time-wait sockets to be re-used for new TCP connections, which defaults to 0, which means shutdown;

Net.ipv4.tcp_tw_reuse = 1

#开启SYN cookies, when there is a SYN wait queue overflow, enable Cookies to handle, can prevent a small number of SYN attacks, the default is 0, indicating off;

Net.ipv4.tcp_syncookies = 1

#表示当keepalive起用的时候, the frequency at which TCP sends keepalive messages. The default is 2 hours 7200, instead of 20 minutes. Unit is seconds

Net.ipv4.tcp_keepalive_time = 1200

#web应用中listen函数的backlog默认会给我们内核参数的net. Core.somaxconn limit to 128,

The Nginx kernel parameter definition of the Ngx_listen_backlog default is 511, so it is necessary to adjust this value

Net.core.somaxconn = 32768

#每个网络接口接收数据包的速率比内核处理这些包的速率快时, the maximum number of packets that are allowed to be sent to the queue

Net.core.netdev_max_backlog = 32768

#系统中最多有多少个TCP套接字不被关联到任何一个用户文件句柄上. If this number is exceeded, the orphan connection is immediately reset and a warning message is printed.

This limit is only to prevent a simple Dos attack, can not rely too much on it or artificially reduce the value, it should increase this value (if the memory is increased)

Net.ipv4.tcp_max_orphans = 3276800

#记录的那些尚未收到客户端确认信息的连接请求的最大值.

#表示SYN队列的长度, default is 1024, increase queue length of 8192, can accommodate more network connections waiting for connections

For connection requests that still do not have a client acknowledgement, the maximum number that needs to be saved in the queue. For systems that exceed 128Mb of memory, the default value is 1024,

The lower 128Mb is 128. If the server is overloaded frequently, try increasing this number. Warning! If you set this value to be greater than 1024,

It is best to modify the tcp_synq_hsize, inside the include/net/tcp.h to keep the tcp_synq_hsize*16 (SYN flood attack using the TCP protocol to scatter the handshake,

Forgery of false source IP addresses sends a large number of Tcp-syn semi-open connections to the target system, eventually causing the target system socket queue resource to be exhausted and unable to accept new connections.

In order to cope with this attack, modern UNIX systems generally use multi-connection queue processing to buffer (rather than solve) this attack,

is to use a basic queue to handle a normal fully connected application (connect () and accept ()), which is to hold a semi-open connection separately from another queue.

This dual-queue approach, combined with some other system kernel measures (such as syn-cookies/caches), can be used to mitigate small-scale SYN flood attacks more effectively (fact-proving)

Net.ipv4.tcp_max_syn_backlog = 8192

#时间戳可以避免序列号的卷绕. A 1Gbps link will definitely encounter a previously used serial number.

Timestamps allow the kernel to accept this "exception" packet. You need to turn it off here.

Timestamps is used in other things to guard against forged sequence numbers. A 1G broadband line may re-encounter

The old sequence number of the Out-of-line value (if it was the last time it was generated). Timestamp will let it know it's an ' old bag '.

(This file indicates whether it is enabled to enable calculation of RTT in a more precise way than a time-out, RFC 1323); This option should be enabled for better performance. ) The default value is 1

Net.ipv4.tcp_timestamps = 0

#为了打开对端的连接, the kernel sends a SYN and comes with an ACK that responds to the previous syn.

The second handshake in the so-called three-time handshake. This setting determines the number of Syn+ack packets sent before the kernel abandons the connection.

Net.ipv4.tcp_synack_retries = 2

#在内核放弃建立连接之前发送SYN包的数量

Net.ipv4.tcp_syn_retries = 2

#在内核放弃建立连接之前发送SYN包的数量.

Net.ipv4.tcp_syn_retries = 1

#net. Ipv4.tcp_tw_len = 1

#如果套接字由本端要求关闭, this parameter determines how long it remains in the fin-wait-2 state.

The peer may go wrong and never close the connection, or even accidentally become a machine. The default value is 60 seconds. 2.2 The normal value of the kernel is 180 seconds,

You can press this setting, but remember that even if your machine is a light-loaded Web server,

There is also the risk of memory overflow due to a large number of dead sockets, fin-wait-2 is less dangerous than fin-wait-1 because it can only eat up to 1.5K of memory, but their lifetime is longer.

Net.ipv4.tcp_fin_timeout = 30

# TCP Read buffer, reference to optimized values: 32768 436600 873200 min, default, max

Net.ipv4.tcp_rmem = 32768 436600 873200

Min: The amount of memory reserved for the TCP socket to receive buffering,

Even if there is a memory tension, the TCP socket will have at least so much memory to receive buffering, with a default value of 8K.

Default: The amount of memory reserved for the TCP socket to receive buffering, which, by defaults, affects the Net.core.wmem_default value used by other protocols.

This value determines the TCP window size of 65535 in the case of Tcp_adv_win_scale, Tcp_app_win, and tcp_app_win=0 default values. The default value is 87380

Max: The maximum amount of memory used for the TCP socket receive buffer. This value does not affect Net.core.wmem_max, and the "static" selection parameter so_sndbuf is not affected by this value.

The default value is 128K. The default value is 87380*2 bytes.

(as you can see, the. Max setting is preferably twice times the default, which is the main increase for NAT, 51200 131072 204800 in My network)

# TCP Write buffer, reference optimization value: 8192 436600 873200 min, default, max

Net.ipv4.tcp_wmem = 8192 436600 873200

Min: Reserve the minimum amount of memory used to send buffers for the TCP socket. Each TCP socket can be used after it is recommended. Default value is 4096 (4K)

Default: The amount of memory reserved for the TCP socket for sending buffers, which, by defaults, affects the Net.core.wmem_default values used by other protocols.

Generally lower than the value of Net.core.wmem_default. The default value is 16384 (16K).

Max: The maximum amount of memory used for the TCP socket send buffer. This value does not affect Net.core.wmem_max, and the "static" selection parameter so_sndbuf is not affected by this value.

The default value is 131072 (128K). (for the server, increasing the value of this parameter is helpful for sending data in my network environment, modified in order to 51200 131072 204800)

Net.ipv4.tcp_mem = 94500000 91500000 92700000 Low, pressure, high

# There are also 3 values, meaning:

NET.IPV4.TCP_MEM[0]: Below this value, TCP has no memory pressure.

NET.IPV4.TCP_MEM[1]: Under this value, enter the memory pressure phase.

NET.IPV4.TCP_MEM[2]: Above this value, TCP refuses to allocate the socket.

The above memory units are pages, not bytes. A reference to the optimization value is: 786432 1048576 1572864

Low: TCP does not consider freeing memory when TCP uses a number of memory pages that are below this value.

(Ideally, this value should match the 2nd value assigned to TCP_WMEM-this 2nd value indicates that the maximum page size is multiplied by the maximum number of concurrent requests divided by the page size (131072 * 300/4096).) )

Pressure: When TCP uses more memory pages than this value, TCP attempts to stabilize its memory usage, enter pressure mode,

Exits the pressure state when memory consumption is below the low value. (Ideally this value should be the maximum amount of total buffer size that TCP can use (204800 * 300/4096).) )

High: Allows all TCP sockets the amount of pages used to queue buffered datagrams.

(If this value is exceeded, the TCP connection will be rejected, which is why you should not make it too conservative (512000 * 300/4096).)

In this case, the value provided is very large, it can handle many connections, is expected 2.5 times times, or so that the existing connection can transmit 2.5 times times the data.

On my network is 192000 300000 732000)

In general, these values are calculated based on the amount of system memory at system startup.

Net.core.wmem_default = 8388608

Net.core.rmem_default = 8388608

Net.core.rmem_max = 16777216 #最大socket读buffer, reference optimization value: 873200

Net.core.wmem_max = 16777216 #最大socket写buffer, reference optimization value: 873200

To summarize:

These parameters are recommended to be opened only on servers with very large traffic, which can have significant effects. General traffic is small on the server, there is no need to set these several parameters.

Net.ipv4.tcp_keepalive_time = 1200

Net.ipv4.ip_local_port_range = 10000 65000

Net.ipv4.tcp_max_syn_backlog = 8192

Net.ipv4.tcp_max_tw_buckets = 5000

This article is from "Learning to live like a tree" blog, please be sure to keep this source http://laobaiv1.blog.51cto.com/2893832/1952732

Linux OS Network kernel optimization