Linux optimized reprint

1. Open File Descriptors

Linux in System invocation, a large number of system calls are dependent on the file descriptor, and the file descriptor can be assigned to the maximum size of the process is defined by resource constraints.

Therefore, if the file descriptor size is insufficient, or there is an unhealthy network connection (socket count), the file IO is not closed and the file descriptor is released. It can also be said that file Operator, which causes errors in too many open files.

This error is common, so OP is almost always optimized.

Use the following command to view system-related configuration:

Ulimit-u Viewing open files settings

Ulimit-a View All Configurations

Ulimit-u 65535 temporarily modifies open files to 65535

Ulimit-n 65536 maximum number of files that users can open at the same time (max Open file)

Lsof-p PID ID View the currently open file resource for a process

The long-term modifications that are valid for all users and the session are:

$ sudo vim/etc/security/limits.conf

# Allow all users to open 100000 files

# Alternatively, replace * with an explicit username

* Soft Nofile 100000 #限制单个进程最大文件句柄数

* Hard Nofile 100000

Also need to modify/etc/sysctl.conf, set the maximum file handle number of the whole system, run sysctl-p effective

#/etc/sysctl.conf

# Increase System File descriptor limit

Fs.file-max = 100000

2TCP Time Wait interval specifies the length of a socket that is forcibly closed while waiting for a fin packet to be sent from the sender. If TCP does not enter the closed state, the active shutdown will enter time_wait after the last ACK is sent.

This is most common in cache software, but other servers exist as well.

Net.ipv4.tcp_fin_timeout = 30 holds the Fin-wait-2 status time.

Net.ipv4.tcp_keepalive_time = 1200

Net.ipv4.tcp_syncookies = 1

Net.ipv4.tcp_tw_reuse = 1 turn on reuse. Allow time-wait sockets to be re-used for new TCP connections, default 0, which means shutdown

Net.ipv4.tcp_tw_recycle = 1 turns on fast recycling of time-wait sockets in TCP connections, which defaults to 0, which means shutdown.

Net.ipv4.ip_local_port_range = 1024 65000

# indicates the range of ports used for an outward connection.

# is small by default: 32768 to 61000, 18000 to 65535.

Net.ipv4.tcp_max_syn_backlog = 8192 Indicates the length of the SYN queue, with the default of 1024, which can accommodate more network connections waiting to be connected.

Net.ipv4.tcp_max_tw_buckets = 5000

# indicates that the system maintains the maximum number of time_wait sockets simultaneously.

# If the time_wait socket is exceeded, it will be cleared immediately and the warning message printed. The default is 180000, which changes to 5000.

Understanding Server TCP Status

10

Syn_send the request server to establish a connection.

The syn_received server received a SYN from the client.

The established client receives a SYN from the server and the session is established.

LISTEN start receiving connections on the server side.

Fin_wait_1 indicates the end of the activity.

The timed_wait client enters this state after the activity ends.

Close_wait indicates passive shutdown. The server only receives the first blade from the client.

The Fin_wait_2 client receives confirmation from the server of its first FIN.

Last_ack sends its own fin after entering this state.

The CLOSED server receives an ACK from the client and the connection is closed.

Attached to Zhao Yan and i whoisd the above summary

Can be optimized through the system's sysctl.conf configuration file

1, reduce the time in the Fin-wait-2 connection state, so that the system can handle more connections.

Net.ipv4.tcp_fin_timeout = 2

If the request is closed by the local side, this parameter determines how long it remains in the fin-wait-2 state.

The default value is 60 seconds, which can go wrong and never close the connection, or even accidentally.

The normal value of the kernel is 180 seconds, you can press this setting, but remember, even if your machine is a light-load Web server, there is a large number of dead sockets and memory overflow risk, fin-wait-2 is less dangerous than fin-wait-1, Because it can only eat up to 1.5K of memory, but they have a longer lifetime.

2. The following two parameters can resolve too many time_wait problems in a large number of connected Web (cache) servers in the production scenario.

Net.ipv4.tcp_tw_reuse = 1

means to turn on reuse. Allows time-wait sockets to be re-used for new TCP connections, which by default is 0 for shutdown.

3, open time-wait reuse and recycling function.

Net.ipv4.tcp_tw_recycle = 1

Indicates that the fast retract function of time-wait sockets on a TCP connection is turned on, and the default is 0, which means shutdown.

4, when the KeepAlive, TCP sends keepalive message frequency, the default is 2 hours, changed to 20 minutes.

Net.ipv4.tcp_keepalive_time = 600

5. Allow the system to open the port range

Net.ipv4.ip_local_port_range = 4000 65000

Represents the range of ports used for an outward connection. Small by default: 32768 to 61000, 4000 to 65000.

6. Increase the maximum number of SYN half connections supported by the system (default 1024)

Net.ipv4.tcp_max_syn_backlog = 16384

Represents the length of the SYN queue, which defaults to 1024, and a larger queue length of 16384, which can accommodate the maximum number of network connections waiting to be connected.

7, the system at the same time to maintain the maximum number of time_wait

Net.ipv4.tcp_max_tw_buckets = 360000

Indicates that the system maintains the maximum number of time_wait at the same time, and if this number is exceeded, time_wait is immediately cleared and the warning message is printed. The default is 180000, which changes to 5000.

For Apache, Nginx and other servers, the parameters of the last few lines can be very good to reduce the number of time_wait, but for squid, the effect is not small. This parameter can control the maximum number of time_wait and avoid the squid server being dragged to death by a large number of time_wait.

8, the route cache refresh frequency, when a route fails after how long to jump to another route, the default is 300.

Net.ipv4.route.gc_timeout = 100

9. The number of SYN packets sent before the kernel abandons the connection.

Net.ipv4.tcp_syn_retries = 1

10. Reduce the number of System SYN connection retries (default is 5)

Net.ipv4.tcp_synack_retries = 1

In order to open the connection to the end, the kernel sends a SYN and comes with an ACK that responds to the previous syn.

The second handshake in the so-called three-time handshake. This setting determines the number of Syn+ack packets sent before the kernel abandons the connection.

11, set the system to the maximum number of tracking TCP connections limit

Net.ipv4.ip_conntrack_max = 25000000

A complete sysctl.conf file

#/etc/sysctl.conf

# Increase System File descriptor limit

Fs.file-max = 100000

# discourage Linux from swapping idle processes to disk (default = 60)

Vm.swappiness = 10

# Increase Ephermeral IP ports

Net.ipv4.ip_local_port_range = 10000 65000

# increase Linux autotuning TCP buffer limits

# Set Max to 16MB for 1GE and 32M (33554432) or 54M (56623104) for 10GE

# Don ' t set tcp_mem itself! Let the kernel scale it based on RAM.

Net.core.rmem_max = 16777216

Net.core.wmem_max = 16777216

Net.core.rmem_default = 16777216

Net.core.wmem_default = 16777216

Net.core.optmem_max = 40960

Net.ipv4.tcp_rmem = 4096 87380 16777216

Net.ipv4.tcp_wmem = 4096 65536 16777216

# make the time_wait sockets due to more clients,

# and allow them to being reused if we run out of sockets

# Also Increase the Max packet backlog

Net.core.netdev_max_backlog = 50000

Net.ipv4.tcp_max_syn_backlog = 30000

Net.ipv4.tcp_max_tw_buckets = 2000000

Net.ipv4.tcp_tw_reuse = 1

Net.ipv4.tcp_fin_timeout = 10

# Disable TCP slow start on idle connections

Net.ipv4.tcp_slow_start_after_idle = 0

# If your servers talk UDP, also up these limits

Net.ipv4.udp_rmem_min = 8192

Net.ipv4.udp_wmem_min = 8192

# Disable Source Routing and redirects

net.ipv4.conf.all.send_redirects = 0

net.ipv4.conf.all.accept_redirects = 0

Net.ipv4.conf.all.accept_source_route = 0

# LOG packets with impossible addresses for security

Net.ipv4.conf.all.log_martians = 1