TCP parameter optimization for high load server kernel

Net.ipv4.tcp_mem

The memory that the kernel allocates to the TCP connection, in page,1 Page = 4096 Bytes, can be viewed with commands:

#getconf PAGESIZE

4096

Net.ipv4.tcp_mem = 196608 262144 393216

The first number indicates that when TCP uses less than 196608 page, kernel does not intervene in any of its

The second figure indicates that when TCP uses more than 262144 pages, kernel will enter "memory pressure" pressure mode

The third number indicates that when the pages used by TCP exceed 393216 (equivalent to 1.6GB of memory), they are reported as: Out of socket memory

The above values apply to 4GB memory machines, for 8GB memory machines, the following parameters are recommended:

Net.ipv4.tcp_mem = 524288 699050 1048576 (TCP connection uses up to 4GB memory)

Net.ipv4.tcp_rmem and Net.ipv4.tcp_wmem

The read, write buffer memory size allocated for each TCP connection, in units of byte

Net.ipv4.tcp_rmem = 4096 8192 4194304

Net.ipv4.tcp_wmem = 4096 8192 4194304

The first number indicates the minimum memory allocated for the TCP connection

The second number indicates the default memory allocated for the TCP connection

The third number indicates the maximum memory allocated for a TCP connection

Generally according to the default value allocation, the above example is read and write are 8KB, a total of 16KB

The number of connections that 1.6GB TCP memory can hold, approximately 1600mb/16kb = 100K = 100,000

The number of connections that 4.0GB TCP memory can hold, approximately 4000mb/16kb = 250K = 250,000

Net.ipv4.tcp_max_orphans

The maximum number of orphan sockets (orphan sockets), the unit is a

Net.ipv4.tcp_max_orphans = 65536

Represents a maximum of 65,536

Note: When the number of orphans seen by Cat/proc/net/sockstat reaches about half of Net.ipv4.tcp_max_orphans, it is reported as: Out of socket memory

See Kernel source code:

Include/net/tcp.h:

268 Static inline bool Tcp_too_many_orphans (struct sock *sk, int shift)

269 {

percpu_counter struct *OCP = sk->sk_prot->orphan_count;

271 int orphans = percpu_counter_read_positive (OCP);

272

273 if (Orphans << shift > Sysctl_tcp_max_orphans) {

274 orphans = percpu_counter_sum_positive (OCP);

275 if (Orphans << shift > Sysctl_tcp_max_orphans)

276 return true;

277}

278

279 if (sk->sk_wmem_queued > Sock_min_sndbuf &&

280 atomic_long_read (&tcp_memory_allocated) > Sysctl_tcp_mem[2])

281 return true;

282 return false;

283}

Reference http://blog.tsunanet.net/2011/03/out-of-socket-memory.html

For Net.ipv4.tcp_max_orphans = 65536, when the orphans reaches 32,768, an out of socket memory is reported, which takes up Ram 32K*64KB=2048MB=2GB

(Each orphan socket can occupy up to 64KB of memory), which may actually be smaller

Net.ipv4.tcp_orphan_retries

Orphan socket pre-retry the number of retries, heavy load Web server recommended Small

Net.ipv4.tcp_orphan_retries = 1

Setting a smaller value can effectively reduce the number of orphans (net.ipv4.tcp_orphan_retries = 0 is not expected to retry)

Net.ipv4.tcp_retries2

The number of active TCP connection retransmissions is considered to be dropped, and the connection is discarded. Default value: 15, recommended to 2 or 3.

Net.ipv4.tcp_synack_retries

Syn/ack phase of the TCP three handshake, retry count, default 5, set to 2-3

Net.ipv4.tcp_fin_timeout = 30

Time-out for TCP connections in the Fin_wait state

Net.ipv4.tcp_tw_reuse = 1

Net.ipv4.tcp_tw_recycle = 1

TIME_WAIT state socket for fast recovery, recycling

Net.ipv4.tcp_max_syn_backlog = 4096

TCP connection SYN Queue size

Net.core.netdev_max_backlog = 2048

The queue size of the network device's transceiver packet

Net.ipv4.tcp_syncookies = 1

TCP syn Cookies to protect against DDoS attacks and to prevent SYN queues from being fully occupied

Reference documents:

http://jaseywang.me/2012/05/09/%E5%85%B3%E4%BA%8E-out-of-socket-memory-%E7%9A%84%E8%A7%A3%E9%87%8A-2/

Http://blog.tsunanet.net/2011/03/out-of-socket-memory.html

http://rdc.taobao.com/blog/cs/?p=1062

TCP parameter optimization for high load server kernel