Linux Server solutions for lost packets

Troubleshooting:

The morning suddenly received Nagios server check_icmp alarm, alarm shows a Web server intranet network problems. Because that server Mounted intranet NFS, so intranet network uses Nagios check_icmp to do monitoring.

Quickly log on to the server for troubleshooting. First Use ping LAN IP way to view the connectivity of the intranet, ping the process of the loss of packet phenomenon, the information is as follows:

Bytes from 10.1.1.1:icmp_seq=34 ttl=255 time=0.928 ms

Bytes from 10.1.1.1:icmp_seq=35 ttl=255 time=1.01 ms

Ping:sendmsg:Operation not permitted

Ping:sendmsg:Operation not permitted

Show Ping is not allowed, strange, the firewall clearly opened the ICMP protocol. There are problems to look at the log first, the log file will generally have records, tail–f/var/log/messages, found a large number of the following content:

Sep 09:11:21 dowload_server1 kernel:printk:261 messagessuppressed.

Sep 09:11:21 dowload_server1 kernel:ip_conntrack:table full,dropping Packet

The discovery is that the current number of sessions is full, so there is a packet loss phenomenon. Here to Ip_conntrack to do a simple introduction: Ip_conntrack represents the Connection Tracking database (conntrack), on behalf of the NAT machine tracking the number of connections, the connection tracking table can accommodate how many records are controlled by a variable, It can be set by the IP-SYSCTL function in the kernel. Each trace connection table consumes 350 bytes of kernel storage space, which fills the default space for a long time, so what is the default space? In the memory of 64MB machine is 4096, memory is 128MB is 8192, memory is 256MB is 16384

View the current number of sessions by using the following command:

Cat/proc/net/ip_conntrack | Wc–l do not use, accounting for the CPU

or use:

Cat/proc/sys/net/ipv4/netfilter/ip_conntrack_count

Use the following command to view the maximum number of sessions set

Cat/proc/sys/net/ipv4/ip_conntrack_max

Solution:

found that the maximum number of sessions has actually been reached, Google found that the maximum number of sessions can be directly adjusted to the user, the command is:

echo "102400" >/proc/sys/net/ipv4/ip_conntrack_max

After executing this command, the ping is no longer missing the package. However, this setting is not persisted and the settings are lost when the system restarts, so you need to save to/etc/sysctl.conf, add in/etc/sysctl.conf: Net.ipv4.ip_conntract_max = 102400, and then execute/ Sbin/sysctl–p Refresh kernel parameters, if error: "Net.ipv4.ip_conntract_max" is a unknown key error, need to load ip_conntract module, use Modprobe ip_ Conntrack loading, using Lsmod | Grepip_conntrack See if the module is loaded.

Ultimate Solution:

In order to completely solve this problem, there is a need to set another thing, that is, the session connection timeout variable, this parameter set too long will cause the number of session connections, the default is set to 432,000 seconds, it is obviously too large, the following command to set a smaller point:

Echo 21600>/proc/sys/net/ipv4/netfilter/ip_conntrack_tcp_timeout_established

Set to 21600 is 6 hours, which automatically clears the invalid link after 6 childhood. Remember to add this phrase to the/etc/rc.local file in the automatic startup file.