Linux io/System/memory performance tuning Finishing

Why do you want performance tuning?

Most Linux distributions are designed to be fully compatible with most computers in the marketplace. This is a fairly mixed set of hardware (hard drive, video card, NIC, etc.). So Red Hat, Suse,mandriva and some other distribution vendors have chosen some conservative settings to make sure the installation is successful.
Simply put: Your distribution is running well, but it can work better!
For example, there may be an advanced hard drive with specific special features that may not be enabled in the case of a standard configuration.
Tuning of the disk subsystem

For Linux EXT3/4, a parameter that is helpful in almost all cases is to turn off file system access time, and under/etc/fstab to see if your filesystem has noatime parameters. Atime is the most recent time to access a file, and whenever you access a file, The underlying file system must record this timestamp. Because system administrators rarely use atime, disabling it can reduce disk access time. The way to disable this feature is to add the Noatime option in the fourth column of/etc/fstab.
Network tuning

TCP Connection Retention Management:
If a connection is idle within the specified time of this parameter, the kernel initiates a probe to the remote host

Net.ipv4.tcp_keepalive_time = 7200

The time interval for a live probe sent by the kernel to a remote host

NET.IPV4.TCP_KEEPALIVE_INTVL = 75

The maximum number of times the kernel sends a live probe, if the number of probes is greater than this, and the remote host is not reached, the connection is closed and local resources are released
One connection 7200s idle, the kernel will retry every 75 seconds, if 9 consecutive time to give up. This causes a connection to be discarded after 2h11min, reducing the value to minimize the resources that are used by the failed connection, and to the new connection.

Net.ipv4.tcp_keepalive_probes = 9

TCP connection Management:
Represents the maximum number of packets that are allowed to be sent to a queue when the rate at which packets are received per network interface is faster than the rate at which the kernel processes these packets.

net.core.netdev_max_backlog=3000

Control the queue length of the TCP SYN for each port, the connection request from the client needs to be queued until the server accepts it, and if the number of connection requests is greater than that, the connection request is discarded and the client is unable to connect to the server, and the general server should increase this value

Net.ipv4.tcp_max_syn_backlog = 1024

Controls the number of times the kernel resend the response to a Ack,syn segment of a socket (the second handshake of three times), reducing this value to detect connection failure attempts from the remote host as early as possible

Net.ipv4.tcp_synack_retries = 5

Control the number of times the kernel resend data to a remote host that has established connections, reducing this value and detecting connection failures as early as possible

Net.ipv4.tcp_retries2 = 15

The SYN cookie is a modification of the three handshake protocol on the TCP server side, which is specifically designed to prevent SYN flood attacks. The rationale is that when a TCP server receives a TCP SYN packet and returns a TCP Syn+ack packet, it does not allocate a dedicated data area, but calculates a cookie value based on the SYN packet. When a TCP ACK packet is received, the TCP server checks the legality of the TCP ACK packet against that cookie value. If it is legal, then assign a dedicated data area to handle future TCP connections.

Net.ipv4.tcp_syncookies = 1

The following SYSCTL command is used to change security settings, but it can also prevent network performance from falling. These commands are set to the default values.

Turn off the following parameters to prevent hackers from attacking server IP addresses

Sysctl-w net.ipv4.conf.eth0.accept_source_route=0
Sysctl-w net.ipv4.conf.lo.accept_source_route=0
Sysctl-w net.ipv4.conf.default.accept_source_route=0
Sysctl-w net.ipv4.conf.all.accept_source_route=0

The following command causes the server to ignore redirects from the server being included in the gateway. Because redirects can be used to attack, we only accept redirects with reliable sources

Sysctl-w Net.ipv4.conf.eth0.secure_redirects=1
Sysctl-w Net.ipv4.conf.lo.secure_redirects=1
Sysctl-w Net.ipv4.conf.default.secure_redirects=1
Sysctl-w Net.ipv4.conf.all.secure_redirects=1

Alternatively, you can configure to accept or reject any ICMP redirects. ICMP redirection is the mechanism by which information is transmitted by the device. For example, when the gateway receives the Internet data from the host network, the gateway can send redirect information to a host. The gateway checks the routing table for the address of the next gateway, and the second gateway routes the datagram to the destination network. To turn off these redirection commands are as follows:

Sysctl-w net.ipv4.conf.eth0.accept_redirects=0
Sysctl-w net.ipv4.conf.lo.accept_redirects=0
Sysctl-w net.ipv4.conf.default.accept_redirects=0
Sysctl-w net.ipv4.conf.all.accept_redirects=0

If this server is not a router, it will not send redirects, so you can turn off the feature:

Sysctl-w net.ipv4.conf.eth0.send_redirects=0
Sysctl-w net.ipv4.conf.lo.send_redirects=0
Sysctl-w net.ipv4.conf.default.send_redirects=0
Sysctl-w net.ipv4.conf.all.send_redirects=0

Configure the server to refuse to accept broadcast storms or Smurf attacks attacks:

Sysctl-w Net.ipv4.icmp_echo_ignore_broadcasts=1

Ignore all ICMP packets or pings:

Sysctl-w net.ipv4.icmp_echo_ignore_all=1

Some routers send invalid responses to broadcast frames, each generating a warning and generating a log in the kernel. These responses can be ignored:

Sysctl-w Net.ipv4.icmp_ignore_bogus_error_responses=1

The commands below are used to tune a server with a very large number of connections.
Indicates open reuse. Allow time-wait sockets to be reconnected to a new TCP connection by default of 0

Net.ipv4.tcp_tw_reuse = 1

Indicates a quick recycle of time-wait sockets on a TCP connection, which defaults to 0

Net.ipv4.tcp_tw_recycle = 1

Tcp_fin_timeout
The default value is 60
For a socket connection that is disconnected from the end, TCP remains in the Fin_wait_2 state for a time. The other side may disconnect or never end the connection or the unexpected process dies. The default value is 60 seconds. It used to be 180 seconds in the 2.2 version of the kernel. Can you set this value? But you need to be aware of it? If your machine is a heavily loaded Web server, you might want to risk memory being filled with a large number of invalid datagrams? Fin-wait-2 sockets is less dangerous than fin-wait-1 because they eat up to 1.5K of RAM? But they exist longer. In addition reference Tcp_max_orphans.
The reason for the generation of close_wait state
If the server program Apache is in the close_wait state, it means that the socket is passively closed!
Assuming that the client side actively disconnects the current connection, the two sides will need four packet to close the TCP connection:
Client-> fin-> Server
Client <---ACK <---Server
The client side is in the fin_wait_2 state, and the server program is in the Close_wait state.
Client <---FIN <---Server
The Server then sends the fin to client,server for the Last_ack state.
Client---> ack-> Server
The client responds to the ACK, then the server's socket is actually placed in the closed state.
The Server program is in a close_wait state, not a last_ack state, indicating that the fin has not been sent to the client, so there may be a lot of data to send or something else to do before closing the connection, causing the fin packet to not be sent.
Usually, a close_wait will last for at least 2 hours. If a rogue specially wrote a program to give you a bunch of close_wait, consuming resources, then it is usually not until the release of the moment, the system has been resolved to collapse.
Change this value before carefully monitoring to avoid memory overflow due to dead sockets.

Sysctl-w net.ipv4.tcp_fin_timeout=30

For all queues (i.e. system), set maximum system send cache (WMEM) and receive cache (RMEM) to 8MB

Sysctl-w net.ipv4.core.wmem_max=8388608
Sysctl-w net.ipv4.core.rmem_max=8388608

Use the following command to adjust TCP send and receive caching. This command has three values: minimum, initial, and maximum values:

Sysctl-w net.ipv4.tcp_rmem= "4096 87380 8388608"
Sysclt-w net.ipv4.tcp.wmem= "4096 87380 8388608"

The third value must be less than or equal to Wmem_max and Rmem_max.

The number of Half-open connections can increase when the server is overloaded or there are many clients that have long delay connection failures. This is very common for Web servers, especially when there are many dial-up customers. These half-open connections are saved in the backlog (backlog) connections queue.
Set this value at a minimum of 4096 (the default is 1024). Even if the server does not receive such a connection, setting this value can also prevent denial-of-service (Syn-flood) attacks

Sysctl-w net.ipv4.tcp_max_syn_backlog=4096

Set Ipfrag parameters, especially for NFS and samba servers. Here, we can set the maximum and minimum memory to be used to regroup IP fragments. When the Ipfrag_high_thresh value is assigned, the fragment is discarded until the Ipfrag_low_thres value is reached.
When a TCP packet transfer error occurs, the defragmentation begins. Valid packets remain in memory, while corrupted packets are forwarded. For example, set the available memory range from 256 MB to 384 MB

Sysctl-w net.ipv4.ipfrag_low_thresh=262144
Sysctl-w net.ipv4.ipfrag_high_thresh=393216

Network Security settings
Preventing TCP SYN Flood attacks

TCP SYN Flood is a common and effective remote (denial of service) attack, which destroys TCP three handshake through certain operations to establish a normal connection, consumes and consumes system resources, and makes the host system that provides the TCP service not functioning properly. Since TCP SYN Flood is an attack on server servers through the bottom of the network, it can change its IP address arbitrarily and not be recognized by other devices on the network, which makes it difficult for the Network Crime Department to trace the source of the crime.

System Check
In general, some simple steps can be checked to determine if the system is suffering from a TCP SYN flood attack
1, the server can not provide normal TCP services. The connection request was denied or timed out
2, through the Netstat-an command to check the system, found that there are a large number of SYN_RECV connection status

Settings for Iptables
Prevent synchronization Pack Floods (Sync Flood)

Iptables-a forward-p tcp--syn-m limit--limit 1/s-j ACCEPT

There are people writing.

Iptables-a input-p tcp--syn-m limit--limit 1/s-j ACCEPT

–limit 1/s Limit syn concurrency by 1 times per second and can be modified according to your needs

Prevent various port scans

Iptables-a forward-p tcp--tcp-flags syn,ack,fin,rst rst-m limit--limit 1/s-j ACCEPT

Ping flood Attack (ping of Death)

Iptables-a forward-p ICMP--icmp-type echo-request-m limit--limit 1/s-j ACCEPT

Tuning of the memory subsystem

Tuning the memory subsystem is not easy and requires constant monitoring to ensure that changes in memory do not adversely affect other subsystems of the server. If you want to change the virtual memory parameters (in/PROC/SYS/VM), it is recommended that you change only one parameter at a time and then monitor the effect.

The adjustments to virtual memory include the following items:

Configure how the Linux kernel updates dirty buffers to disk
Disk buffers are used for staging disk data. Disk buffers are slow relative to memory. Therefore, if the server uses this type of memory, performance can be problematic. When the data in the buffer is completely dirty, use:

Sysctl-w vm.bdflush= "30 500 0 0 500 3000 60 20 0"

Vm.bdflush has 9 parameters, but it is recommended to change only 3 of them:
1 nfract, maximum percentage of buffer allowed for Bdflush daemon before writing to disk for queued
2 Ndirty, the value of the maximum buffer immediately written for Bdflush. If this value is large, Bdflush will need more time to complete the disk's data update.
7 Nfract_sync, the maximum percentage of buffer change dirty before synchronization occurs

Configure KSWAPD daemon To specify the number of memory pages for Linux

Sysctl-w vm.kswapd= "1024 32 64"

The three parameters are described as follows:
–tries_base is equivalent to four times times the number of "pages" that the kernel has each time. For systems with a lot of exchange information, adding this value can improve performance.
–tries_min is the smallest number of pages that KSWAPD swaps out each time.
–swap_cluster is the number of pages that are KSWAPD. A small number increases disk I/O performance, and a large number may negatively affect the request queue.

If you want to make changes to these parameters, use the tool Vmstat to check the performance impact. Other virtual memory parameters that can improve performance are:
_ Buffermem
_ Freepages
_ Overcommit_memory
_ Page-cluster
_ Pagecache
_ Pagetable_cache
Tuning of File subsystems

Ulimit-a is used to display current user process restrictions.
Linux for each user, the system limits its maximum number of processes. To improve performance, you can set the maximum number of processes per Linux user based on device resources, and I set the maximum number of processes for a Linux user to 10,000: www.111Cn.net

Ulimit-u 10000

For Java applications that require many socket connections and keep them open, it is a good idea to modify the number of files that each process can open by using Ulimit-n XX, which defaults to 1024.

Ulimit-n 4096 increases the number of files that can be opened per process to 4096, and defaults to 1024

Some important settings for other suggestions that are set to unrestricted (unlimited) are:
Length of data segment: Ulimit-d Unlimited
Maximum memory size: Ulimit-m Unlimited
Stack size: Ulimit-s Unlimited
CPU Time: Ulimit-t Unlimited
Virtual Memory: Ulimit-v Unlimited
Temporarily, it is appropriate to log on to a shell session through the Ulimit command.

Permanently, by adding a corresponding Ulimit statement to a file read by the login shell, a shell-specific user resource file, such as:
1, the maximum number of processes to unlock the Linux system and maximum file open limit:
Vi/etc/security/limits.conf
# Add the following line

* Soft Noproc 11000
* Hard Noproc 11000
* Soft Nofile 4100
* Hard Nofile 4100

Description: * representative for all users
Noproc represents the maximum number of processes
Nofile is the maximum number of file open

2, let SSH accept login program login, easy to SSH client view ulimit-a resource restrictions:
A, Vi/etc/ssh/sshd_config
Change the value of Userlogin to Yes and remove the # annotation
b, restart the SSHD service:
/etc/init.d/sshd restart

3, modify all Linux user's environment variable file:

Vi/etc/profile
Ulimit-u 10000
Ulimit-n 4096
Ulimit-d Unlimited
Ulimit-m Unlimited
Ulimit-s Unlimited
Ulimit-t Unlimited
Ulimit-v Unlimited

Sometimes in the program to open a number of files, analysis, the general default number of systems is 1024, (with ULIMIT-A can see) for normal use is enough, but for the program, it is too little.

Modify 2 files.
1.vi/etc/security/limits.conf
Plus:

* Soft Nofile 8192
* Hard Nofile 20480

2./etc/pam.d/login

Session required/lib/security/pam_limits.so

Also ensure that the/etc/pam.d/system-auth file has the following content

Session required/lib/security/$ISA/pam_limits.so

This line ensures that this restriction is enforced by the system.

3. Bash_profile for general users.

Ulimit-n 1024

Re-login OK
Optimization steps:

Modify the/etc/profile file and add:

Ulimit-u 10240
Ulimit-n 4096
Ulimit-d Unlimited
Ulimit-m Unlimited
Ulimit-s Unlimited
Ulimit-t Unlimited
Ulimit-v Unlimited

Modify/etc/rc.d/rc.local and add:

Echo ' 999999′>/proc/sys/fs/file-max
Echo ' 999999′>/proc/sys/fs/inode-max

(1G Memory value modified to: 65535 2G Memory Value modified to: 131072 4G Memory Value modified to: 262144)

Modify the/etc/sysctl.conf file and add:

Net.core.rmem_default = 8388608
Net.core.rmem_max = 8388608
Net.core.wmem_default = 8388608
Net.core.wmem_max = 8388608
Net.ipv4.tcp_timestamps = 0
Net.ipv4.tcp_sack =1
net.ipv4.tcp_window_scaling = 1
net.core.netdev_max_backlog=3000

#Modify I-node

sys.fs.file-max= 65535
sys.fs.inode-max= 65535

#Set System Memory

Vm.bdflush= "30 500 0 0 500 3000 60 20 0"
Vm.kswapd= "1024 32 64"

#Disable hackattack!

Net.ipv4.conf.eth0.accept_source_route=0
Net.ipv4.conf.lo.accept_source_route=0
Net.ipv4.conf.default.accept_source_route=0
Net.ipv4.conf.all.accept_source_route=0
Net.ipv4.conf.lo.accept_redirects=0
Net.ipv4.conf.all.accept_redirects=0
Net.ipv4.conf.eth0.accept_redirects=0
Net.ipv4.conf.default.accept_redirects=0
Net.ipv4.conf.lo.secure_redirects=0
Net.ipv4.conf.all.secure_redirects=0
Net.ipv4.conf.eth0.secure_redirects=0
Net.ipv4.conf.default.secure_redirects=0
Net.ipv4.conf.eth0.send_redirects=0
Net.ipv4.conf.lo.send_redirects=0
Net.ipv4.conf.default.send_redirects=0
Net.ipv4.conf.all.send_redirects=0
Net.ipv4.tcp_syncookies=1
Net.ipv4.icmp_echo_ignore_broadcasts=1
Net.ipv4.icmp_ignore_bogus_error_responses=1

#Web Servers

Net.ipv4.tcp_tw_reuse=1
Net.ipv4.tcp_tw_recycle=1
Net.ipv4.tcp_fin_timeout=30
net.ipv4.tcp_keepalive_time=1800
net.ipv4.core.wmem_max=16777216
net.ipv4.core.rmem_max=16777216
net.ipv4.tcp_rmem= "4096 87380 8388608"
net.ipv4.tcp.wmem= "4096 87380 8388608"
net.ipv4.tcp_max_syn_backlog=8192

Firewall security:

Iptables-a forward-p tcp--syn-m limit--limit 1/s-j ACCEPT

There are people writing.

Iptables-a input-p tcp--syn-m limit--limit 1/s-j ACCEPT

–limit 1/s Limit syn concurrency by 1 times per second and can be modified according to your needs

Prevent various port scans

Iptables-a forward-p tcp--tcp-flags syn,ack,fin,rst rst-m limit--limit 1/s-j ACCEPT

Ping flood Attack (ping of Death)

Iptables-a forward-p ICMP--icmp-type echo-request-m limit--limit 1/s-j ACCEPT

Note: (Relatively radical adjustment of network parameters) www.111Cn.net

# use TCP syncookies when needed

Net.ipv4.tcp_syncookies = 1

# Enable TCP Window Scaling

net.ipv4.tcp_window_scaling = 1

# increase TCP Max buffer size

Net.core.rmem_max = 16777216
Net.core.wmem_max = 16777216

# increase Linux autotuning TCP buffer limits

Net.ipv4.tcp_rmem = 4096 87380 8388608
Net.ipv4.tcp_wmem = 4096 65536 8388608

# Increase number of ports available

Net.ipv4.ip_local_port_range = 1024 65000