"Reprint" Apache View the number of connections and limit the current number of connections

Cause: A server on the line, has always been a slow access to the situation, click a link for more than 2 seconds to open, according to our estimates of the number of visitors, the server should not respond so slow, so need to address this problem analysis, to solve the site visit too slow.

Analysis:

1, first of all, when the page access slows down, use the top command to view the server load situation, found that the load is not high, the initial estimate is not a program problem.
2, then, look at the number of httpd in the thread, Ps-aux | grep httpd | Wc-l found that the number of threads has reached the maximum value of Apache settings. It is concluded that the number of visitors to the site caused the visit too slow.
3, in order to verify, view the number of connections and the current number of connections, respectively, are
Netstat-ant | grep $IP: 80 | Wc-l
Netstat-ant | grep $IP: 80 | grep EST | Wc-l
As it turns out, the number of connections is particularly great, far exceeding our estimated value.
4, at the beginning, the server's MPM configuration is not particularly familiar, that the modification of the server configuration can solve the problem. The main configuration includes prefork mode or work mode configuration, and here are some simple introductions.

Prefork mode:
The default configuration for Apache working in Prefork mode:
<ifmodule mpm_prefork_module>
Serverlimit 2000
Startservers 5 #指定服务器启动时建立的子进程数量
Minspareservers 5 #指定空闲子进程的最小数量
Maxspareservers #指定空闲子进程的最大数量
MaxClients #指定同一时间客户端最大接入请求的数量 (number of concurrent threads per process), any requests exceeding this limit will enter the waiting queue, and once a connection is released, the request in the queue will be serviced
Maxrequestsperchild 0 #指定每个子进程在其生存周期内允许伺服的最大请求数量, the default is 10000,0, which means that the child process never ends.
</IfModule>
The Prefork control process creates a process, waits a second, continues to create two, waits a second, continues to create four, after the "startservers" child process is initially established, to meet the needs of minspareservers settings ... This increases the number of processes created, up to 32 per second, until the value of the Minspareservers setting is met. This pattern can reduce system overhead to increase performance by eliminating the need to generate new processes when requests arrive.
Maxspareservers sets the maximum number of idle processes, and if the number of idle processes is greater than this value, Apache automatically kill some redundant processes. This value should not be set too large, but if the value set is smaller than Minspareservers, Apache will automatically adjust it to minspareservers+1. If the site load is large, consider increasing both minspareservers and maxspareservers.
MaxClients is one of the most important of these directives, setting the request that Apache can handle at the same time, which is the most influential parameter to Apache performance. The default value of 150 is far from sufficient, and if the total number of requests has reached this value (which can be confirmed by Ps-ef|grep Httpd|wc-l), then the subsequent request will be queued until a processed request has been completed. This is the main reason that there are a lot of system resources left and the HTTP access is slow. Although theoretically the larger the value, the more requests can be processed, but the Apache default limit is not greater than 256. In Apache2, you can increase maxclients by serverlimit instructions without recompiling Apache.
Although by setting serverlimit, we can add the maxclients very large, but often counterproductive, the system consumes all memory. Take one server as an example: Memory 2G, each Apache process consumes about 0.5% (can be confirmed by PS aux) of memory, that is, 10M, so that the server run up to 200 Apache process will consume all the memory system, so, Set MaxClients to be cautious.
Worker mode:
The default configuration for Apache, which works in worker mode, is:
<ifmodule mpm_worker_module>
Startservers 2
MaxClients 150
Minsparethreads 25
Maxsparethreads 75
Threadsperchild 25
Maxrequestsperchild 0
</IfModule>
The Worker generates a "Startservers" subprocess by the main control process, each containing a fixed number of threadsperchild threads, and each thread processes the request independently. Again, in order not to generate a thread when the request arrives,
Minsparethreads and Maxsparethreads set the minimum and maximum number of idle threads, while maxclients sets the maximum total number of simultaneous clients. If the total number of threads in the existing child process does not meet the load, the control process derives the new child process.
The maximum default values for Minsparethreads and Maxsparethreads are 75 and 250, respectively. These two parameters have little effect on the performance of Apache, and can be adjusted according to the actual situation.
Threadsperchild is the most performance-related instruction in worker mpm.
The maximum default value for Threadsperchild is 64, and 64 is not enough if the load is large. To explicitly use the THREADLIMIT directive, its maximum default value is 20000.
The total number of requests that can be processed concurrently in worker mode is determined by the total number of child processes multiplied by the Threadsperchild value and should be greater than or equal to maxclients. If the load is large and the number of existing child processes is not satisfied, the control process derives the new child process. The default maximum number of child processes is 16, which also requires an explicit declaration of serverlimit (maximum is 20000). It is important to note that if Serverlimit is explicitly declared, then the value multiplied by threadsperchild must be greater than or equal to maxclients, and maxclients must be an integer multiple of threadsperchild, otherwise     Apache will automatically adjust to a corresponding value. The server Apache uses the prefork mode of work, the corresponding adjustment to the maxclients, found that the service started very short time, the number of connections can reach the maximum.
5, then think of the need to see users are access to those pages, the configuration of the Access_log open, found that more than 85% of the access is directly accessed by the resource file, which determines that users may use multi-threaded download tools, or these resources have suffered hotlinking.
6, found the problem, the solution is better to do. Thought of two methods:
A, the thread limit for connecting to a single IP does not allow multithreading to connect resources.
For IP restrictions, I used the Mod_limitipconn module. The benefits of this module are simple, and the disadvantage is that it is not possible to set up a separate folder or file, and it does not support virtual hosts.
After the module is installed in Apache, the following paragraphs are added to the configuration file to take effect:
Extendedstatus on
< Ifmodule mod_limitipconn.c >
< Location/> # All Virtual Host/directory
Maxconnperip 3 # Only 3 concurrent connections per IP allowed
Noiplimit image/* # Don't make IP restrictions on pictures
</location >
< Location/mp3 > #/mp3 directory for all hosts
Maxconnperip 1 # Only one connection request per IP allowed
Onlyiplimit audio/mpeg Video # This limit is only for videos and audio format files
</location >
</ifmodule >
B, add URL rewrite to prevent hotlinking.
One important way to prevent hotlinking is to judge the request's refer, but if you use some browsers to make a request to remove the refer, or disguise it, there's nothing you can do about it. But there seems to be a more advanced approach, or you can implement this function.
After installing Apache's mod_rewrite module, add it in the Apache configuration file
Rewriteengine on
Rewritecond%{http_referer}!^http://linuxidc.com/.*$ [NC]
Rewritecond%{http_referer}!^http://linuxidc.com$ [NC]
Rewritecond%{http_referer}!^http://www.linuxidc.com/.*$ [NC]
Rewritecond%{http_referer}!^http://www.linuxidc.com$ [NC]
Rewriterule. *\. (gif|jpg|swf) $ http://www.linuxidc.com/about/nolink.png [R,NC]

This allows hotlinking requests to be redirected to an error page, reducing the pressure that downloads bring to the server

Another article: Talking about Apache optimization

Saturday the Company a server load is too large, restart
And then landed up and looked at the next httpd.conf and server status.
Net-nap | grep:80 | Wc-l found a large number of connections of more than 10,000
Net-nap | grep:80 | grep time_wait | Wc-l found and the total number of connections is very close to about 9,000
Net-nap | grep:80 | grep time_wait | grep an IP | Wc-l found that some IP time_wait number is actually more than 600

Checked the httpd.conf.
Found timeout KeepAlive off
Infer a large number of time_wait because TIMEOUT time is too long, single IP time_wait number is because KeepAlive off

Modify Configuration
TIMEOUT 10
KeepAlive on
Maxkeepaliverequests #注这里数字要大一些, but much smaller than Apache allows the total number of clients
KeepAliveTimeout 10

Restart Apache after Net-nap | grep:80 | Wc-l down to more than 200.

The next night when the peak of the observation server, in addition to occasional load high some accidents, all normal

This morning I added a page output compression module to the server to speed up data transfer and improve the throughput of the Web server.
Apache version httpd-2.0.54
CD modules
CD filters
$APACHE _home/bin/apxs-i. -i-a-c-o mod_deflate.so mod_deflate.c
Cd.. /metadata/
$APACHE _home/bin/apxs-i. -i-a-c-o mod_headers.so MOD_HEADERS.C

VI $APACHE _home/conf/httpd.conf
Increase

# Insert Filter
Setoutputfilter DEFLATE
# Netscape 4.x has some problems ...
Browsermatch ^MOZILLA/4 gzip-only-text/html
# Netscape 4.06-4.08 have some more problems
Browsermatch ^mozilla/4\.0[678] No-gzip
# MSIE masquerades as Netscape, but it is fine
Browsermatch \bmsie!no-gzip!gzip-only-text/html
# Don ' t compress images
Setenvifnocase request_uri \. (?: gif|jpg|jpe?g|png) $ no-gzip dont-vary

# Make sure proxies don ' t deliver the wrong content
Header Append Vary user-agent env=!dont-vary


So the page output compression module is installed, at night when the load is high to see the effect of the operation
Mod_deflate module can be refined to set the output of how large to a large range of files before the compression output, I this is just a simple configuration, the specific other configuration to see the Apache documents