TOMCAT7 concurrency and number of threads

Recently has been addressing a problem on the line, performance is:

Tomcat will have a peak every morning, the peak of concurrency reached more than 3000, the final result is the Tomcat thread pool full, log see a lot of requests over 1s.

Server performance is good, the Tomcat version is 7.0.54, configured as follows:

<executor name= "Tomcatthreadpool" nameprefix= catalina-exec-"maxthreads=" minsparethreads= "    <connector executor= "Tomcatthreadpool" port= "8084"       protocol= " Org.apache.coyote.http11.Http11AprProtocol "               connectiontimeout=" 60000 "               keepalivetimeout=" 30000               " maxkeepaliverequests= "8000"               maxhttpheadersize= "8192"               uriencoding= "UTF-8"               enablelookups= "false"               acceptcount= "               disableuploadtimeout=" "True"               redirectport= "8443"/>

After the thread dump see actually in the runnable state of a few threads, most of the threads are in the timed_waiting state:

So everyone began to tangle about why threads were up to 3000, and found that even if the spikes were over the number of threads did not fall down. The first thing we think about is:

The processing of the backend application is slow and "blocked" causes the number of front-end threads to rise up.

However, after optimizing a version of the line found that although the situation has improved, but the final thread pool will reach the maximum value of 3000.

================================== Split Line =========================================

The above is a large background, the middle of the process is omitted, directly with you say at present I get the conclusion:

1, the first is why the thread does not release the problem?

Simply say I verify that the tomcat (7.0.54) thread pool probably works

• When Tomcat starts without a request, the number of threads (all referring to the thread pool) is 0;
• Once requested, Tomcat initializes the number of threads set by Minsaprethreads;
• Tomcat does not actively shrink the thread pool unless it is determined that no request is made, and Tomcat shrinks the thread pool to the size of the minsparethreads setting;
• The previous version of TOMCAT6 has a maxsparethreads parameter, but has been removed in 7, so Tomcat does not release more than idle threads as long as there is only one request in front of it.

As for why Tomcat removed the maxsparethreads parameter, I think it is also due to performance considerations, the constant shrinking thread pool performance is certainly not high, while the extra threads are waiting for the benefit that a new request comes in immediately can be processed.

• Also, a large number of Tomcat threads are waiting without consuming the CPU, but consume some JVM storage.

Add: There is a bit of a problem with the red line above, further verifying that this behavior is found only when using keep-alive (both client and server support), and if the client is not using keep-alive then the thread will be recycled as the TCP connection is released.

Keep-alive related parameters in Tomcat:

Maxkeepaliverequests:

The maximum number of HTTP requests which can be pipelined until the connection are closed by the server. Setting this attribute to 1 would disable http/1.0 keep-alive, as well as http/1.1 keep-alive and pipelining. Setting This to-1 would allow an unlimited amount of pipelined or keep-alive HTTP requests. If not specified, this attribute is set to 100.

KeepAliveTimeout:

The number of milliseconds this Connector would wait for another HTTP request before closing the connection. The default value is to use the value of that have been set for the ConnectionTimeout attribute. Use a value of-1 to indicate no (i.e. infinite) timeout.

2. Why is the thread pool full?

This is the core of my current struggle. Whether or not the performance of the application is slow, I now conclude that there is a relationship, but the key is concurrency.

• The number of threads in the tomcat thread pool is related to your momentary concurrency, such as MaxThreads set to 1000, and when the instant concurrency reaches 1000 then Tomcat will have 1000 threads to handle, which is a little too fast for your application.

So how many threads does it take to get a number of tomcat? This is also related to Tomcat's several parameter settings, to see the official explanation is the most reliable:

MaxThreads:

The maximum number of request processing threads to being created by this Connector, which therefore determines the maximum n Umber of simultaneous requests that can be handled. If not specified, this attribute is set to 200. If an executor are associated with this connector, this attribute is ignored as the connector would execute tasks using the Executor rather than an internal thread pool.

MaxConnections:

The maximum number of connections that the server would accept and process at any given time. When this number has been reached, the server would accept, but not process, one further connection. This additional connection is blocked until the number of connections being processed falls below maxconnections at which Point the server would start accepting and processing new connections again. Note that once the limit have been reached, the operating system may still accept connections based on the acceptCount setting. T He default value varies by connector type. For BIO The default was the value of MaxThreads unless an Executor was used in which case the default would be the value of M Axthreads from the executor. For NIO, the default is 10000 . For apr/native, the default is 8192 .

......

Acceptcount:

The maximum queue length for incoming connection requests if all possible request processing threads is in use. Any requests received if the queue is full would be refused. The default value is 100.

Minsparethreads:

The minimum number of threads always kept running. If not specified, the default of is 10 used.

I simply understand that:

The maximum number of threads that the MAXTHREADS:TOMCAT thread pool can afford;

Maxconnections:tomcat requests (connections) that can be processed concurrently;

Acceptcount:tomcat maintain the maximum number of columns;

Minsparethreads:tomcat the thread pool size initialized or the Tomcat thread pool will have at least as many threads.

It is easy to confuse the two parameters of MaxThreads and MaxConnections:

MaxThreads refers to the number of threads that the tomcat thread pool can play, while MaxConnections is the number of concurrent connections that Tomcat can handle in a flash. For example maxthreads=1000,maxconnections=800, assuming a moment of concurrency of 1000, then eventually the number of Tomcat threads will be 800, that is, processing 800 requests at the same time, the remaining 200 into the queue "queued", If acceptcount=100, then 100 requests will be rejected.
Note: According to the foregoing, only the concurrency that moment Tomcat will be 800 threads processing requests, but after stability, a moment may be only a few threads in the runnable state, most of the thread is timed_waiting, if your application processing time fast enough. So the number of threads that really decide which Tomcat is the most likely to reach is maxconnections this parameter and the number of concurrent numbers, and when the number of concurrent numbers exceeds this parameter the request is queued, and the speed of the response depends on your program performance.

TOMCAT7 concurrency and number of threads