Details Tomcat connection number and thread pool

Details Tomcat connection number and thread pool

Objective

When using Tomcat, you often encounter configuration issues such as number of connections, threads, and to really understand these concepts, you must first understand the Tomcat Connector (Connector).

In the previous article in the Tomcat configuration file Server.xml wrote: Connector's main function is to receive connection requests, create request and response objects for and the request to exchange data The Distribution line Chengyang engine (that is, the servlet container) to process the request and pass the resulting request and response objects to engine. When engine finishes processing the request, it also returns the response to the client through connector.

It can be said that the servlet container handles requests that require connector for scheduling and control, and connector is the backbone of Tomcat processing requests, so connector configuration and usage have an important impact on Tomcat performance. This article will start with connector to discuss some important issues related to connector, including Nio/bio mode, thread pool, number of connections, etc.

Depending on the protocol, Connector can be divided into HTTP Connector, AJP Connector, etc., this article only discusses HTTP Connector.

I. Nio, Bio, APR

1, connector of protocol

Connector uses a different protocol when processing HTTP requests. Different Tomcat versions support different protocol, with the most typical protocol including bio, NIO, and APR (TOMCAT7 support these 3, TOMCAT8 increased support for NIO2, In Tomcat8.5 and Tomcat9.0, the support for bio is removed.

The bio is blocking IO, the name implies that the blocking Io;nio is non-blocking io, then non-blocking io. and APR is Apache portable Runtime, is the Apache portable runtime, the use of local libraries can achieve high scalability, high-performance; Apr is the preferred mode for running high concurrent applications on Tomcat, but you need to install APR, apr-utils, Tomcat-native and other bags.

2. How to specify protocol

Connector uses which protocol can be specified either through the Protocol property in the <connector> element, or by using the default value.

The specified protocol value and the corresponding protocol are as follows:

http/1.1: Default value, the protocol used is related to the Tomcat version

Org.apache.coyote.http11.Http11Protocol:BIO

Org.apache.coyote.http11.Http11NioProtocol:NIO

Org.apache.coyote.http11.Http11Nio2Protocol:NIO2

Org.apache.coyote.http11.Http11AprProtocol:APR

If protocol is not specified, the default value http/1.1 is used, which means the following: in Tomcat7, automatically select the use of bio or APR (if you find the local library that APR needs, use APR, otherwise use bio); in Tomcat8, Automatically select the use of NiO or APR (if you find the local library your APR needs, use APR, otherwise use NIO).

3, what is the difference between Bio/nio

The approximate process of processing requests, whether bio or Nio,connector, is the same:

Receive a connection in the Accept queue (when the client sends a request to the server, the OS puts the connection into the Accept queue if the client has made a connection to the OS three times); Gets the requested data in the connection, generates the request, invokes the Servlet container to process the request ; return to response. In order to facilitate the description of the following, first of all clarify the relationship between the connection and request: The connection is the TCP level (transport layer), the corresponding socket, the request is HTTP level (application layer), must rely on TCP connection implementation, a TCP connection may transfer multiple HTTP requests.

In the connector of the bio implementation, the primary entity that handles the request is the Jioendpoint object. Jioendpoint maintains the acceptor and Worker:acceptor receive sockets and then finds idle thread processing sockets from the worker thread pool, and acceptor blocks if the worker thread pool does not have an idle thread. Where worker is Tomcat's own thread pool, the principle is similar to worker if other thread pools are configured through <Executor>.

In the connector of NIO implementations, the primary entity that handles the request is the Nioendpoint object. Nioendpoint In addition to include acceptor and worker, or poller, the processing process as shown in the following figure (Image source: http://gearever.iteye.com/blog/1844203).

Acceptor receives the socket, instead of directly using the thread in worker to process the request, it sends the request to Poller, and Poller is the key to achieving NIO. Acceptor sends requests to poller to be implemented through queues, using a typical producer-consumer model. In Poller, a selector object is maintained, and when Poller is removed from the queue, it is registered in the selector, and then through the selector, the readable socket is found and the corresponding request is processed using the thread in the worker. Like bio, worker can also be replaced by a custom thread pool.

As can be seen from the above process, in the process of nioendpoint processing request, whether it is acceptor receive socket, or thread processing request, still use blocking mode, but in "read the socket and handed to the worker in the thread" in this process, Implemented using Non-blocking NIO, which is the main difference between the NIO model and the Bio mode (other differences have less impact on performance and are temporarily omitted). This difference, in the case of large concurrent volume, can bring about a significant increase in tomcat efficiency:

Most HTTP requests currently use a long connection (http/1.1 default keep-alive is True), while a long connection means that a TCP socket does not immediately release if no new request arrives after the current request has been completed. But wait for timeout and then release. If you use bio, the process of "reading the socket and handing it to a thread in worker" is blocked, which means that when the socket waits for the next request or waits for release, the worker thread that handles the socket will be occupied and unable to release ; Therefore, the number of sockets that Tomcat can handle at the same time cannot exceed the maximum number of threads, and performance is greatly limited. With NIO, the process of reading the socket and handing it to a thread in worker is non-blocking, and when the socket waits for the next request or waits for release, it does not occupy the worker thread, so the number of sockets that Tomcat can handle is much larger than the maximum number of threads. Concurrency performance is greatly improved.

Two, 3 parameters: Acceptcount, MaxConnections, MaxThreads

Review the process of Tomcat processing requests: Receive connections in the Accept queue (when a client sends a request to the server, if the client establishes a connection with the OS three handshake, the OS places the connection in the Accept queue), gets the requested data in the connection, generates the request ; calls the servlet container to process the request; Returns response.

Correspondingly, several parameter functions in the connector are as follows:

1, Acceptcount

The length of the accept queue, when the number of connections in the Accept queue reaches Acceptcount, the queue is full and incoming requests are rejected. The default value is 100.

2, MaxConnections

The maximum number of connections that Tomcat receives and processes at any time. When the number of connections received by Tomcat reaches MaxConnections, the acceptor thread does not read the connection in the Accept queue, and the threads in the accept queue are blocked until the number of connections that Tomcat receives is less than maxconnections. If set to-1, the number of connections is unrestricted.

The default value is related to the protocol used by the connector: The default value for NiO is 10000,apr/native 8192, and the default for bio is MaxThreads (if executor is configured, the default value is executor MaxThreads).

Under Windows, the MaxConnections value of the apr/native is automatically adjusted to the integer multiple of the maximum of 1024 below the set value, and if set to 2000, the maximum value is actually 1024.

3, MaxThreads

The maximum number of request processing threads. The default values are TOMCAT7 and 8. If the connector is bound to executor, this value is ignored because the connector will use the bound executor instead of the built-in thread pool to perform the task.

MaxThreads Specify the maximum number of threads, not the actual number of running CPUs; in fact, the maxthreads size is much larger than the CPU core number. This is because the thread that is processing the request may have very little time to compute, and most of the time may be blocked, such as waiting for the database to return data, waiting for the hard disk to read and write data, and so on. So, at some point, only a handful of threads are actually using the physical CPU, and most threads are waiting, so the number of threads is far greater than the physical core.

In other words, Tomcat can keep the CPU busy by using a much larger number of threads than the CPU core, greatly increasing CPU utilization.

4. Parameter setting

(1) The setting of MaxThreads is related to the application characteristics and the CPU core quantity of the server. As you can see from the previous introduction, the number of maxthreads should be much larger than the CPU core, and the larger the CPU core, the greater the maxthreads should be, and the less intensive the CPU in the application (the denser the IO), the larger the maxthreads should be so that the CPU can be fully utilized. Of course, the MaxThreads value is not the larger the better, if the maxthreads is too large, then the CPU will spend a lot of time to switch to the thread, the overall efficiency will be reduced.

(2) The setting of MaxConnections is related to Tomcat's operating mode. If Tomcat is using bio, then the MaxConnections value should be the same as maxthreads, and if Tomcat uses NIO, it is similar to Tomcat's default value. The MaxConnections value should be much larger than maxthreads.

(3) As you can see from the previous introduction, although the number of connections that Tomcat can handle at the same time is maxconnections, the number of connections that can be received at the same time in the server is Maxconnections+acceptcount. The setting of the Acceptcount is related to how the application wishes to react when the connection is too high. If the setting is too large, the request waiting time for the next entry will be long, and if the setting is too small, the subsequent incoming request immediately returns to connection refused.

Third, thread pool executor

The executor element represents a thread pool in Tomcat that can be shared by other components, and the component needs to specify the thread pool through the Executor property to use the line pool.

Executor is an inline element of the service element. In general, the thread pool is the connector component, and the executor element should be placed in front of connector in order for connector to use the thread pool. Examples of executor and connector configurations are as follows:

<executor name= "Tomcatthreadpool" Nameprefix = "catalina-exec-" maxthreads= "the" minsparethreads= "4"/>

<connector executor= "Tomcatthreadpool" port= "8080" protocol= "http/1.1" connectiontimeout= "20000" 8443 "acceptcount=" 1000 "/>

The main properties of executor include:

Name: tag for this thread pool

MaxThreads: Maximum number of active threads in the thread pool, default (TOMCAT7 and 8)

Minsparethreads: The minimum number of threads maintained in the thread pool, with a minimum value of 25

MaxIdleTime: The maximum time a thread is idle to close a thread when idle exceeds this value (unless the number of threads is less than minsparethreads), the unit is MS, the default value is 60000 (1 minutes)

Daemon: Background thread, default value True

ThreadPriority: Thread priority, default value 5