Detailed Tomcat connection number and thread pool

Objective

When using Tomcat, you will often encounter configuration problems such as the number of connections, the number of threads, and to really understand these concepts, you must first understand the Tomcat Connector (Connector).

In the previous article detailed in the Tomcat configuration file Server.xml: The main function of connector is to receive connection requests, create request and response objects for exchanging data with the requester side Then assign the line Chengyang engine (the servlet container) to handle the request and pass the resulting request and response object to the engine. When the engine finishes processing the request, it also returns the response to the client via connector.

It can be said that the Servlet container processing requests, which need connector for scheduling and control, connector is the backbone of Tomcat processing requests, so the configuration and use of connector have an important impact on Tomcat performance. This article starts with connector and discusses some important issues related to connector, including Nio/bio mode, thread pool, number of connections, and so on.

Depending on the protocol, Connector can be divided into HTTP Connector, AJP Connector, etc., this article only discusses HTTP Connector.

The protocol of Nio, Bio, APR1 and connector

Connector uses a different protocol when processing an HTTP request. Different versions of Tomcat support different protocol, of which the most typical protocol include bio, NIO, and APR (TOMCAT7 supports these 3 kinds, Tomcat8 adds support for NIO2, To Tomcat8.5 and Tomcat9.0, the support for bio was removed).

Bio is blocking Io, as the name implies is the blocking Io;nio is non-blocking io, then non-blocking IO. The APR is the Apache Portable runtime, which is the Apache portable runtime that enables high scalability and high performance with local libraries, and APR is the preferred mode for running high concurrency applications on Tomcat, but requires the installation of APR, Apr-utils, Tomcat-native and other bags.

2. How to Designate protocol

Connector which protocol you use, you can specify it through the protocol attribute in the <connector> element, or you can use the default value.

The specified protocol value and the corresponding protocol are as follows:

• http/1.1: Default value, the protocol used is related to the Tomcat version
• Org.apache.coyote.http11.Http11Protocol:BIO
• Org.apache.coyote.http11.Http11NioProtocol:NIO
• Org.apache.coyote.http11.Http11Nio2Protocol:NIO2
• Org.apache.coyote.http11.Http11AprProtocol:APR

If protocol is not specified, the default value of http/1.1 is used, meaning the following: in Tomcat7, auto-selection uses bio or APR (Apr is used if you find the local library required for APR, otherwise using bio); in Tomcat8, Automatically choose to use NiO or APR (Apr is used if you find the local library required for APR, otherwise NIO is used).

3. What is the difference between Bio/nio

The approximate process for processing requests, either bio or Nio,connector, is the same:

Receive a connection in the Accept queue (when a client sends a request to the server, if the client establishes a connection with the OS to complete three handshakes, the OS places the connection in the Accept queue), gets the requested data in the connection, generates the request, and invokes the servlet container to process the request ; return to response. In order to facilitate the following instructions, first clarify the relationship between the connection and the request: The connection is the TCP level (transport layer), the corresponding socket, the request is the HTTP level (application layer), must rely on the TCP connection implementation, a TCP connection may transmit multiple HTTP requests.

In the connector implemented by bio, the primary entity that processes the request is the Jioendpoint object. Jioendpoint maintains the acceptor and worker:acceptor receive sockets, then finds the idle thread processing socket from the worker thread pool, and if the worker thread pool does not have idle threads, acceptor will block. Where the worker is the thread pool that Tomcat comes from, and if other thread pools are configured through <Executor>, the principle is similar to worker.

In the connector implemented by NIO, the primary entity that processes the request is the Nioendpoint object. In addition to including acceptor and workers in Nioendpoint, the poller is used, as shown in the process (picture Source: http://gearever.iteye.com/blog/1844203).

Acceptor receives the socket, instead of directly using the thread in the worker to process the request, it sends the request to Poller first, and Poller is the key to implementing NIO. Acceptor sends requests to poller through the queue, using a typical producer-consumer model. In Poller, a selector object is maintained, and when the Poller is removed from the queue, it is registered to the selector, and then by traversing the selector, the readable socket is found and the corresponding request is processed using the thread in the worker. Like bio, workers can also be replaced by a custom thread pool.

As can be seen in the process of nioendpoint processing the request, whether it is the acceptor receive socket, or the thread processing request, the use is still blocking mode, but in the "read the socket and to the worker thread" in this process, Using non-blocking NIO implementations, this is the most significant difference between the NIO model and the Bio mode (other differences have little effect on performance and are temporarily omitted). This difference, in the case of large concurrency, can lead to a significant increase in tomcat efficiency:

Most HTTP requests currently use a long connection (http/1.1 default keep-alive is true), whereas a long connection means that a TCP socket will not be released immediately after the current request has ended without a new request coming in. But wait for a timeout before releasing it. If you use bio, the process of "reading a socket and handing it to a worker" is blocked, which means that the working thread that handles the socket is always occupied and cannot be freed while the socket is waiting for the next request or for release. , so Tomcat can handle at the same time no more sockets than the maximum number of threads, and the performance is severely limited. With NIO, the process of "reading a socket to a thread in a worker" is non-blocking, and when the socket is waiting for the next request or waiting to be released, it does not occupy the worker thread, so tomcat can process more sockets than the maximum number of threads at the same time. Concurrency performance is greatly improved.

Two, 3 parameters: Acceptcount, MaxConnections, MaxThreads

Look again at the process of Tomcat processing requests: receiving connections in the Accept queue (when a client sends a request to the server, if the client establishes a connection with the OS to complete three handshakes, the OS puts the connection in the Accept queue), gets the requested data in the connection, generates the request ; Call the servlet container to process the request; return response.

Correspondingly, several parameters in the connector function as follows:

1, Acceptcount

The length of the accept queue; When the number of connections in the Accept queue reaches Acceptcount, the queue is full, and incoming requests are rejected. The default value is 100.

2, MaxConnections

The maximum number of connections that Tomcat receives and processes at any given time. When the number of connections that Tomcat receives reaches maxconnections, the acceptor thread does not read the connections in the Accept queue, and the threads in the accept queue are blocked until the number of connections that Tomcat receives is less than maxconnections. If set to-1, the number of connections is not restricted.

The default value is related to the protocol used by the connector: The default value for NiO is 10000,apr/native is 8192, and the default value for bio is MaxThreads (if executor is configured, the default value is executor MaxThreads).

Under Windows, the MaxConnections value of the apr/native is automatically adjusted to the integer multiple of the maximum of 1024, as set to 2000, the maximum value is actually 1024.

3, MaxThreads

The maximum number of request processing threads. The default value is $ (TOMCAT7 and 8 are yes). If the connector is bound to executor, this value is ignored because the connector uses the bound executor instead of the built-in thread pool to perform the task.

MAXTHREADS Specifies the maximum number of threads, not the actual number of running CPUs; in fact, the size of the maxthreads is much larger than the number of CPU cores. This is because the thread that handles the request can be used to calculate very little time, most of the time it may be blocking, such as waiting for the database to return data, waiting for the hard disk to read and write data, etc. So, at some point, only a handful of threads are actually using the physical CPU, and most threads are waiting, so the number of threads is much larger than the number of physical cores.

In other words, Tomcat can get the CPU busy by using a much larger number of threads than the CPU core, greatly improving CPU utilization.

4. Parameter setting

(1) The setting of the maxthreads is related to the characteristics of the application, and also to the number of CPU cores of the server. As can be known from the previous introduction, the number of maxthreads should be much larger than the number of CPU cores, and the larger the CPU cores, the larger the maxthreads should be, the less dense the CPU in the application (the denser the IO), the larger the maxthreads should be so that it can take full advantage of the CPU. Of course, the value of maxthreads is not greater, the better, if the maxthreads is too large, then the CPU will spend a lot of time for the thread switching, the overall efficiency will be reduced.

(2) The MaxConnections settings are related to Tomcat's operating mode. If Tomcat is using bio, then the value of maxconnections should be the same as maxthreads, and if Tomcat is using NIO, it is similar to Tomcat's default value. The MaxConnections value should be much larger than maxthreads.

(3) As can be known through the previous introduction, although Tomcat can handle the number of connections is maxconnections, but the server can receive at the same time the number of connections is maxconnections+acceptcount. The setting of the Acceptcount is related to how the application would like to react in the case of high connections. If the setting is too large, the subsequent incoming request waits a long time, and if the setting is too small, the subsequent incoming request returns connection refused immediately.

Third, thread pool executor

The executor element represents a thread pool in Tomcat, which can be shared by other components, and the component needs to specify that thread pool through the Executor property to use the pooling.

Executor is an inline element of the service element. In general, the connector component is used with the thread pool, and in order for connector to use the thread pool, executor elements should be placed in front of connector. Examples of configuration of executor and connector are as follows:

12	<executor name= "Tomcatthreadpool" nameprefix = "catalina-exec-" maxthreads= minsparethreads= /> < Connector executor= "Tomcatthreadpool" port= "8080" protocol= "http/1.1" connectiontimeout= "20000" redirectport= "8443" acceptcount= "" " />

The main properties of executor include:

• Name: tag for this thread pool
• MaxThreads: The maximum number of active threads in the thread pool, with a default value of (TOMCAT7 and 8)
• Minsparethreads: The minimum number of threads kept in the thread pool, the minimum value is 25
• MaxIdleTime: The maximum time the thread is idle when the thread is closed when idle exceeds the value (unless the number of threads is less than minsparethreads), the unit is MS, and the default value is 60000 (1 minutes)
• Daemon: Whether background thread, default value True
• ThreadPriority: Thread priority, default value 5
• Nameprefix: Thread name prefix, thread pool threads name: nameprefix+ Thread number

Iv. Viewing the current status

The above describes the number of Tomcat connections, the concept of the number of threads, and how to set up, which shows how to view the number of connections and threads in the server.

Viewing the status of the server is broadly divided into two scenarios: (1) using off-the-shelf tools and (2) using Linux commands directly.

Out-of-the-box tools, such as the Jconsole tool that comes with the JDK, can easily view thread information (in addition to CPU, memory, class, JVM basic information, etc.), Tomcat's own manager, the toll tool new relic, and more. Jconsole is the interface for viewing thread information:

Here's how to view the number of connections and threads in the server from the Linux command line.

1. Number of connections

Assuming that Tomcat receives an HTTP request with a port of 8083, you can use the following statement to view the connection situation:

1	netstat –nat | grep 8083

The results are as follows:

As can be seen, there is a connection in the listen state, listen to the request, in addition, there are 4 established connections (established) and 2 waiting for a shutdown of the connection (close_wait).

2. Threads

The PS command can view the status of a process, such as executing the following command:

1	ps –e | grep java

Results such as:

As you can see, only one process of information is printed; 27989 is a thread id,java is a Java command that executes. This is because starting a tomcat, all of the internal work is done in this process, including the main thread, the garbage collection thread, the acceptor thread, the request processing thread, and so on.

You can see how many threads are in the process with the following command, where NLWP meaning is number of light-weight process.

1	ps –o nlwp 27989

You can see that there are 73 threads inside the process, but 73 does not exclude threads that are in an idle state. To get the number of threads that really are in running, you can do this with the following statement:

1	ps -eLo pid ,stat | grep 27989| grep running | wc -l

Where Ps-elo PID, stat can find all the threads and print the process number and the current state of the thread, the two GREP commands filter the process number and the thread state respectively, and the number of WC statistics. Among them, Ps-elo PID, stat | The results of the grep 27989 output are as follows:

Only part of the result is shown, and SL means that most threads are in an idle state.

Reference documents

• Tomcat 7.0 Official documentation
• Tomcat 8.0 Official documentation
• Tomcat 8.5 Official documentation
• Tomcat maxthreads maxconnections acceptcount parameter description
• Tomcat Architecture Analysis (Connector BIO implementation)
• Tomcat Architecture Analysis (Connector NIO implementation)
• Why are the Tomcat default thread pool size so large?
• Howto Find Tomcat Current thread count

Detailed Tomcat connection number and thread pool