Details about tomcat connection count and thread pool, and details about tomcat Thread Pool

Details about tomcat connection count and thread pool, and details about tomcat Thread Pool
Preface

When using tomcat, you often encounter configuration problems such as the number of connections and the number of threads. To really understand these concepts, you must first understand Tomcat Connector (Connector ).

The previous article details the Tomcat configuration file server. written in xml: the main function of Connector is to receive connection requests, create Request and Response objects for exchanging data with the Request side, and assign threads to let the Engine (that is, Servlet container) to process the Request, and send the generated Request and Response object to the Engine. After the Engine processes the request, it also returns the response to the client through Connector.

It can be said that the Servlet container needs Connector to schedule and control requests,ConnectorIs TomcatRequest Processing trunkTherefore, the configuration and use of ctor have an important impact on Tomcat performance. This article will start with ctor and discuss some important issues related to Connector, including NIO/BIO mode, thread pool, and number of connections.

Depending on the Protocol, ctor can be divided into HTTP Connector and AJP Connector. This article only discusses HTTP Connector.

Directory

1. Nio, Bio, APR

1. protocol of Connector

2. How to Select protocol

3. What is the difference between BIO/NIO?

2. Three parameters: acceptCount, maxConnections, and maxThreads

1. acceptCount

2. maxConnections

3. maxThreads

4. parameter settings

Iii. Thread Pool Executor

4. view the current status

1. Number of connections

2. threads

References

1. Protocols of Nio, Bio, APR1, and Connector

Connector uses different protocols to process HTTP requests. Different Tomcat versions support different protocols. The most typical protocols include BIO, NIO, and APR. Tomcat7 supports these three types. Tomcat8 supports NIO2, in Tomcat8.5 and Tomcat9.0, the BIO support is removed ).

BIO is Blocking IO, as its name implies, it is blocking IO; NIO is Non-Blocking IO, it is Non-blocking IO. APR is Apache Portable Runtime, a Portable Runtime Library of Apache, which can be used to achieve high scalability and performance. Apr is the preferred mode for running highly concurrent applications on Tomcat, however, you must install packages such as apr, apr-utils, and tomcat-native.

2. How to specify protocol

Which protocol does the Connector use? You can specify the protocol attribute in the <connector> element or use the default value.

The specified protocol value and corresponding protocol are as follows:

• HTTP/1.1: default value. The protocol used is related to the Tomcat version.
• Org. apache. coyote. http11.Http11Protocol: BIO
• Org. apache. coyote. http11.Http11NioProtocol: NIO
• Org. apache. coyote. http11.Http11Nio2Protocol: NIO2
• Org. apache. coyote. http11.Http11AprProtocol: APR

If no protocol is specified, the default value HTTP/1.1 is used. The default value is as follows: in Tomcat 7, BIO or APR is automatically selected. (If the local database required by APR is found, APR is used, otherwise, BIO is used. In Tomcat8, NIO or APR is automatically selected. (If the local database required by APR is found, APR is used. Otherwise, NIO is used ).

3. What is the difference between BIO/NIO?

Whether BIO or NIO, the general process for Connector to process requests is the same:

In acceptReceive connections in the queue (when the client sends a request to the serverAfter the three-way handshake is established, the OSPut the connection into acceptQueue); obtain the request data in the connection and generate the request; Call servletContainer Processing request; response.To facilitate subsequent instructions, first clarify the relationship between the connection and the request: the connection is at the TCP layer (transport layer), corresponding to the socket, and the request is at the HTTP layer (Application Layer ), it must depend on the implementation of TCP connections. a TCP connection may transmit multiple HTTP requests.

In BIO-implemented Connector, the main object for processing requests is the JIoEndpoint object. The JIoEndpoint maintains the Acceptor and Worker: Acceptor to receive the socket, and then finds the idle thread to process the socket from the Worker thread pool. If there is no idle thread in the worker thread pool, the Acceptor will be blocked. Worker is the thread pool that comes with Tomcat. If other thread pools are configured through <Executor>, the working principle is similar to that of Worker.

In the NIO ctor implemented by NIO, the main object for processing requests is the NIoEndpoint object. In addition to the Acceptor and Worker, the NIoEndpoint also uses Poller, as shown in the process (Image Source: http://gearever.iteye.com/blog/1844203 ).

 

After receiving the socket, the Acceptor does not directly use the thread in the Worker to process the request, but sends the request to Poller first, which is the key to implementing NIO. The Acceptor sends a request to Poller through the queue implementation, using a typical producer-consumer mode. In poroller, a Selector object is maintained. When poroller fetches the socket from the queue, it registers it to the Selector. Then, it traverses the Selector to find the readable socket, and use the thread in the Worker to process the corresponding request. Similar to BIO, Worker can also be replaced by a custom thread pool.

Through the above process, we can see that during the NIoEndpoint's request processing process, whether the Acceptor receives the socket or the thread processes the request, it still uses the blocking method; however, in the process of "reading the socket and handing it to the thread in the Worker", non-blocking NIO is used, this is the main difference between the NIO mode and the BIO mode (other differences have little impact on performance and will not be mentioned for the time being ). However, this difference can significantly improve Tomcat efficiency when the concurrency is large:

Currently, most HTTP requests use persistent connections (by default, keep-alive is true for HTTP/1.1). Persistent connections mean that after a TCP socket ends the current request, if no new request arrives, the socket will not be immediately released, but will be released after timeout. If BIO is used, the process of "reading the socket and handing it to the thread in the Worker" is blocked, which means that when the socket is waiting for the next request or waiting for release, the worker threads that process this socket will remain occupied and cannot be released. Therefore, the number of sockets that Tomcat can simultaneously process cannot exceed the maximum number of threads, and the performance is greatly limited. Using NIO, the process of "reading the socket and handing it to the thread in Worker" is non-blocking. When the socket is waiting for the next request or waiting for release, it does not occupy the working thread, therefore, the number of sockets that Tomcat can process at the same time is much larger than the maximum number of threads, and the concurrency performance is greatly improved.

2. Three parameters: acceptCount, maxConnections, and maxThreads

Review the Tomcat request processing process.: InAcceptReceive connections in the queue (when the client sends a request to the serverAfter the three-way handshake is established, the OSPut the connection into acceptQueue); obtain the request data in the connection and generate the request; Call servletContainer Processing request; response.

Correspondingly, several parameter functions in Connector are as follows:

1. acceptCount

Length of the accept queue. When the number of connections in the accept queue reaches acceptCount, the queue is full and incoming requests are rejected. The default value is 100.

2. maxConnections

The maximum number of connections that Tomcat receives and processes at any time. When the number of connections received by Tomcat reaches maxConnections, the Acceptor thread will not read connections in the accept queue. At this time, the threads in the accept queue will be blocked until Tomcat receives fewer connections than maxConnections. If it is set to-1, the number of connections is not limited.

The default value is related to the protocol used by the connector: NIO defaults to 10000, APR/native defaults to 8192, and BIO defaults to maxThreads (if Executor is configured, the default value is maxThreads of Executor ).

In windows, the maxConnections value of APR/native is automatically adjusted to an integer multiple of the maximum value below 1024. If it is set to 2000, the maximum value is actually 1024.

3. maxThreads

The maximum number of request processing threads. The default value is 200 (both Tomcat 7 and 8 ). If the execuctor is bound to Executor, this value is ignored because the execuctor ctor uses the bound Executor instead of the built-in thread pool to execute the task.

MaxThreads specifies the maximum number of threads rather than the actual number of running CPUs. In fact, maxThreads is much larger than the number of CPU cores. This is because the request processing thread may be used for computing in a very small amount of time, and most of the time may be blocked, such as waiting for the database to return data, and waiting for the hard disk to read and write data. Therefore, at a certain time point, only a few threads are actually using the physical CPU, and most threads are waiting. Therefore, the number of threads is much larger than the number of physical cores.

In other words, by using a much larger number of threads than the CPU core, Tomcat can make the CPU busy and greatly improve the CPU utilization.

4. parameter settings

(1) The setting of maxThreads is related to both the characteristics of the application and the number of CPU cores of the server. We can see from the previous introduction that the number of maxThreads should be much larger than the number of CPU cores. The larger the number of CPU cores, the larger the number of maxThreads. The less intensive the CPU (I/O intensive) in the application, the larger the maxThreads, to make full use of the CPU. Of course, the value of maxThreads is not as large as possible. If maxThreads is too large, the CPU will spend a lot of time on thread switching and the overall efficiency will be reduced.

(2) the settings of maxConnections are related to the running mode of Tomcat. If tomcat uses BIO, the value of maxConnections should be consistent with that of maxThreads. If tomcat uses NIO, it is similar to the default value of Tomcat. The value of maxConnections should be much larger than that of maxThreads.

(3) According to the previous introduction, although the number of connections that tomcat can process at the same time is maxConnections, the number of connections that the server can receive at the same time is maxConnections + acceptCount. The setting of acceptCount is related to the response that the application wants when the connection is too high. If the value is too large, the wait time for the subsequent requests will be long. If the value is too small, the subsequent requests will immediately return connection refused.

Iii. Thread Pool Executor

The Executor element represents the thread pool in Tomcat, which can be shared by other components. To use this thread pool, the component must specify the thread pool through the executor attribute.

Executor is an embedded element of the Service element. Generally, the ctor component is used to use the thread pool. To enable the ctor to use the thread pool, the Executor element should be placed before the Connector. The Executor and Connector configurations are as follows:

<Executor name="tomcatThreadPool" namePrefix ="catalina-exec-" maxThreads="150" minSpareThreads="4" /><Connector executor="tomcatThreadPool" port="8080" protocol="HTTP/1.1" connectionTimeout="20000" redirectPort="8443" acceptCount="1000" />

Executor has the following attributes:

• Name: Mark of the thread pool
• MaxThreads: Maximum number of active threads in the thread pool. The default value is 200 (both Tomcat 7 and 8)
• MinSpareThreads: Minimum number of threads maintained in the thread pool. The minimum value is 25.
• MaxIdleTime: Maximum idle time of a thread. When the idle time exceeds this value, the thread is closed (unless the number of threads is smaller than minSpareThreads). The Unit is ms. The default value is 60000 (1 minute)
• Daemon: Specifies whether the background thread is used. The default value is true.
• ThreadPriority: Specifies the thread priority. The default value is 5.
• NamePrefix: the prefix of the thread name. The thread name in the thread pool is namePrefix + thread number.

4. view the current status

The following describes how to set the Tomcat connection count and thread count. The following describes how to view the number of connections and thread count on the server.

There are two solutions to view the server status: (1) using a ready-made tool and (2) directly using Linux commands.

Ready-made tools, such as JDK's built-in jconsole tool, can conveniently view thread information (In addition, you can also view CPU, memory, class, JVM basic information, etc.), Tomcat's built-in manager, billing tools such as New Relic. Is the interface for jconsole to view thread information:

 

The following describes how to view the number of connections and the number of threads on the server through the Linux Command Line.

1. Number of connections

If Tomcat receives an http request on port 8083, you can use the following statement to view the connection information:

netstat –nat | grep 8083

The result is as follows:

It can be seen that there is a connection in the listen status, listening for requests; in addition, there are four ESTABLISHED connections (ESTABLISHED) and two waiting for closed connections (CLOSE_WAIT ).

2. threads

The ps command can view the process status, for example, execute the following command:

ps –e | grep java

The result is as follows:

We can see that only the information of one process is printed; 27989 is the thread id, and java is the java Command executed. This is because starting a tomcat, all internal work is completed in this process, including the main thread, garbage collection thread, Acceptor thread, and request processing thread.

Run the following command to view the number of threads in the process. The nlwp indicates number of light-weight process.

ps –o nlwp 27989

We can see that there are 73 threads in the process, but 73 does not exclude threads in the idle state. To obtain the real number of running threads, you can use the following statement:

ps -eLo pid ,stat | grep 27989 | grep running | wc -l

Among them, ps-eLo pid and stat can find all threads and print the process number and current state of the thread. Two grep commands filter the process number and state of the thread respectively. wc counts the number of threads. Ps-eLo pid, stat | grep 27989 output is as follows:

Only some results are shown in the figure. Sl indicates that most threads are idle.

References

Tomcat 7.0 official documentation

Tomcat 8.0 official documentation

Tomcat 8.5 official documentation

Tomcat maxThreads maxConnections acceptCount parameter description

Tomcat Architecture Analysis (implemented by connector BIO)

Tomcat Architecture Analysis (NIO ctor NIO implementation)

Why is the tomcat default thread pool size so large?

Howto find Tomcat current thread count