Implementation Algorithm Analysis of Several thread pools [reprinted], thread algorithm analysis

Implementation Algorithm Analysis of Several thread pools [reprinted], thread algorithm analysis

Original article address

Content

• Preface
• Thread Pool meaning
• Technical Points of Thread Pool
• Section
• Reference source code

   
However, a framework (a "service" Framework) basically involves the thread pool issue. Although you may not use it directly, this is because the framework has helped you complete this work.
    
Why do we need a thread pool? Now, if you write a service program without using the concurrent or parallel method, I am sorry for the 4-core, 8-core, or even more CPU cores (physical kernel, logic kernel ). If you need to create a thread each time, destroy the thread after use, and consume a lot of system performance, it is more appropriate that during program initialization, create all threads at one time, so that you can use them directly when necessary ~
    
Although the framework provides a thread pool and does not need to be written by itself, understanding the thread pool can at least greatly improve your programming capabilities. Maybe, not every project requires a framework. If the project is not that complex, you still need to write a simple thread pool.
 

Preface

Before reading the source code of the thread pool, I always felt that the thread pool was the most advanced technology in the framework. After research, we found that the implementation of the thread pool is so delicate. This article analyzes the essential principles and composition of the thread pool from a technical perspective, and analyzes the source code implementation of JDK, Jetty6, Jetty8, and Tomcat, it is of some guiding significance for business scenarios that want to understand the nature of thread pools, better use of thread pools, or customize their own thread pools.

Thread Pool meaning

• Reuse: systems such as Web servers require a large number of internal threads to process client requests, while the response time for a single request is usually relatively short, in this case, the creation and destruction of threads in a large number of Java local calls based on the operating system will become a bottleneck and a waste of resources in the system. If the thread pool technology is used, the worker thread can be reused, that is, multiple tasks can be processed during the life cycle of a worker thread creation and destruction, in general, the frequency and time of thread creation and destruction are reduced, improving the system performance.
• Throttling: limited server resources and high concurrency settings that exceed the server performance become a burden on the system, resulting in a large amount of CPU consumption, such as context switching and memory overflow. The thread pool technology can be used to control the maximum number of concurrent tasks and the maximum number of tasks processed by the system, so as to implement flow control well and ensure that the system will not crash.
• Function: JDK thread pools are flexible and provide many functions. In some scenarios, thread pools are used based on functions.

Technical Points of Thread Pool

From the perspective of internal implementation, the thread pool technology can be divided into the following six key points:

Implementation

Worker workers structure and concurrency Protection

Structure of the queue to be processed

JDK

HashSet is used to store worker workers, and concurrent protection is implemented through ReentrantLock. Each worker is a Runnable interface.

The block queue that implements the interface BlockingQueue is used to store the jobs to be processed, and the queue is used as the constructor parameter, so that the business can flexibly expand the queue of the custom thread pool. Services can also use JDK's own synchronous blocking queue SynchronousQueue, bounded queue ArrayBlockingQueue, unbounded queue blockingqueue, and priority queue PriorityBlockingQueue.

Jetty6

The HashSet storage worker workers is also used to implement concurrent HashSet protection through an object synchronized. Every worker is actually a Thread extension.

An array is used to store the Runnable job object to be processed. The array initialization capacity is _ maxThreads. The Variable _ queued is used to calculate the number of jobs to be processed in the current system, that is, the array length. When the maximum value of the array is exceeded, the _ maxThreads capacity is extended. Therefore, the array is always large enough and the capacity is unbounded. Synchronization is also implemented using synchronized as an object.

Jetty8

The concurrentjavasqueue storage worker workers is used. The CAS-based JDK improves the concurrency efficiency and reduces the complexity of thread pool concurrency protection. The atomic variable AtomicInteger is used to collect statistics on the number of workers.

Similar to JDK, BlockingQueue-based blocking queues are used to store jobs to be processed, and queue types can be imported into the parameters of the thread pool constructor. At the same time, Jetty8 does not set the queue type by default. Two types of queues can be automatically set: ArrayBlockingQueue, which cannot be resized, and BlockingArrayQueue, which can be customized and extended by Jetty.

Tomcat

Implement ThreadPoolExecutors Based on JDK and reuse JDK services

Reuse JDK services

Example of a thread pool initialization and processing business job Algorithm

Implementation	Thread Pool Construction and worker Initialization	Process Business jobsAlgorithm
JDK	1. Flexible initialization Based on Multiple construction parameters. Several core parameters are as follows: CorePoolSize: number of core workers MaximumPoolSize: Maximum number of workers KeepAliveTime: The survival time of idle workers when the number of core workers exceeds. WorkQueue: The BlockingQueue interface mentioned earlier. 2. By default, the worker is not started after initialization. The worker is started only when a request is sent. You can call the thread pool interface to start several worker threads with core work in advance, or multiple worker threads with business expectations.	1. When the number of worker workers is lower than the core worker Number corePoolSize, a worker will be created first to process the job. If the job is processed successfully, a result will be returned. 2. When the number of worker workers is higher than the number of core workers, jobs are put into the queue to be processed first, and the processing ends when the job is put into the queue successfully. 3. If the number of workers is smaller than the maximum number of workers maximumPoolsize, a new worker is created to process the job. 4. Reject
Jetty6	1. You can also set multiple parameters: _ SpawnOrShrinkAt: Expansion/contraction Threshold Value _ MinThreads: Minimum number of workers _ MaxThreads: Maximum number of workers _ MaxIdleTimeMs: Maximum idle time-out period for idle workers 2. Start _ minThreads worker threads directly after Initialization	1. Find the idle worker and dispatch the job. 2. If there are no idle workers, save the job to the array to be processed. 3. When the number of pending jobs in the array is identified to exceed the expansion threshold, the expansion worker will process the job. 4. Otherwise, it will not be processed.
Jetty8	1. The configuration parameters are similar to Jetty6, removing the _ spawnOrShrinkAt threshold parameter. 2. Start _ minThreads worker threads directly after Initialization	It is very simple to directly team jobs to be processed.
Tomcat	1. Build Method Based on JDK Thread Pool 2. Start the worker upon request	The processing method reuse JDK, but the JDK function is extended before submission, enabling the ability to count the number of submittedCount submissions.

Example of worker increase/decrease mechanism of thread pool worker

Implementation	Add algorithms to workers	Worker Reduction Algorithm
JDK	1. When the number of workers to be processed is lower than the core Worker Number corePoolSize. 2. When a job is to be processed, the number of workers exceeds the maximum number of workers and the number of workers in the queue to be processed fails. 3. When the core Worker Number update interface of the Business Call thread pool is found to be resized, the number of workers is increased.	1. There are no jobs in the queue of tasks to be processed and the number of worker workers exceeds the core Worker Number corePoolSize. 2. There is no job in the queue of tasks to be processed and the number of workers allowed is smaller than the core worker parameter is true. In this scenario, at least one worker thread is retained.
Jetty6	1. When the thread pool is started, _ minThreads worker threads will be started. 2. the number of jobs to be processed is higher than the threshold parameter and the number of workers does not reach the maximum value. 3. When you call the thread pool interface setMinThreads to update the minimum number of workers, the number of workers is increased as needed.	When the following three conditions are met, the number of workers is reduced: 1. No jobs to be processed in the list of pending tasks 2. The number of workers exceeds the minimum number of workers _ minThreads 3. The number of idle worker threads exceeds the threshold.
Jetty8	1. Start the minimum worker parameter worker thread when the thread pool is started 2. The number of idle or idle workers is smaller than the total number of jobs to be processed. 3. When you call the thread pool interface setMinThreads to update the minimum number of workers	When the following three conditions are met, the number of workers is reduced: 1. No jobs to be processed in the queue of pending tasks 2. The total number of worker workers exceeds the minimum worker parameter configuration _ minThreads 3. the idle time of the worker thread times out.
Tomcat	Add worker algorithms with JDK	Reuse JDK to reduce algorithm and customize extension delay parameters. When the parameter is exceeded, an exception is thrown out to terminate the thread pool worker.

Summary

Compared with several thread pool implementations, JDK implementation is the most flexible, functional, and scalable. Tomcat is based on JDK thread pool Function Extension implementation, while reusing the original business, it also expands its own business. Jetty6 is a self-tailored thread pool service. It is coupled with many complex business logic of the thread pool to the thread pool class. The logic is relatively complicated and the scalability is also very poor. Compared with Jetty6, Jetty8 simplifies implementation. It uses the synchronous containers and atomic variables in JDK, And the implementation method is closer and closer to JDK.

Reference source code

• JDK source code: java. util. concurrent. ThreadPoolExecutor
• Jetty6 source code class: org. mortbay. thread. QueuedThreadPool
• Jetty8 source code class: org. eclipse. jetty. util. thread. QueuedThreadPool
• Tomcat source code: org. apache. tomcat. util. threads. ThreadPoolExecutor
• Python Thread Pool
• Quartz. Net SimpleThreadPool and ZeroSizeThreadPool