[Java concurrent programming] 7. Thread Pool and java concurrent Thread Pool

[Java concurrent programming] 7. Thread Pool and java concurrent Thread Pool
1.Why use Thread Pool

Many server applications, such as Web servers, database servers, file servers, or email servers, are designed to process a large number of short tasks from some remote sources. Requests arrive at the server in a certain way, which may be through the network protocol (such as HTTP, FTP, or POP), through the JMS queue, or possibly by polling the database. Regardless of how requests arrive, server applications often encounter a situation where a single task is processed for a short period of time and the number of requests is huge.

A simple model for building a server application is to create a new thread whenever a request arrives and then serve the request in the new thread. In fact, this method works well for prototype development, but if you try to deploy a server application that runs in this way, the serious shortage of this method is obvious. One of the shortcomings of each request corresponding to a thread (thread-per-request) method is that the overhead of creating a new thread for each request is high; it takes more time and system resources to create and destroy a server that creates a new thread for each request than to process actual user requests.

In addition to the overhead of creating and destroying threads, active threads also consume system resources. Creating too many threads in a JVM may cause the system to use up memory or "over-switching" due to excessive memory consumption ". To prevent resource insufficiency, server applications need some methods to limit the number of requests processed at any given time point.

The thread pool provides a solution for thread lifecycle overhead and resource insufficiency. By reusing threads for multiple tasks, the overhead created by threads is apportioned to multiple tasks. The advantage is that the thread already exists when the request arrives, so the delay caused by thread creation is accidentally eliminated. In this way, the application can immediately respond to the request. In addition, adjust the number of threads in the thread pool appropriately, that is, when the number of requests exceeds a certain threshold, force any other new requests to wait, wait until a thread is obtained for processing to prevent resource insufficiency.

2. Risks of using the thread pool

Although the thread pool is a powerful mechanism for building multi-threaded applications, using it is not risky. Applications built with the thread pool are vulnerable to all concurrency risks that any other multi-threaded applications may suffer, such as synchronization errors and deadlocks. It is also vulnerable to a few other risks that are specific to the thread pool, such as deadlocks related to the pool, insufficient resources, and thread leakage.

2.1 Deadlock

Any multi-threaded application has the risk of deadlock. When every one of a group of processes or threads is waiting for an event that can only be caused by another process in the group, we will say that this group of processes or threads are deadlocked. The simplest case of A deadlock is that thread A holds the exclusive lock of object X and waits for the lock of object Y, while thread B holds the exclusive lock of object Y while waiting for the lock of object X. Unless there is a way to break the lock wait (Java locks do not support this method), the deadlocked thread will wait forever.

Although there is a deadlock risk in any multi-threaded program, the thread pool introduces another deadlock possibility. In that case, all the pool threads are executing the task that is waiting for the execution result of another task in the blocked queue, but this task cannot be run because there is no unused thread. When the thread pool is used to simulate many interactive objects, the simulated objects can send queries to each other. These queries are then executed as queued tasks, and the query objects are waiting for response synchronously, this will happen.

2.2 Insufficient resources

One advantage of the thread pool is that it is generally executed well compared to other alternative Scheduling Mechanisms (which we have discussed. However, this is only true if the thread pool size is adjusted appropriately. Threads consume a large amount of resources, including memory and other system resources. In addition to the memory required by the Thread object, each Thread requires two potentially large execution call stacks. In addition, the JVM may create a local thread for each Java thread, which consumes additional system resources. Finally, although the scheduling overhead for switching between threads is small, if there are many threads, Environment switching may seriously affect the program performance.

If the thread pool is too large, the resources consumed by those threads may seriously affect the system performance. Switching between threads will waste time, and exceeding the number of threads you actually need may cause resource shortage because the pool thread is consuming some resources, these resources may be used more effectively by other tasks. In addition to the resources used by the thread itself, the work performed during service requests may require other resources, such as JDBC connections, sockets, or files. These resources are limited, and too many concurrent requests may also be invalid. For example, JDBC Connections cannot be allocated.

2.3 Concurrency Error

The thread pool and other queuing systems depend on the wait () and Y () methods, which are difficult to use. If the encoding is incorrect, the notification may be lost, causing the thread to remain idle even if there is work in the queue to be processed. Be especially careful when using these methods. It is better to use existing implementations that are known to work, such as the util. concurrent package.

2.4 Thread Leakage

A serious risk in various thread pools is thread leakage. When a thread is removed from the pool to execute a task, but the thread does not return to the pool after the task is completed, this will happen. A thread leakage occurs when the task throws a RuntimeException or Error. If the pool classes do not capture them, the thread will exit and the size of the thread pool will be permanently reduced by one. When this happens many times, the thread pool will eventually be empty and the system will stop because there is no available thread to process the task.

Some tasks may always wait for some resources or input from the user, and these resources cannot be guaranteed to become available, and the user may have already gone home. Such tasks will be permanently stopped, these stopped tasks also cause the same problems as thread leaks. If a thread is permanently consumed by such a task, it is actually removed from the pool. For such tasks, you should either give them their own threads or let them wait for a limited time.

2.5 Request overload

It is only possible that a request will crush the server. In this case, we may not want to queue every incoming request to our work queue, because tasks waiting for execution in the queue may consume too many system resources and cause resource shortage. In this case, it is up to you to decide what to do. In some cases, you can simply discard the request and retry the request later based on a higher level protocol, you can also use a response indicating that the server is temporarily busy to reject the request.

3. Guidelines for Effective Use of thread pools

As long as you follow several simple guidelines, the thread pool can be an extremely effective method for building server applications:

Do not queue tasks waiting for other task results synchronously. This may lead to the deadlock in the form described above. In the deadlock, all threads are occupied by some tasks, and these tasks are waiting for the result of queuing tasks in turn, these tasks cannot be executed because all threads are busy.

Be careful when using a commonly used thread for operations that may take a long time. If the program has to wait for a resource such as I/O to complete, specify the maximum wait time and whether the task will expire or be requeued for execution later. This ensures that some progress will be achieved by releasing a thread to a task that may be successfully completed.

Understand the task. To effectively adjust the thread pool size, you need to understand the tasks that are being queued and what they are doing. Are they CPU-bound? Are they I/O-restricted (I/O-bound? Your answers will affect how you adjust your application. If you have different task classes and these classes have different features, it may be meaningful to set multiple work queues for different task classes, so that you can adjust each pool accordingly.

4. Thread pool size settings

Adjusting the thread pool size is basically to avoid two types of errors: Too few threads or too many threads. Fortunately, for most applications, there is a lot of room between too many and too few.

Recall: There are two main advantages of using threads in an application. Although you are waiting for slow operations such as I/O, you can continue processing and use multi-processor. In applications running on N processor machines with computing restrictions, adding additional threads when the number of threads is close to N may improve the total processing capability, when the number of threads exceeds N, adding additional threads does not work. In fact, too many threads may even reduce performance because it will lead to additional Environment switching overhead.

The optimum size of the thread pool depends on the number of available processors and the nature of tasks in the work queue. If there is only one working queue on a system with N processors, all of which are computing tasks, when the thread pool has N or N + 1 threads, the maximum CPU utilization is generally obtained.

For tasks that may need to wait for I/O to complete (for example, tasks that read HTTP requests from sockets), the pool size must exceed the number of available processors, because not all threads are working all the time. By using the summary analysis, You can estimate the ratio of the waiting time (WT) of a typical request to the service time (ST. If we call this ratio WT/ST, we need to set about N * (1 + WT/ST) for a system with N processors) threads to keep the processor fully utilized.

CPU utilization is not the only consideration for adjusting the thread pool size. As the thread pool grows, you may encounter restrictions on the scheduler, available memory, or other system resources, for example, the number of sockets, opened file handles, or database connections.

5. Several common thread pools5.1 NewCachedThreadPool

Create a cache thread pool. If the thread pool length exceeds the processing requirement, free threads can be recycled flexibly. If no thread pool can be recycled, a new thread is created.

This type of thread pool features:

• There is almost no limit on the number of worker threads created (in fact, there are also restrictions, the number is Interger. MAX_VALUE), so that you can flexibly add threads to the thread pool.
• If a task is not submitted to the thread pool for a long time, that is, if the worker thread is idle for a specified time (1 minute by default), the worker thread is automatically terminated. After the termination, if you submit a new task, the thread pool creates a new working thread.
• When using CachedThreadPool, be sure to control the number of tasks. Otherwise, the system is paralyzed because a large number of threads run simultaneously.

The sample code is as follows:

 1 package test; 2 import java.util.concurrent.ExecutorService; 3 import java.util.concurrent.Executors; 4 public class ThreadPoolExecutorTest { 5  public static void main(String[] args) { 6   ExecutorService cachedThreadPool = Executors.newCachedThreadPool(); 7   for (int i = 0; i < 10; i++) { 8    final int index = i; 9    try {10     Thread.sleep(index * 1000);11    } catch (InterruptedException e) {12     e.printStackTrace();13    }14    cachedThreadPool.execute(new Runnable() {15     public void run() {16      System.out.println(index);17     }18    });19   }20  }21 }

5.1 NewFixedThreadPool

Creates a thread pool with a specified number of worker threads. A worker thread is created every time a task is submitted. If the number of worker threads reaches the maximum initial number of the thread pool, the submitted tasks are saved to the pool queue.

FixedThreadPool is a typical and excellent thread pool, which has the advantages of improving program efficiency and saving the overhead consumed when creating a thread. However, when the thread pool is idle, that is, when no task can be run in the thread pool, it will not release the working thread and occupy certain system resources.

The sample code is as follows:

 

package test;import java.util.concurrent.ExecutorService;import java.util.concurrent.Executors;public class ThreadPoolExecutorTest { public static void main(String[] args) {  ExecutorService fixedThreadPool = Executors.newFixedThreadPool(3);  for (int i = 0; i < 10; i++) {   final int index = i;   fixedThreadPool.execute(new Runnable() {    public void run() {     try {      System.out.println(index);      Thread.sleep(2000);     } catch (InterruptedException e) {      e.printStackTrace();     }    }   });  } }}

 

Because the thread pool size is 3 and each task outputs an index and sleep for 2 seconds, three numbers are printed every two seconds.
It is best to set the size of a fixed-length thread pool based on system resources such as Runtime. getRuntime (). availableProcessors ().

 

5.1 NewSingleThreadExecutor

Create a single-threaded Executor, that is, only create a unique worker thread to execute the task. It only uses a unique worker thread to execute the task, ensuring that all tasks are in the specified Order (FIFO, LIFO, priority) execution. If this thread ends abnormally, another thread will replace it to ensure sequential execution. The biggest feature of a single working thread is that each task can be executed sequentially, and no multiple threads are active at any given time.

The sample code is as follows:

 

package test;import java.util.concurrent.ExecutorService;import java.util.concurrent.Executors;public class ThreadPoolExecutorTest { public static void main(String[] args) {  ExecutorService singleThreadExecutor = Executors.newSingleThreadExecutor();  for (int i = 0; i < 10; i++) {   final int index = i;   singleThreadExecutor.execute(new Runnable() {    public void run() {     try {      System.out.println(index);      Thread.sleep(2000);     } catch (InterruptedException e) {      e.printStackTrace();     }    }   });  } }} 

 

5.1 NewScheduleThreadPool

Creates a fixed-length thread pool and supports scheduled and periodic task execution. It also supports scheduled and periodic task execution.

The execution is delayed by 3 seconds. The following is an example code:

 

package test;import java.util.concurrent.Executors;import java.util.concurrent.ScheduledExecutorService;import java.util.concurrent.TimeUnit;public class ThreadPoolExecutorTest { public static void main(String[] args) {  ScheduledExecutorService scheduledThreadPool = Executors.newScheduledThreadPool(5);  scheduledThreadPool.schedule(new Runnable() {   public void run() {    System.out.println("delay 3 seconds");   }  }, 3, TimeUnit.SECONDS); }}

 

It indicates that execution is performed every 3 seconds after a delay of 1 second. The sample code for regular execution is as follows:

 

package test;import java.util.concurrent.Executors;import java.util.concurrent.ScheduledExecutorService;import java.util.concurrent.TimeUnit;public class ThreadPoolExecutorTest { public static void main(String[] args) {  ScheduledExecutorService scheduledThreadPool = Executors.newScheduledThreadPool(5);  scheduledThreadPool.scheduleAtFixedRate(new Runnable() {   public void run() {    System.out.println("delay 1 seconds, and excute every 3 seconds");   }  }, 1, 3, TimeUnit.SECONDS); }}