Thread Pool Model

/// // Threa_pool_model ///////// //////////////////////////////////////// //// task {task (ARG) {_ Arg = ARG;} process (); Private: _ ARG;} thread: thread (threadpool) {_ touchthreadpool = threadpool} thread: threadfun (ARG :( this )) {threadpool = This-_ touchthreadpool () while (thread. task = threadpool. gettask () onstart () process () // write the thread processing event process to the thread function. Onend () This. block () setselftofreethreadlist ()} thread: onstart () {} thread: Porcess () {task. posscess ();} thread: onend () {// free current task;} threadpool. addtask (task) {If (hasfreethread) thread = threadpool-> getthread (); thread. task = taskthread. run (); else if (threadcount <maxthreadcount) // create new thread to process else: bufftask (task );} /// // epoll + thread_pool ///////////// /// // Semi-synchronous/semi-asynchronous mode event: task {FD; type; process () {If (! Eventhandlers [ID]) {eventhandlers [ID] = new eventhandler ()} eventhandlers [ID]. Handler (this) handler is equivalent to synchronizing (completing the task in one thread. Upper-layer application collections). The thread pool is equivalent to asynchronous (multiple tasks are performed at the same time and multiple threads work at the same time (underlying I/O)} eventhandler {handler (Event) {Switch (event. type) Case (r) Case (w) Case (ERR)} trigger_r () trigger_w () trigger_err ()} eventhandlers [FD] = eventhandlerthreadpool {push (task )} epoll_dispatcher {run () {FDS = epoll_waite (); For (FD) {e = event (FD, type) threadpool. push (e) ;}} threadpool should be able to process all types of tasks. Yes, all types --- "class, as long as the task or event or other belongs to the same system, you can also use the process interface (interface-oriented programming instead of implementation) Epoll_wait (I/O thread) thread in threadpoll (Work thread) Half SYN/half Asyn has other ideas to reuse producers and consumers.

Other things: http://www.ibm.com/?works/cn/java/j-jtp0730/

Risks of using the thread pool

Although the thread pool is a powerful mechanism for building multi-threaded applications, using it is not risky. Applications built with the thread pool are vulnerable to all concurrency risks that any other multi-threaded applications may suffer, such as synchronization errors and deadlocks. It is also vulnerable to a few other risks that are specific to the thread pool, such as deadlocks related to the pool, insufficient resources, and thread leakage.

Deadlock

Any multi-threaded application has the risk of deadlock. When every one of a group of processes or threads is waiting for an event that can only be caused by another process in the group, we will say that this group of processes or threadsDeadlock. The simplest case of a deadlock is that thread a holds the exclusive lock of object X and waits for the lock of object y, while thread B holds the exclusive lock of object y while waiting for the lock of object X. Unless there is a way to break the lock wait (Java locks do not support this method), the deadlocked thread will wait forever.

Although there is a deadlock risk in any multi-threaded program, the thread pool introduces another deadlock possibility. In that case, all the pool threads are executing the task that is waiting for the execution result of another task in the blocked queue, but this task cannot be run because there is no unused thread. When the thread pool is used to simulate many interactive objects, the simulated objects can send queries to each other. These queries are then executed as queued tasks, and the query objects are waiting for response synchronously, this will happen.

Insufficient resources

One advantage of the thread pool is that it is generally executed well compared to other alternative Scheduling Mechanisms (which we have discussed. However, this is only true if the thread pool size is adjusted appropriately. Threads consume a large amount of resources, including memory and other system resources. BesidesThreadIn addition to the memory required by the object, each thread requires two potentially large execution call stacks. In addition, the JVM may create a local thread for each Java thread, which consumes additional system resources. Finally, although the scheduling overhead for switching between threads is small, if there are many threads, Environment switching may seriously affect the program performance.

If the thread pool is too large, the resources consumed by those threads may seriously affect the system performance. Switching between threads will waste time, and exceeding the number of threads you actually need may cause resource shortage because the pool thread is consuming some resources, these resources may be used more effectively by other tasks. In addition to the resources used by the thread itself, the work performed during service requests may require other resources, such as JDBC connections, sockets, or files. These resources are limited, and too many concurrent requests may also be invalid. For example, JDBC Connections cannot be allocated.

Concurrency Error

Thread pools and other queuing mechanisms rely onwait()Andnotify()Method. Both methods are difficult to use. If the encoding is incorrect, the notification may be lost, causing the thread to remain idle even if there is work in the queue to be processed. Be especially careful when using these methods; even experts may make mistakes on them. It is better to use existing implementations that are known to be able to work. For example, in the following example, you do not need to write your own pool to discussutil.concurrentPackage.

Thread Leakage

A serious risk in various thread pools is thread leakage. When a thread is removed from the pool to execute a task, but the thread does not return to the pool after the task is completed, this will happen. A thread leakage occurs when a task throwsRuntimeExceptionOrError. If the pool classes do not capture them, the thread will exit and the size of the thread pool will be permanently reduced by one. When this happens many times, the thread pool will eventually be empty and the system will stop because there is no available thread to process the task.

Some tasks may always wait for some resources or input from the user, and these resources cannot be guaranteed to become available, and the user may have already gone home. Such tasks will be permanently stopped, these stopped tasks also cause the same problems as thread leaks. If a thread is permanently consumed by such a task, it is actually removed from the pool. For such tasks, you should either give them their own threads or let them wait for a limited time.

Request overload

It is only possible that a request will crush the server. In this case, we may not want to queue every incoming request to our work queue, because tasks waiting for execution in the queue may consume too many system resources and cause resource shortage. In this case, it is up to you to decide what to do. In some cases, you can simply discard the request and retry the request later based on a higher level protocol, you can also use a response indicating that the server is temporarily busy to reject the request.

Back to Top

Guidelines for Effective Use of thread pools

As long as you follow several simple guidelines, the thread pool can be an extremely effective method for building server applications:

• Do not queue tasks waiting for other task results synchronously. This may lead to the deadlock in the form described above. In the deadlock, all threads are occupied by some tasks, and these tasks are waiting for the result of queuing tasks in turn, these tasks cannot be executed because all threads are busy.

• Be careful when using a commonly used thread for operations that may take a long time. If the program has to wait for a resource such as I/O to complete, specify the maximum wait time and whether the task will expire or be requeued for execution later. This ensures that by releasing a thread to a task that may be successfully completedSomeProgress.

• Understand the task. To effectively adjust the thread pool size, you need to understand the tasks that are being queued and what they are doing. Are they CPU-bound? Are they I/O-restricted (I/O-bound? Your answers will affect how you adjust your application. If you have different task classes and these classes have different features, it may be meaningful to set multiple work queues for different task classes, so that you can adjust each pool accordingly.

Back to Top

Adjust the pool size

Adjusting the thread pool size is basically to avoid two types of errors: Too few threads or too many threads. Fortunately, for most applications, there is a lot of room between too many and too few.

Recall: There are two main advantages of using threads in an application. Although you are waiting for slow operations such as I/O, you can continue processing and use multi-processor. In applications running on N processor machines with computing restrictions, adding additional threads when the number of threads is close to N may improve the total processing capability, when the number of threads exceeds N, adding additional threads does not work. In fact, too many threads may even reduce performance because it will lead to additional Environment switching overhead.

The optimum size of the thread pool depends on the number of available processors and the nature of tasks in the work queue. If there is only one working queue on a system with N processors, all of which are computing tasks, when the thread pool has n or N + 1 threads, the maximum CPU utilization is generally obtained.

For tasks that may need to wait for I/O to complete (for example, tasks that read HTTP requests from sockets), the pool size must exceed the number of available processors, because not all threads are working all the time. By using the summary analysis, You can estimate the ratio of the waiting time (wt) of a typical request to the service time (ST. If we call this ratio WT/ST, we need to set about N * (1 + WT/ST) for a system with N processors) threads to keep the processor fully utilized.

CPU utilization is not the only consideration for adjusting the thread pool size. As the thread pool grows, you may encounter restrictions on the scheduler, available memory, or other system resources, for example, the number of sockets, opened file handles, or database connections.

Thread Pool Model