Linux Multithreading Practice (9)--design and implementation of simple thread pool

Technical background of thread pool

In object-oriented programming, creating and destroying objects is time-consuming, because creating an object takes memory resources or more resources . More so in Java, virtual machines will attempt to track each object so that it can be garbage collected after the object is destroyed. So one way to improve the efficiency of the service process is to minimize the number of objects created and destroyed, especially the resource-intensive object creation and destruction . How to use the existing objects to serve (more than a different task) is a key problem that needs to be solved, in fact, this is the cause of some "pooling resources" technology. For example, you are familiar with the database connection pool is to follow this idea, the thread pool technology described in this article also conforms to this idea.

at present, some famous big companies are particularly bullish on this technology, and have already applied the technology in their products. For example, IBM's Websphere,iona Orbix 2000 in Sun's Jini, Microsoft's MTS (Microsoft Transaction Server 2.0), COM +, and so on.

Do you also want to apply the technology to the server program now?

How thread pooling technology can improve the performance of server programs

The server program I mentioned refers to a program that accepts requests from clients and can process requests, not just those that accept requests from network clients.

Multithreading technology mainly solves the problem of multiple threads in the processor unit, which can significantly reduce the idle time of the processor unit and increase the throughput capacity of the processor unit. However, if you use multithreading incorrectly, you increase the processing time for individual tasks. A simple example can be cited:

Assume that the time to complete a task on a single server is T

T1 when the thread was created

T2 time to execute a task in a thread, including the time required to synchronize between threads

T3 the time the thread was destroyed

Obviously t = t1+t2+t3. Note that this is an extremely simplified hypothesis.

    you can see T1, T3 is the overhead of multithreading itself, and we aspire to Reduce the time spent on T1,T3 by , thus reducing t time. However, users of some threads do not notice this, so the threads are created or destroyed frequently in the program, This results in a considerable proportion of T1 and T3 in T (in the traditional multithreaded server model: Once a request arrives, a new thread is created, the thread executes the task, and the thread exits after the task has finished executing. This is the "instant Create, instant destroy" strategy. Although the time to create a thread has been greatly shortened compared to the creation process, the server will be in a state of creating threads and destroying threads if the task being committed to the thread is a short execution time and the number of executions is very frequent. This overhead is not negligible, especially when the thread is executing very, very short. )。 Obviously this is the highlight of the thread's weakness (T1,T3), not the merit (concurrency).

Thread Pooling technology is a technique that focuses on how to shorten or adjust t1,t3 time to improve the performance of server programs. it places t1,t3 on the start and end of the server program, or some idle time period ( after the application is started, a certain number of threads are created and placed in an idle queue. These threads are in a blocking state, which occupies only a bit of memory and does not occupy the CPU. When the task arrives, the thread pool selects an idle thread to run the task into this thread. When all threads are in the process of processing tasks, the thread pool automatically creates a certain number of new threads to handle more tasks. The thread does not exit after the execution of the task, but continues to wait for the next task in the threads pool. When most threads are in a blocking state, the thread pool automatically destroys a subset of the threads, reclaiming system resources, so that there is no t1,t3 overhead when the server program processes the client request.

the thread pool not only adjusts the time period generated by the T1,T3, but it also significantly reduces the number of threads created. Let's look at an example:

Suppose a server handles 50,000 requests a day, and each request requires a separate thread to complete. We compare the total number of threads that result from threading pool technology and the servers that are detrimental to thread pooling technology. Thread pooling, the number of threads is generally fixed, so the total number of threads does not exceed the number of threads in the thread pool or the upper limit (hereinafter referred to as the thread-pool size), and if the server does not utilize the thread pool to process these requests, the total number of threads is 50000. The general thread pool size is far less than 50000. Therefore , server programs that utilize the thread pool do not waste time processing requests in order to create 50000, thereby increasing efficiency .

Implementation of a simple thread pool

The following is an implementation of a simple thread pool, which uses the following scenarios:

1. Before the program starts, the thread pool is initialized, at which point there are no threads, and the AddTask method needs to be called to add tasks to the thread pool;

2. If the thread pool has idle (waiting) threads at this time, no new threads will be created, thus eliminating the T1, T3 time;

3. Create and start a thread if there is no waiting thread in the thread pool at this point (because the thread is definitely not in the waiting state at this point in the thread pool) and the number of threads in the thread pool does not reach the threshold at this point;

4. If the number of threads in the thread pool has reached the threshold at this point, then only the thread that is currently executing the task can wait until it finishes executing its current task and then takes the new task out of the task queue and executes it;

The thread pool consists mainly of two files, a threadpool.h header file and a threadpool.cpp source file. There are important comments in the source code, which is not analyzed.

Thread entry function//this is actually equivalent to a consumer thread, the constant consumption task (execution Task) void *thread_routine (void *args) {//sets the sub-thread to a detached state so that the main thread can not jion Pthread_de    Tach (Pthread_self ());    printf ("*thread 0x%lx is starting...\n", (unsigned Long) pthread_self ());    ThreadPool *pool = (ThreadPool *) args;        Wait for the task to arrive, and then perform the task while (true) {bool Timeout = false;        Pool->ready.lock ();        When waiting, it means that the idle thread is more than one + + pool->idle; The condition variables in Pool->ready have three functions://1. Wait for task queue to arrive//2. Wait for the thread pool to destroy the notification//3. Ensure that threads can be destroyed (thread exits) when waiting for timeout Hile (Pool->taskqueue.empty () && pool->quit = = False) {printf ("Thread 0x%lx is waiting..            . \ n ", (unsigned Long) pthread_self ());                Wait Waitseconds if (0! = pool->ready.timedwait (pool->waitseconds)) {//If wait timeout                printf ("Thread 0x%lx is wait timeout ... \ n", (unsigned Long) pthread_self ());                Timeout = true; Break out of the loop, continue down execution, willGo to the 1th if break at the bottom of the line;        }}//condition is ripe (when waiting for the end), the thread starts to perform a task or is thread destroyed, then the idle thread is one less--pool->idle; Status 3. If the wait time-out (typically at this time the task queue is empty) if (timeout = = True && pool->taskqueue.empty ()) {--pool-            >counter;            Unlock and then jump out of the loop, directly destroy the thread (exit thread) Pool->ready.unlock ();        Break             }//Status 2. If you wait for the thread to destroy the notification, and the task finishes executing if (Pool->quit = = True && pool->taskqueue.empty ()) {            --pool->counter; If there is no thread, send a notification to the thread pool//Tell the thread pool that there are no threads in the pool if (Pool->counter = = 0) pool->ready.sig            NAL ();            Unlock and then jump out of the Loop Pool->ready.unlock ();        Break }//Status 1. If there is a task, then perform the task if (!) ( Pool->taskqueue.empty ())) {//Remove task from Team head for processing threadpool::task_t *t = Pool->taskqueue.front            ();            Pool->taskqueue.pop (); It takes a certain amount of time//unlock to perform the task to facilitate itsHis producers can continue to produce tasks, other consumers can also consume tasks pool->ready.unlock ();            Processing Task T->run (T->args);        Delete T;    }}//After jumping out of the loop, print exit information and then destroy thread printf ("Thread 0x%lx is exiting...\n", (unsigned Long) pthread_self ()); Pthread_exit (NULL);}

The AddTask function//Add task function, similar to a producer, keeps the task generated, hangs on the task queue, waits for the consumer thread to consume void Threadpool::addtask (callback_t run, void *args) {    /** 1. Build the task and add the task to the task queue tail **/    task_t *newtask = new task_t {run, args};    Ready.lock ();   Note that the shared variable    taskqueue.push (NewTask) needs to be protected with a mutex.    /** 2. Let the thread begin to perform the task **/    startTask ();    Ready.unlock ();//Unlock for the task to start executing}

Thread start function void Threadpool::starttask () {    ///If there is a waiting thread, wake one of them and let it perform the task if    (Idle > 0)        ready.signal ();    Without waiting for the thread, and the current number of threads is not yet up to the threshold, we need to create a new thread    else if (counter < MaxThreads)    {        pthread_t tid;        Pthread_create (&tid, NULL, Thread_routine, this);        + + counter;    }}

destructor Threadpool::~threadpool () {    //If it has been called, return directly if    (quit = = true)        ;    Ready.lock ();    Quit = true;    if (Counter > 0)    {        //For waiting state, send them a notification,/        /These waiting threads will receive a notification,        //Then exit directly        if (Idle > 0)            ready.broadcast ();        They do not receive these notifications for the thread that is executing the task,        //They need to wait for them to finish the task while        (Counter > 0)            ready.wait ();    }    Ready.unlock ();}

Complete source code:http://download.csdn.net/download/hanqing280441589/8449049

Discussion on the advanced thread pool

There are some problems with the simple thread pool, such as if there are a large number of clients that require the server to serve it, but because the thread pool's worker threads are limited, the server can only serve some customers, and other client-submitted tasks can only wait for processing in the task queue . Some system designers may be dissatisfied with this situation, because their response time requirements for the server program are more stringent, so the system design may suspect the feasibility of thread pooling technology, but the thread pool has a corresponding solution. Tuning the optimization thread pool size is an issue to be addressed by advanced thread pooling. The following solutions are mainly available:

Scenario One: Dynamically increasing worker threads

the ability to dynamically change the number of worker threads in some advanced thread pools is typically provided to accommodate bursts of requests. Once the request becomes less, the number of worker threads in the thread pool is progressively reduced. Of course, thread increases can take a proactive approach, in which a batch of worker threads is added instead of a request to create a thread. Batch creation is a more efficient way. The scenario should also limit the upper and lower limits of the number of worker threads in the thread pool. Otherwise, this flexible approach becomes a bad way or a disaster, because frequent thread creation or a large number of threads in a short time will deviate from the original intent of using the thread pool-reducing the number of threads created.

Example: Jini in the TaskManager, is a smart thread pool manager, it is the dynamic increase of worker threads. SQL Server uses a single-process multi-threaded (multi-thread) system structure, 1024 number of thread pools, and dynamic threading allocations, theoretically limiting 32767.

Scenario Two: Optimize the number of worker threads

If you do not want the online pool to apply complex policies to ensure that the number of worker threads meets the requirements of your application, you will need to count the number of requests based on statistical principles, such as how many tasks require processing during peak hours on average within a second. And according to the system's ability to withstand and customer tolerance to balance the estimation of a reasonable thread pool size. The size of the thread pool is really hard to determine, so sometimes it simply uses empirical values.

Example: The size of the MTS thread pool is fixed at 100.  

Scenario Three: One server provides multiple thread pools

This scenario is used in a number of complex system architectures. This allows different thread pools to be processed according to different tasks or task priorities.

For example, COM + uses multiple thread pools.

Each of the three options has advantages and disadvantages. Different scenarios may be used in different applications or simply combine these three scenarios to solve practical problems.

Application scope of thread pool technology and problems to be paid attention to

Here are some of the thread pool applications I've summed up, which may not be comprehensive.

Application scope of thread pool:

(1) a large number of threads are required to complete the task, and the time to complete the task is relatively short . It is very appropriate to use thread pooling techniques to perform tasks such as Web server requests. Because a single task is small and the number of tasks is huge, you can imagine the number of clicks on a popular website. But for long-time tasks, such as a Telnet connection request, the thread pool's advantages are not obvious. Because the Telnet session time is much larger than the thread's creation time.

(2) for performance -demanding applications, such as requiring the server to respond quickly to customer requests.

(3) Receive a large number of bursts of requests, but do not allow the server to generate a large number of threading applications. Sudden large number of customer requests, in the absence of a thread pool, will generate a large number of threads, although theoretically most of the operating system threads maximum is not a problem, a short period of time to produce a large number of threads may cause memory to reach the limit, and the "OutOfMemory" error occurs.

Conclusion

This article is just a brief introduction to thread pooling technology. It can be seen that threading pool technology is significant for the performance improvement of server programs. Thread pool technology has a wide application foreground in server field. It is hoped that this technology can be applied to your multithreaded service program.

Note: This is the transformation of a blog online: the Java version of the thread pool has been transformed into a Linux-based C + + version, the original link is:http://www.ibm.com/developerworks/cn/java/ l-threadpool/, if the reader's interest is Java, please step in this, solemnly recommend to you, this is a very good article, thank you!

Linux Multithreading Practice (9)--design and implementation of simple thread pool