C ++ thread pool-multi-thread development in Linux

Why Thread Pool

Currently, most network servers, including Web servers, email servers, and database servers, share the same thing: a large number of connection requests must be processed per unit time, but the processing time is relatively short.

The server model we use in the traditional multithreading solution is to create a new thread for the thread to execute the task once the request is received. After the task is executed, the thread exits. This is the policy of "instant creation and destruction. Although the thread creation time has been greatly shortened compared with the creation process, if the task submitted to the thread is executed for a short time, and the execution times are extremely frequent, the server will be in the state of creating and destroying threads continuously.

We divide the thread Execution Process in the traditional solution into three processes: T1, T2, and T3. T1: thread creation time t2: thread execution time, including thread synchronization time t3: thread destruction time so we can see that the proportion of thread overhead is (t1 + T3) /(t1 + T2 + T3 ). If the thread execution time is short, the overhead may be about 20%-50%. If the task is executed frequently, this overhead cannot be ignored. In addition, the thread pool can reduce the number of created threads. Generally, the concurrent threads allowed by the thread pool have an upper bound. If the number of concurrent threads needs to exceed the upper bound, some threads will wait. In the traditional scheme, if the number of requests is 2000 at the same time, the system may need to generate 2000 threads in the worst case. Although this is not a large number, some machines cannot meet this requirement. Therefore, the appearance of the thread pool focuses on reducing the overhead brought by the thread pool itself. The thread pool uses the pre-created technology. After the application is started, a certain number of threads (N1) will be created immediately and put into the idle queue. These threads are all in the congested State, which does not consume CPU, but takes up a small amount of memory space. When the task arrives, the buffer pool selects an idle thread to pass the task to this thread for running. When N1 threads are processing tasks, the buffer pool automatically creates a certain number of new threads for processing more tasks. After the task is executed, the thread does not exit, but continues to wait for the next task in the pool. When the system is idle, most threads remain in the paused state. The thread pool automatically destroys some threads and recycles system resources.

Based on this pre-creation technology, the thread pool allocates the overhead of thread creation and destruction to specific tasks. The more execution times, the smaller the thread overhead shared by each task, but we may also need to consider the overhead of synchronization between threads.

Build thread pool frameworkGenerally, a thread pool must have the following components:

• Thread pool manager:Used to create and manage thread pools
• Worker thread: Actually executed thread in the thread pool
• Task Interface: Although the thread pool is used to support network servers in most cases, we abstract the tasks executed by the thread to form the task interface, so that the thread pool is irrelevant to the specific task.
• Task queue: The concept of thread pool is specific to the implementation, it may be a data structure such as a queue or a linked list, where the execution thread is saved.

The universal thread pool framework we implement consists of five important parts: cthreadmanage, cthreadpool, cthread, cjob, and cworkerthread. In addition, the framework also includes the class cthreadmutex and ccondition used for thread synchronization. Cjob is the base class of all tasks. It provides an interface to run. All Task classes must inherit from the class and implement the run method. The specific task logic is implemented in this method. Cthread is the packaging of threads in Linux. It encapsulates the attributes and methods most frequently used by Linux threads. It is also an abstract class and is the base class of all thread classes and has an interface run. Cworkerthread is the Thread class actually scheduled and executed. It inherits from cthread and implements the run method in cthread. Cthreadpool is a thread pool class that stores, releases, and schedules threads. Cthreadmanage is the direct interface between the thread pool and the user, which shields the specific internal implementation. Cthreadmutex is used for mutual exclusion between threads. Ccondition is the encapsulation of condition variables for synchronization between threads.

Cthreadmanage directly deals with the client. It accepts the initial number of threads to be created and accepts the tasks submitted by the client. The task here is a specific non-Abstract task. Cthreadmanage actually calls cthreadpool operations. The cthreadpool creates a specific thread and distributes the tasks submitted by the client to cworkerthread. The cworkerthread actually executes the specific tasks.
 

Understanding System ComponentsThe following describes the components in the system separately. Cthreadmanagecthreadmanage provides the simplest method. Its class definition is as follows: Class cthreadmanage {PRIVATE: cthreadpool * m_pool; int m_numofthread; protected: public: cthreadmanage (); cthreadmanage (INT num); Virtual ~ Cthreadmanage (); void setparallelnum (INT num); void run (cjob * job, void * jobdata); void terminateall (void) ;}; m_pool points to the actual thread pool; m_numofthread is the number of concurrent threads allowed during initial creation. In addition, the run and terminateall methods are also very simple, but they simply call some related methods of cthreadpool. The specific implementation is as follows: cthreadmanage () {m_numofthread = 10; m_pool = new cthreadpool (m_numofthread);} cthreadmanage: cthreadmanage (INT num) {m_numofthread = num; m_pool = new cthreadpool (m_numofthread);} cthreadmanage ::~ Cthreadmanage () {If (null! = M_pool) delete m_pool;} void cthreadmanage: setparallelnum (INT num) {m_numofthread = num;} void cthreadmanage: Run (cjob * job, void * jobdata) {m_pool-> Run (job, jobdata);} void cthreadmanage: terminateall (void) {m_pool-> terminateall ();} CthreadThe cthread class encapsulates thread operations in Linux. It is the base class of all threads and an abstract class. It provides an abstract interface run, all cthreads must implement the run method. Cthread is defined as follows: Class cthread {PRIVATE: int m_errcode; semaphore m_threadsemaphore; // The inner semaphore, which is used to realize unsigned long m_threadid; bool m_detach; // The thread is detached bool m_createsuincluded; // If suspend after creating char * m_threadname; threadstate m_threadstate; // The state of the thread protected: void seterrcode (INT errcode) {m_errcode = errcode;} static void * th Readfunction (void *); Public: cthread (); cthread (bool createsu0000ded, bool detach); Virtual ~ Cthread (); Virtual void run (void) = 0; void setthreadstate (threadstate state) {m_threadstate = State;} bool terminate (void ); // terminate the threa bool start (void); // start to execute the thread void exit (void); bool wakeup (void); threadstate getthreadstate (void) {return m_threadstate ;} int getlasterror (void) {return m_errcode;} void setthreadname (char * thrname) {strcpy (m_threadname, thrname);} Char * Ge Tthreadname (void) {return m_threadname;} int getthreadid (void) {return m_threadid;} bool setpriority (INT priority); int getpriority (void); int getconcurrency (void ); void setconcurrency (INT num); bool detach (void); bool join (void); bool yield (void); int self (void );}; the thread status can be divided into four types: idle, busy, suspended, and terminated (including normal exit and abnormal exit ). Currently, the Linux thread library does not support the suspension operation. Therefore, the suspension operation here is similar to the suspension operation. If the thread does not want to execute the task immediately after it is created, we can "pause" it. If it needs to be run, it will wake up. It must be noted that once the thread starts to execute the task, it cannot be suspended and it will continue to execute the task until it is completed. Operations related to the Thread class are very simple. The thread execution entry starts from the start () function. It will call the threadfunction and threadfunction to call the actual run function to execute the actual task. Cthreadpoolcthreadpool is the container of the thread. Generally, it can be implemented as a stack, one-way queue, or two-way queue. In our system, we use STL vector to store threads. The implementation code of cthreadpool is as follows: Class cthreadpool {friend class cworkerthread; private: Unsigned int m_maxnum; // The Max thread num that can create at the same time unsigned int m_availlow; // The min num of idle thread that shoule kept unsigned int m_availhigh; // The Max num of idle thread that kept at the same time unsigned int m_availnum; // The normal thread num of idle num; unsigned int m_initnum; // normal thread num; protected: cworkerthread * getidlethread (void); void appendtoidlelist (cworkerthread * jobthread ); void Merge (cworkerthread * idlethread); void movetoidlelist (cworkerthread * busythread); void deleteidlethread (INT num); void createidlethread (INT num); Public: cthreadmutex m_busymutex; // when visit busy list, use m_busymutex to lock and unlock cthreadmutex m_idlemutex; // when visit idle list, use m_idlemutex to lock and unlock cthreadmutex tables; // when visit job list, use Cases to lock and unlock cthreadmutex m_varmutex; ccondition m_busycond; // m_busycond is used to sync busy thread list ccondition m_idlecond; // operation is used to sync idle thread list ccondition limit; // m_jobcond is used to sync job list ccondition m_maxnumcond;

Vector <cworkerthread *> m_threadlist; vector <cworkerthread *> m_busylist; // thread list vector <cworkerthread *> m_idlelist; // idle list cthreadpool (); cthreadpool (INT initnum ); virtual ~ Cthreadpool (); void setmaxnum (INT maxnum) {m_maxnum = maxnum;} int getmaxnum (void) {return m_maxnum;} void setavaillownum (INT minnum) {m_availlow = minnum ;} int getavaillownum (void) {return m_availlow;} void setavailhighnum (INT high) {m_availhigh = highnum;} int getavailhighnum (void) {return m_availhigh;} int getactualavailnum (void) {return m_availnum;} int getallnum (void) {return m_threadlist.size ();} int getbusynum (void) {return m_busylist.size ();} void setinitnum (INT initnum) {m_initnum = initnum;} int getinitnum (void) {return m_initnum;} void terminateall (void); void run (cjob * job, void * jobdata );};

 

Cworkerthread * cthreadpool: getidlethread (void) {While (m_idlelist.size () = 0) m_idlecond.wait (); m_idlemutex.lock (); If (m_idlelist.size ()> 0) {cworkerthread * thr = (cworkerthread *) m_idlelist.front (); printf ("Get idle thread % d \ n", Thr-> getthreadid (); m_idlemutex.unlock (); return thr;} m_idlemutex.unlock (); return NULL ;}

// Create num idle thread and put them to idlelist void cthreadpool: createidlethread (INT num) {for (INT I = 0; I <num; I ++) {cworkerthread * thr = new cworkerthread (); Thr-> setthreadpool (this); appendtoidlelist (THR); m_varmutex.lock (); m_availnum ++; m_varmutex.unlock (); thr-> Start (); // begin the thread, the thread wait for job }}

Void cthreadpool: Run (cjob * job, void * jobdata) {assert (job! = NULL); // If the busy thread num adds to m_maxnum, So we shoshould wait if (getbusynum () = m_maxnum) m_maxnumcond.wait (); If (m_idlelist.size () <m_availlow) {If (getallnum () + m_InitNum-m_IdleList.size () <m_maxnum) createidlethread (m_InitNum-m_IdleList.size (); else createidlethread (m_MaxNum-GetAllNum ();} cworkerthread * idlethr = getidlethread (); if (idlethr! = NULL) {idlethr-> m_workmutex.lock (); movetobusylist (idlethr); idlethr-> setthreadpool (this); job-> setworkthread (idlethr ); printf ("job is set to thread % d \ n", idlethr-> getthreadid (); idlethr-> setjob (job, jobdata );}} there are two linked lists in the cthreadpool. One is the idle linked list and the other is the busy linked list. The idle linked list stores all idle processes. When a thread executes a task, its status changes to busy. It also deletes it from the idle linked list and moves it to the busy linked list. In the cthreadpool constructor, we will execute the following code: For (INT I = 0; I <m_initnum; I ++) {cworkerthread * thr = new cworkerthread (); appendtoidlelist (THR); Thr-> setthreadpool (this); Thr-> Start (); // begin the thread, the thread wait for job} in this code, we will create m_initnum threads. After creation, we will call appendtoidlelist to put it into the idle linked list. Because no tasks are distributed to these threads, the thread will suspend itself after executing start. In fact, the number of threads in the thread pool is not static, and it will automatically scale according to the execution load. Therefore, in the cthreadpool, set four variables: m_initnum: Number of threads in the thread pool at the time of creation. M_maxnum: Maximum number of concurrent threads allowed in the current thread pool. M_availlow: the minimum number of Idle threads allowed in the current thread pool. If the number of Idle threads is lower than this value, the load may be too heavy. In this case, it is necessary to increase the number of idle thread pools. In implementation, we always adjust the number of threads to m_initnum. M_availhigh: the maximum number of Idle threads allowed in the current thread pool. If the number of Idle threads is higher than this value, the current load may be light, and redundant Idle threads will be deleted, after deletion, the number of adjustments is also m_initnum. M_availnum: the number of actually existing threads in the thread pool. The value is between m_availhigh and m_availlow. If the number of threads is always between m_availlow and m_availhigh, the threads do not need to be created or deleted to maintain a balance. Therefore, how to set values of m_availlow and m_availhigh to keep the maximum possible equilibrium state of the thread pool is an issue that must be considered in the thread pool design. After receiving a new task, the thread pool must first check whether there is sufficient idle pool available. The check is divided into three steps: (1) check whether the current busy thread has reached the set maximum m_maxnum. If yes, it indicates that no idle thread is available, in addition, a new thread cannot be created. Therefore, you must wait until a thread completes execution and return it to the idle queue. (2) If the current number of Idle threads is smaller than the minimum number of Idle threads we set m_availlow, we must create a new thread. By default, the number of created threads should be m_initnum, therefore, the number of created threads should be (the number of Idle threads and m_initnum). However, there is a special case that you must consider that the number of existing threads plus the number of created threads may exceed m_maxnum, therefore, we must treat thread creation differently. If (getallnum () + m_InitNum-m_IdleList.size () <m_maxnum) createidlethread (m_InitNum-m_IdleList.size (); else createidlethread (m_MaxNum-GetAllNum (); if the total number after creation does not exceed m_maxnum, then, the created thread is m_initnum. If it is exceeded, only (m_maxnum-Total number of current threads) will be created. (3) Call the getidlethread method to find Idle threads. If no idle thread exists, the task is suspended. Otherwise, the task is assigned to the thread and moved to the busy queue. After the thread is executed, it will call the movetoidlelist method to move it into the idle linked list, and call the m_idlecond.signal () method to wake up the thread that may be blocked in getidlethread. The cjobcjob class is relatively simple. It encapsulates the basic attributes and methods of the task. The most important thing is the run method. The Code is as follows: Class cjob {PRIVATE: int m_jobno; // The num was assigned to the job char * m_jobname; // The job name cthread * m_pworkthread; // The Thread associated with the job public: cjob (void); Virtual ~ Cjob (); int getjobno (void) const {return m_jobno;} void setjobno (INT jobno) {m_jobno = jobno;} Char * getjobname (void) const {return m_jobname ;} void setjobname (char * jobname); cthread * getworkthread (void) {return m_pworkthread;} void setworkthread (cthread * pworkthread) {m_pworkthread = pworkthread;} virtual void run (void * PTR) = 0 ;};

 Example of thread pool usageSo far, we have provided a simple thread pool framework unrelated to specific tasks. This framework is very simple to use. All we need to do is to derive the cjob class and implement the tasks to be completed in the run method. Then the job is submitted to cthreadmanage for execution. The following is a simple example program class cxjob: Public cjob {public: cxjob () {I = 0 ;}~ Cxjob () {} void run (void * jobdata) {printf ("The job comes from cxjob \ n"); sleep (2) ;}}; class cyjob: public cjob {public: cyjob () {I = 0 ;}~ Cyjob () {}void run (void * jobdata) {printf ("The job comes from cyjob \ n") ;}}; main () {cthreadmanage * manage = new cthreadmanage (10); For (INT I = 0; I <40; I ++) {cxjob * job = new cxjob (); manage-> Run (job, null);} Sleep (2); cyjob * job = new cyjob (); manage-> Run (job, null ); manage-> terminateall ();} both cxjob and cyjob are inherited from the job class and all implement the run interface. Cxjob simply prints the statement "The job comes from cxjob", and cyjob only prints "The job comes from cyjob", and then sleeps for 2 seconds. In the main program, we initially create 10 working threads. Then execute the cxjob and cyjob for 40 times.Postscript for thread pool usageSuitable for use in thread pools

In fact, the thread pool is not omnipotent. It has specific application scenarios. The thread pool is designed to reduce the impact of thread overhead on applications. The premise is that the thread overhead cannot be ignored compared with the thread execution task. If the thread overhead is negligible relative to the thread task overhead, the benefits of the thread pool are not obvious, such as for the FTP server and telnet server, generally, it takes a long time to transfer files, and the overhead is large. In this case, using a thread pool may not be an ideal method. We can choose the "instant creation, instant destruction" policy.

In short, the thread pool is usually suitable for the following scenarios: (1) Processing Tasks frequently per unit time and short processing time (2) high real-time requirements. If the thread is created after the task is accepted, the real-time requirement may not be met. Therefore, the thread pool must be used for pre-creation. (3) It is necessary to often face high emergency events, such as web servers. If football is broadcast, the server will have a huge impact. In this case, if the traditional method is adopted, a large number of threads must be continuously generated and destroyed. In this case, dynamic thread pools can be used to avoid this situation. In conclusion, this article provides a simple and general implementation of a thread pool unrelated to the task. This thread pool can greatly simplify the development of multithreading in Linux.