Linux C + + thread pool Framework (Favorites)

In this paper, we give a general Thread Pool Framework, the framework abstracts high-level tasks related to thread execution, making it irrelevant to specific execution tasks. In addition, the thread pool has dynamic scalability, it can automatically adjust the number of thread pool threads according to the severity of the task. At the end of the article, we'll give you a simple example program, and we'll see how easy it is to perform multithreaded tasks through the thread pool framework.


1. Why the thread pool is needed

Most of the current Web servers, including Web servers, email servers, and database servers, all have one thing in common: a large number of connection requests must be processed within a unit of time, but the processing time is relatively short.
The server model we used in the traditional multithreaded scenario is that once the request is received, a new thread is created, and the thread executes the task. After the task has finished executing, the thread exits, which is the policy of "create immediately, destroy immediately." Although the time to create a thread has been greatly shortened compared to the creation process, if the task submitted to the thread is execution time is short and the execution is extremely frequent, the server will be in a constant state of creating threads and destroying the threads.
We divide the thread execution process in the traditional scenario into three processes: T1, T2, T3.
T1: Thread creation time
T2: Thread execution time, including synchronization of threads, etc.
T3: Thread Destroy Time
Then we can see that the cost of the thread itself is (T1+T3)/(T1+T2+T3). If the thread executes for a short time, this may account for about 20% to 50% of the cost. This overhead is not negligible if the task is performed frequently.
In addition, the thread pool can reduce the number of threads created. Usually the thread pool allows concurrent threads with an upper bound, and if the number of concurrent threads exceeds the upper bound, a portion of the thread will wait. In the traditional scenario, if the number of simultaneous requests is 2000, then the system may need to produce 2000 threads at worst. Although this is not a large number, some machines may not be able to achieve this requirement.
So the thread pool appears to be focused on reducing the overhead of the thread pool itself. The thread pool takes a pre-built technique and, after the application starts, creates a number of threads (N1) immediately and puts it into the idle queue. These threads are in a blocking (suspended) state, do not consume the CPU, but occupy a small amount of memory space. When the task arrives, the buffer pool selects an idle thread and runs the task into this thread. When N1 threads are working on a task, the buffer pool automatically creates a number of new threads to handle more tasks. The thread does not quit after the task has finished executing, but continues to remain in the pool waiting for the next task. When the system is more idle, most of the threads have been paused, the thread pool automatically destroys some of the threads and reclaims the system resources.
Based on this pre-built technology, the thread pool distributes the overhead of threading creation and destruction itself to individual tasks, and the more executions, the less the thread itself spends on each task, but we may also need to consider the overhead of synchronizing between threads.

2. Build thread pool Framework

The general thread pool must have the following components:
　　 Thread pool Manager: Used to create and manage thread pools
　　 Worker Threads: Threads actually executing in the thread pool
　　 Task Interface: Although a thread pool is most often used to support a network server, we abstract the tasks that the thread performs to form the task interface, so that the thread pool is independent of the specific task.
　　 Task Queues: The concept of a thread pool can be specific to the implementation of a queue, a data structure such as a list, where the execution thread is saved.
The universal thread pool Framework We implement consists of five important parts cthreadmanage,cthreadpool,cthread,cjob,cworkerthread, In addition, the framework includes classes Cthreadmutex and ccondition that are used by thread synchronization.
Cjob is the base class for all tasks that provide an interface run, and all task classes must inherit from that class while implementing the Run method. The specific task logic is implemented in this method.
Cthread is a Linux thread wrapper that encapsulates the properties and methods most often used by Linux threads, an abstract class that is the base class for all thread classes and has an interface run.
CWorkerThread is a thread class that is actually scheduled and executed, which inherits from Cthread and implements the Run method in Cthread.
CThreadPool is a thread pool class that is responsible for saving threads, freeing threads, and scheduling threads.
Cthreadmanage is a direct interface between the thread pool and the user, which masks the concrete implementation of the internal.
Cthreadmutex is used for mutual exclusion between threads.
Ccondition is the encapsulation of the conditional variable, used for synchronization between threads.
The inheritance relationships of their classes are shown in the following illustration: (To ADD)
The sequence of the thread pool is simple, as shown in the following figure: (to ADD).
Cthreadmanage deals directly with clients, accepting the initial number of threads that need to be created and accepting the tasks that the client submits. The task here is a concrete, non-abstract task. The inside of the cthreadmanage is actually called CThreadPool related operations. CThreadPool Create specific threads and distribute the tasks submitted by the client to Cworkerthread,cworkerthread to actually perform specific tasks.

3. Understanding System Components

Let's take a look at each component in the system separately.
　　Cthreadmanage
The Cthreadmanage feature is very simple, providing the simplest method, with classes defined as follows:

Class Cthreadmanage {
private:
    cthreadpool* M_pool;
    int m_numofthread;
Public:
    cthreadmanage ();
    cthreadmanage (int num);
    Virtual ~cthreadmanage ();

    void setparallelnum (int num); 
    void Run (cjob* job,void* jobdata);
    void Terminateall (void);

Where M_pool points to the actual thread pool; M_numofthread is the number of concurrent threads that are allowed to be created at the initial creation time. The run and Terminateall method is also very simple, just a simple call to cthreadpool some related methods. Its specific implementation is as follows:

Cthreadmanage::cthreadmanage () {
    m_numofthread = ten;
    M_pool = new CThreadPool (m_numofthread);
}

cthreadmanage::cthreadmanage (int num) {
    m_numofthread = num;
    M_pool = new CThreadPool (m_numofthread);
}

Cthreadmanage::~cthreadmanage () {
    if (NULL!= m_pool)
        delete m_pool;
}

void cthreadmanage::setparallelnum (int num) {
    m_numofthread = num;
}

void Cthreadmanage::run (cjob* job,void* jobdata) {
    m_pool->run (job,jobdata);
}

void Cthreadmanage::terminateall (void) {
    m_pool->terminateall ();
}

　　Cthread

The Cthread class implements the encapsulation of the Linux thread operation, which is the base class for all threads and an abstract class that provides an abstract interface run, and all cthread must implement the Run method. The definition of Cthread is as follows:

Class Cthread {Private:int m_errcode; Semaphore M_threadsemaphore; 
    The inner semaphore, which is used to realize unsigned long m_threadid; BOOL M_detach; The thread is detached bool m_createsuspended;
    If suspend after creating char* m_threadname; ThreadState m_threadstate;
    The state of the thread protected:void seterrcode (int errcode) {m_errcode = Errcode;}
Static void* threadfunction (void*);
    Public:cthread ();
    Cthread (bool Createsuspended,bool detach);

    Virtual ~cthread ();
    virtual void Run (void) = 0;
    void Setthreadstate (ThreadState state) {m_threadstate = state;} BOOL Terminate (void); Terminate the threa bool Start (void);
    Start to execute the thread void Exit (void);
    BOOL Wakeup (void);
    ThreadState getthreadstate (void) {return m_threadstate;}
    int GetLastError (void) {return m_errcode;}
    void SetThreadName (char* thrname) {strcpy (m_threadname,thrname);} char* getthreadname (void) {return M_threaDname;}
    int GetThreadId (void) {return m_threadid;}
    BOOL SetPriority (int priority);
    int getpriority (void);
    int getconcurrency (void);
    void setconcurrency (int num);
    BOOL Detach (void);
    BOOL Join (void);
    BOOL Yield (void);
int Self (void); };

The state of a thread can be divided into four types, idle, busy, suspended, terminated (including normal exit and abnormal exit). Because the current Linux line threading does not support pending operations, our pending operation here is similar to a pause. If the thread does not want to perform the task immediately after it is created, then we can "pause" it and wake if it needs to be run. It is important to note that once a thread begins to perform a task, it cannot be suspended and will continue to perform the task to completion.
The related operation of the thread class is very simple. The thread's execution entry begins with the start () function, which calls the function threadfunction,threadfunction and then calls the actual run function to perform the actual task.

　　CThreadPool

CThreadPool is the bearer container of a thread, which can generally be implemented as a stack, a one-way queue, or a two-way queue. In our system we use STL vectors to save threads. The implementation code for the CThreadPool is as follows:

Class CThreadPool {friend class CWorkerThread; private:unsigned int m_maxnum;//the max thread num that can crea Te at the same time unsigned int m_availlow; The min num of idle thread that shoule kept unsigned int m_availhigh; The max num of idle thread that kept in the same time unsigned int m_availnum;
    The normal thread num of idle num; unsigned int m_initnum;
Normal thread num; 
    protected:cworkerthread* getidlethread (void);
    void Appendtoidlelist (cworkerthread* jobthread);
    void Movetobusylist (cworkerthread* idlethread);
    void Movetoidlelist (cworkerthread* busythread);
    void deleteidlethread (int num);
void createidlethread (int num); Public:cthreadmutex M_busymutex; When visit busy List,use M_busymutex to lock and unlock cthreadmutex; When visit idle List,use M_idlemutex to lock and unlock cthreadmutex;
   When visit job List,use M_jobmutex to lock and unlock Cthreadmutex M_varmutex; Ccondition M_busycond; M_busycond is used to sync busy thread list ccondition M_idlecond; M_idlecond is used to sync idle thread list ccondition M_idlejobcond;
    M_jobcond is used to sync job list ccondition M_maxnumcond;
    Vector<cworkerthread*> m_threadlist; Vector<cworkerthread*> m_busylist; Thread List vector<cworkerthread*> m_idlelist;
    Idle List CThreadPool ();
    CThreadPool (int initnum); 
    Virtual ~cthreadpool ();
    void Setmaxnum (int maxnum) {m_maxnum = Maxnum;}
    int Getmaxnum (void) {return m_maxnum;}
    void Setavaillownum (int minnum) {m_availlow = Minnum;}
    int Getavaillownum (void) {return m_availlow;}
    void Setavailhighnum (int highnum) {m_availhigh = Highnum;}
    int Getavailhighnum (void) {return m_availhigh;}
    int Getactualavailnum (void) {return m_availnum;}
    int Getallnum (void) {return m_threadlist.size ();}
    int Getbusynum (void) {return m_busylist.size ();} void Setinitnum (int initnum) {m_initnum = InitnUm;}
    int Getinitnum (void) {return m_initnum;}
    void Terminateall (void);
void Run (cjob* job,void* jobdata);

};
    Cthreadpool::cthreadpool () {m_maxnum = 50;
    M_availlow = 5; 
    M_initnum=m_availnum = 10;
    M_availhigh = 20;
    M_busylist.clear ();
    M_idlelist.clear ();
        for (int i=0;i<m_initnum;i++) {cworkerthread* thr = new CWorkerThread ();
        Thr->setthreadpool (this);
        Appendtoidlelist (THR);
    Thr->start ();
    } cthreadpool::cthreadpool (int initnum) {assert (initnum>0 && initnum<=30);
    M_maxnum = 30;
    M_availlow = initnum-10>0?initnum-10:3; 
    M_initnum=m_availnum = Initnum;
    M_availhigh = initnum+10;
    M_busylist.clear ();
    M_idlelist.clear ();
        for (int i=0;i<m_initnum;i++) {cworkerthread* thr = new CWorkerThread ();
        Appendtoidlelist (THR);
        Thr->setthreadpool (this); Thr->start (); Begin the Thread,the thread wait for job} CThreadPool:: ~cthreadpool () {Terminateall ();} void Cthreadpool::terminateall () {for (int i=0;i < m_threadlist.size (); i++) {cworkerthread* thr = M_thread
        List[i];
    Thr->join ();
} return;
    } cworkerthread* Cthreadpool::getidlethread (void) {while (M_idlelist.size () ==0) m_idlecond.wait ();
    M_idlemutex.lock ();
        if (m_idlelist.size () > 0) {cworkerthread* thr = (cworkerthread*) m_idlelist.front ();
        printf ("Get Idle thread%d/n", Thr->getthreadid ());
        M_idlemutex.unlock ();
    return thr;
    } m_idlemutex.unlock ();
return NULL; //add an idle thread to idle list void Cthreadpool::appendtoidlelist (cworkerthread* jobthread) {M_idlemutex.lock ()
    ;
    M_idlelist.push_back (Jobthread);
    M_threadlist.push_back (Jobthread);
M_idlemutex.unlock (); //move and idle thread to busy thread void Cthreadpool::movetobusylist (cworkerthread* idlethread) {M_busymutex.loc
    K (); M_busylist.push_back (Idlethread);
    m_availnum--;
    
    M_busymutex.unlock ();
    M_idlemutex.lock ();
    Vector<cworkerthread*>::iterator POS;
    pos = Find (M_idlelist.begin (), M_idlelist.end (), idlethread);
    if (pos!=m_idlelist.end ()) m_idlelist.erase (POS);
M_idlemutex.unlock ();
    } void Cthreadpool::movetoidlelist (cworkerthread* busythread) {m_idlemutex.lock ();
    M_idlelist.push_back (Busythread);
    m_availnum++;
    M_idlemutex.unlock ();
    M_busymutex.lock ();
    Vector<cworkerthread*>::iterator POS;
    pos = Find (M_busylist.begin (), M_busylist.end (), busythread);
    if (Pos!=m_busylist.end ()) m_busylist.erase (POS);
    M_busymutex.unlock ();
    M_idlecond.signal ();
M_maxnumcond.signal (); }//create num idle thread and put them to idlelist void cthreadpool::createidlethread (int num) {for (int i=0;i<nu
        m;i++) {cworkerthread* thr = new CWorkerThread ();
        Thr->setthreadpool (this);
        Appendtoidlelist (THR);
    M_varmutex.lock ();    m_availnum++;
        M_varmutex.unlock (); Thr->start (); Begin the Thread,the thread wait for job} void CThreadPool::D eleteidlethread (int num) {printf (' Enter into C
    ThreadPool::D eleteidlethread/n ");
    M_idlemutex.lock ();
    printf ("Delete num is%d/n", num);
        for (int i=0;i<num;i++) {cworkerthread* thr;
            if (m_idlelist.size () > 0) {thr = (cworkerthread*) m_idlelist.front ();
        printf ("Get Idle thread%d/n", Thr->getthreadid ());
        } vector<cworkerthread*>::iterator Pos;
        pos = Find (M_idlelist.begin (), M_idlelist.end (), THR);
        if (Pos!=m_idlelist.end ()) m_idlelist.erase (POS);
        m_availnum--;
        printf ("The Idle thread available num:%d/n", m_availnum);
    printf ("The Idlelist num:%d/n", m_idlelist.size ());
} m_idlemutex.unlock ();

    } void Cthreadpool::run (cjob* job,void* jobdata) {assert (job!=null); If the busy thread num adds to M_maxnum,so We should wait if (getbusynum () = = M_maxnum) m_maxnumcond.wait (); if (M_idlelist.size () <m_availlow) {if (Getallnum () +m_initnum-m_idlelist.size () < M_maxnum) Create
        Idlethread (M_initnum-m_idlelist.size ());
    Else Createidlethread (M_maxnum-getallnum ());
    } cworkerthread* Idlethr = Getidlethread ();
        if (Idlethr!=null) {idlethr->m_workmutex.lock ();
        Movetobusylist (IDLETHR);
        Idlethr->setthreadpool (this);
        Job->setworkthread (IDLETHR);
        printf ("Job is set to Thread%d/n", Idlethr->getthreadid ());
    Idlethr->setjob (Job,jobdata); }
}

There are two linked lists in the CThreadPool, one is an idle list, and one is a busy list. The idle list holds all the idle processes, and when the thread executes the task, its status becomes busy, removed from the free list, and moved to the busy list. In the CThreadPool constructor, we will execute the following code:

for (int i=0;i<m_initnum;i++) {
    cworkerthread* thr = new CWorkerThread ();
    Appendtoidlelist (THR);
    Thr->setthreadpool (this);
    Thr->start (); Begin the Thread,the thread wait for job
}

In this code, we'll create m_initnum threads that call appendtoidlelist into the idle list after they are created, and because there are no tasks currently being distributed to these threads, the thread suspends itself when it executes start.
In fact, the number of threads in the thread pool is not static, and it scales automatically based on the execution load. To do this, set four variables in CThreadPool:
M_initnum: The number of threads in the thread pool when you create a life.
M_maxnum: The maximum number of concurrent threads allowed to exist in the current thread pool.
M_availlow: The minimum number of idle threads allowed in the current thread pool, if the number of idle is below this value, indicating that the load may be too heavy, it is necessary to increase the number of free thread pools at this time. In the implementation we always adjust the thread to M_initnum.
M_availhigh: The maximum number of idle threads allowed in the current thread pool, if the number of idle is higher than this, indicating that the current load may be lighter, the extra idle threads are deleted, and the adjusted number is m_initnum after deletion.
M_availnum: The number of threads that actually exist in the current thread pool, with values between M_availhigh and M_availlow. If the number of threads is always maintained between M_availlow and M_availhigh, the thread does not need to be created or deleted to remain in a balanced state. So how to set the value of M_availlow and M_availhigh, so that the thread pool is the most likely to maintain a balanced state, is the thread pool design must consider the problem.
Thread pool After accepting the new task, the thread pool first checks to see if there are enough free pools available. The check is divided into three steps:
(1) checks whether the currently busy thread has reached the maximum set value m_maxnum, and if so, indicates that no free threads are currently available and that no new threads can be created, so you must wait until the thread is finished and return to the idle queue.
(2) If the current number of idle threads is less than the minimum number of idle M_availlow we have set, we must create a new thread, by default, the number of threads created should be m_initnum, so the number of threads created should be (current number of idle threads and m_ Initnum); However, there is a special case to consider, that is, the total number of existing threads plus the number of threads created may exceed m_maxnum, so we must treat the creation of threads differently.

if (Getallnum () +m_initnum-m_idlelist.size () < M_maxnum)
    Createidlethread (M_initnum-m_idlelist.size ());
else
    Createidlethread (M_maxnum-getallnum ());

If the total is not more than m_maxnum after creation, the created thread is M_initnum, and if it is exceeded, only (m_maxnum-the total number of current threads) is created.
(3) Call the Getidlethread method to find an idle thread. If there are currently no idle threads, suspend, or assign the task to the thread and move it into the busy queue.
When the thread finishes executing, it calls the Movetoidlelist method to move into the free list, which also calls the M_idlecond.signal () method, which wakes up threads that may be blocked in Getidlethread ().
　　Cjob
The Cjob class is relatively simple, encapsulating the basic properties and methods of the task, the most important of which is the Run method, which is as follows:

Class Cjob {
private:
    int m_jobno;//the num is assigned to the job
    char* m_jobname;//the job name
    Cthre Ad *m_pworkthread; The thread associated with the job public
:
    cjob (void);
    Virtual ~cjob ();