Design a simple thread pool in Linux

Definition

What is a thread pool? To put it simply, the thread pool is a bunch of threads that have already been created. Initially, they are all in the idle waiting state. When a new task needs to be processed, take an idle waiting thread from this pool to process the task. When the processing is complete, the thread is put back into the pool for later tasks. When all threads in the pool are busy, there are no available idle waiting threads in the thread pool. In this case, create a new thread as needed and place it in the pool, or notify the task thread pool to be busy. Try again later.

 

Why use a thread pool?

We say that the creation and destruction of threads is lightweight than the creation and destruction of processes, but when our tasks require a large number of thread creation and destruction operations, this consumption will become quite large. For example, when you design a stress Performance Testing Framework, You need to generate a large number of concurrent operations continuously. This is the time when the thread pool can help you well.The advantage of thread pool is thread reuse.After a task is processed, the current thread can directly process the next task instead of creating it after destruction,It is very suitable for Continuous Generation of a large number of concurrent tasks.

 

Working principle of Thread Pool

The working process of each thread in the thread pool is as follows:

Figure 1: thread Workflow

The task of the thread pool is responsible for the creation, destruction, and transmission, wakeup, and waiting of these threads and task processing parameters.

1. Create several threads and place them in the thread pool.

2. When the task reaches, take the idle thread from the thread pool

3. Get the idle thread and process the task immediately

4. Otherwise, create a new thread, place it into the thread pool, and execute 3.

5. If creation fails or the thread pool is full, choose to return an error based on the design policy or place the task in the processing queue for processing.

6. Destroy the thread pool

 

Figure 2: How the thread pool works

 

Thread Pool Design Data Structure Design Task Design

typedef struct tp_work_desc_s TpWorkDesc;typedef void (*process_job)(TpWorkDesc*job);struct tp_work_desc_s {         void *ret; //call in, that is arguments         void *arg; //call out, that is return value};

Where, tpworkdesc is the task parameter description, Arg is the parameter passed to the task, and RET is the return value after the task is processed;

Process_job function is a prototype of the task processing function. Each task processing function should be defined as a parameter and then passed to the thread pool for processing, the thread pool selects an idle thread to call this function for task processing;

 

Thread Design

typedef struct tp_thread_info_s TpThreadInfo;struct tp_thread_info_s {         pthread_t thread_id; //thread id num         TPBOOL is_busy; //thread status:true-busy;flase-idle         pthread_cond_t thread_cond;         pthread_mutex_t thread_lock;         process_job proc_fun;         TpWorkDesc* th_job;         TpThreadPool* tp_pool;};

Tpthreadinfo is a description of a thread.

Thread_id is the ID of the thread;

Is_busy indicates whether the thread is busy;

Thread_cond is used to wake up and wait for task processing;

Thread_lock, used for task locking, used for conditional variables waiting for locking;

Proc_fun is the callback function address of the current task;

Th_job is the parameter information of the task;

The tp_pool is the pointer of the thread pool;

 

Thread Pool Design

typedef struct tp_thread_pool_s TpThreadPool;struct tp_thread_pool_s {         unsigned min_th_num; //min thread number in the pool         unsigned cur_th_num; //current thread number in the pool         unsigned max_th_num; //max thread number in the pool         pthread_mutex_t tp_lock;         pthread_t manage_thread_id; //manage thread id num         TpThreadInfo* thread_info;         Queue idle_q;         TPBOOL stop_flag;};

The tpthreadpool is a description of the thread pool.

Min_th_num is the minimum number of threads in the thread pool. During thread pool initialization, min_th_num threads are created;

Cur_th_num indicates the number of existing threads in the thread pool;

Max_th_num indicates the maximum number of threads that can exist in the thread pool;

Tp_lock is used for mutual exclusion during thread pool management;

Manage_thread_id is the management thread ID of the thread pool;

Thread_info points to the thread pool data. Here, an array is used to store the thread information in the thread pool. The size of the array is max_th_num;

Idle_q is a queue that stores idle thread pointers in the thread pool and is used to quickly obtain Idle threads from the thread pool;

Stop_flag is used to destroy the thread pool. When stop_flag is false, it indicates that the current thread pool needs to be destroyed. All busy threads exit after processing the current task;

Algorithm Design thread pool creation and initialization thread Creation

At the beginning of creation, the thread pool thread capacity ceiling is max_th_num, and the initial capacity is min_th_num;

TpThreadPool *tp_create(unsigned min_num, unsigned max_num) {TpThreadPool *pTp;pTp = (TpThreadPool*) malloc(sizeof(TpThreadPool));memset(pTp, 0, sizeof(TpThreadPool));//init member varpTp->min_th_num = min_num;pTp->cur_th_num = min_num;pTp->max_th_num = max_num;pthread_mutex_init(&pTp->tp_lock, NULL);//malloc mem for num thread info structif (NULL != pTp->thread_info)free(pTp->thread_info);pTp->thread_info = (TpThreadInfo*) malloc(sizeof(TpThreadInfo) * pTp->max_th_num);memset(pTp->thread_info, 0, sizeof(TpThreadInfo) * pTp->max_th_num);return pTp;}

Thread Initialization

TPBOOL tp_init(TpThreadPool *pTp) {int i;int err;TpThreadInfo *pThi;initQueue(&pTp->idle_q);pTp->stop_flag = FALSE;//create work thread and init work thread infofor (i = 0; i < pTp->min_th_num; i++) {pThi = pTp->thread_info +i;pThi->tp_pool = pTp;pThi->is_busy = FALSE;pthread_cond_init(&pThi->thread_cond, NULL);pthread_mutex_init(&pThi->thread_lock, NULL);pThi->proc_fun = def_proc_fun;pThi->th_job = NULL;enQueue(&pTp->idle_q, pThi);err = pthread_create(&pThi->thread_id, NULL, tp_work_thread, pThi);if (0 != err) {perror("tp_init: create work thread failed.");clearQueue(&pTp->idle_q);return FALSE;}}//create manage threaderr = pthread_create(&pTp->manage_thread_id, NULL, tp_manage_thread, pTp);if (0 != err) {clearQueue(&pTp->idle_q);printf("tp_init: creat manage thread failed\n");return FALSE;}return TRUE;}

The number of threads in the initial thread pool is min_th_num, Which is initialized one by one;

Put the initialized Idle threads into the idle queue one by one;

Create a management thread to monitor the status of the thread pool and reclaim excessive thread resources;

 

Thread Pool closure and destruction

void tp_close(TpThreadPool *pTp, TPBOOL wait) {unsigned i;pTp->stop_flag = TRUE;if (wait) {for (i = 0; i < pTp->cur_th_num; i++) {pthread_cond_signal(&pTp->thread_info[i].thread_cond);}for (i = 0; i < pTp->cur_th_num; i++) {pthread_join(pTp->thread_info[i].thread_id, NULL);pthread_mutex_destroy(&pTp->thread_info[i].thread_lock);pthread_cond_destroy(&pTp->thread_info[i].thread_cond);}} else {//close work threadfor (i = 0; i < pTp->cur_th_num; i++) {kill((pid_t)pTp->thread_info[i].thread_id, SIGKILL);pthread_mutex_destroy(&pTp->thread_info[i].thread_lock);pthread_cond_destroy(&pTp->thread_info[i].thread_cond);}}//close manage threadkill((pid_t)pTp->manage_thread_id, SIGKILL);pthread_mutex_destroy(&pTp->tp_lock);//free thread structfree(pTp->thread_info);pTp->thread_info = NULL;}

When the thread pool is closed, you can choose whether to wait for the tasks being processed. If yes, it will wake up all tasks, wait until all tasks are completed, and then return. If not, it will immediately kill all threads and then return,Note: This may cause an error when the task is interrupted!

 

Task Processing

TPBOOL tp_process_job(TpThreadPool *pTp, process_job proc_fun, TpWorkDesc *job) {TpThreadInfo *pThi ;//fill pTp->thread_info's relative work keypthread_mutex_lock(&pTp->tp_lock);pThi = (TpThreadInfo *) deQueue(&pTp->idle_q);pthread_mutex_unlock(&pTp->tp_lock);if(pThi){pThi->is_busy =TRUE;pThi->proc_fun = proc_fun;pThi->th_job = job;pthread_cond_signal(&pThi->thread_cond);DEBUG("Fetch a thread from pool.\n");return TRUE;}//if all current thread are busy, new thread is created herepthread_mutex_lock(&pTp->tp_lock);pThi = tp_add_thread(pTp);pthread_mutex_unlock(&pTp->tp_lock);if(!pThi){DEBUG("The thread pool is full, no more thread available.\n");return FALSE;}DEBUG("No more idle thread, created a new one.\n");pThi->proc_fun = proc_fun;pThi->th_job = job;//send cond to work threadpthread_cond_signal(&pThi->thread_cond);return TRUE;}

When a new task arrives, the thread pool first checks whether there are available Idle threads. If yes, the thread pool uses the idle thread to process the task and returns true. If not, create a new thread and use it to process the task. If the thread fails, false is returned, indicating that the thread pool is busy or has an error.

static void *tp_work_thread(void *arg) {pthread_t curid;//current thread idTpThreadInfo *pTinfo = (TpThreadInfo *) arg;//wait cond for processing real job.while (!(pTinfo->tp_pool->stop_flag)) {pthread_mutex_lock(&pTinfo->thread_lock);pthread_cond_wait(&pTinfo->thread_cond, &pTinfo->thread_lock);pthread_mutex_unlock(&pTinfo->thread_lock);//processpTinfo->proc_fun(pTinfo->th_job);//thread state be set idle after work//pthread_mutex_lock(&pTinfo->thread_lock);pTinfo->is_busy = FALSE;enQueue(&pTinfo->tp_pool->idle_q, pTinfo);//pthread_mutex_unlock(&pTinfo->thread_lock);DEBUG("Job done, I am idle now.\n");}}

The above function is a task processing function, which will always process and wait for the wake-up status until the new task arrives or the thread is destroyed, and then call the task processing callback function to process the task; when the task processing is complete, you are placed in the idle queue for processing by the next task.

TpThreadInfo *tp_add_thread(TpThreadPool *pTp) {int err;TpThreadInfo *new_thread;if (pTp->max_th_num <= pTp->cur_th_num)return NULL;//malloc new thread info structnew_thread = pTp->thread_info + pTp->cur_th_num; new_thread->tp_pool = pTp;//init new thread's cond & mutexpthread_cond_init(&new_thread->thread_cond, NULL);pthread_mutex_init(&new_thread->thread_lock, NULL);//init status is busy, only new process job will call this functionnew_thread->is_busy = TRUE;err = pthread_create(&new_thread->thread_id, NULL, tp_work_thread, new_thread);if (0 != err) {free(new_thread);return NULL;}//add current thread number in the pool.pTp->cur_th_num++;return new_thread;}

The above function is used to add a new thread to the thread pool. This function will be called when there are no Idle threads in the thread pool.

The function will create a new thread and set its status to busy (it will be used to execute the task immediately ).

Thread Pool Management

The management of the thread pool is mainly to monitor the overall busy state of the thread pool. When most threads in the thread pool are idle, the management thread will destroy a certain number of Idle threads as appropriate, this reduces the consumption of system resources by the thread pool.

 

The design here holds that when the number of Idle threads exceeds 1/2 of the number of threads in the thread pool, the thread pool will be in idle state and some threads in the thread pool can be destroyed as appropriate, to reduce the overhead of the thread pool on system resources.

 

Thread Pool status Calculation

Here, the value of busy_threshold is 0.5, that is, when the number of Idle threads exceeds half, 0 is returned, indicating that the overall state of the thread pool is idle. Otherwise, 1 is returned, indicating that the thread pool is busy.

int tp_get_tp_status(TpThreadPool *pTp) {float busy_num = 0.0;int i;//get busy thread numberbusy_num = pTp->cur_th_num - pTp->idle_q.count;DEBUG("Current thread pool status, current num: %u, busy num: %u, idle num: %u\n", pTp->cur_th_num, (unsigned)busy_num, pTp->idle_q.count);//0.2? or other num?if (busy_num / (pTp->cur_th_num) < BUSY_THRESHOLD)return 0;//idle statuselsereturn 1;//busy or normal status}

Thread destruction Algorithm

1. dequeue an idle thread pointer from the idle queue. This pointer points to an item in the thread information array. For example, here is P;

2. Destroy this thread

3. Copy the last entry of the thread information array to the position P.

4. Reduce the number of thread pools by one, namely, cur_th_num --

Figure 3: thread destruction

 

TPBOOL tp_delete_thread(TpThreadPool *pTp) {unsigned idx;TpThreadInfo *pThi;TpThreadInfo tT;//current thread num can't < min thread numif (pTp->cur_th_num <= pTp->min_th_num)return FALSE;//pthread_mutex_lock(&pTp->tp_lock);pThi = deQueue(&pTp->idle_q);//pthread_mutex_unlock(&pTp->tp_lock);if(!pThi)  return FALSE;//after deleting idle thread, current thread num -1pTp->cur_th_num--;memcpy(&tT, pThi, sizeof(TpThreadInfo));memcpy(pThi, pTp->thread_info + pTp->cur_th_num, sizeof(TpThreadInfo));//kill the idle thread and free info structkill((pid_t)tT.thread_id, SIGKILL);pthread_mutex_destroy(&tT.thread_lock);pthread_cond_destroy(&tT.thread_cond);return TRUE;}

Thread pool monitoring

The thread pool is monitored by a Management thread. The management thread calculates the status of the thread pool at intervals and destroys some threads according to the status of the thread pool, reduce the consumption of system resources.

 

static void *tp_manage_thread(void *arg) {TpThreadPool *pTp = (TpThreadPool*) arg;//main thread pool struct instance//1?sleep(MANAGE_INTERVAL);do {if (tp_get_tp_status(pTp) == 0) {do {if (!tp_delete_thread(pTp))break;} while (TRUE);}//end for if//1?sleep(MANAGE_INTERVAL);} while (!pTp->stop_flag);return NULL;}

Program testing

So far, our design requires a test program for verification. So we write down such a piece of code.

#include <stdio.h>#include <unistd.h>#include "thread_pool.h"#define THD_NUM 10 void proc_fun(TpWorkDesc *job){int i;int idx=*(int *)job->arg;printf("Begin: thread %d\n", idx);sleep(3);printf("End:   thread %d\n", idx);}int main(int argc, char **argv){TpThreadPool *pTp= tp_create(5,10);TpWorkDesc pWd[THD_NUM];int i, *idx;tp_init(pTp);for(i=0; i < THD_NUM; i++){idx=(int *) malloc(sizeof(int));*idx=i;pWd[i].arg=idx;tp_process_job(pTp, proc_fun, pWd+i);usleep(400000);}//sleep(1);tp_close(pTp, TRUE);free(pTp);printf("All jobs done!\n");return 0;}

Execution result:

 

Source code download

Address: https://sourceforge.net/projects/thd-pool-linux/

Remarks

The thread pool design is relatively simple and there are still many bugs. You are welcome to give suggestions for improvement.

 

Fixed:

2011/08/04:

The tp_close function adds the queue clearing operation. See the source code comments section.

Void tp_close (tpthreadpool * PTP, tpbool wait) {unsigned I; PTP-> stop_flag = true; If (wait) {for (I = 0; I <PTP-> cur_th_num; I ++) {pthread_cond_signal (& PTP-> thread_info [I]. thread_cond) ;}for (I = 0; I <PTP-> cur_th_num; I ++) {pthread_join (PTP-> thread_info [I]. thread_id, null); pthread_mutex_destroy (& PTP-> thread_info [I]. thread_lock); pthread_cond_destroy (& PTP-> thread_info [I]. thread_cond) ;}} else {// close work threadfor (I = 0; I <PTP-> cur_th_num; I ++) {kill (pid_t) PTP-> thread_info [I]. thread_id, sigkill); pthread_mutex_destroy (& PTP-> thread_info [I]. thread_lock); pthread_cond_destroy (& PTP-> thread_info [I]. thread_cond) ;}}// close manage threadkill (pid_t) PTP-> manage_thread_id, sigkill); abort (& PTP-> tp_lock); clearqueue (& PTP-> idle_q ); /** Add the queue here to clear ** // free thread structfree (PTP-> thread_info); PTP-> thread_info = NULL ;}

The preceding operation will cause a segment error because the queue deletes the Element Free. In fact, our element is the address of an element in the array, which will lead to a segment error. The element release function callback is added to the queue section in the source code. If this function is set to null or empty (nothing is done), no free operation will be performed when the element is deleted. For the complete source code, please download it from the address above.

When initializing a thread pool, you must set the element release function to null. For details, see the source code comments section.

Tpbool tp_init (tpthreadpool * PTP) {int I; int err; tpthreadinfo * pthi; initqueue (& PTP-> idle_q, null ); /** set the element release function to null during initialization **/PTP-> stop_flag = false; // create work thread and init work thread infofor (I = 0; I <PTP-> min_th_num; I ++) {pthi = PTP-> thread_info + I; pthi-> tp_pool = PTP; pthi-> is_busy = false; pthread_cond_init (& pthi-> thread_cond, null); pthread_mutex_init (& pthi-> thread_lock, null); pthi-> proc _ Fun = def_proc_fun; pthi-> th_job = NULL; enqueue (& PTP-> idle_q, pthi); err = pthread_create (& pthi-> thread_id, null, tp_work_thread, pthi ); if (0! = ERR) {perror ("tp_init: create work thread failed. "); clearqueue (& PTP-> idle_q); Return false; }}// create manage threaderr = pthread_create (& PTP-> manage_thread_id, null, tp_manage_thread, PTP ); if (0! = ERR) {clearqueue (& PTP-> idle_q); printf ("tp_init: creat manage thread failed \ n"); Return false;} return true ;}

Some source code of the queue header file is attached here:

#ifndef __QUEUE_H_#define __QUEUE_H_#include <pthread.h>typedef struct sNode QNode;typedef struct queueLK Queue;typedef void * EType;typedef void (*free_data_fun)(void *data);struct sNode {EType * data;struct sNode *next;};struct queueLK {struct sNode *front;struct sNode *rear;free_data_fun free_fun;unsigned count;pthread_mutex_t lock;};void initQueue(Queue *hq, free_data_fun pff);int enQueue(Queue *hq, EType x);EType deQueue(Queue *hq);EType peekQueue(Queue *hq);int isEmptyQueue(Queue *hq);void clearQueue(Queue *hq);