Multi-task Programming in linux-Thread Pool

In an embedded system environment, due to the characteristics of system resources and tasks, multithreading has become an important way to implement multi-task processing. in some common application environments, such as Web servers, Email servers, and database servers, there is one thing in common: many concurrent connection requests must be processed per unit of time, however, the processing time is relatively short. the server model we use in the traditional multi-threaded solution is to create a new thread to run the task once the request is received. after the task is executed, the thread exits. This is the policy of "instant creation and destruction. although the thread creation time has been greatly shortened compared with the creation process, if the task submitted to the thread is executed for a short period of time, and the execution times are frequent, the server will be in the state of creating and destroying threads continuously.
The thread pool is the most outstanding model that uses a multi-thread solution to improve system performance. It pre-creates a certain number of worker threads to concurrently process the system, this improves the system running efficiency. because the thread pool can make the system adopt lightweight and controllable system resources to maximize the system's concurrent processing capability, many applications use the thread pool model.
In addition, threads can limit the number of Created processes. generally, the concurrent threads allowed by the thread pool have an upper bound. If the number of concurrent threads needs to exceed the upper bound, some threads will wait. in the traditional scheme, if the request data is 200 at the same time, the system may need to generate 200 threads in the worst case. although this is not a large number, the system does not meet this requirement.
There are many thread pool models. There are three common models: the thread pool model controlled by the task queue, the thread pool model controlled by the working thread, and the thread pool model controlled by the main control thread. this article describes how to implement the thread pool model controlled by task queues.
The thread pool model controlled by the task queue is to schedule the thread pool concurrently through the task queue, as shown in. A thread pool is composed of a pre-created task queue and a group of working threads. The task queue stores work objects. after the thread pool is started, the worker thread obtains the task object from the task queue by polling. Because no task object exists in the task queue during initialization, the semaphore is 0, all worker threads are in the blocking status. the main control thread puts the task object in the task queue and adds the semaphore to 1. This way, the semaphore will wake up a blocked working thread (the operating system determines which blocked working thread to wake up ). after the worker thread wakes up, it obtains a task object from the task queue and executes the task. After the task is executed, the worker thread will access the semaphore again. If the semaphore is greater than 0, the worker thread will continue to obtain and execute the task object from the task queue, knowing that the semaphore is equal to 0, then the worker thread will be blocked again.

The thread model of task queue control is to control the thread scheduling in the thread pool through the signal in the task queue.

This routine consists of five files: tpool. h, tpool. c, log. c, log. h, testpool. c. where tpool. h, tpool. c is the core file that implements the thread pool. h defines the data structure of the thread pool and worker thread, and methods such as creating a thread pool and adding a worker thread. tpool. c is the specific implementation code. log. c, log. h provides a method to generate a record file and record custom information in the file. testpool. c is the test program of the thread pool.

This example consists of multiple source files. The compilation command is as follows:

The source code of this example is given below. The first is the definition file tpool. h of the thread pool:

/* ------------------------------------------------------------------------- * Tpool. h-thread pool definition * ------------------------------------------------------------------------- */# ifndef _ TPOOL_H _ # define _ TPOOL_H _ # include <stdio. h> # include <pthread. h>/* Working thread linked list */typedef struct tpool_work {void (* handler_routine) ();/* task function pointer */void * arg; /* task function parameters */struct tpool_work * next;/* next task linked list */} tpool_work_t;/* thread pool struct */typedef struct tpool {int num_threads; /* Maximum number of threads */int max_queue_size;/* Maximum number of task chain tables */int do_not_block_when_full;/* Whether to block when the linked list is full */pthread_t * threads; /* thread pointer */int cur_queue_size; tpool_work_t * queue_head;/* linked list header */tpool_work_t * queue_tail;/* linked list tail */pthread_mutex_t queue_lock; /* linked list mutex quantity */pthread_cond_t queue_not_full;/* linked list condition quantity-less than */pthread_cond_t queue_not_empty; /* linked list condition quantity-null */int queue_closed; int shutdown;} tpool_t;/* initialize the connection pool */extern tpool_t * tpool_init (int num_worker_threads, \ int max_queue_size, int limit ); /* Add a working thread */extern int tpool_add_work (tpool_t * pool, void (* routine) (), void * arg ); /* clear the thread pool */extern int tpool_destroy (tpool_t * pool, int finish); # endif/* _ TPOOL_H _*/

/* ------------------------------------------------------------------------- * Tpool. c-thread pool implementation * ----------------------------------------------------------------------------- */# include <stdio. h> # include <stdlib. h> # include <string. h> # include <pthread. h> # include "tpool. h "# include" log. h "/* worker thread */void * tpool_thread (void * tpool ); ********************* * *******/tpool_t * tpool_init (in T num_worker_threads, \/* Number of threads in the thread pool */int max_queue_size, \/* Maximum number of tasks */int do_not_block_when_full)/* Whether to block full tasks */{int I, rtn; tpool_t * pool; lprintf (log, INFO, "init pool begin... \ n ");/* Create thread pool struct */if (pool = (struct tpool *) malloc (sizeof (struct tpool) = NULL) {lprintf (log, FATAL, "Unable to malloc () thread pool! \ N "); return NULL;}/* set the thread pool Architecture Body Member */pool-> num_threads = num_worker_threads;/* Number of worker threads */pool-> max_queue_size = max_queue_size; /* maximum task linked list length */pool-> do_not_block_when_full = do_not_block_when_full; /* Wait for the task linked list to be full * // * generate the thread pool cache */if (pool-> threads = (pthread_t *) malloc (sizeof (pthread_t) * num_worker_threads )) = NULL) {lprintf (log, FATAL, "Unable to malloc () thread info array \ n"); return NULL;}/* initialize the task chain Table */pool-> cur_queue_size = 0; pool-> queue_head = NULL; pool-> queue_tail = NULL; pool-> queue_closed = 0; pool-> shutdown = 0; /* initialize the mutex variable. The condition variable is used for synchronization between threads */if (rtn = pthread_mutex_init (& (pool-> queue_lock), NULL ))! = 0) {lprintf (log, FATAL, "pthread_mutex_init % s", strerror (rtn); return NULL ;} if (rtn = pthread_cond_init (& (pool-> queue_not_empty), NULL ))! = 0) {lprintf (log, FATAL, "pthread_cond_init % s", strerror (rtn); return NULL ;} if (rtn = pthread_cond_init (& (pool-> queue_not_full), NULL ))! = 0) {lprintf (log, FATAL, "pthread_cond_init % s", strerror (rtn); return NULL ;} if (rtn = pthread_cond_init (& (pool-> queue_empty), NULL ))! = 0) {lprintf (log, FATAL, "pthread_cond_init % s", strerror (rtn); return NULL;}/* create all threads */for (I = 0; i! = Num_worker_threads; I ++) {if (rtn = pthread_create (& (pool-> threads [I]), NULL, tpool_thread, (void *) pool ))! = 0) {lprintf (log, FATAL, "pthread_create % s \ n", strerror (rtn); return NULL;} lprintf (log, INFO, "init pthread % d! \ N ", I);} lprintf (log, INFO," init pool end! \ N "); return pool ;}

The tpool_add_work function adds a worker thread to the thread pool. because the pre-created thread cannot do any work, the pre-created thread will actually work only after an appropriate task is assigned.

Int tpool_add_work (tpool_t * pool, \/* thread pool pointer */void (* routine) (void *), \/* worker thread function pointer */void * arg) /* worker thread function parameter */{int rtn; tpool_work_t * workp;/* Current worker thread */if (rtn = pthread_mutex_lock (& pool-> queue_lock ))! = 0) {lprintf (log, FATAL, "pthread mutex lock failure \ n"); return-1 ;} /* access the task linked list in an exclusive manner */if (pool-> cur_queue_size = pool-> max_queue_size) & \ (pool-> do_not_block_when_full )) {if (rtn = pthread_mutex_unlock (& pool-> queue_lock ))! = 0) {lprintf (log, FATAL, "pthread mutex lock failure \ n"); return-1 ;}return-1 ;} /* Wait for the task linked list to release space for the new thread */while (pool-> cur_queue_size = pool-> max_queue_size )&&(! (Pool-> shutdown | pool-> queue_closed) {if (rtn = pthread_cond_wait (& (pool-> queue_not_full), & (pool-> queue_lock )))! = 0) {lprintf (log, FATAL, "pthread cond wait failure \ n"); return-1 ;}} if (pool-> shutdown | pool-> queue_closed) {if (rtn = pthread_mutex_unlock (& pool-> queue_lock ))! = 0) {lprintf (log, FATAL, "pthread mutex lock failure \ n"); return-1 ;}return-1 ;} /* allocate the worker thread struct */if (workp = (tpool_work_t *) malloc (sizeof (tpool_work_t) = NULL) {lprintf (log, FATAL, "unable to create work struct \ n"); return-1;} workp-> handler_routine = routine; workp-> arg = arg; workp-> next = NULL; if (pool-> cur_queue_size = 0) {pool-> queue_tail = pool-> queue_head = workp; if (rtn = pthread _ Cond_broadcast (& (pool-> queue_not_empty )))! = 0) {lprintf (log, FATAL, "pthread broadcast error \ n"); return-1 ;}} else {pool-> queue_tail-> next = workp; pool-> queue_tail = workp;} pool-> cur_queue_size ++;/* release exclusive to the task linked list */if (rtn = pthread_mutex_unlock (& pool-> queue_lock ))! = 0) {lprintf (log, FATAL, "pthread mutex lock failure \ n"); return-1 ;}return 0 ;}

When the thread pool exits, you need to release the resources used, including the following five steps:

Int tpool_destroy (tpool_t * pool, int finish) {int I, rtn; tpool_work_t * cur;/* Current worker thread */lprintf (log, INFO, "destroy pool begin! \ N ");/* release exclusive to the task linked list */if (rtn = pthread_mutex_lock (& (pool-> queue_lock )))! = 0) {lprintf (log, FATAL, "pthread mutex lock failure \ n"); return-1 ;}/ * Step 1, set the thread exit flag */lprintf (log, INFO, "destroy pool begin 1! \ N "); if (pool-> queue_closed | pool-> shutdown) {if (rtn = pthread_mutex_unlock (& (pool-> queue_lock )))! = 0) {lprintf (log, FATAL, "pthread mutex lock failure \ n"); return-1 ;}return 0 ;}/ * Step 2, prohibit new tasks from joining the task linked list */lprintf (log, INFO, "destroy pool begin 2! \ N "); pool-> queue_closed = 1; if (finish) {while (pool-> cur_queue_size! = 0) {if (rtn = pthread_cond_wait (& (pool-> queue_empty), & (pool-> queue_lock )))! = 0) {lprintf (log, FATAL, "pthread_cond_wait % d \ n", rtn); return-1 ;}}/ * Step 3, set the thread pool destruction mark */lprintf (log, INFO, "destroy pool begin 3! \ N "); pool-> shutdown = 1; if (rtn = pthread_mutex_unlock (& (pool-> queue_lock )))! = 0) {lprintf (log, FATAL, "pthread mutex unlock failure \ n"); return-1 ;}/ * Step 4, wait for all created threads to exit */lprintf (log, INFO, "destroy pool begin 4! \ N "); if (rtn = pthread_cond_broadcast (& (pool-> queue_not_empty )))! = 0) {lprintf (log, FATAL, "pthread_cond_boradcast % d \ n", rtn); return-1 ;} if (rtn = pthread_cond_broadcast (& (pool-> queue_not_full )))! = 0) {lprintf (log, FATAL, "pthread_cond_boradcast % d \ n", rtn); return-1 ;}for (I = 0; I <pool-> num_threads; I ++) {if (rtn = pthread_join (pool-> threads [I], NULL ))! = 0) {lprintf (log, FATAL, "pthread_join % d \ n", rtn); return-1 ;}/ * Step 5, release the memory space occupied by the thread pool */free (pool-> threads); while (pool-> queue_head! = NULL) {cur = pool-> queue_head-> next; pool-> queue_head = pool-> queue_head-> next; free (cur);} free (pool ); lprintf (log, INFO, "destroy pool end! \ N "); return 0 ;}

The tpool_thread function defines the function of the worker thread. Only one line of code is related to the actual task:

(*(my_work->handler_routine))(my_work->arg); 

That is, execute the my_work-> handler_routine pointer pointing to the function, and input the my_work-> arg parameter. Other steps are various settings and preparations for executing this task.

Void * tpool_thread (void * tpool) {tpool_work_t * my_work; tpool_t * pool = (struct tpool *) tpool; (;;) {/* loop in the thread */pthread_mutex_lock (& (pool-> queue_lock);/* If the task list is 0 and the thread pool is not closed, the thread waits, until the task arrives */while (pool-> cur_queue_size = 0 )&&(! Pool-> shutdown) {pthread_cond_wait (& (pool-> queue_not_empty), & (pool-> queue_lock);}/* Whether the thread pool has been closed, if the thread pool is closed, the thread automatically closes */if (pool-> shutdown) {pthread_mutex_unlock (& (pool-> queue_lock); pthread_exit (NULL ); /* The thread exit status is empty. The main thread does not capture the status of each sub-thread */} my_work = pool-> queue_head; pool-> cur_queue_size --;/* removes the task linked list header, the modification task is being processed */if (pool-> cur_queue_size = 0) pool-> queue_head = pool-> queue_tail = NULL; else pool-> queue_hea D = my_work-> next;/* The task linked list is not full */if ((! Pool-> do_not_block_when_full) & \ (pool-> cur_queue_size = (pool-> max_queue_size-1) {pthread_cond_broadcast (& (pool-> queue_not_full ));} /* The task linked list is empty */if (pool-> cur_queue_size = 0) {pthread_cond_signal (& (pool-> queue_empty ));} pthread_mutex_unlock (& (pool-> queue_lock);/* start thread business processing logic */(* (my_work-> handler_routine) (my_work-> arg ); free (my_work);} return (NULL );}

In order to easily record the working status of the thread pool, a record server is also implemented. the record server implements a syslog-like log function. There are three functions in total, which are very simple to use. log. h is the header file of the record server.

/* ------------------------------------------------------------------------- * Log. h record function definition * ------------------------------------------------------------------------- */# ifndef _ LOG_H # define _ LOG_H # include <stdio. h> # include <semaphore. h>/* maximum record length */# define LOGLINE_MAX 1024/* record level */# define DEBUG 1 # define INFO 2 # define WARN 3 # define ERROR 4 # define FATAL 5/* record type */# define LOG_TRUNC 1 <0 # define LOG_N ODATE 1 <1 # define LOG_NOLF 1 <2 # define LOG_NOLVL 1 <3 # define LOG_DEBUG 1 <4 # define LOG_STDERR 1 <5 # define LOG_NOTID 1 <6 typedef struct {int fd; sem_t sem; int flags;} log_t;/** Function Description: Record printing function, which prints records to the logfile of the record file. * Parameter: Return Value of the log_t-log_open () function * level-can be: DEBUG, INFO, WARN, ERROR, FATAL * fmt-recorded content, in the same format as printf () function * return value: 0 is returned for success, and-1 */int lprintf (log_t * log, unsigned int level, char * fmt,...) is returned for failure ,...); /** Function Description: initialization record file the logfile * parameter: fname-record file logfile file name * flags-record format option * LOG_TRUNC-truncated open record file * LOG_NODATE-ignore each row in the record * LOG_NOLF-automatically opens a new row for each record. * LOG_NOLVL-the level of the message is not recorded * LOG_STDERR-send the message to STDERR at the same time * returned value: log_t (> 0) is returned successfully, and NULL */log_t * log_open (char * fname, int flags);/** Function Description: Close the record file * parameter: * log-record file pointer */void log_close (log_t * log); # endif

The record server has no direct relationship with the thread pool. If necessary, you can easily migrate the record server to your project.

The lprintf function is used to print records. The log Pointer Points to a record file opened through the log_open function. All record information will be saved in this file. the level parameter is used to specify the Record Content category. six categories are defined in h, namely DEBUG, INFO, WARN, ERROR, and FATAL, to facilitate the classification of a large number of records.

/* ------------------------------------------------------------------------- * Log. c-record function implementation * ------------------------------------------------------------------------- */# include <stdio. h> # include <unistd. h> # include <semaphore. h> # include <stdarg. h> # include <stdlib. h> # include <string. h> # include <errno. h> # include <time. h> # include <pthread. h> # include <sys/types. h> # include <sys/stat. h> # include <fcntl. H> # include "log. h "int lprintf (log_t * log, unsigned int level, char * fmt ,...) {int fd; int rc; va_list ap; time_t now; char date [50]; static char line [LOGLINE_MAX]; static char threadnum [10]; int cnt; static char * levels [6] = {"[(bad)]", "[debug]", "[info]", "[warn]", "[error]", "[fatal]"}; if (! Log) return-1; if (! (Log-> flags & LOG_DEBUG) & level = DEBUG) return 0; fd = log-> fd;/* date */if (! (Log-> flags & LOG_NODATE) {now = time (NULL); strcpy (date, ctime (& now); date [strlen (date) -6] = ''; date [strlen (date)-5] = '\ 0';}/* thread Number */if (! (Log-> flags & LOG_NOTID) {sprintf (threadnum, "(TID: % lu)", pthread_self ();} cnt = snprintf (line, sizeof (line ), "% s", log-> flags & LOG_NODATE? "": Date, log-> flags & LOG_NOLVL? "": (Level> FATAL? Levels [0]: levels [level]), log-> flags & LOG_NOTID? "": Threadnum); va_start (ap, fmt); vsnprintf (line + cnt, sizeof (line)-cnt, fmt, ap ); /* if the input log is too long, it will automatically intercept */va_end (ap); line [sizeof (line)-1] = '\ 0'; if (! (Log-> flags & LOG_NOLF) {/* chomp (line); * // * strcpy (line + strlen (line), "\ n "); */} sem_wait (& log-> sem);/* use signals for synchronization */rc = write (fd, line, strlen (line )); if (log-> flags & LOG_STDERR) write (2, line, strlen (line); sem_post (& log-> sem); if (! Rc) errno = 0; return rc ;}

log_t *log_open( char *fname, int flags )    {    log_t *log = malloc(sizeof(log_t));    if(!log)     {        fprintf(stderr, "log_open: Unable to malloc()");        goto log_open_a;    }    log->flags=flags;    if( !strcmp(fname,"-") )     {        log->fd = 2;    }    else    {        log->fd = open(fname, O_WRONLY|O_CREAT|O_NOCTTY |                 (flags&LOG_TRUNC ? O_TRUNC : O_APPEND) , 0666);    }    if( log->fd == -1 )    {        fprintf(stderr, "log_open: Opening logfile %s: %s", fname, strerror(errno));        goto log_open_b;    }    if( sem_init(&log->sem, 0, 1) == -1 )    {        fprintf(stderr, "log_open: Could not initialize log semaphore.");        goto log_open_c;    }    return log;log_open_c:    close(log->fd);log_open_b:    free(log);log_open_a:    return NULL;}

The log_close function is used to close an open record file. This function is usually executed when the function exits to ensure that all records are correctly written to the record file.

void log_close( log_t *log ) {    sem_wait(&log->sem);    sem_destroy(&log->sem);    close(log->fd);    free(log);    return;}

/* ------------------------------------------------------------------------- * Testpool. c-thread pool Test Program * --------------------------------------------------------------------------- */# include <pthread. h> # include "log. h "# include" tpool. h "log_t * log;/* process global log file handle */* task */void thread (void * arg) {char * ptr = (char *) arg; sleep (1); printf ("hello world! % S \ n ", ptr);} int main (int argc, char * argv []) {tpool_t * pool; /* thread pool pointer * // * Open the record file */log = log_open ("test. log ", 0);/* Create a thread pool with 100 working threads and a maximum of 200 task queues */pool = tpool_init (100,200, 1); int I; /* Open the record file */* Add 100 tasks */for (I = 0; I <100; I ++) tpool_add_work (pool, thread, "test! "); Sleep (10);/* terminate thread pool */tpool_destroy (pool, 1);/* close record file */log_close (log); pthread_exit (NULL );}

This example demonstrates how to create a thread pool and how to record the running state of the program. It has some usage significance and can be applied to the actual project with a slight modification. I can understand the design ideas and skills and greatly improve my programming skills.