A Linux under the C-thread pool implementation (turn) __linux

1. Thread Pool Fundamentals

In a traditional server architecture, there is often a total listening thread listening for new users to connect to the server, whenever a new user enters, the server opens a new thread user to process the user's packet. This thread only serves this user, and when the user closes the connection with the server, the server destroys the thread. However, the frequent opening up and destruction of threads has greatly occupied the resources of the system. And in the case of a large number of users, the system will waste a lot of time and resources in order to open up and destroy threads. The thread pool provides a solution to the contradiction between the large number of external users and the limited resources of the server. Thread pool and a traditional user corresponding to a thread of the processing method is different, its basic idea is in the beginning of the program in memory to open some threads, the number of threads is fixed, they form a separate class, shielding the external operation, The server simply has to give the packet to the thread pool. When a new customer request arrives, instead of creating a new thread to serve it, instead, select an idle thread from the pool to request a service for the new customer, and after the service has finished, the thread enters the free thread pool. If no thread is idle, the packet is temporarily accumulated, waiting for the thread pool to be idle before processing. The overhead of creating and destroying thread objects is reduced by reusing existing thread objects for multiple tasks. When the client requests, the thread object already exists and can increase the response time of the request, thus improving the performance of the system service as a whole.

In general, implementing a thread pool consists mainly of the following components:

1 Thread Manager: Used to create and manage thread pools.

2 worker thread: The thread in the thread pool that actually executes the task. A fixed number of threads are created beforehand in the pool when the thread is initialized, and these initialized threads are generally idle and generally do not occupy the CPU and take up less memory space.

3 Task interface: Each task must implement the interface, when the thread pool in the task queue, there are executable tasks, the idle work thread is tuned to execute (thread's idle and busy is achieved by mutual exclusion, similar to the setting flag in the previous article), abstraction of the task to form an interface, you can do the thread pool and the specific task independent.

4 Task queue: To store the tasks that are not handled, provide a buffer mechanism to achieve this structure there are several methods, commonly used in the queue, the main use of advanced first out principle, the other is a linked list of data structure, you can dynamically allocate memory space for it, the application is more flexible, the following is used in the linked list.

The following is not to repeat the Baidu "thread pool technology in the application of concurrent server" wrote very detailed.

Turn from: http://blog.csdn.net/zouxinfox/article/details/3560891

When do I need to create a thread pool? Simply put, if an application needs to create and destroy threads frequently, and the task takes a very short time to execute, the overhead of creating and destroying threads should not be overlooked, and this is the chance for the thread pool to appear. If the thread creation and destruction times are negligible compared to the task execution time, there is no need to use the thread pool.

The following is a thread pool created by the C language in the Linux system. The thread pool maintains a task list (each cthread_worker structure is a task).
The Pool_init () function creates a max_thread_num thread in advance, and each thread holds the Thread_routine () function. In this function
while (pool->cur_queue_size = = 0) {pthread_cond_wait (& (Pool->queue_ready),& (Pool->queue_lock)); Indicates that if there are no tasks in the task list, the thread is blocking the wait state. Otherwise, the task is removed from the queue and executed.

The Pool_add_worker () function adds a task to the thread pool's task list, and then wakes up a blocking-state thread by calling Pthread_cond_signal (& Pool->queue_ready) (If any )。

The Pool_destroy () function is used to destroy the thread pool, and the tasks in the thread pool task list are not executed again, but the running thread will always run out of the task.

#include <stdio.h> #include <stdlib.h> #include <unistd.h> #include <sys/types.h> #include < pthread.h> #include <assert.h>/* * * All tasks running and waiting in the thread pool are a cthread_worker * because all tasks are in the chain, so it is a linked list structure/typedef struct
    Worker {/* callback function, which is invoked when the task is run, and may also be declared as other forms/void * (*process) (void *arg);

The parameters of void *arg;/* callback function * * struct worker *next;



} Cthread_worker;
    /* Thread pool structure */typedef struct {pthread_mutex_t queue_lock;

    pthread_cond_t Queue_ready;

    /* Linked list structure, all waiting task in thread pool/cthread_worker *queue_head;
    /* Whether to destroy the thread pool */int shutdown;
    pthread_t *threadid;
    * * Number of active threads allowed in the thread pool/int max_thread_num;

/* The number of tasks currently waiting for queues */int cur_queue_size;



} Cthread_pool;
int Pool_add_worker (void * (*process) (void *arg), void *arg);


void *thread_routine (void *arg);
Share resource static Cthread_pool *pool = NULL;

    void Pool_init (int max_thread_num) {pool = (Cthread_pool *) malloc (sizeof (Cthread_pool)); Pthread_mutex_init (& (Pool->queUe_lock), NULL);

    Pthread_cond_init (& (Pool->queue_ready), NULL);

    Pool->queue_head = NULL;
    Pool->max_thread_num = Max_thread_num;

    pool->cur_queue_size = 0;

    Pool->shutdown = 0;
    Pool->threadid = (pthread_t *) malloc (max_thread_num * sizeof (pthread_t));
    int i = 0; for (i = 0; i < Max_thread_num i++) {pthread_create (& (Pool->threadid[i)), NULL, Thread_routine,n
    ULL); }/* Add task to thread pool/int Pool_add_worker (void * (*process) (void *arg), void *arg) {/* to construct a new task * * * Cthread_worker *n
    Ewworker = (Cthread_worker *) malloc (sizeof (Cthread_worker));
    newworker->process = process;
    Newworker->arg = arg;
    Newworker->next = null;/* Don't forget to Empty/Pthread_mutex_lock (& (Pool->queue_lock));
    /* Add the task to the wait queue/* Cthread_worker *member = pool->queue_head;
        if (member!= null) {while (Member->next!= null) member = member->next; Member->next= Newworker;
    else {pool->queue_head = Newworker;

    ASSERT (Pool->queue_head!= NULL);
    pool->cur_queue_size++;
    Pthread_mutex_unlock (& (Pool->queue_lock));
    /* Well, wait for the queue to have a task, wake up a waiting thread; Note If all threads are busy, this sentence has no effect/pthread_cond_signal (& (Pool->queue_ready));
return 0; /* Destroy the thread pool and wait for the task in the queue to no longer be executed, but the running thread will continue to run the task and then exit the/int Pool_destroy () {if (Pool->shutdown) return-1;/

    * Prevent two calls * * Pool->shutdown = 1;

    /* Wake up all waiting threads, thread pool to destroy/Pthread_cond_broadcast (& (Pool->queue_ready));
    /* Blocking wait for the thread to exit, otherwise it will become a zombie/int i;
    for (i = 0; i < pool->max_thread_num; i++) Pthread_join (Pool->threadid[i], NULL);

    Free (pool->threadid);
    /* Destroy wait Queue * * Cthread_worker *head = NULL;
        while (Pool->queue_head!= NULL) {head = pool->queue_head;
        Pool->queue_head = pool->queue_head->next;
    Free (head); }/* Condition variable and mutex also don't forget to destroy */PTHREad_mutex_destroy (& (Pool->queue_lock));
    
    Pthread_cond_destroy (& (Pool->queue_ready));
    Free (pool);
    /* Destroy the pointer after the empty is a good habit of * * pool=null;
return 0;
    } void * Thread_routine (void *arg) {printf ("Starting thread 0x%x\n", pthread_self ());
        while (1) {Pthread_mutex_lock (& (Pool->queue_lock)); /* If the wait queue is 0 and the thread pool is not destroyed, it is in a blocking state; Note that Pthread_cond_wait is an atomic operation that will unlock before waiting, and will be unlocked after awakening (pool->cur_queue_size = 0 &&!pool->shutd
            Own) {printf ("Thread 0x%x is waiting\n", pthread_self ());
        Pthread_cond_wait (& (Pool->queue_ready), & (Pool->queue_lock));
            }/* thread pool to destroy */if (Pool->shutdown) {/* encountered Break,continue,return and other jump statements, do not forget to unlock the first * * *
            Pthread_mutex_unlock (& (Pool->queue_lock));
            printf ("Thread 0x%x'll exit\n", pthread_self ());
        Pthread_exit (NULL); printf ("Thread 0X%x is starting to work\n ", pthread_self ());
        /*assert is a good helper for debugging/assert (pool->cur_queue_size!= 0);
        
        ASSERT (Pool->queue_head!= NULL);
        /* Wait Queue Length minus 1 and remove the header element from the list. * * pool->cur_queue_size--;
        Cthread_worker *worker = pool->queue_head;
        Pool->queue_head = worker->next;

        Pthread_mutex_unlock (& (Pool->queue_lock));
        /* Call callback function, execute task//(* (worker->process)) (WORKER-&GT;ARG);
        Free (worker);
    worker = NULL;
/* This sentence should be not up to/pthread_exit (NULL); //Below is test code void * myprocess (void *arg) {printf ("ThreadID is 0x%x, working on Task%d\n", Pthread_self (), * (i
    NT *) ARG);
Sleep (1);/* Rest a second, extended task execution time/return NULL; the int main (int argc, char **argv) {pool_init (3);///////////////////////////////////int *workingnum = (
    int *) malloc (sizeof (int) * 10);
    int i;
        for (i = 0; i < i++) {Workingnum[i] = i; POol_add_worker (Myprocess, &workingnum[i]);
    /* Wait for all tasks to be completed/* sleep (5);

    /* Destroy thread pool/Pool_destroy ();
    Free (workingnum);
return 0;
 }

Put all of the above code into the threadpool.c file,
Compile command on Linux input
$ gcc-o ThreadPool Threadpool.c-lpthread

The following are the results of the operation
Starting thread 0xb7df6b90
Thread 0xb7df6b90 is waiting
Starting thread 0xb75f5b90
Thread 0xb75f5b90 is waiting
Starting thread 0xb6df4b90
Thread 0xb6df4b90 is waiting
Thread 0xb7df6b90 is starting to work
ThreadID is 0xb7df6b90, working on task 0
Thread 0xb75f5b90 is starting to work
ThreadID is 0xb75f5b90, working on Task 1
Thread 0xb6df4b90 is starting to work
ThreadID is 0xb6df4b90, working on Task 2
Thread 0xb7df6b90 is starting to work
ThreadID is 0xb7df6b90, working on task 3
Thread 0xb75f5b90 is starting to work
ThreadID is 0xb75f5b90, working on Task 4
Thread 0xb6df4b90 is starting to work
ThreadID is 0xb6df4b90, working on Task 5
Thread 0xb7df6b90 is starting to work
ThreadID is 0xb7df6b90, working on task 6
Thread 0xb75f5b90 is starting to work
ThreadID is 0xb75f5b90, working on task 7
Thread 0xb6df4b90 is starting to work
ThreadID is 0xb6df4b90, working on task 8
Thread 0xb7df6b90 is starting to work
ThreadID is 0xb7df6b90, working on task 9
Thread 0xb75f5b90 is waiting
Thread 0xb6df4b90 is waiting
Thread 0xb7df6b90 is waiting
Thread 0xb75f5b90 'll exit
Thread 0xb6df4b90 'll exit
Thread 0xb7df6b90 'll exit