Task Pool management and actuators

1 background

A business platform for background real-time processing, usually based on the data input and output, according to the time axis decomposition into different stages or different granularity of the logical task, and each of the data to be processed we are called tasks or messages. The relationship between tasks can be divided into two categories: a upstream and downstream parent-child relationship, B can run in parallel sibling relationships. A task set with upstream and downstream relationships has a logical or data dependency, which means that the downstream tasks can be performed after the task is completed, and that the siblings have no logical effect on each other and can run in parallel.

Whether it is a business scenario for any of the above scenarios, we need a management class whose responsibility is to manage a bunch of threads and the same set of tasks to be executed. In the JDK there are ready-made management class Threadpoolexecutor, then in C + + look at a similar implementation:

2 tasks and Task pools

2.1 Quests

Whether it is a message or business data, it can be expressed in an abstract way:

      struct Data_pair      {         char *data;          int len;      }

2.2 Task Pool

The task's cache is expressed in a queue:

Std::queue<data_pair*> _queue;

2.3 Task Submission Portal

  intCqueuethread::writedata (void*data,intLen) {        if(data = = NULL | | Len <=0) {            returnexit_failure; } Data_pair*item =NewData_pair (); Item->data = (Char*)malloc(len); ASSERT (item->data! =NULL); memcpy (Item-data, data, Len); Item->len =Len; _mutex.Lock();        _queue.push (item);        _mutex.signal ();        _mutex.unlock (); returnexit_success; }

3 thread Pool    

3.1 Thread Encapsulation

C + + is similar to the JDK inside the thread class package Cthread

{classCthread { Public:    /** * Constructor function*/Cthread () {tid=0; PID=0; }     /** * Starting a thread and starting to run*/    BOOLStart (Runnable *r,void*a) {runnable=R; Args=A; return 0= = Pthread_create (&tid, NULL, Cthread::hook, This); }     /** * Wait for thread to exit*/    voidjoin () {if(TID) {Pthread_join (tid, NULL); Tid=0; PID=0; }    }     /** * Get Runnable Object * * @return Runnable*/Runnable*getrunnable () {returnrunnable; }     /** * Get callback parameters * * @return args*/    void*Getargs () {returnargs; }       /** * get thread's process ID*/    intGetpid () {returnpid; }     /** * Thread's callback function **/     Static void*hook (void*Arg) {Cthread*thread = (cthread*) Arg; Thread->pid =Gettid (); if(thread->getrunnable ()) {Thread->getrunnable ()->run (thread, thread->Getargs ()); }         return(void*) NULL; }   Private:       /** * Get the TID number*/#ifdef _syscall0Static_syscall0 (Pid_t,gettid)#else    Staticpid_t Gettid () {returnStatic_cast<pid_t>(Syscall (__nr_gettid));} #endif Private: pthread_t tid; //pthread_self () ID    intpid//process ID of the threadRunnable*runnable; void*args;}; }

View Code

3.2 Thread Pool

The capacity of the parallel processing pool is determined by the number of threads, defined as follows:

Cthread *_thread; int _threadcount;

4 Actuator

4.1 Execution start

intCdefaultrunnable::start () {if(_thread! = NULL | | _threadcount <1) {Tbsys_log (ERROR,"start Failure, _thread:%p, ThreadCount:%d", _thread, _threadcount); return 0; } _thread=NewCthread[_threadcount]; if(NULL = =_thread) {Tbsys_log (ERROR,"Create _thread object failed, ThreadCount:%d", _threadcount); return 0; }    inti =0;  for(; i<_threadcount; i++)    {        if(!_thread[i].start ( This, (void*)((Long))) {returni; }    }returni;}

4.2 Execution

The actuator contains the execution of the specific business:

voidCqueuethread::run (Cthread *thread,void*args) {        intThreadindex = (int)((Long) (args)); _mutex.Lock();  while(!_stop) {             while(_stop = =0&&_queue.empty ())            {_mutex.wait (); }            if(_stop) { Break; } Data_pair*item =_queue.front ();            _queue.pop ();            _mutex.unlock (); if(Item! =NULL) {                if(_handler) {_handler->handlequeue (Item->data, item->Len, Threadindex, _args); }                if(item->data) {                     Free(item->data); }                 Free(item); } _mutex.Lock();   } _mutex.unlock (); 

5 Sample Code

Cmyhandler handler;    Cqueuethread Queuethread (3, &handler, NULL);    Queuethread.start ();     Char data[1024x768];      for (int i=1; i<=mwritecount; i++) {        int"data_%05d  ", i);        Queuethread.writedata (data, Len+1);    }    Queuethread.wait ();

Reference:

http://code.taobao.org/p/tfs/src/

Task Pool management and actuators