POSIX multithreaded Programming _ pipelined work routines

#include <pthread.h> #include "errors.h" typedef struct STAGE_TAG{&LT;SPAN style= "White-space:pre" ></span >//assembly line Work cell structure pthread_mutex_t mutex;<span style= "White-space:pre" ></span>//the mutex that protects the current working cell data pthread_ cond_t avail;<span style= "White-space:pre" ></span>//wait for the current unit of work to store data for available condition variables pthread_cond_t ready;<span Style= "White-space:pre" ></span>//condition variable that waits for the current unit of work to process new data int Data_ready;<span style= "White-space:pre" > </span>//indicates the status of the current unit of work (0 means data is out of date, 1 means data is valid) long Data;<span style= "White-space:pre" ></span >//data stored in the current unit of work pthread_t Thread;<span style= "White-space:pre" ></span>//the thread where the current work cell is located idstruct stage_ Tag *next;<span style= "White-space:pre" ></span>//a pointer to the next work order unit in the pipeline}stage_t;typedef struct pipe_tag{< Span style= "White-space:pre" ></span>//pipelined structural body pthread_mutex_tmutex;<span style= "White-space:pre" > </span>//data stage_t*head;<span style= "White-space:pre" ></span>//to add new data or read pipelining resultsPipeline first work order original pointer Stage_t*tail;<span style= "White-space:pre" ></span>//pipeline's last work Tanyuan pointer ( Save data after pipeline processing of the resulting data) int Stages;<span style= "White-space:pre" ></span>//the number of units in the pipeline int Active;<span style= number of work units in the "White-space:pre" ></span>//pipeline that are processing data}pipe_t;int pipe_send (stage_t *stage,long data) {< Span style= "White-space:pre" ></span>//the function that transmits new data to the work order pointed to by the stage int Status;<span style= "White-space: Pre "></span>//Save the call state of the thread function (a return value of 0 means the call succeeds or the call fails to print error message) status = Pthread_mutex_lock (&stage->mutex); <span style= "White-space:pre" ></span>//attempted to lock the work Tanyuan of the stage point to the mutex if (status! = 0) return Status;while (stage- >data_ready) {<span style= "white-space:pre" ></span>//wait condition variable ready to send current work order signal status of new data processing = Pthread_ Cond_wait (&stage->ready,&stage->mutex); <span style= "font-family:arial, Helvetica, Sans-serif;" > If Data_ready is 1, that is, the data stored in the current work order is valid data, wait for the processing to complete </span>if (status! = 0) {<span style= "White-space:pre" ></ Span&gT;//<span style= "font-family:arial, Helvetica, Sans-serif;" > Otherwise, indicates that the current unit of work stored data is out-of-date data and can process new incoming data data</span>pthread_mutex_unlock (&stage->mutex); return status;}} Stage->data = Data;<span style= "white-space:pre" ></span>//Update current work order original stored data is Datastage->data_ready = 1 ; <span style= "White-space:pre" ></span>//updated data is valid data, Data_ready set to 1status = Pthread_cond_signal (& Stage->avail); <span style= "White-space:pre" ></span>//and sends a avail signal to notify the waiting thread that the data for the unit of work is available valid data if ( Status! = 0) {pthread_mutex_unlock (&stage->mutex); return status;} Status = Pthread_mutex_unlock (&stage->mutex); <span style= "White-space:pre" ></span>// Unlocks the mutex of the current unit of work return Status;<span style= "White-space:pre" ></span>//returns the function call state}void *pipe_stage (void *arg) { <span style= "White-space:pre" ></span>//thread entry function, arg for current work cell struct pointer stage_t *stage = (stage_t*) arg;<span Style= "White-space:pre" ></span>//j convert arg to stage_t* type and assign value to stagestage_t *next_stage= Stage->next;<span style= "White-space:pre" ></span>//assigns a pointer to the next work cell in the stage to Stage_nextint status;< Span style= "White-space:pre" ></span>//Ibid. statusstatus = Pthread_mutex_lock (&stage->mutex);< Span style= "White-space:pre" ></span>//started to work, attempting to lock the current mutex if (status! = 0) Err_abort (status, "Lock pipe Stage"); while (true) {<span style= "White-space:pre" ></span>//loops over the data in the current work-order primitive while (stage->data_ready! = 1) { <span style= "White-space:pre" ></span>//wait condition variable avail send current data available signal status = Pthread_cond_wait (&stage- >avail,&stage->mutex)///The start of the wait will unlock the bound mutex if (status! = 0) Err_abort (status, "Wait for previous stage");} Pipe_send (Next_stage, Stage->data + 1); <span style= "White-space:pre" ></span>//will work on the current data (here is the data+1) After the result is sent to the next working unit processing Stage->data_ready = 0;<span style= "White-space:pre" ></span>//data is passed to the next unit of work after the current moment expires, Data_ready set to 0status = Pthread_cond_signal (&stage->ready); <span style= "White-space:pre" ></span>//and sends a ready signal to notify the waiting thread that the unit of work can receive and process the new data if (status! = 0) Err_abort (status, "Wake next Stage");}} int Pipe_create (pipe_t * pipe,int stages) {<span style= ' white-space:pre ' ></span>//function to create workflow Stages represents the number of work units int Pipe_index;<span style= "White-space:pre" ></span>//the currently created work Tanyuan index stage_t **link = & Pipe->head, *new_stage, *stage;//the various pointer variables used to create the linked list int status;<span style= "White-space:pre" ></span>// Ibid. statusstatus = Pthread_mutex_init (&pipe->mutex,null); <span style= "White-space:pre" ></span>/ /Initialize the mutex to protect the workflow list if (status! = 0) Err_abort (status, "Init pipe Mutex");p ipe->stages = Stages;pipe->active = 0;< Span style= "White-space:pre" ></span>//the number of working units currently processing data initialized to 0for (pipe_index = 0; Pipe_index <= stages; pipe_ index++) {<span style= "White-space:pre" ></span>//loop creates the data node of the work cell new_stage = (stage_t*) malloc (sizeof ( stage_t) <span style= "White-space:pre" ></span>//allocates memory space if (new_stage = = NULL) errno_abort ("Allocate Stage "); statUS = Pthread_mutex_init (&new_stage->mutex,null); <span style= "White-space:pre" ></span>// Here are some of the initialization work if (status! = 0) Err_abort (status, "Init stage Mutex"); status = Pthread_cond_init (&new_stage->avail, NULL), if (status! = 0) Err_abort (status, "Init avail condition"); status = Pthread_cond_init (&new_stage->ready, NULL); if (status! = 0) Err_abort (status, "Init ready Condition"); New_stage->data_ready = 0;*link = New_stage;link = &amp ; new_stage->next;} *link = (stage_t*) null;<span style= "white-space:pre" ></span>pipe->tail = new_stage;for (stage = pipe- >head;stage->next! = Null;stage = stage->next) {<span style= "White-space:pre" ></span>// Create Thread status = Pthread_create (&stage->thread,null,pipe_stage, (void*) stage) for each unit of work, and if (status! = 0) Err_abort ( Status, "Create pipe Stage"); return 0;} int Pipe_start (pipe_t *pipe, Long value) {<span style= "white-space:pre" ></span>//send newly added data to Workflow processing int Status;status = Pthread_mutex_lock (&Pipe->mutex); <span style= "White-space:pre" ></span>//attempted to lock a mutex to add new data if (status! = 0) Err_abort (status, " Lock pipe Mutex ");p ipe->active++;<span style=" White-space:pre "></span>//if the mutex is locked, the current first unit of work can work, Add the count of work to a status = Pthread_mutex_unlock (&pipe->mutex); <span style= "White-space:pre" ></span>// Unlock the mutex if (status! = 0) Err_abort (status, "Unlock pipe Mutex");p ipe_send (pipe->head,value); <span style= " White-space:pre "></span>//sends data to the first unit of work to process return 0;} int Pipe_result (pipe_t *pipe,long *result) {<span style= "White-space:pre" ></span>//get the function of processing result stage_t * Tail = Pipe->tail;long Value;<span style= "White-space:pre" ></span>//save result data int empty = 0;<span style= The "White-space:pre" ></span>//determines whether the data in the workflow is empty flagint status;status = Pthread_mutex_lock (&pipe->mutex); <span style= "White-space:pre" ></span>//attempted to lock the pipeline mutex if (status! = 0) Err_abort (status, "Lock pipe Mutex"); if (pipe->active <= 0) <span style= "White-space:pre" ></span>//if the working unit in the current work is less than or equal to 0empty = 1;<span style= "White-space:pre" ></span> Set empty to 1elsepipe->active--;<span style= "White-space:pre" ></span>//otherwise reduce the number of working units in one job (because it takes a data out.) Status = Pthread_mutex_unlock (&pipe->mutex); <span style= "White-space:pre" ></span>// Unlock the line mutex if (status! = 0) Err_abort (status, "Unlock pipe Mutex"); if (empty) <span style= "White-space:pre" ></ span>//if the data in the pipeline is empty, the fetch data fails, and the function returns 0return 0;pthread_mutex_lock (&tail->mutex); <span style= " White-space:pre "></span>//attempted to lock the mutex of the last unit of work while (!tail->data_ready) <span style=" White-space:pre " ></span>//waits for the last unit of work to send a signal pthread_cond_wait (&tail->avail,&tail->mutex) where the data stored is available as a valid number of moments; result = Tail->data;<span style= "White-space:pre" ></span>//the valid data in the last unit of work is taken out into result Tail->data _ready = 0;<span style= "White-space:pre" ></span>//the last work Tanyuan data is out, the data in it is out of date, Data_ready is set to 0pthread_ Cond_signal (&Amp;tail->ready) <span style= "White-space:pre" ></span>//and sends a signal to other waiting threads, The last unit of work can process new data pthread_mutex_unlock (&tail->mutex); <span style= "White-space:pre" ></span>// Lock the last work cell mutex return 1;<span style= "White-space:pre" ></span>//get data successfully, return 1}int main (int argc, char *argv[] {<span style= "white-space:pre" ></span>//main function pipe_t my_pipe;<span style= "White-space:pre" ></ span>//defines the workflow variable long value,result;<span style= "White-space:pre" ></span>//value for data to be processed, result for the processing result int Status;char line[128];<span style= "white-space:pre" ></span>//input cache Pipe_create (&my_ Pipe, <span style= "White-space:pre" ></span>//Create a new workflow (the workflow that creates 10 units of work by default) printf ("Enter integer values, or \ "=\" for Next result\n ") <span style=" White-space:pre "></span>//hint information while (true) {<span style=" White-space:pre "></span>//Loop waits for the user to enter and process data printf (" data> "); <span style=" White-space:pre "></ span>//Prompt if (fgets (line,sizeof (line), stdin) = = NULL) exit (0); <span style= "white-space:pre" ></span>//user input end exit if (line) <= 1) continue;if (strlen (line) <= 2 && line[0] = = ') {<span style= "White-space:pre" ></span>/ /If input = indicates the processing result in the Read pipeline if (Pipe_result (&my_pipe,&result)) <span style= "White-space:pre" ></span>// If the read result is successful printf ("Result is%ld\n", result); <span style= "White-space:pre" ></span>//prints the results Else<span Style= "White-space:pre" ></span>//otherwise read fails printf ("Pipe is empty\n"); <span style= "White-space:pre" > </span>//print hint, workflow is empty}else{if (sscanf (line, "%ld", &value) < 1) <span style= "White-space:pre" ></ span>//reads the data from the cache fprintf (stderr, "Enter an Integer value\n"); else Pipe_start (&my_pipe,value); <span style= " White-space:pre "></span>//call function to process the newly added data}}}

This routine is also a sample of how the workflow is described in the POSIX multithreaded programming book, which has been repeated many times before figuring out the logic, especially the relationship between the two conditional variables ready avail and the Data_ready flag.

There are a few things to note:

1> because the workflow creates 10 units of work by default, each unit of work has a single thread (except for the last unit of work, it is responsible for saving the processing results), so there is a total of 11 data to be processed at the same time, and if more data is entered, The work units in the entire workflow are waiting for the next unit of work to emit a signal to receive new data through ready, while the work order principle referred to by tail is waiting for the main thread to read out the data. When the main thread is plugged in to new data and the work unit waiting for the head is signaled by ready to accept the new data, a deadlock occurs and the entire process is suspended.

2> the mutex in each stage_t only protects data for the current unit of work, The condition variable is also. Where the ready condition variable is sent by itself to the previous work unit to receive signals, representing the current work order in the original data has expired, notify the previous work order to the original can be the new data to its processing; Avail condition change the amount is sent to the next work order to receive the signal, representing the current work cell data has been processed, the next data unit can be taken to continue the next processing. This may be a bit of a detour, but also very well understood: b->c These two units, if the data processed in C has not been handed over to the next job as a unit, that is, the data in C is not expired, B will block, Wait for the data in C to pass to the next unit of work processing (at which time the data in C expires) and send a signal through ready to notify B that the new data can be given to C processing, then B will unblock and continue. If the data in C is already out of date, the data in B is is not handled well (for example, there is no new data to join, or is processing new data in the process), then C will block waiting for B to send a signal through avail notification c,b have new processed data (that is, valid data) can be given to C processing, At this point C will not contact the blocking to continue working.     So ready and avail these two conditional variables are associated, and finally, the combination is a complete workflow of the signal transmission system.

3> It is easy to explain the 1th reason according to the 2nd: because there are 11 data in the current workflow and have never been read out, the data that was first added to the workflow D1 waiting to be read in the work unit G11 referred to by tail. The second data D2 to join the workflow waits for G11 to send a signal to process the new data through Ready in the tenth work order G10. So the thread where the G10 is located is in a blocking state. At this time the D3 G9 is also waiting for G10 to send the ready signal to receive the D3 for processing, so the G9 is also in a blocking state of the thread, and so on, D11 is waiting for G1 in the ready signal, thread G2 blocked, because This, at this time the user input of the 12th data D12 in the main function call Pipe_start () to be processed, Pipe_start () will give the data to Pipe_send (PIPE->HEAD,D12) processing, and Pipe_ Send blocks on the condition variable that waits for the G1 to send ready signals, that is, the main thread is blocked and the entire process is suspended.

I am still learning, in view of the level of limited, if there are errors please advise, thank you!

POSIX multithreaded Programming _ pipelined work routines