The problem of producer and consumer in multithreaded programming

Producer-Consumer issues (Producer-consumer problem), also known as limited buffering issues (Bounded-buffer problem), are a classic problem in the multithreading world, and can be described as: two or more threads sharing the same buffer, One or more of these as "producers" will continuously add data to the buffer, and one or more of the data to be taken from the buffer as "consumers". The key to this problem is to ensure that the "producer" cannot continue adding data after the buffer is full, and that the "consumer" cannot take the data again after the buffer is empty.

This problem must be troublesome in the case of multiple "producers" and "consumers", so take a look at only one "producer" and one "consumer" and one element buffer. This situation can be simplified to:

1. Taking the data from the buffer and adding data to the buffer requires the mutex to be kept in sync, so it is necessary to use a critical section or a mutex to achieve it;
2. The producer waits for the buffer to "have space" (since there is only one element in the buffer, so it is equivalent to waiting for the buffer to be empty) in order to add data, and the same consumer cannot fetch the data when the buffer is empty. Both processes require an event or semaphore to be notified.

Considering the above two-point requirement, we can design the following ideas and algorithms:

/* Algorithm 1 for the producer: */
{WaitForSingleObject (hempty, INFINITE);　　WaitForSingleObject (Hmutex, inifinite);    /* Producer's activities */ReleaseMutex (HMUTEX); ReleaseSemaphore (Hfull, 1, NULL);
}/* Algorithm for Consumers 1: */
{WaitForSingleObject (hfull, INFINITE);　　WaitForSingleObject (Hmutex, inifinite);    /* Consumer Activities */ReleaseMutex (HMUTEX); ReleaseSemaphore (Hempty, 1, NULL);
}

Of course, the mutual exclusion of producers and consumers is not hmutex and the same is true for entercriticalsection critical sections. Take an example to see how to apply producer-consumer algorithms:

typedefstruct _message_queue/*Data structures for Message Queuing*/{IntThreadId;IntMsgtype[max_number];IntCount HANDLE Hfull; HANDLE Hempty; HANDLE Hmutex;} Message_queue;/*Send a message similar to "producer"*/void Send_mseesge (int threadId, message_queue* pqueue,Intmsg) {assert (NULL! =Pqueue);if (threadId! = pqueue->THREADID)Return; WaitForSingleObject (pqueue->Hempty, INFINITE); WaitForSingleObject (pqueue->Hmutex, INFINITE); Pqueue->msgtype[pqueue->count + +] =Msg ReleaseMutex (pqueue->Hmutex); ReleaseSemaphore (Pqueue->hfull,1, NULL);}  /* Receive message, similar to "consumer" */void Get_message (message_queue* pqueue, int* msg) {assert (NULL! = Pqueue && NULL! = msg); WaitForSingleObject (pqueue->hfull, INFINITE); WaitForSingleObject (pqueue->Hmutex, INFINITE); *msg = Pqueue->msgtype[pqueue->count--]; ReleaseMutex (pqueue->Hmutex); ReleaseSemaphore (Pqueue->hempty, 1, NULL);}           

Figuring out a simple pattern of a producer and a consumer as well as a buffer of elements, let's look at the case where the consumer has two and the buffer can hold four elements:

//1 producers, 2 consumers, 4 buffers # include <stdio.h>#include <process.h>#include <windows.h>Constint end_produce_number =8;//Number of products producedConstint buffer_size =4;//Number of buffersint g_buffer[buffer_size];//Buffer poolIntG_i, G_j; Critical_section G_cs;//Signal volume and critical segmentHANDLE G_hsemaphorebufferempty, G_hsemaphorebufferfull;//Producer Thread function unsignedInt__stdcall Producerthreadfun (PVOID PM) {for (int i =1; I <= End_produce_number; i++) {//Wait for the "buffer to have space left" signal!WaitForSingleObject (G_hsemaphorebufferempty, INFINITE);//Mutually exclusive access buffer EnterCriticalSection (&G_CS); G_buffer[g_i] =I G_i = (g_i +1)%Buffer_size; LeaveCriticalSection (&G_CS);//Notify consumers of "buffer data"! ReleaseSemaphore (G_hsemaphorebufferfull,1, NULL); } printf ("The producer finishes the task and the thread ends up running \ n");Return0;}//Consumer threading Function unsignedInt__stdcall Consumerthreadfun (PVOID PM) {while (True) {//Waiting for "data in buffer" signalWaitForSingleObject (G_hsemaphorebufferfull, INFINITE);//Mutually exclusive access buffer EnterCriticalSection (&G_CS);if (g_buffer[g_j] = = End_produce_number)//End Flag{LeaveCriticalSection (&G_CS);//Notify other consumers of new data (end sign) releasesemaphore (G_hsemaphorebufferfull,1, NULL);Break; } G_j = (G_j +1)%Buffer_size; LeaveCriticalSection (&G_CS); Sleep (50);//Some other work-to-do ReleaseSemaphore (G_hsemaphorebufferempty,1, NULL);//Add a space to the buffer}Return0;}IntMain () {InitializeCriticalSection (&G_CS);//Initializes the semaphore, one record has the number of buffers for the product, and the other records the number of empty buffers G_hsemaphorebufferempty = CreateSemaphore (NULL,4,4, NULL);//Specifies that there are four "remaining spaces" in the initial state of the buffer g_hsemaphorebufferfull = CreateSemaphore (NULL,0,4, NULL); G_i =0; G_j =0; memset (G_buffer,0,sizeof(G_buffer));Constint threadnum =3; HANDLE Hthread[threadnum];//Producer Thread hthread[0] = (HANDLE) _beginthreadex (NULL,0, Producerthreadfun, NULL,0, NULL);//Consumer threading hthread[1] = (HANDLE) _beginthreadex (NULL,0, Consumerthreadfun, NULL,02 ] = (HANDLE) _beginthreadex (null, 0, Consumerthreadfun, NULL, 0for (int i = 0; I < Threadnum; I++// Destroy Semaphore and critical segment g_cs); return 0;}   

The above code is very simple, a producer has been waiting for "buffer space" this signal, and two consumers have been waiting for the "buffer data" this signal is OK! When manipulating buffers, take mutex mutex actions to prevent collisions. Note that the number of initial semaphores is specified at the beginning of the creation of the semaphore, and the number of semaphores determines the size of the buffer. Each time the WaitForSingleObject will reduce the signal volume one, and each time the ReleaseSemaphore will make the signal volume plus one.

　　

Summary:

The issue of "producer-consumer" can only be considered in two ways:

1. The producers and consumers should be mutually exclusive to the buffer operation;
2. The producer waits for the signal " buffer has space " to add data, and the consumer waits for the " buffer to have data " signal to fetch the data.

The problem of producer and consumer in multithreaded programming