Producer/consumer model

 

[0]: overview

Today we are going to introduce the "producer/consumer model", which can be used in many development fields. This mode is very important. I plan to introduce it in several posts. Today, I am going to talk about literacy. If you know more about this mode, skip this literacy post and read the next post (about the application of this mode ).

You may have heard of this in the 23 modes of the four-person gang (gof! In fact, the 23 classic gof models are mainly based on OO (from the title of design patterns: Elements of reusable object-oriented software ). The pattern can be oo or non-oo pattern.

★Introduction

Let's get down to the truth! In the actual software development process, we often encounter the following scenarios: a module is responsible for generating data, and the data is processed by another module (the module here is in a broad sense, can be a class, function, thread, process, etc ). The module that generates data is visually called a producer, and the module that processes data is called a consumer.

Abstract producers and consumers alone is not enough to be a producer/consumer model. This mode also requires a buffer between the producer and consumer as a mediation. The producer puts the data into the buffer, while the consumer extracts the data from the buffer. The approximate structure is shown in.

In order not to be too abstract, let's give an example of sending a mail (although it is no longer a good time to send a mail, this example is more appropriate ). Suppose you want to send a letter, the general process is as follows:

1. You write the letter-equivalent to the producer's data

2. You put the mail in the mailbox-the producer puts the data in the buffer zone

3. The Postman extracts the mail from the mail box, which is equivalent to taking the data out of the buffer zone by the consumer.

4. The postman takes the mail to the post office for corresponding processing-equivalent to the Consumer processing data

★Advantages

Some colleagues may ask: what is the use of this buffer zone? Why not let the producer directly call a function of the consumer and pass the data directly? What can we do with such a buffer?

In fact, this is very exquisite, and it has the following benefits.

◇ Decoupling

Assume that the producer and consumer are two classes. If the producer directly calls a method of the consumer, the producer will depend on the consumer (that is, coupling ). In the future, if the consumer's code changes, it may affect the producer. If both of them depend on a buffer zone, the coupling between the two is reduced accordingly.

In the preceding example, if you do not use a mailbox (that is, a buffer zone), you must directly send the mail to the postman. Some people may say that it is quite easy to directly give the postman? In fact, it is not easy. You must know who is a postman to give the mail to him (just wear the uniform on your body. If someone is fake, it will be miserable ). This produces dependencies with you and the postman (equivalent to strong coupling between producers and consumers ). In case the postman changes people one day, You Need To Know it again (equivalent to modifying the producer code due to consumer changes ). The mailbox is relatively fixed, and the cost of relying on it is relatively low (equivalent to weak coupling with the buffer zone ).

◇ Concurrency)

Another drawback is that a producer directly calls a method of a consumer. Because function calls are synchronous (or blocked), the producer has to wait until the consumer's method is returned. In case the consumer processes data slowly, the producer will waste a good time.

After the producer/consumer mode is used, the producer and consumer can be two independent concurrent subjects (Common concurrency types include process and thread, the following post will talk about applications under two concurrent types ). The producer can output the next data as soon as the data is lost to the buffer zone. Basically, there is no need to rely on the processing speed of the consumer.

In fact, this mode was mainly used to handle concurrency issues.

From the mail example. If there is no mailbox, you have to stand at the intersection with a letter and wait for the postman to come over (equivalent to the producer blocking); or the postman has to ask door-to-door who wants to send a mail (equivalent to consumer polling ). No matter which method is used, it's pretty good.

◇ Support for idle and busy periods

The buffer has another benefit. If the speed of data manufacturing is fast and slow, the benefits of the buffer zone will be reflected. When data is created quickly, the consumer cannot process the data. unprocessed data can be temporarily stored in the buffer zone. When the producer's manufacturing speed slows down, the consumer will slowly process it.

In order to fully reuse the information, let's take the mail example as an example. Assume that the postman can only take away 1000 messages at a time. If a greeting card is sent on Valentine's Day (or Christmas), more than 1000 emails need to be sent. At this time, the mailbox buffer zone will be used. The postman saves the delayed emails in the inbox and takes them again when they arrive.

With so much saliva, I hope that students who do not know much about the producer/consumer model can understand what it is. In the next post, let's talk about how to determine the data unit.

In addition, to facilitate reading, sort the directories of this series of posts as follows:

1. How to determine data units

2. Queue Buffer

3. Queue Buffer

4. Dual-buffer zone

5 ,......

[1]: How to determine a data unit?

Since the previous post has already completed literacy, we should start to talk about some specific programming technical issues. However, before entering the specific technical details, we must first understand the question: how to determine the data unit? Only by clearly analyzing data units can we develop the technical design later.

★What is a data unit?

What is data unit pinching? Simply put, each producer put in a buffer zone is a data unit. Each consumer fetches data from the buffer zone. For the example sent in the previous post, we can regard each letter as a data unit.

However, this introduction is too simple to help everyone analyze this stuff. So let's take a look at the features that data units need. After understanding these features, it is easy to analyze what is suitable for data units from complicated business logic.

★Features of data units

To analyze data units, consider the following features:

◇ Associate with Business Objects

First, data units must be associated with certain business objects. When considering this issue, you must have a deep understanding of the business logic of the current producer/consumer model to make appropriate judgments.

Because the business logic of "Sending mail" is relatively simple, everyone can easily determine what a data unit is. But in real life, it is often not so optimistic. Most of the business logic is complex. The business objects contained in the logic are of various levels and different types. In this case, it is difficult to make a decision.

This step is very important. If the wrong business object is selected, the complexity of subsequent programming and coding implementation will be greatly increased, increasing the development and maintenance costs.

◇ Integrity

Integrity means to ensure the integrity of the data unit during transmission. Either the entire data unit is transmitted to the consumer or completely not to the consumer. Partial transfer is not allowed.

For a mail, you cannot put half a mail into a mail box. Similarly, the postman can neither take the mail from the mail box, nor just take out a part of the mail.

◇ Independence

The so-called independence means that each data unit is not mutually dependent. The transmission failure of a data unit should not affect the unit that has completed the transmission, nor affect the unit that has not yet been transmitted.

Why does the transmission fail? If the producer's production speed exceeds the consumer's processing speed for a period of time, it will cause the buffer to grow and reach the upper limit, and the subsequent data units will be discarded. If data units are independent of each other and the producer's speed drops, subsequent data units will not be affected. Otherwise, if some coupling exists between data units, the discarded data units will affect the processing of other units in the future, which will complicate the program logic.

For a mail, the loss of a mail will not affect the delivery of subsequent mail; of course, it will not affect the delivery of delivered mail.

◇ Granularity

As mentioned above, data units must be associated with certain business objects. So do data units and business objects need to be interconnected one by one? In many cases, it is indeed a one-to-one correspondence.

However, sometimes n business objects may be packaged into a data unit for performance and other reasons. Then, how to set the N value is a matter of granularity. The granularity is exquisite. A large granularity may cause some waste; a small granularity may cause performance problems. The balance of granularity should be based on many factors, as well as some experience values.

Or an example of sending a mail. If the granularity is too small (for example, set to 1), the postman will retrieve only one letter at a time. If there are too many letters, you have to go back and forth for a long time.

If the granularity is too large (for example, set it to 100), the sender must wait until 100 messages are collected to put them into the mailbox. If you rarely write a letter at ordinary times, you have to wait for a long time.

Some may ask: Can the granularity of the producer and consumer be set to different sizes (for example, set the sender to 1 and the postman to 100 ). Of course, this can be done theoretically, but in some cases it will increase the complexity of Program Logic and code implementation. We may discuss the specific technical details later.

Well, this is the topic of data units. I hope that through this post, everyone can understand what the data unit is. Next post, let's talk about the technical implementation of "queue-based buffer zone.

[2]: queue Buffer

After the preparation of the previous two posts, I finally began to talk about some specific programming technologies. Different buffer types and concurrency scenarios have a great impact on specific technical implementation. To make it easy for everyone to understand, let's first introduce the most traditional and common methods. That is, a single producer corresponds to a single consumer and uses the queue (FIFO) as the buffer.

Regarding the concurrency scenario, in the previous post, "is the process still a thread? It is a problem !" The advantages and disadvantages of processes and threads have been discussed. Therefore, the process and thread modes will be mentioned in the introduction of various buffer types at the same time.

★Thread mode

Let's take a look at the example of using a queue in a concurrent thread and its advantages and disadvantages.

◇ Memory Allocation performance

In the thread mode, each producer and consumer is a thread. The producer writes data to the queue header (push), and the consumer reads data from the end of the queue (POP ). When the queue is empty, the consumer will take a moment (take a break); when the queue is full (up to the maximum length), the producer will take a moment. The entire process is not complex.

So what are the problems in the above process? A major problem is the performance overhead of memory allocation. For common queue implementation: heap memory allocation may be involved in each push; in each pop operation, heap memory release may be involved. If producers and consumers are diligent and frequently push and pop, the memory allocation overhead will be considerable. For memory allocation overhead, students who use Java can refer to the post "Java performance optimization [1]" in the previous days. For students who use C/C ++, we must be clear about the underlying operating system mechanism. We should know that allocating heap memory (new or malloc) will incur lock overhead and user/core switching overhead.

What should I do? Let's take a look at the following breakdown for the "producer/consumer mode [3]: Ring buffer ".

◇ Synchronization and mutex Performance

In addition, because the two threads share a queue, it will naturally involve such painstaking tasks as synchronization, mutex, deadlock, and so on. Fortunately, the course "Operating System" has a detailed introduction to this. Should I still have some impressions? You don't have to worry about this course. There are a lot of online introductions (such as "here"). Let's just take a look. Today we are not talking about the details.

The performance overhead of synchronization and mutex will be discussed in detail. In many cases, the use of such things as semaphores and mutex also has a great deal of overhead (in some cases, it may also lead to user/core switching ). If producers and consumers are both diligent, these expenses should not be underestimated.

What should I do? Please refer to the following breakdown for "producer/consumer mode [4]: Dual-buffer ".

◇ Suitable for queue scenarios

I criticized the shortcomings of the queue just now. Is the queue method useless? None. Because queue is a common data structure, most programming languages have built-in queue support (for details, see "here "), some languages even provide thread-safe queues (for example, arrayblockingqueue introduced by JDK 1.5 ). Therefore, developers can seize the opportunity to avoid re-inventing the wheel.

Therefore, if your data traffic is not large, the benefits of using the queue buffer are obvious: clear logic, simple code, and easy maintenance. It is in line with the KISS Principle.

★Process Method

After the thread method is completed, we will introduce the process-based concurrency.

The cross-process producer/consumer mode relies heavily on the specific inter-process communication (IPC) mode. However, IPC has a wide variety of names, which cannot be listed one by one (after all, the saliva is limited ). Therefore, we have chosen several cross-platform and programming languages that support a large number of IPC methods.

◇ Anonymous Pipeline

It is felt that the MPs queue is the most queue-like IPC type. The producer process puts data into the write end of the pipeline; the consumer process extracts data from the read end of the pipeline. The effect of the entire process is very similar to that of using a queue in a thread. The difference is that you don't have to worry about thread security, memory allocation, and other things when using pipelines (the operating system is done for you in the dark ).

The MPs queue is divided into named MPs queues and anonymous MPs queues. Today, we primarily talk about anonymous MPs queues. Because named pipelines vary greatly in different operating systems (for example, Win32 and POSIX, the API interfaces and functions of named pipelines are significantly different; some platforms do not support named pipelines, for example, Windows CE ). Apart from operating system problems, for some programming languages (such as Java), named pipelines cannot be used. So I generally do not recommend this.

In fact, the APIs of anonymous pipelines on different platforms are also different (for example, the usage of Win32 createpipe and POSIX pipe is very different ). However, we can only use standard input and standard output (hereinafter referred to as stdio) for inbound and outbound data. Then, use the shell pipeline to associate the producer process with the consumer process (if you have never heard of this method, you can see "here "). In fact, many operating systems (especially POSIX-style) Come with commands that fully utilize this feature to implement data transmission (such as more and grep ).

There are several advantages to doing so:

1. Basically, all operating systems support using MPs queues in shell mode. Therefore, it is easy to implement cross-platform.

2. Most programming languages can operate stdio, so cross-programming languages are easy to implement.

3. As mentioned earlier, the MPs queue method saves the trouble of thread security. This helps reduce development and debugging costs.

Of course, this method also has its own shortcomings:

1. producer and consumer processes must be on the same host and cannot communicate with each other. This disadvantage is obvious.

2. This method works well in one-to-one scenarios. But if you want to extend to one-to-many or multiple-to-one, it will be a bit tricky. Therefore, the scalability of this method requires a discount. If similar extensions are to be considered in the future, this disadvantage is obvious.

3. Because the MPs queue is created by shell, the processes on both sides are invisible (the program only sees stdio ). In some cases, the program is not easy to manipulate the pipeline (such as adjusting the buffer size of the pipeline ). This disadvantage is not obvious.

4. Finally, data can only be transmitted in one way. Fortunately, in most cases, the consumer process does not need to transmit data to the producer process. If you really need information feedback (from the consumer to the producer), it will be difficult. You may have to consider replacing the IPC method.

By the way, let's take a look at the following points:

1. Read and Write operations on stdio are blocked. For example, if there is no data in the MPs queue, the read operation of the consumer process will stop until the MPs queue refresh the data.

2. Because stdio has its own buffer zone (this buffer zone is the same as the MPs queue buffer zone), it sometimes causes some unpleasant phenomena (for example, the producer process outputs data, but the consumer process does not read it immediately ). For more information, see "here ".

◇ Socket (TCP Mode)

TCP-based socket communication is another IPC method similar to the queue. It also ensures the arrival of data in sequence; it also has a buffer mechanism. This is also cross-platform and cross-language. It is similar to the shell pipeline method just introduced.

What are the advantages of socket over Shell Pipe operators? It has the following advantages:

1. the socket mode can be cross-machine (easy to implement distributed ). This is the main advantage.

2. the socket method facilitates future expansion into multiple-to-one or one-to-many. This is also the main advantage.

3. the socket can be configured with blocking and non-blocking methods, which are flexible to use. This is a secondary advantage.

4. Socket supports two-way communication, which facilitates consumer feedback.

Of course, there are advantages and disadvantages. Compared with the preceding shell pipeline method, using socket programming is more complex. Fortunately, our predecessors have already done a lot of work and developed many socket communication libraries and frameworks for everyone (such as C ++ ace library and Python twisted ). With the help of these third-party libraries and frameworks, the socket method is quite easy to use. Since the usage of a specific network communication library is not the focus of this series, I will not elaborate on it here.

Although TCP is more reliable than UDP in many aspects, due to the inherent unpredictability of cross-machine communication (for example, the network cable may be pulled incorrectly by a silly X, and the network may fluctuate greatly ), in terms of program design, we still need to retain more hands. What should I do? Thread-based "producer/consumer mode" can be introduced in the producer process and consumer process ". This sounds like a tongue twister. To make it easy to understand, draw a picture to show everyone.

The key to doing so is to divide the code into two parts: the production line and consumption thread are codes related to the business logic (irrelevant to the communication logic ); the sending and receiving threads are communication-related codes (independent from the business logic ).

The benefits are obvious, as shown below:

1. Be able to cope with temporary network faults. You can continue to work after the network fault is rectified.

2. How to Handle network faults (for example, try to reconnect after disconnection) only affects the sending and receiving threads, and does not affect the production line and consumption threads (business logic ).

3. The specific socket method (blocking and non-blocking) only affects the sending and receiving threads, and does not affect the production line and consumption threads (business logic part ).

4. It does not rely on the sending and receiving buffers of TCP itself. (The default TCP buffer size may not meet the actual requirements)

5. Changes in business logic (such as changes in business requirements) do not affect the sending and receiving threads.

For the last article above, I would like to say a few more words. If multiple processes in the entire business system adopt the above model, you may be able to refactor one by one: cutting the line between the business logic code and the communication logic code, encapsulate the unrelated parts of the business logic into a communication middleware (which seems to be a good choice :-). If everyone is interested in this stuff, they will post a chat later.

[3]: Ring Buffer

The previous post mentioned the possible performance problems and solutions of the queue buffer zone: The Ring buffer zone. This topic is described here today.

In order to prevent someone from giving us a "overly-designed" big hat, we declare that only when the storage space is allocated/released frequently and has a significant impact, you should consider the use of the ring buffer. Otherwise, we recommend that you use the most basic and simple queue buffer. Note: The "storage space" mentioned in this article includes not only memory, but also storage media such as hard disks.

★Ring buffer vs queue Buffer

◇ External interfaces are similar

Before introducing the circular buffer, let's review common queues. A common queue has a write end and a read end. When the queue is empty, the reading end cannot read data. When the queue is full (maximum size), the writing end cannot write data.

For users, the ring buffer is the same as the queue buffer. It also has a write end (for push) and a read end (for pop), as well as a buffer that is "full" and "empty. Therefore, switching from the queue buffer to the ring buffer can be a smooth transition for users.

◇ Different internal structures

Although the two have similar external interfaces, the internal structure and operation mechanism are very different. The internal structure of the queue is not long-winded here. This section focuses on the internal structure of the ring buffer.

Everyone can think of the Reading end (R) and writing end (w) of the circular buffer zone as two people chasing the stadium runway (r chasing W ). When r catches up with W, the buffer zone is empty. When w catches up with R (W is a lap longer than R), the buffer zone is full.

For the sake of image, find a picture and make slight changes, as shown below:

It can be seen that all push and pop operations in the ring buffer are performed in a fixed storage space. When the queue buffer is pushed, storage space may be allocated to store new elements. When pop is used, the storage space of discarded elements may be released. Therefore, compared with the queue mode, the ring mode reduces the allocation and release of the storage space used by the buffer elements. This is a major advantage of the ring buffer.

★Implementation of circular buffer

If you already have a ready-made ring buffer available and you are not interested in the internal implementation of the ring buffer, you can skip this section.

◇ Array vs linked list

The internal implementation of the ring buffer can be implemented based on arrays (arrays here refer to contiguous buckets) or linked lists.

An array is a one-dimensional continuous linear structure in physical storage. It can be allocated once during initialization, which is an advantage of the array method. However, to use an array to simulate the ring, you must logically connect the header and tail of the array. When traversing the array sequentially, special processing is required for the tail element (the last element. When accessing the next element of the tail element, You need to return to the Header element (0th elements ). As shown in:

Using a linked list is the opposite of an array: The linked list saves the special processing of the first and last links. However, the linked list is cumbersome during initialization, and in some cases (such as the cross-process IPC mentioned later) it is not easy to use.

◇ Read/write operations

Two indexes must be maintained in the ring buffer, corresponding to the write side (W) and the read side (r) respectively ). When writing (push), first make sure the ring is not full, then copy the data to the elements corresponding to W, and finally point w to the next element; when reading (POP, first, make sure that the ring is not available, then return the corresponding elements of R, and then R points to the next element.

◇ Judge "empty" and "full"

The above operations are not complex, but there is a small trouble: When the Blank Ring and the full ring, R and W both point to the same location! In this way, you cannot determine whether it is "empty" or "full ". There are two ways to solve this problem.

Method 1: Keep an element unused.

When the ring is empty, R and W overlap. When W is faster than R and there is an element gap between R, it is considered that the ring is full. When the storage space occupied by elements in the ring is large, this method is very earthy (a waste of space ).

Method 2: maintain additional variables

If you do not like the above method, you can also use additional variables to solve the problem. For example, you can use an integer to record the number of elements saved in the current ring (this integer> = 0 ). When R and W overlap, you can use this variable to know whether it is "null" or "full ".

◇ Element storage

Because the circular buffer itself is to reduce the overhead of storage space allocation, the type of elements in the buffer must be selected. Store Data of the value type as much as possible, instead of data of the pointer type. Because pointer-type data can cause the distribution and release of storage space (such as heap memory), the effect of the ring buffer is compromised.

★Application scenarios

The implementation mechanism inside the ring buffer is introduced just now. According to the practice of the previous post, we will introduce the use of online and process methods.

If your programming language and Development Library have a ready-made and mature ring buffer, it is strongly recommended that you use a ready-made library instead of re-creating the wheel. If you cannot find the ready-made library, consider your own implementation. If you are a trainer in your spare time, let alone.

◇ Used for concurrent threads

Similar to the queue buffer in a thread, thread security should also be considered in the circular buffer in the thread. Unless the library of the ring buffer you are using has helped you achieve thread security, you still have to do it yourself. The circular buffer in the thread mode is used a lot, and there are also a lot of related online information. The following is a rough introduction.

For C ++ programmers, it is strongly recommended to use the circular_buffer template provided by boost. This template was first introduced in boost 1.35. In view of boost's position in the C ++ community, everyone should be able to use this template with confidence.

For C programmers, you can look at the open-source project circbuf. However, this project is based on the GPL Protocol and is not very good. It is not very active. There is only one developer. Everyone should be cautious! We recommend that you use it for reference only.

For C # programmers, refer to an example in codeproject.

◇ Used for Concurrent Processes

The circular buffer between processes seems to have few ready-made libraries available. Everyone had to do it by themselves.

It is applicable to the IPC type of inter-process ring buffering. Common examples include shared memory and files. In these two methods, ring buffering is usually implemented using arrays. The program allocates a fixed-length storage space in advance, and then the specific read/write operations, judgment of "null" and "full", element storage and other details can be referred to the above.

The shared memory mode has good performance and is suitable for scenarios with large data traffic. However, some languages (such as Java) do not support shared memory. Therefore, this method has some limitations in the multi-language collaborative development system.

The file method supports very well in programming languages, and almost all programming languages support file operations. However, it may be limited by the performance of disk I/O. Therefore, the file method is not suitable for fast data transmission. However, for some situations where "data units" are very large, the file method is worth considering.

For the ring buffer between processes, we also need to consider the synchronization and mutex among processes. We will not go into detail here.

Next post, let's talk about the use of dual-buffer.