Producer/consumer Mode 2: How to Determine the data unit?

This article turned from programming blog: http://program-think.blogspot.com/2009/03/producer-consumer-pattern-1-data.html

 

★What is a data unit?
Simply put, each producer put in a buffer zone is a data unit. Each consumer fetches data from the buffer zone. For sending example, we can regard each letter as a data unit.

 

However, this introduction is too simple to help everyone analyze this stuff. So let's take a look at the features that data units need. After understanding these features, it is easy to analyze what is suitable for data units from complicated business logic.

 

★Features of data units
To analyze data units, consider the following features:

 

◇ Associate with Business Objects
First, data units must be associated with certain business objects. When considering this issue, you must have a deep understanding of the current producer/consumer modelBusiness LogicTo make a proper judgment.

 

Because the business logic of "Sending mail" is relatively simple, everyone can easily determine what a data unit is. But in real life, it is often not so optimistic. Most of the business logic is complex. The business objects contained in the logic are of various levels and different types. In this case, it is difficult to make a decision.

 

Although this step is difficult, it is very important! If you select the wrong business object, the complexity of subsequent programming and coding implementation will be greatly increased, increasing the development and maintenance costs.

 

◇ Integrity
Integrity means to ensure the integrity of the data unit during transmission. OrWholeThe data unit is either passed to the consumer, or completely not to the consumer. Not AllowedPartTransfer situation.

 

For mail, youNoPut half a mail in the mail box. Similarly, the postman picks up the mail from the mail boxNoOnly part of the email is provided.

 

◇ Independence
Independence means that data units are not mutually dependent and a data unit fails to be transmitted.NoAffects the data units that have completed transmission.NoAffects untransmitted data units.

 

Why does the transmission fail? If the producer's production speed exceeds the consumer's processing speed for a period of time, it will cause the buffer to continue to grow and reach the upper limit, then it is very bad (some data units will be ruthlessly abandoned ). If data units are independent of each other and the producer's speed drops, subsequent data units will not be affected. Otherwise, if some coupling exists between data units, the discarded data units will affect the processing of other units in the future, which will complicate the program logic.

 

For a mail, the loss of a mail will not affect the delivery of subsequent mail; of course, it will not affect the delivery of delivered mail.

 

◇ Granularity
As mentioned above, data units must be associated with certain business objects. So do data units and business objects need to be interconnected one by one? In many cases, it is indeed a one-to-one correspondence.

 

However, sometimes n business objects may be packaged into a data unit for performance and other reasons. Then, how to set the N value is a matter of granularity. The granularity is exquisite. A large granularity may waste space; a small granularity may affect time performance. The balance of granularity should be based on many factors, as well as some experience values.

 

Is an example of sending a mail. If the granularity is too small (for example, set to 1), the postman will retrieve only one letter at a time. If there are too many letters, you have to go back and forth for a long time. If the granularity is too large (for example, set it to 100), the sender must wait until 100 messages are collected to put them into the mailbox. If you rarely write a letter at ordinary times, you have to wait for a long time.

 

Some may ask: Can the granularity of the producer and consumer be set to different sizes (for example, set the sender to 1 and the postman to 100 ). Of course, this can be done theoretically, but in some cases it will increase the complexity of Program Logic and code implementation. Later, when discussing specific technologies, everyone should be able to understand the importance of data units.

 

OK. This is the topic of data unit. I hope that through this post, everyone can understand what the data unit is. Next post, let's talk about the technical implementation of "queue-based buffer zone.