A wonderful comparison of throughput, latency, Semaphore, and mutex lock

We know that there are many concepts in computers that are not easy to understand. sometimes, a good analogy can beat many explanations. The following are two wonderful metaphors I have seen. let's share them with you. The first analogy is about throughput and latency.

We know that there are many concepts in computers that are not easy to understand. sometimes, a good analogy can beat many explanations. The following are two wonderful metaphors I have seen. let's share them with you.

The first analogy is about throughput and latency. If you want to optimize the network performance, you must know these two concepts. they seem simple, but they are not. I believe that many people, including me, once thought that high throughput means low latency, and high latency means lower throughput. The following metaphor can be used to explain that this viewpoint is totally incorrect. This metaphor comes from here. I will make a general translation (non-verbatim translation ).

We can compare the packets sent over the network to the ATM for money on the street. It takes one minute for each person to use the ATM from the beginning to get the money. so the delay here is 60 seconds. what about the throughput? Of course it is 1/60 people/second. Now the bank has upgraded their ATM operating system, and every user can withdraw money in 30 seconds! The latency is 30 seconds, and the throughput is 1/30 persons/second. Well understood, but the previous problems still exist, right? Don't panic. let's take a look.

Because there are a lot of people who come to get money nearby, now the bank has decided to add an ATM here. There are two ATM machines in total. Now, four people can get the money in one minute. it will take 30 seconds to get the money in front of the ATM even if you go to the queue to get the money! That is to say, the delay has not changed, but the throughput has increased! It can be seen that the throughput does not need to be increased by reducing the latency.

Now, the bank has made a new decision to improve the service: each customer who receives the money must enter a questionnaire next to the customer after obtaining the money, which also takes 30 seconds. Now, if you get the money, it will take 60 seconds from using the ATM to completing the questionnaire! In other words, the delay is 60 seconds. Throughput has not changed! Four people can come in within one minute! It can be seen that the latency increases while the throughput does not change.

From this metaphor, we can see that latency measures the length of time that each customer (each application) feels, while throughput measures the processing efficiency of the entire bank (the entire operating system, is two completely different concepts. In the author's original words:

In short, the throughput is a function of how many stages are in parallel while latency is a function of how many are in series when there are multiple stages in the processing. the stage with the lowest throughput determines the overall throughput.

Just as banks do not only need to improve their own efficiency to satisfy their customers, but also try to shorten their time spent on banking operations, the operating system should not only make the network throughput as high as possible, in addition, the latency of each application to send data should be minimized. These are two different goals.

Another analogy is to explain the difference between semaphore and mutex. This metaphor was originally from here. I first translated it and then improved it.

A mutex is the key to a public toilet. A person can get the key when using the toilet, and then give it to the next person in the queue.

Semaphores are the number of keys in the restroom that no one uses. all the keys in the restroom are the same. For example, four toilets have the same keys and locks. The semaphore value is the number of keys, which is 4 at the beginning. When one person comes in, the number will be one less. if four toilets are full, the semaphore will be 0, and 1 will be added when one person goes out, and the key will be handed over to the next person in the queue.

This metaphor is not very good, especially it cannot explain the difference between binary semaphores and mutex locks! I have improved this metaphor. The mutex lock is similar to the above. it should be noted that when you get the key, you are its owner, and others cannot open the toilet door.

What is the Semaphore? It is a large public toilet with several locations in it. there is a sign that can be turned on at the door of the outside, saying "full" and "available". when there is still a blank location in it, the person in the queue does not need to flip this sign until the last person in the queue has to set it to "full" when there is no position, then the person who goes out must turn the sign to "available", if necessary.

Well understood, right? So how does it explain binary semaphores? That is to say, when the restroom can only accommodate one person, everyone should turn the sign at the door to "full" and "available" when going out ". The difference between the lock and the mutex lock can be seen immediately. The brand of the flip can be turned over by others outside, and the lock can only be opened by those who take the key!

Of course, the reason why semaphores are translated into "signals" is still reasonable, because it (the sign at the toilet door) indicates the state of resources (toilet space), and mutex lock is the lock, it actually locks resources. This is more obvious in the case of producers and consumers.