Performance Analysis of queuing theory-little law & Utilization Law

To understand the performance of a system, we usually refer to some metric values (metric). We will discuss how to calculate these metric values. Little Law (queuing theory: liter law) and Utilization Law are common rules of Performance Engineering (part of system engineering). They are both mathematical theories and can be used as the theoretical basis of performance computing. The detailed analysis of the two rules is beyond my personal knowledge, so I will only talk about how to apply them.

Before that, I have written about the performance of the storage system-bandwidth computing, which applies little law and Utilization Law. You can refer to it.

Little law is developed by Philip M. morse proposed in 1954 that the formula is L = λ W, but it was not proved at the time. Therefore, it is assumed that it is correct and has been used by people, philip hopes that the reader can overturn this relationship if it is not applicable. John Little proved in 1961 that there was no such case that it was not applicable to little law, providing theoretical evidence of the correctness of little law.

Little law defines (taking supermarkets as an example) the average number of customers observed over a long period of time in a stable system.L= Valid customer arrival rate observed over a long period of timeλ* Average time spent by each customer in the systemW, That is, L = λ W. This rule applies to any system, or even a subsystem in a system, such as a bank's customer queue. The only requirement is that the system must be stable and non-preemptible.

To better understand little law, consider a small supermarket with only one counter. The counter can only serve one customer at a time. If everyone buys things, the workflow of this system can be simplified: store-> browse-> checkout-> exit, This is a stable system. If the [customer enters the supermarket rate> customer leaves the supermarket rate], that is, arrival rate> exit rate, the system will start to be unstable, because the waiting customer queue will gradually become infinitely long.

• Little law tells us: Average number of customers in supermarkets l = Customer Arrival Rate λ * the customer's stay time in the timeout period W, that is, L = λ W
• Assume that the customer reaches the rate of 10 persons/hour, and each person stays in the supermarket for an average of 0.5 hours, then the number of customers in the supermarket at any point in timeL = 10*0.5 = 5 persons
• Assume that the customer is starting to flock to 20 people/hour. If the customer's stay remains unchanged for 0.5 hours, the supermarket must be able to accommodateL = 20*0.5 = 10 personsOtherwise, five more people will have to wait outside the supermarket. If the supermarket cannot be expanded temporarily, it can only accommodate five people. In order to make the five more people do not have to wait outside the supermarket, customer stay time must be shortenedW = 5/20 = 0.25 hours.

To sum up, in the case of an increase in traffic, in order to ensure the stable operation of the system (maintain an acceptable queue length and thus ensure an acceptable response time ), that is, supermarkets can guarantee the ability to serve five customers at any time point. According to the little law Law, there are two solutions:

• 1. Expand the supermarket capacity (build a larger supermarket)
• Second, reduce the customer's stay time (improve counter efficiency, such as increasing the number of counters and training counter staff)

========================================================== ================

Performance is not mentioned at all. Don't worry. If we apply the same example to the disk system, that is:

L maximum number of requests allowed to ensure stable disk operation l

L I/O Request reaches speed λ

L disk processing time of each I/O W

Therefore, the maximum number of I/O Requests allowed to ensure the stable operation of a disk L = the speed of I/O Requests x the time it takes for the disk to process each I/O W, that is, L = λ W. Similarly, for an I/O controller, the arrival rate must be <service rate, or service time must be <internal arrival time, otherwise, the processing capability of the I/O controller cannot meet excessive I/O requests, which will inevitably lead to performance degradation.

Combined with Utilization Law (not introduced, directly applied, it can be used to describe the I/O controller utilization), the formula is u = λ * rs

L u = I/O controller Utilization

L RS = service time, that is, the average time for the Controller to process an I/O. For disks, service time rs = seek time + rotation delay + internal transmission rate (the rate at which data is transmitted from a single track on a disk to the buffer), which is usually a fixed value, depends on the physical characteristics of the disk.

L λ = arrival rate

Little law + Utilization Law can be used to derive the following formula (the derivation process is omitted and used directly)

L average response time r = RS/(1-u)

L average queue length NQ = u ^ 2/(1-u)

With the above formula, we can consider such a disk system, where λ = 100 per second and rs = 8 ms.

L disk utilization u = RS/Ra = 8/10 = 0.8 or 80%

L response time r = RS/(1-u) = 8/(1-0.8) = 40 ms

L average queue length NQ = u ^ 2/(1-u) = 0.8 ^ 2/(1-0.8) = 3.2

L the waiting time of a request in the queue = [u * RS] or [Response Time-service time] = 40-8 = 32 Ms

If the Controller's processing capability is doubled, the service time and utilization will be halved, RS = 4 ms, u = 40%. In this case, the response time r can be greatly reduced. Similarly, if the processing capability is halved, the service time and utilization will greatly increase. There is a very important concept here, that is, we often mention that as the utilization rate increases to a certain point, if it continues to rise, the response time will increase exponentially, that isR and u are not linear., Let's analyze [response time r = RS/(1-u)] to see why:

Average response time r = RS/(1-u), RS is a fixed value. It can be seen that when u = 1, that is, when the controller is saturated, [response time r] approaches infinity, this is a Limit Concept. When U tends to be 1, RS is a constant, so r tends to be infinite. Therefore, a controller with a 100% utilization rate is the bottleneck, which forces I/O serialization (I/O serialization ), that is, each I/O must wait in the queue until the I/O in front of it is processed. Once the queue is infinitely increased (buffer usually has a control mechanism to prevent the infinite growth of the queue, such as fiber channel BB credit, TCP window, Ethernet pause, etc.), the response time will rise sharply.

We usually think thatWhen the utilization rate reaches 70%, the future growth will cause the performance to decline exponentially, rather than linearly.. Why is it non-linear? You can try to draw a functional image of R = RS/(1-u). When U tends to 1 gradually, r tends to be infinite, the vertical gradient is x = 1. However, I don't understand how 70% is calculated. It may be that the derivative of U in the range of [0, 1) is determined by comparing the slope change rate? In fact, when u is set to 0.9, 0.99, 0.999 ,...., at 0.9999999, you will find R = 10rs, 100rs, 1000rs ,...., RS and R increased exponentially. When the U value is 0.1, 0.11, or 0.111, the R increase is much slower. As the value of U increases, the increasing trend of R also increases in a non-linear manner. 70% may be calculated by means of mathematics, or it may be a relatively good experience. If you are good at mathematics, you can explain it.

Performance Analysis of queuing theory-little law & Utilization Law