One of LoadRunner's not telling you-descriptive statistics and performance result analysis

In LoadRunner, what does the 90% response time mean? What is the role of this value in performance analysis? This article tries to use the most concise text to answer this question, and draws out the application of the "descriptive statistics" method in performance test result analysis.

 

Why is there 90% user response time? This is because it is not enough to have an average transaction response time when evaluating the results of a test. Why? You can try to think about whether the average transaction response time meets the performance requirement, which means that the system performance has met the requirements of most users?

If there are two groups of test results, the response time is {1, 3, 5, 10, 16} and {5, 6, 7, 8, 9} respectively, their average values are 7, which of the following tests do you think is more satisfactory?

For example, in one test, a total of 100 requests were responded. The minimum response time was 0.02 seconds, the maximum response time was 110 seconds, and the average transaction response time was 4.7 seconds, do you think that the deviation between the minimum and maximum response time will lead to the untrustworthy average itself?

To answer the above questions, let's first look at a table:

 

 

The table above contains several different columns, which have the following meanings:

Region IDPage requested during test

NumNumber of successful responses

MeanAverage response time of all successful requests

Std DevStandard deviation (the role of this value will be highlighted in the next article)

MinMinimum response time

50 th (60/70/80/90/95 th)If the response time is sorted in ascending order, 50% of the request response time is within this range. The following 60/70/80/90/95 th also means the same.

MaxMaximum response time

I want to read the explanations of the table and columns above. You can understand what I mean without much talk. I will sort out the conclusive items:

1. 90% the user response time can be set in LoadRunner. You can change it to 80% or 95%;

2. This table is not directly provided in LoadRunner. You can export the raw data in lr to excel and usePercentileThe function is simple to calculate the Response Time Distribution of user requests of different percentages;

3. From the preceding table, for home page, the average transaction response time (mean) is only the same as the response time of 70% users. That is to say, if we determine that the response time of the home page should be within 5 seconds, the average transaction response time is satisfactory, but in fact, the response time of 10-20% of user requests is greater than this value. The same is true for page 1. If we are sure that the request for page 1 should receive a response within 3 seconds, although the average transaction response time meets the requirements, the response time of 20-30% of user requests exceeds our requirements;

4. You can add 96/97/98/99/99.9/99.99/Th after 95 th, and draw a curve using the Excel chart function to show the distribution of system response time more clearly. At this time, you may find that the maximum value is only one thousandth or even one thousandth of the occurrence probability, and 1‰ of the response time of user requests is within the range defined by performance requirements;

5. if you want to use this method to evaluate the system performance, it is recommended to keep your test scenario running longer as much as possible, because when you obtain more test data, the closer the distribution curve of the response time is to the actual situation;

6. When determining performance requirements, you can use the average transaction response time to measure the system performance. You can also use 90% or 95% user response time as the measurement standard. They do not conflict with each other. In fact, when defining the performance requirements of some systems, request failures within a certain range are acceptable;

7. the content mentioned above is not related to the tool. As long as you can get the original response time record, whether it is using LoadRunner, jmeter, or opensta, you can use these methods and ideas to evaluate the performance of your system.

 

In fact, there are more things in the Performance Testing field that are not specifically described by commercial or open-source testing tools-in other words, performance testing is not enough to have only tools. We also need more knowledge in other fields, such as mathematics and statistics, to help us better analyze performance data and find the truth hidden under those data.

Welcome to fellow experts for bumping bricks and shoot bricks ^_^

One of LoadRunner's not telling you-descriptive statistics and performance result analysis