One of LoadRunner's not telling you-descriptive statistics and performance result analysis

Copyright Notice: This article can be reproduced, but it shall not be used for any commercial purposes or other purposes for profit without my permission. I reserve all rights to this article. If you need to reprint the content, Please retain this copyright statement and ensure the integrity of this article. You can also understand the hard work and respect the work of the author.

Author: Chen Lei(Jackei)

Email:Jackeichan@gmail.com

Blog:Http://jackei.cnblogs.com

 

LoadRunnerIn90% What does response time mean? What is the role of this value in performance analysis? This article tries to use the most concise text to answer this question, and draws out the application of the "descriptive statistics" method in performance test result analysis.

 

Why?90% USER response time? This is because it is not enough to have an average transaction response time when evaluating the results of a test. Why? You can try to think about whether the average transaction response time meets the performance requirement, which means that the system performance has met the requirements of most users?

if there are two groups of test results, the response time is {1 , 3 , 5 , 10 , 16 } and {5 , 6 , 7 , 8 , 9 } , Their average values are 7 what test results do you think are more satisfactory?

For example, if one test is performed, a total100Requests are responded. The minimum response time is0.02Seconds, the maximum response time is110Second, the average transaction response time is4.7Seconds, do you think that the deviation between the minimum and maximum response time will cause the average itself to be untrusted?

To answer the above questions, let's first look at a table:

The table above contains several different columns, which have the following meanings:

Region IDPage requested during test

NumNumber of successful responses

MeanAverage response time of all successful requests

Std DevStandard deviation (the function of this value will be in the nextArticle)

MinMinimum response time

50 th (60/70/80/90/95 th) If the response time is sorted in ascending order, then 50 % of the request response time is within this range. 60/70/80/90/95 th the same meaning

MaxMaximum response time

I want to read the explanations of the table and columns above. You can understand what I mean without much talk. I will sort out the conclusive items:

1. 90 % USER Response Time: LoadRunner is configurable. You can change it to 80 %or 95 %;

2. for this table, LoadRunner is not provided directly, you can set LR export original data to Excel , and use Excel percentile the function is simple to calculate the Response Time Distribution of user requests of different percentages.

3. As shown in the preceding table Home Page For example, the average Transaction Response Time (Mean) Only 70 % The user response time is consistent. That is to say, if we are sure Home Page The response time should be in 5 Within seconds, the average transaction response time is satisfactory, but there are actually 10-20 % Of user request response time is greater than this value; Page 1 The same is true Page 1 The request must be in 3 Response within seconds. Although the average transaction response time meets the requirements 20-30 % Of user request response times exceed our requirements;

4. you can go to 95 Th continue to add 96/97/98/99/99.9/99.99 th , and use Excel the chart function draws a curve to show the distribution of system response time more clearly. At this time, you may find that the maximum value is only one in 1‰ or even one in, and 99 % of user request response time is within the scope defined by performance requirements;

5. if you want to use this method to evaluate the system performance, A recommended practice is to keep your test scenario running longer as much as possible, because the more test data you obtain, the closer the distribution curve of the response time is to the actual situation;

6. when determining performance requirements, you can use the average transaction response time to measure the system performance. Use 90 %or 95 % USER response time is used as the measurement standard, and they do not conflict. In fact, when defining performance requirements for some systems, request failures within a certain range are acceptable.

7.The content mentioned above is not related to the tool. As long as you can get the original response time recordLoadRunnerOrJmeterOrOpenstaYou can use these methods and ideas to evaluate the performance of your system.

 

In fact, there are more things in the Performance Testing field that are not specifically described by commercial or open-source testing tools-in other words, performance testing is not enough to have only tools. We also need more knowledge in other fields, such as mathematics and statistics, to help us better analyze performance data and find the truth hidden under those data.

Welcome to fellow experts for bumping bricks and shoot bricks ^_^

 

In the future, please continue to close LoadRunner's second question --STD .(Standard Deviation) InPerformance Testing and Analysis.

 

 

Click here to learn about the creation progress, view the document directory, or browse completed articles.