Scenario-Based Performance Test Design

The "test case scenario designed for testing purposes" is mainly based on the experience of the test designer. However, it is still necessary to participate in the user's actual scenarios, the actual use case is the basis for designing all test cases. For example, in some business systems, although the historical data backup cycle is one year, the design of a big data test case still contains a one-month, half-year simulation test of the data volume of the system, these are typical scenarios of users.

From the above, we can see that the performance test case design should first analyze the typical scenarios in the user's reality, and then design according to the typical scenarios. In actual projects, analysis scenarios do not analyze a specific type of scenario in isolation. Instead, they combine two or more types of scenarios for analysis and design, in this way, we choose more typical scenarios and save some test costs.

 

Example

The following uses a video-on-demand website shown in Figure 1 as an example to discuss the design details of various test cases one by one. Figure 1 shows the main business and application scenarios of the VOD website.

How to determine the user's system usage

Determining the user's usage of the system is the basis for the specific data of the design case, subsequent concurrent user data design, fatigue strength design, and various scenario design depend on the analysis results of user usage system conditions. To analyze user usage, we often use two methods: on-site investigation and system log analysis. * On-site user investigation. It is actually to communicate with the user to determine the composition of the user's personnel. This method is applicable when the user group is fixed and the target test system is not before production. * Analyze system logs. In many cases, you cannot grasp the details of the system you are using through communication with users, especially the website Business System of Figure 1, because the usage of the system by target users is uncertain. When users are scattered and on-site investigation is difficult, you can use the System Log Analysis Method to supplement the user's on-site investigation information. Most systems perform log management on the user's usage of the system, so logs can be analyzed. The log analysis method is applicable to systems that have been put into production or trial run. In the specific design process, the two methods are generally used in combination. Figure 1 The online VOD system obtains the test data through two methods: by communicating with users, we can obtain the general information about the system used by maintenance personnel throughout the country, then, by analyzing the logs of the system for one month, we can find out the usage of the system by other users. Finally, we can get the usage diagram of the system. Some people may ask: Why does it show all user usage through log analysis? There are two main reasons: First, log analysis may not necessarily generate all usage conditions and may lead to deviations. For example, repeated logon to the system and registration of multiple accounts will affect the statistical results; second, log analysis is often more costly than user research because it involves development work.

Concurrent user quantity Design

The Design of concurrent users, especially the maximum number of concurrent users, has always been a hot topic in many online testing forums. Before designing the number of concurrent users, you must first understand how to determine the maximum number of concurrent users in the system. The following example shows how to estimate the maximum number of concurrent users. The number of users used in an OA system is 600, which is not expected to change significantly during the use period. the maximum number of online users of the system is about 200 based on the analysis logs, and the maximum number of concurrent users is analyzed. Step 1: The maximum number of concurrent users in the OA system is generally 5 ~ Between 20%, the usage frequency of this system is not high, so we estimate that the maximum number of concurrent users is 15% of the total number of users, based on experience, we can conclude that the first maximum number of concurrent users is 90 (600 × 15% = 90). Step 2: Generally, the number of concurrent users in the OA system is about 30% of the number of online users, based on our experience, we can obtain the second maximum number of concurrent users 60 (200 × 30% = 60). Step 3: Compare the preceding results, and finally take 90 as the maximum number of concurrent users. After evaluating the maximum number of concurrent users, you can design the number of users to be simulated in each case.

Table 1 shows that the design of the number of concurrent users is very simple. It is basically designed based on the percentage of the maximum number of concurrent users. For example, you can design the number of concurrent users according to the increase of 20% of the maximum number of users, until the maximum number of concurrent users is reached. Based on the above content, we can see that the number of concurrent users is designed to be flexible. You don't have to stick to formulas and forms, as long as you fully consider the user's current and future growth trends.

Design of the System in different time periods

The scenarios in different time periods are closer to the user's usage, which is also the basis for designing the core module and the concurrent performance test cases of the combination module. For example 1, The on-demand online movie system has different usage situations every two hours. Therefore, we need to design some typical scenarios. The data sources for Scenario Analysis in different time periods are mainly requirement analysis and log analysis results. As shown in figure 1, it is easy to see how concurrent users of different modules in different time periods. With the above information, you can design a combined module test case for each time period. The following figure shows the on-demand online movie System "0 ~ A test case for the 2-Point scenario. The number of concurrent users in the preceding scenario is just an example. For details about how to design the maximum number of concurrent users, refer to "number of concurrent users" and "Business Model Design" in this section. There are two basic principles for Scenario Design in different time periods: one is to select a typical scenario for testing, especially for scenarios with a large number of concurrent users; the other is to cover the entire scenario, that is to say, the designed use case should cover a period of time when the stress may be high.

Business Model Design

The design of the business model is a special case of Scenario Design in different time periods. It is also the basis for designing the core module and the concurrent performance test cases of the combination module, the purpose of designing a business model is to focus on the combination of certain functional modules. Generally, the design scenario by time period involves many modules. If the system has a bottleneck caused by the application software, it is difficult to locate it. Therefore, a specific business model is abstracted for case design. Taking the on-demand video streaming system in Figure 1 as an example, we need to design use cases for system maintenance, movie appreciation, and page query and browsing. When designing performance test cases based on business models and time periods, it involves how to design the number of concurrent users for each module. The maximum number of concurrent users of each related module within 24 hours is usually used in combination. For example, the movie browsing mode is between 12 and 12 ~ The test case designed for scenario is as follows: the maximum value of 280 instead of 210 is still obtained within 24 hours, because the system login user is between 10 and ~ The peak reached 280. This principle seems to be in conflict with the method for testing the maximum number of concurrent users. The actual idea is the same, but here we focus on the maximum number of concurrent users in each business module. It can be seen from this that the design of concurrent users cannot be stuck in the form. Note that the increase in the number of users in the software lifetime is not considered here. You can determine the maximum number of concurrent users based on the previous maximum user evaluation method, then, you can design the number of concurrent users for these two performance test cases.Design of big data test cases

Big Data Testing is mainly divided into two types: Big Data Testing caused by historical data and big data testing during running. The following describes how to design data in a performance test case with a large amount of data. The big data test design related to historical data is mainly based on historical scenarios. Generally, the longest migration cycle of system data is determined first. The use of this period value is a typical scenario. During high-volume testing, you can simulate the large amount of data that may be generated during system running. In Example 2, the Online Video-on-Demand System can simulate the situation where a large number of users download or play movies at the same time. This type of test cases are usually analyzed and designed based on the actual situation. It is relatively easy to use the previous design results when testing a large data volume.

Specific test case design

The design of some special test cases, such as fatigue strength testing, maximum user testing, and Capacity Testing, will be based on the user's needs, because the relevant requirements of such test cases are usually very clear. By designing performance tests through analysis scenarios, you can make the performance test cases closer to your actual usage, making it easier to identify system bottlenecks. This method captures the key points of performance testing and is targeted, greatly reducing the testing cost.

Performance Test Category

Performance testing can be divided into two categories according to different scenarios. One is the scenario testing for the purpose of testing, and the other is the scenario testing based on the actual situation of users. Therefore, the design of performance test cases should be oriented to the performance test scenario.Three Common User scenarios

Use Cases of different time periods in a day. The usage of most systems changes over time within the same day. For example, for portals such as Sina and Netease, there may be a large number of email system users when they arrive at work from Monday to Friday morning, there are a large number of users who browse news before or at noon. Scenarios in different periods of system operation. Scenarios in different periods of system operation are the basis for designing performance test cases with large data volumes. With the passage of time, the system's historical data will continue to increase, which will have a great impact on the system response speed. Scenarios in different business models. The same system may be in different business models. For example, many e-commerce systems mainly use browsing mode from three o'clock P.M. to, and mainly use order mode from, the hybrid mode may be used after three o'clock P.M.

 

From: http://www.cnblogs.com/argb/p/3448651.html

Scenario-Based Performance Test Design