Go: Performance test Case design strategy

Performance testing plays an important role in software quality assurance, and it includes a variety of testing content. The same system, different test design and testing process will lead to different results, there will be different interpretation. Reasonable test planning and design is of paramount importance. This paper focuses on how to develop effective performance test cases in combination with the actual business characteristics of the users.

First, the system business characteristics and user behavior analysis

User behavior reflects the user's use of the system and the application of the background, before the performance test, we need to analyze the user's usage habits, determine the typical business and the time of occurrence of the system. Analyzing user behavior is the first step in designing a performance test case.

1. System Usage Peak time analysis

For many large-scale systems, there is a business focus on the use of the situation, that is, the peak use of the system. This type of system usage spikes can occur over a day, a month, a point in the year, or a time period. For example, in the same day, the usage of most systems changes over time, and the time for different business peaks for this type of system is different. For example, Sina, NetEase and other portals, in the Monday to Friday morning just a job, may be more mail system users, and before work or at noon break time to browse the news users more, and for the general OA system in the morning to read more announcements, Other times may be that many people are not using the system or only a small number of secretaries or leaders are drafting and approving documents. The telecommunications payment system is likely to receive a centralized fee at the end of the month. The special assistance of the Family Planning Commission is generally focused on a particular month of the year.

Determining the peak time of the system can help us to further determine the performance requirements of the system during peak hours, such as the demand for concurrent user support during peak periods, and the demand for business response time during peak periods.

2, System peak business application analysis

At different times of the system, the same system may be in different business modes, so it is necessary to analyze the business application at the peak of the system. The goal is to determine whether the peak business application model is a single business type or a hybrid business type by analyzing peak applications, which is critical to the design of the late test case execution strategy. For example, many e-commerce systems in the morning 8 to 10 to browse the mode of the main, 10 to 3 o'clock in the afternoon in order to the main mode, and after 3 o'clock in the afternoon may be dominated by mixed mode. Therefore, it is necessary to analyze which business applications are typical and stress-heavy business, and then test these business applications separately, so that the system bottleneck can be effectively isolated and located.

Second, the System Performance Index analysis

Reasonable design can not be imagined, but based on the business needs of the system and user habits. In the performance test, the most important two indicators are to determine the number of concurrent users the system needs to withstand, and at a certain user size, the system can provide the application response time.

The following highlights how to design two performance metrics for the number of concurrent users and average transaction response time:

1, concurrent user number design

The number of concurrent users must be calculated based on the maximum number of users that may appear in the real-world use of the system. Here's how to evaluate the maximum number of users based on the maximum number of people in the system or the maximum number of people online.

A) Limit method

For the system has been put into production or the target user group is unsure of the portal site, you can analyze the log, you can also use the number of registered users of the system as the maximum number of users, and then the empirical formula to estimate the maximum number of concurrent users.

b) User Trend analysis

This paper analyzes the future trend of users in the software life cycle, predicts the maximum number of users the system can reach, and estimates the maximum number of concurrent users of the system, which is used for the increasing number of system users.

c) Empirical assessment method

Based on experience, the maximum number of concurrent users of a system can be evaluated, and this method is used for systems with relatively stable and well-defined user numbers.

The design of concurrent users is basically based on the percentage of the maximum number of users of the system, and for a particular use case, it is important to note that the maximum number of concurrent user designs typically does not exceed 30% of the maximum number of users actually used by the previously computed system, unless it is to test the maximum concurrent users that the system can support.

2. Average response time of the transaction

The response time is the time that the user feels the software system spends on its service, which is related to the computer performance, network speed, bandwidth and so on. The average response time index of design transaction is also an important content of performance test case design.
At present, the average response time index design of transactions can be referred to the following 2 ways:

For systems that have explicitly raised response time performance requirements during the requirements analysis and design phase, such as requiring "system response time of not more than 20 seconds", the average response time should be determined on demand.

For systems that do not have a clear performance requirement, the average transaction response time should be based on user perception or demand-side designation. In general, there are 3 seconds within the user will think the response time is faster, 5 seconds or less users think acceptable, more than 8 seconds, the average user will think the response is too slow. Of course, the length of response time is closely related to the type of business, and the response time requirements of the submitted class business operations are generally higher than those of the statistical class.

Other performance indicators can be designed in the same way as the average response time metrics for transactions.

Three, performance test execution Strategy analysis

Performance test execution strategies need to be aware of two points: first, choose a typical business to test, especially to select a large number of concurrent users of the business, the second is to cover the overall, that is, the design of the use case to cover the peak of the system's main business. During performance test execution, it is important to identify the number, scale, and specific behavior of each business participant, which is the basis for the performance test case design. The following highlights how to develop a specific execution strategy for single-business, hybrid-business, and fatigue-strength testing.

1. Single Business test

Performance testing is not likely to test all features, so it is necessary to abstract some specific independent business to design the use case. Independent business actually refers to some core business modules corresponding to the business, these modules often have more complex functions, use more frequently, belonging to the core business and other characteristics. Performance testing for this type of independent business is called a single-service performance test.

User concurrency test is the focus of single business performance testing, user concurrency testing refers to the simulation of a certain number of users at the same time using a core module of the same or different functions, and for a period of time, the system can support the specified user size.

2. Hybrid Business Testing

In real-world applications, there is usually no use of one or several core business modules for all users, that is, every functional module of an application system may be used, so the performance test is to simulate the same operation of multi-user, but also to simulate the different operation of multi-user.

Hybrid business Performance testing is the test that is closest to the user's actual usage and is a necessary part of the performance test. In hybrid business testing, it is often necessary to simulate the combined concurrency of each module according to the user's actual usage ratio. The distinguishing feature of hybrid business testing is that according to the situation of user using the system, the user group is divided into different groups, and the proportion of each group is matched according to the actual situation, and the maximum number of concurrent users of each related module is usually combined.

3. Fatigue Strength test

Fatigue strength testing is the test of a long-running system with a certain load pressure under the condition of normal operation of the system. The main characteristic of the fatigue strength test is the pressure of the target test system for a long time, the aim is to test the stability of the system, the duration is generally more than 1 hours; the fatigue strength test belongs to the continuation of the user concurrent test, so the core content is still the core module user concurrency and the combination module user concurrency, When writing test cases, you need to write multiple test cases under different parameters or load conditions, you can refer to the design content of the mixed business concurrency performance test case, and usually modify the corresponding scene parameters to implement the required test cases.

Summarize

This paper focuses on how to set up effective performance test cases in combination with the actual business characteristics of the users, and to design performance testing by analyzing the actual business characteristics of the users, so that the performance test cases are closer to the actual usage of the users and the system bottleneck is more easily discovered. This method captures the key points of the performance test, achieves the target, and greatly reduces the testing cost.

Go: Performance test Case design strategy