Background Performance Test summary-Test preparation

In the past six months, I have participated in or independently of the project team version of the partial performance test, slowly have some understanding, temporarily do a accumulate, and we do an exchange

In the past six months, I have participated in or independently of the project team version of the partial performance test , slowly have some understanding, temporarily do a accumulate, and we do an exchange

　　The requirement background for performance testing typically comes from the following three scenarios:

The first is the performance of the present network, the project team dedicated to the performance of the transformation. For example, a modified interface, modified by the original synchronous call to asynchronous, or replace the new API, the TCP protocol modified to the UDP protocol, in order to ensure the reliability of the newly replaced API, all need to conduct performance testing

The second is a new system, the operator needs a comprehensive pulse to understand the processing capacity of the system.

The third is the rapid increase in demand, and the system has never done performance testing, the team worried that the system will not be able to withstand the foreseeable short-term, it is required to test personnel to conduct a comprehensive performance test, give a reference data

Depending on the background, we tend to have different ways of preparing, but we can generally get ready from the following aspects.

1, a comprehensive understanding of the system overview

(1) The expected performance index of the system:

For the first 122 cases, there will be a very clear current network performance indicators, such as the previous testing of the ACS, is a new system, the requirements of the specification clearly marked the need to reach 1wtps. And for the third situation, often we need to try to simulate the current network, to obtain data for the operation of reference. For example, the recently tested query throttling engine, which gives the processing power of a single SVR and whether it supports operations such as parallel scaling, is the most concerned data.

(2) The form of communication between the network and the various systems between the networks:

Sometimes our performance transformation is just a small system in the network, which requires us to figure out where the system is in the entire logic processing.

Figure 1

Understanding the location of the tested system throughout the transaction is critical to the design of test Cases and the construction of test environments.

Secondly, it is necessary to understand the communication mode between each module in the network, TCPORUDP, synchronous call or asynchronous call, long connection or short connection.

(3) Various logical branches of the system:

Understanding the logical branches of the system is primarily useful for designing test cases. In our actual work process, time is always very limited, and we improve the efficiency of a very important way is to pay attention to the design of use cases, understand the system of the various logical branches, can be very precise preparation of use cases, directly hit the nature of the problem, reduce the time to explore.

For example, the PSC System Performance Improvement version (1), almost all the business logic to go to the SSP query is limited, but we choose one of the simplest by the peripheral system of the smallest two-level free branch to test, with the shortest cost verification problem, very good to ensure the progress of the test

(4) Combination of internal modules of the system:

More complex systems, will have their own modular combination of methods. We need to understand how the system consists of several modules, the coupling of the various modules, not only for the functional test of the exception test case design is very helpful, for the performance testing of the help is also not to be underestimated.

To give a relatively simple example: AQC system, the system is for external query, the internal module is broadly divided into: network communication layer, request distribution layer, function processing layer. Network communication layer is mainly the use of a network communication components, processing network communications, the request distribution layer dispatch, mainly the network communication Layer queue packet according to the different Cmdcode distribution to the back-end Function processing layer, the function processing layer has a small SVR, each SVR processing a different query request.

If there is a performance requirement is to find that the current network has a query branch performance is not OK, then we need to quickly lock the key module, the bottleneck is likely to exist with the processing of this branch of the SVR.

Secondly, it is important to understand the performance curve and adjust the test scheme when we understand the coupling relationship between the modules and the modules.

2, step on the key points, further deep digging system

System performance indicators, in addition to the most typical indicators that throughput or response time, there are many other indicators we need to pay attention to, such as CPU, memory, IO, database connection operation and so on, so we need to further deep digging system before testing, to find the following key points:

(1) Memory allocation and use. Use of Message Queuing, use of caching

(2) file, network IO operation: Large file read to memory, or memory write file, whether the operation is frequent

(3) CPU-intensive operations: for example, some large memory sequencing

(4) Database operation: Frequent database read and write operations, frequent database connection and so on

(5) Network call: Network delay and the concurrency of the connection

(6) Critical resources: whether there is an improper locking behavior in the multi-process processing mode (7) 3, the design test case to understand the above basic situation, in fact, is to prepare for this step. In solving the first situation

(6) Critical resources: whether there is an improper locking behavior in multi-process processing mode

(7) ...

3. Design Test Cases

Understanding the above basic situation, in fact, is to prepare for this step. It is particularly labor-saving in addressing the performance requirements of the first situation, and experienced performance testers can even determine where the problem is without having to design tests. (Lisa ventured to speculate, though not yet to reach that level)

Test cases of performance testing are not more than functional tests, you can consider a lot of exception testing, because the cost is very small, and the performance of a one-time test case will often need to consume a large amount of resources and times, so the accurate test case is particularly important.

The performance test case design, according to Lisa this period of time accumulation and summary, feel can start from the following aspects.

(1) Select the most appropriate logical branch for testing

Often there will be many branches to meet your test requirements, select the time, you can start from the following two points: A. Affected by the surrounding minimum, B, the least resources consumed.

A point can help us to locate the problem quickly, perhaps the entire logical branch design of the system will be a lot of, we can choose to involve the smallest peripheral system but can contain the system, including the measurement of the branch, of course, the simplest way is to directly test the test system, but there are some problems are not exposed.

b Point can help us to save resources, such as already have a ready environment, in short, I need to prepare for the reduction of things, naturally faster and more efficient, and for the new system or have never conducted performance testing of the old system, in the case of limited time, we also need to combine the actual situation, Select the most important branch that the user requests most to test.

(2) According to the previous analysis, design the most targeted, most effective test cases

For example, I suspect that the cause of AQC system performance degradation is the addition of large memory sequencing for this iteration. For example, the AQC system has a branch that queries all the user's cdkey and then sorts them back to the user. So how do we get this problem verified? At design time can choose two extreme situations extremely combination to test, one is that all users have no cdkey, the other is all users have 500 cdkey, the last one is a combination of the two, By comparison, you can verify that the overall performance is degraded by the occasional cdkey of some users.

Of course, during the testing process, we also need to adjust the test cases according to the existing data, help us to verify the conjecture, clearer positioning problem

(3) Consult experienced performance testers who are often pleasantly surprised.

In the case of limited experience, to begin to deal with the former and the elders to ask, not only can improve work efficiency, but also can grow. But this is probably something that many people ignore. Task to come, do not rush to start, first ask often can expand ideas.

4. Preparation of test environment

The test environment is often built according to the design of the test case. For the first performance testing of the system, our goal is as far as possible and the current network consistent. Can be mainly from the following several aspects of the beginning.

(1) Data: Large data volumes and the diversity of data are often difficult to simulate. Large amounts of data need to write their own scripts to fill the database to a certain extent, if the requirements are high, you can even use the method of data guidance from the current network. Diversity is often more difficult to achieve, need to understand the current network of data diversity and proportions, to achieve the simulation effect

(2) Network latency: This is very relevant to the company's IDC machine management. I had always thought that all the test machines were in an IDC, and later found out that our test machines were distributed in different IDC, just like the operating machines, and that we needed to know the network latency between the existing network operating machines when selecting the machines to deploy. This is especially important when testing the performance of a single logical branch.

(3) Configuration: Log level configuration, number of threads or processes, if the conditions allow, configuration can be upgraded to the machine's hardware configuration, if it can be consistent naturally is the most desirable effect.

Here is a misunderstanding, I have always thought that the best every test environment as far as possible to go with the current network, but after listening to the Yuetangdeng class, found that IBM did not do so, for the previous performance testing system, often that set of environment still exist ( Regardless of whether the environment is consistent with the current network, and the test we are more to verify the system is a performance problem, think with the previous test environment is the same situation, the new iteration version after the replacement of the system performance significantly reduced, can not explain the problem?

After everything is ready, then we can start preparing test tools to execute our test Cases ~ ~ ~

Transferred from: http://www.uml.org.cn/Test/201308025.asp

Background Performance Test summary-Test preparation