Six major steps for Application System Performance Testing

The performance test aims to ensure that the performance of the product can meet the needs of users after it is released. This article introduces the six major steps of the application system performance test based on specific cases.

In the case described in this article, the tested application system is a company's customer information system, which is mainly used to input, modify and query information of global customers, and transfer customer information to the business system. However, during the application process, customers often complain that it is very slow to create or modify a customer information at a specific time point. Normally, only 1 ~ is required to complete this operation ~ 5 seconds, but it takes about 10 minutes for the exception to occur, and the system administrator finds that the system resource usage is relatively low, which is not regular.

In this case, we found that the system has performance problems. The next step is to find out the root cause of system bottlenecks and faults during the performance test, in this work, we should follow the steps below.

1. determine the specific test objectives

Performance Tuning is endless. Therefore, a clear performance tuning goal should be determined before the test. This is also a benchmark for "Evaluating Performance Verification" and a benchmark for testing termination. In this case, the goal is to set 30 concurrent users in the same system environment to 1 ~ Complete various online operations within 5 seconds.

2. Test Requirement Analysis

The purpose of Performance Tuning Test Analysis is to find out factors that may cause system bottlenecks and provide assurance for subsequent test case design. There are many causes for systemic impact. The following key points should be noted:

• Application configuration requirements: such as the overall application framework, third-party components involved, interfaces at the application layer and database layer, and databases used.
• System Configuration Requirements: for example, user client configuration, client-server network configuration, application server or database server operating system.
• User usage requirements: for example, user distribution, which modules are frequently used by users, and what features are there in user-operated data.

This work is complex and requires testers to have insights and sensitivity to potential system bottlenecks. However, after taking over the test, many testers, it soon entered the test case design stage. Practice has proved that this is often counterproductive, either due to a delay in the construction period or project development failure. This process is a critical part of the overall testing process. Performance test analysis has the following characteristics: it focuses on the entire application, or carefully analyzes various external factors around the application, such as the hardware and third-party software involved in the application, it does not go deep into the specific interior of the project. This is because Performance Testing focuses on the entire project and a black box testing method. We are concerned about the problems exposed during the overall running of a project. In this case, we obtain the requirements shown in the table.

3. Test Case Design

The main purpose of this process is to design some reasonable scenarios to verify whether the factors that may affect the performance obtained in the demand analysis stage are factors that cause system bottlenecks. Test Case Design generally includes test strategies, test cases, and test content.

Testing strategies generally include comparison and testing environments and real business operation environments. Real business operation environments may also involve LAN test environments and data center test environments.

The test case is mainly based on the results of the test requirement analysis to develop the system execution method during the test execution. For example, in this case, "five people simultaneously enter different new customer information, and the specific simulation steps ". Pay attention to the following points when designing test cases:

• The procedure of a virtual user should be similar to that of a real user.
• The operation data should be similar to the actual use data of real users. For example, when a user enters customer information in a case, we can design 3 ~ Four virtual users are entering some small customer information, 1 ~ Two virtual users are entering information about major accounts.
• The use frequency of modules should be fully taken into account in case design. In this way, the usage of each module is similar to the actual environment as much as possible during simulation.

The test content includes concurrency performance testing, fatigue strength testing, big data testing, and system resource monitoring, we mainly perform concurrent performance tests and system resource monitoring for performance tuning tests. Monitors the changes of various resources in the system during the concurrent performance test to determine the cause of the performance bottleneck.

4. Prepare, test, and monitor the Script Development Data

The main purpose of test execution and monitoring is to verify whether the system has bottlenecks based on the design scheme and provide various analysis data for test analysis. This process repeats with the following "Test Analysis" process until the system bottleneck is determined.

The author believes that if a testing tool can meet the testing requirements in this process, try to use the testing tool instead of developing the testing program manually, because it is usually time-consuming to do enterprise projects, after all, the test tool is a mature product and has been verified in all aspects. Using tools will shorten the test cycle, and there are many mature test software available on the market. For example, LoadRunner of mercury, robot of IBM, and qaload of compuware. In this case, the author uses mercury's LoadRunner. I will not go into details about some technical details. Here I will mainly mention two points.

First, data preparation. Data preparation must focus on the quality and quantity of data, and avoid waste data that does not conform to the business logic, and the data volume must meet the needs of testing and running. For example, a test requires 100 groups of data, but only 50 groups are actually prepared, resulting in a large deviation in the test execution results.

Second, test execution. In addition to correctly setting various parameters according to the design requirements, you should also pay attention to the existence of functional problems in the system, which is often a blind spot in performance testing ". In principle, you must ensure that the function test is complete before the performance test, but nothing is absolute. Therefore, at the beginning of the performance test, a scenario where the user runs the design will be simulated, it mainly ensures "test script correctness" and "the application system does not have functional problems in the design scenario ". Many performance tests often fail or are extended due to functional problems.

5. Test and Analysis

The main purpose of the test analysis is to determine the location of the System Bottleneck based on the data obtained by the test execution, and find out the real cause of the System Bottleneck. This process is the most technical part, because the data obtained during the test execution process involves various aspects, this case covers network, system, and application knowledge. Testers need to pick out exceptions from these complicated data, the larger the system, the more complicated it is, the higher requirements on testers. However, there are also some tips:

• For testing and analysis, the personnel composition suggestions include: developers, system personnel, and testers. This will fill in technical deficiencies of individuals. In reality, the technology involved in each project may vary, and it is impossible for individuals to be proficient in everything.

• Compare the hop values of a type parameter at different times or the changes of the same type parameter in different scenarios.
• When detecting abnormal changes in a parameter, do not draw conclusions easily. Instead, try to mine other parameter values that may affect this parameter. During long-term testing, we often find that the first so-called bottleneck is caused by other factors.
• One of the most commonly used methods for testing and analysis is division. Based on the information obtained, the problem can be located at a certain layer. However, you can use the exclusion method to locate the specific location.
• Try to use some mature tools to assist in analysis, which will greatly reduce the workload.
• When determining the real performance bottleneck, you can do some small test models to verify and determine the correctness of the analysis.

In this case, after the test results are compared, it is determined that there is a problem at the database layer. But what exactly is the problem? During the analysis, the author adopted many exclusion methods, such as updating the index statistics and changing the table in the database from the page-Level Lock to the row-Level Lock.

Therefore, let's go back and look at the requirements related to the database layer:

1. Because of business needs, many fuzzy queries are required. This operation scans the table. To prevent dirty reads, intention locks are applied to the database application form.

2. Because the customer relationship is complex, there is a multi-Table association during database design.

3. during application development, we used the hiberate component, which is a black box for developers and has some limitations: all associated tables will be updated synchronously during record update, even if the associated table does not need to be updated, the synchronously updated record operation will cover a transaction processing process, which will produce large transaction operations. The SQL optimization technology cannot be used to optimize the SQL statements it generates.

After the study, we found that during the fuzzy query and Key Account Information Input and modification operations, the large transaction SQL generated by the hiberate component causes the database to interlock, which is the bottleneck of the system. In order to verify the correctness of this judgment, I have made a small model to verify.

Assume that there are three tables in the database: A, B, and C. Now there are three virtual Users performing operations on the table at the same time: User vuser1 needs to query the customer information. He only knows the customer's last name, therefore, he used a fuzzy query. User vuser2 is modifying a customer information and is preparing to save it. User vuser3 is querying the customer's office information, and fuzzy query is also required.

The vuser1 operation is executed first, and the table-level intention lock appears in the table scan. In this case, vuser2 needs to modify the corresponding records of tables A, B, and C, the row corresponding to table B and Table C (because it is a row-Level Lock) is successfully locked, and then modified, but Table A cannot be modified. Therefore, vuser2 waits for vuser1 to release the lock; now vuser3 is in, and Table C needs to be queried. Because vuser2 does not release the lock, vuser3 is also waiting. Verification shows that this bottleneck is constantly exposed in the case of a large number of operations and multiple users.

6. System Optimization and Verification

Deliver the obtained analysis data to the development team for optimization. After optimization, it generally needs to be verified again, check whether the results after optimization solve the detected system performance bottlenecks and whether new performance bottlenecks are generated. This work is mainly done by developers. In this case, the hiberate component is removed and changed to be controlled by the application itself, which minimizes the probability of occurrence of large things and negotiates with the business department to reduce the number of fuzzy query operations. When "Performance Evaluation" was performed later, it was confirmed that the system had achieved the expected goal.

 

Note: The original source cannot be indicated because I have seen another person's post in this article. Please forgive me.