Web Performance Test Basics (References)

1.1 Basic concepts

Concurrent users: User concurrency generally occurs in modules that use more frequently, and encountering exceptions is usually a problem with the program.

Number of concurrent users: the number of online users is one of the main bases for calculating the number of concurrent users. = Number of users using the system * (5%~20%)

Concurrency is mainly for the Web server, whether the concurrency is the key to see whether the user's actions on the server has an impact.

Throughput: The sum of the amount of data transferred over the network during a performance test.

Throughput rate: volume/transmission time, amount of data transmitted over the network per unit of time, or the number of client requests processed per unit of time. Throughput is measured in "Requests/sec" or "page Count/second".

CTR: The number of HTTP requests that the user submits to the Web server per second. The greater the click-through rate, the greater the pressure on the server. It is important to analyze the impact of clicks.

Clicking does not refer to a "click" Action of the mouse, because in a single click operation, the client may make multiple HTTP requests to the server.

1.2WEB Performance Test Type

Stress testing: Determine the bottleneck of a system or not receive the performance points requested by the user to obtain the maximum service level that the system can provide.

Load test: Continuously increase the pressure on the system under test until the performance index reaches the limit, response time exceeds the predetermined target, or some resource has reached saturation. This kind of test can find the processing limit of the system and provide the basis for the system tuning.

Big Data Volume Testing: For some systems storage, transmission, statistical query and other services to carry out large data volume testing.

Configuration test: The optimal allocation principle of each resource in the system is found through testing.

Reliability Test: Can apply the CPU resources to maintain 70%-90% utilization of pressure, continuous pressure on the system for 8 hours, and then according to the results of analysis system stability. That is, to load a certain amount of pressure, so that the system runs for some time.

Concurrency testing: Many of the problems found in algorithmic design.

A performance test is a test that is primarily a user-concurrency test.

Performance testing is primarily to identify software problems and hardware bottlenecks.

For the performance of the system to leave about 30% of the expansion space can be.

1.3Web Comprehensive performance test model

Performance test of 1.3.1 expected indicator

Mainly refers to the requirements analysis and design phase proposed some performance indicators.

One or more test cases are written for each metric to verify that the system meets the requirements.

The performance test cases of the expected indicator are usually single-user-oriented, and if the concurrent user content is involved, the design is merged into the concurrent user test case.

1.3.2 Concurrency Performance test

Select a representative, critical business to design the use case, and the user's design should be oriented toward the "module"

User concurrency Performance test is divided into: Independent core module concurrency performance test, combined module concurrency performance test

Standalone Core module Concurrency: Concurrent testing of exactly the same functionality, concurrent testing of exactly the same operation; concurrency of the same/different sub-functions.

For the test case design of the user concurrency performance of the independent core module, some core algorithms or functional problems can be found, such as some multithreading, synchronous concurrency algorithm is difficult to find the problem in single-user mode, and it is more easy to verify the correctness and stability by simulating multi-user concurrent operation.

Core module testing is generally a basic performance test, it is more concerned about the "function" of simulation, generally do not test the server.

Combined module Concurrency: The core module with coupling relationship is combined concurrency test, the concurrent test of Core module group with independent and internal coupling, and concurrency test based on user scenario.

The composite module test generally discovers the functional problems of the interface, and discovers the comprehensive performance problem as soon as possible.

In practice, all types of users will correspond to a set of modules, the equivalent of different business groups in the concurrent access system, to take full account of the actual scenario, such as the monthly charge management system in the month of the 10th and so on the charges peaks and other scenarios.

When writing the combination module user concurrency Performance test case, not only to consider the user scenario, but also pay attention to the use of concurrent points, concurrency point refers to a certain number of users to start the same function or operation point of time, a set of test scenarios usually contain multiple concurrency points, so that the core module the same function or operation of the real concurrency.

1.3.3 Independent Business Performance test

The independent business actually refers to some core business modules corresponding to the business. These modules often have more complex functions, more frequent use, and belong to the core business and other characteristics. The main test is that these modules and performance-related algorithms, but also to test the response of such modules to concurrent users.

User concurrency testing is the key test content of the core business module.

1.3.4 Combination Business Performance test

Is the closest to the user's actual use of the test, but also the core content of performance testing.

The prominent feature of combination concurrency is that the user group is divided into different user groups according to the users ' use of the system, and the proportion of each group is matched according to the actual situation.

User concurrency testing is the core of the composite business performance test. The prominent feature of "combination" concurrency is that users are divided into different groups according to the user's use of the system, and the proportion of each group is matched according to the actual situation.

1.3.5 Network Performance Test

To accurately show how the bandwidth, latency, load, and port changes affect the response time of the user. The relationship between the number of users of the application system and the network bandwidth is mainly tested.

The best way to adjust performance is to combine soft and hard.

1.3.6 Big Data Volume test

Mainly for the database has special requirements of the system testing, mainly divided into three kinds:

1. Real-time Big data volume: simulates the real-time large amount of data when the user is working, the main purpose is to test the user more or some business to produce a large amount of data, the system can run stably.

2. Limit state test: The test system is mainly used for a period of time when the system accumulates a certain amount of data, whether the normal operation of the business

3. The first two kinds of combination: The test system has accumulated a large amount of data, some real-time production of large data volume of the module can be stable to work.

Design of large data volume test case: 1, Big Data volume test due to historical data and 2 run-time Big Data volume test

First, determine the longest migration period of the system data and select some of the preceding core modules or the concurrent user test cases of the combination module as its main content.

1.3.7 Server Performance Test

The main purpose of the performance test is to find the system bottleneck and solve it under the premise of good software function, while the software and server are the two major sources of bottleneck, so the performance of the server is monitored and the performance of the server is evaluated when the user concurrency performance test, fatigue strength and large data volume performance test.

The server performance test case design is to identify the performance counters to be collected and associate them with the previous tests.

1.3.8 the principle of design performance test Case attention:

To meet the expected performance metrics test case requirements, there is no need to design more content, because the more use cases, the higher the cost of execution.

Must obey the overall performance test strategy, must not consider only from the technical angle design "the comprehensive" test case, "comprehensive" should be to meet own test request as the standard.

Proper cropping principle

Only according to the characteristics of the actual project to develop a reasonable performance test strategy, write the appropriate performance test cases, and in the test implementation of flexibility to be able to do a good job in performance testing.

Test plan: Mainly includes test scope, test environment, introduction of test plan, risk analysis, etc., the test plan should be reviewed before it can take effect.

Test report: Mainly includes test process record, test analysis result, system adjustment suggestion, etc.

Test Experience Summary: Continuous summing up work experience is the basis of building a learning team, practice-summary-re-practice

2.1 Coordination between personnel

Customer representative: To understand the background of a number of projects, such as customer requirements for software performance, whether to focus on performance testing, these are the basis for the development of performance testing strategies.

Requirements Analyst: Determine which business is the core business, lay a good foundation for the test cases related to writing the core business modules, and they are more aware of the user community composition and the extended goals of the system, which are the data sources for the design performance test.

Architect: Understand the structure of the system so that the design of the performance test cases in the "appropriate" place pressure.

2.2 Range Determination of performance tests

The test items or test requirements are scored, according to the comprehensive score to determine the test content included in the performance test, scoring elements mainly include customer focus, performance risk, testing costs, etc., performance risk mainly refers to if the performance testing requirements, the production system may be potential risks.

The degree of customer attention or high performance risk should be divided into the test scope.

Number	Test requirements	Performance risk (10 points)	User focus (10 points)	Cost investment (10 points)	Score
1	Data volume Test for one year of system operation	7	10	6	23
2	......	......

2.3 Business analysis of the target system

Determine the core modules of the system: the business is more complex or users use more frequently

Determine the coupling relationship between the modules: a clear understanding of the core modules between the data transmission mode, by determining how the interface between modules, can realistically simulate the situation of multiuser concurrency, in particular, can determine the user concurrency when some of the algorithm is correct.

Analysis system pressure points: Most users use more frequent or large data traffic places.

2.4 User and scene analysis

One, based on the user's actual use of the scene test, two, for special testing purposes (extensibility, stability) and design of the scene test.

Determine how many typical users are in the system, the approximate number of users per class, and the percentage of users at different time periods to use the system. The more common user scenarios are the following three types:

Usage scenarios for different time periods in a day

The system runs at different times of the scene

User scenarios in different business models

2.5 Overall planning

Performance Test planning focuses on project management elements such as time, quality, and cost.

2.5.1 Common Performance testing tools

LoadRunner: is a load test tool that predicts system behavior and performance, and many companies currently perform the preferred tool for performance testing.

One of the Rational Performance:rational series features is very powerful and competitive with LoadRunner.

Qaload:compu Ware Company's products

Webload: A tool dedicated to Web performance testing

Was: Full name is Microsoft Web Application Stress tool, Microsoft offers free performance testing tools

Apache JMeter: Open-source Performance testing tools

Opensta: Open-source Performance testing tools

2.5.2 Test Results Record specification management

The test result data is the main basis for analyzing the bottleneck of the system, a large number of test result files should be standardized management and the naming standard of the unified file. For example: 2007-1-12-dbtest-oracleserver-50-once

2.5.3 test environment Management and maintenance

Perform performance tests to avoid damaging the user environment, and to pre-establish the appropriate backup/recovery strategy so that the system can revert to the pre-test state when an accident occurs.

Performance testing is likely to generate a lot of junk data, eliminating spam is the first job to test the knot

The test also monitors the use of the test machine, unless the resources used to ensure that the scene consumes more than the load capacity of the test machine, otherwise you should carefully monitor the test machine, because once the test machine bottlenecks, all test results are meaningless.

2.5.4 test Analysis and experience summary

Focus on performance testing planning and design, test case design, testing tools and techniques, performance analysis and so on.

Performance test Case Design analysis: usability, execution effect, execution time, should also analyze the use case design method, design ideas and so on.

For bottlenecks: Application systems, databases, Web servers and so on sometimes due to incorrect configuration parameters lead to system performance is not high, can accumulate experience to solve this problem, in order to quickly solve the problem in the future.

1. With the increase of the pressure, the throughput curve increases to a certain time, there is a slow change, even a flat state, it is likely to indicate that the network bandwidth bottleneck. Similarly, when the pressure increases, the CTR/tps curve changes slowly or flat trend, it is likely that the server began to bottleneck.

2. Throughput rates are strongly correlated with TPS: If the change in throughput and TPS is generally consistent with the increase in pressure, it increases together, indicating that there is no significant performance bottleneck in the system under test pressure.

3.1 Steps for performance analysis

1. First, the response time is the starting point for analyzing performance. Review the response time to determine whether the user's performance expectations are met.

2. The bottleneck of the investigation system is in the network link or in the server link.

For server analysis, it mainly involves application, Web server, database server, operating system and so on.

First of all, the response time of business or user transaction should be analyzed, according to the test results to analyze which business is really slow, and then analyze the processing of web resources, finally, the response time of the composition elements of the page decomposition.

3.1.1 User Transaction Analysis

1. View the Results Overview chart: View average response time for transactions, and transaction pass rates

2. View transactions Overview chart and transaction average response time analysis diagram: View transaction pass and failure values to determine whether the program algorithm is a problem or the server has a memory leak.

3. Number of transactions per second analysis graph: The actual transaction load of the system at any given moment can be determined. When the number of transactions per second is found to be reduced, more in-depth analysis is needed, together with server monitoring data.

4. Transactional Performance Summary diagram: Focus on the average and maximum execution time of a transaction, and if its scope is not within the user's acceptable time frame, cause analysis is required.

5. Transaction response time and load analysis diagram: A combination of running virtual user and average transaction response time graph, through which we can see the relationship between the transaction response time and the number of users at any point, thus mastering the performance data of the system in user concurrency, providing reference for extending the application system. Useful for analyzing test scenarios with gradient loads.

6. Transaction response Time Distribution map: pre-defined minimum and maximum transaction response times acceptable to related transactions, you can use this graph to determine whether server performance is within acceptable range.

3.1.2web Resource Analysis

1. Click-through chart: clicks per second, that is, the CTR graph shows the number of HTTP requests that a virtual user submits per second to the Web server during the run of the scene, to assess the amount of load generated by the virtual user based on the number of clicks, and to compare it to the average transaction response time graph to see the impact of clicks on transactional performance.

A drop in System Ctr usually indicates that the server is responding slowly.

2. Throughput graph: Shows the throughput of the server per second during a scene run. A unit of measure is a byte that represents the amount of data that a virtual user obtains from the server on any given second.

CTR: Number of HTTP requests processed by the server per second

Throughput Rate: The total amount of data that the client obtains from the server per second.

HTTP response per second graph can also return to other types of status code information, through the analysis of the status code, you can determine the server under pressure operating conditions.

The common HTTP status code: from 200-505 has its meaning. Example 202: The request has been accepted, but the processing has not been completed.

3. Connections per second graph: shows the number of newly established TCP/IP connections per second during the run. The number of new connections should be a fraction of the number of clicks per second, ideally many HTTP requests should use the same connection instead of opening a new connection for each request.

3.1.3 Page Element subdivision

It allows you to drill down into problematic elements such as slow-downloading images or broken links on your site.

Page Decomposition Overview: You can display the response of a particular transaction in the test process, and then analyze whether the related transaction is running properly.

1. Download time Breakdown: View a static GIF picture and dynamic JSP code.

2. Component breakdown (varies over time): You can select different elements to see the change curve of its download time during testing. For pages that require more control on the client side, it is easy to see which controls are unstable or time-consuming by analyzing the control's response time.

3. Download time breakdown (varies over time): View JSP page The main time spent in such as Receive,first buffer,connection.

Download time Breakdown: Macro, statistical analysis of the response time of page elements throughout the test process

Download time breakdown (over time): microscopic, showing the statistical results of the response time of the page elements in each second of the scene during operation.

4. First buffer time breakdown (over time): You can see whether the page run time is mainly spent on the server or on the network transmission.

Server analysis typically starts with a Web server and a database server.

The first step in server analysis is to analyze the results of the Test tool's monitoring of the Web server and database server-related counters, and then determine whether the Web service is slower or slower to process under pressure.

Web Service is slow: view the various parameters of the Web server configuration, such as the maximum number of connections, the maximum memory, etc. is set reasonable. View memory, CPU, hard disk

Slow data processing: The database configuration is typically problematic, or the hardware resource configuration is too low. such as Oracle, to see the memory configuration, operating mode and other information.

4.1 Database Tuning Strategy

1. Modifying the wording of the effect speed in an SQL statement

2. Add or modify an index

To create an index on a connection between tables

Indexing for lookups

When working with indexes, you can achieve better results by following these guidelines

First: A composite index that is generally built on multiple fields is better than multiple indexes built on a single field, and an index created from a value matching criterion also needs to follow the same principle:

Second: When you create a composite index, exact matching fields precede the non-exact match fields, and fields with a large range of values placed in front of a field with a small range of values can improve query speed, such as the ID field should be placed in front of the gender field.

Third: The index is not the more the better, when the database records more, it means that the cost of the database will be very large, thereby reducing the performance of other aspects of the database.

3. Adjust the system parameters of the corresponding database (System-generated tuning, usually done by the manufacturer's cooperation)

General checks are: Complex statement support, large object feature support, concurrency query performance, throughput, data migration (export backup).

4.2weblogic/oracle Correlation Analysis

Primary monitoring:%processor, avalable Mbytes (free memory), JVM memory, connection Delay time (database connection pool When database connection is established)

Oracle runs the Platform AIX Monitoring (Unix), CPU usage (CPU utilization), disk traltic (disk load), Page-in,page-out rate usage.

and Oracle itself related reports: Tearful buffer tuning cache, application I/O operations.

4.3 Performance test case design to be based on user language

That is to meet user requirements and relatively comprehensive performance test cases, design to be based on the "user language", easy to understand the user-readable, outline form of test cases, so the technical language involved is not many, the user is easy to read. This allows the user to make many suggestions for improvement during the field testing phase, and agrees to adjust the use case (nearly half of the use case) to save costs for the post-test execution.

One of the characteristics of performance testing implementation is not strictly follow the test case to execute, usually in the project to make certain adjustments to the user, and then to perform, for the test case to adjust, delete, modify, add, it is very normal, basic cost to design and execution.

Web Performance Test Basics (References)