Performance Test on half-Bucket Water

First of all, in my career, there are not many opportunities to perform performance tests, and I found that the performance bottleneck is less frequently. Specifically, there are only two performance tests. With the promotion of large-scale distributed systems, especially web apps and cloud computing, performance testing is becoming more urgent. Today, I will talk about performance testing in half a bucket. We welcome you to discuss it together.

What is performance testing?
Performance testing uses automated testing tools to simulate a variety of normal, peak, and abnormal load conditions to test the system's performance indicators.
Both load testing and stress testing are performance tests, which can be combined.
Through the load test, determine the system performance under various workloads, the goal is to test the changes in the system performance indicators when the load increases gradually.
Stress testing is to determine the bottleneck of a system or the performance points that cannot be received to obtain the maximum service level tests that the system can provide.

The China Software Evaluation Center has summarized the performance test into three aspects: test the client performance of applications, test the network performance of applications, and test the server performance of applications.
Under normal circumstances, the combination of the three aspects is effective and reasonable to achieve comprehensive analysis of system performance and Forecast of bottlenecks.
Note that three aspects are mentioned here, but we usually focus on the performance of the server and ignore the performance of the client and network.

Here are some of my experiences:
I. Purpose/Requirement
Why is performance testing or performance testing required?
1. the first release of a new project/product requires a benchmark test.
2. The product in use restructured a module/module using new technologies and requires an evaluation.
3. The number of users of products in use has exploded. The number of users has increased from millions to tens of millions.
4. If one or more pages are added, You need to perform a test on the page latency.
Before the test starts, you need to prepare for the purpose of this performance test and determine the data to be collected and the performance indicators to be followed.

2. Environment deployment
Performance testing requires a clean environment, including hardware, network, operating system, tested system, and database.
Before you start, you need to prepare the environment. It is best to simulate the online system. At the same time, this is also a process of eliminating interference and creating a structural diagram.
1. What kind of server is used for hardware? Ideally, the server is the same as the production environment. Do I need to configure Server Load balancer?
2. Network, whether to simulate various networks, whether dual NICs are required, and whether the internal network will affect the normal use of other employees
3. For Windows/Linux operating systems, whether to use default settings or reference settings, which operating system services need to be disabled, and how to set ulimit in Linux
4. The tested system invites architects to participate in the architecture of the system, whether the Web server-IIS/Apache/Tomat is used, and what is their optimal configuration?
5. Database: whether the intra-site search database is separated from the order database, whether the database is cached, and whether the database uses the master-slave mode

Iii. scenarios and load modes
What scenarios need to be simulated for this performance test:

1. Shopping website seckilling activities, using constant load mode, set the set point, user concurrency

2. The normal use of the system is analyzed from the daily monitoring system. For example, if the shopping peak is noon, the single-step load mode (Ram-up/down) is used ), during each interval, the number of user increases, and the peak time is constant load. After the peak time, the user starts to decrease.

3. The system was revised to use the target-based load mode to set the target based on historical data, or for example, set the maximum CPU/memory to 85%.

Only when the scenario and load mode are determined can the use case/script be correctly generated.

Iv. Performance indicators
What performance indicators and data need to be collected for this performance test:
1. CPU usage
2. memory usage
3. QPS
4. Response Time
5. Network I/O
6. file IO
7. Database Io
8. Maximum number of supported users
Not all of the above indicators need to be collected and determined based on specific scenarios.

V. Performance testing tools
When it comes to performance testing, the first concept of many testers is tools, such as commercial popular LoadRunner and open-source popular jmeter. However, few people pay attention to the four points mentioned above, without the above design analysis, the performance bottleneck cannot be analyzed only when the tool runs out of the result. The unreliable data results will greatly increase troubleshooting, these data are often questioned.
Use of tools:

Parameterization
Protocol selected by LoadRunner
LoadRunner Association
LoadRunner Controller/agent
Jmeter remote testing
The skill of using a tool is only a small part of the performance test.

Is it necessary to use a large tool for each performance test? Can the response time of a page use other lightweight tools, such as some browser plug-ins httpwatch, Yahoo yslow, and Google speed tracer;
Can you write some targeted gadgets by yourself and define the actual page load time based on the actual situation, not just the server response time.

Vi. Data collection and analysis
The ultimate goal of performance testing is to analyze whether the system has bottlenecks through data collection. Therefore, data collection and analysis is a very important process.
The analysis process requires the participation of team members, such as architects, DBAs, and developers. It is a long-term process that generates different data comparisons by adjusting the test script.

7. Two problems I found
1. dual Nic problems, a new version of the project, after the launch, there will always be a machine crash or a significant decline in performance at the peak, and finally found that the server only uses a single Nic, this is obviously because the above mentioned exclusion interference is not well done (the configuration is not checked), and this performance test fails.
2. sub-process crash, 64-bit system benchmark test, through continuous communication with team members, eliminate various set interference, determine hardware and software configurations, add profile tools, and comparison with 32-bit systems.

VIII. Subsequent Optimization
Performance testing is just the beginning. The ultimate goal of performance testing is to discover and solve the bottleneck of the system, which involves optimization. The optimization process is often considered in the system design stage.
1. obtain data Asynchronously
2. Create a cache
3. Distributed
4. File Decomposition

9. Recommendation

1. User experience-based performance testing, not metrics
   more close to the user use scenario performance testing, the main tools used in the use of IBM Rational teststudio
   http://www.cnblogs.com/mayingbao/archive/2008/04/11/1148510.html

2. detailed description of the software performance test process and case analysis (second edition)

3. what do you need to know about performance testing?
   http://www.cnblogs.com/jackei/archive/2008/06/27/1231307.html
   http://www.cnblogs.com/jackei/category/32808.html