Problems in the stress test

Problems in the stress testWhat is a stress test

Software stress testing is a basic quality assurance behavior that is part of every important software testing effort. The basic idea of software stress testing is simple: not to run manual or automated tests under normal conditions, but to run tests with fewer computers or poor system resources. The resources typically used for software stress testing include internal memory, CPU availability, disk space, and network bandwidth.

Stress testing covers, performance tests, load tests, concurrency tests, and so on, and these test points are often interwoven and coupled together.

There are problems with stress tests.

I summed up a few more points:

1. The operating system is installed by default, and the stress test is implemented without any optimization
2. The impact of disk IO on software is not considered
3. No consideration of the impact of network bandwidth on software
4. Network software Testing, without regard to TCP features
5. Various timeout parameter optimizations
6. Test Client not optimized
7. Concurrency understood incorrectly
8. Web servers, databases, and so on server not optimized

If the above items are not optimized, the stress test data is basically without any reference value, and any one without optimization will cause your stress test data to deviate. Let me explain the following article by clause:

1. Operating system problems operating system is a popular software, factory optimization is for the public, it is impossible for a domain to do a separate optimization. So our first step is to optimize the operating system. Linux system optimizes kernel parameters, Windows system optimizes the registry, and so on.

2. Disk IO This is where the bottleneck is most likely, often the CPU has not reached its limit and the disk is overwhelmed.

3. Network IO is the same as disk IO

4. TCP connection almost all B/S, c/s software is multi-threading, or multi-process technology. This technology has a feature, the developer will be programmed to the thread can auto-scaling mode, the open process will start a small number of threads, when the connection continues to increase, a gradual increase in threads, with the end of the thread run, the thread gradually reduced. Such a design would make more efficient use of hardware resources and give the hardware resources to other processes when the program is idle. Rare software is designed to open a service exclusive resource. So the test software to do stress testing, not a lot of requests at a time, but to adopt a gradual increase in the way, otherwise the first test will have a concurrent failure to respond, resulting in test data deviation. You can also make a few more pressure requests (let multithreading work), from the third start to record the test data, two times before the test data.

Tip: Another problem is TCP connection multiplexing, which is also an important configuration to think about. If this is not configured, I would like to test the data will be biased

1. The timeout parameter timeout parameter is a very important parameter in the stress test, for example, the connection timeout from the Web to the database is 60 seconds, and if there is an SQL query that is longer than 300 seconds, then the subsequent requests will continue to queue up, and when the number of connections reaches the maximum database connection, all subsequent requests fail. Usually our web server time-out will not exceed 30 seconds, sometimes I set to 10 seconds, once the timeout occurs, rather let the connection timeout, do not let him affect the overall service.

2. Client a lot of network software needs to send a stress test request from the client, so the client's optimization is also necessary, otherwise the client pressure cannot go out, the service side pressure does not come.

3. Concurrent, many people think that concurrency is the maximum number of connections at the same time, which is wrong. If you have written multithreaded programs, you will find that multi-threaded runs are regular. are queued in sequence and are not running at all. So concurrency is the sum of connections that are done in relative time, such as concurrency per second, concurrency per minute, and so on, usually in seconds. We currently use the operating system called time-sharing operating system, the characteristics of this system is likely to achieve multi-user, multi-tasking. The operating system queues the process (priority) polling to run, except that it is too fast to allow you to think that multiple processes are running concurrently.

4. Server Optimization main b/S software stress test, WEB, cache, database and so on server, all need to optimize to the best state

Why do stress tests

If these problem elements are taken into account during the software design phase, the development phase is strictly enforced. So it's almost impossible to develop some software to do this stress test.

So in the software design phase to consider, flexibility, scalability, reliability and performance, but also consider high availability and load balancing.

Software optimization is accompanied by development, continuous integration, continuous testing, and continuous deployment.

Where to do stress tests

Some software requires a closed environment test and cannot be tested in a shared resource environment. So it is necessary to do VLAN isolation, and even independent routers and switches are tested in the closed network.

(when) What time does the stress test

Any time you can do stress testing, why do I put the "time" focus? Currently affected by the Earth's rotation, often leap seconds, you do not consider this issue.

Participants in the stress test process (WHO)

1. Operation and Maintenance Department
2. Development Department
3. Testing Department

How to do stress tests

Let's give you some examples of stress testing methods that are limited in length, and I'm Just giving you a clue.

Before testing you need some monitoring tools, the fact that monitors the server's resource changes.

For example, the WEB server stress test, the test scenario is nginx:

Worker_processes  8;            Number of processors    worker_rlimit_nofile 65530;     Allow maximum number of open files    worker_connections  4096;       The maximum number of connections is    keepalive_timeout  ;          Open multiplexing Connection    gzip on  ;                       Compress transfer data

How to test it? Do you want to live to maximize performance? Or the Relativity energy? What we usually need is a good relative performance to meet the demand, not the maximum performance. Why is it? Because to live to maximize performance is to make a lot of configuration sacrifices, such as logging off, prohibit access time, and so on.

According to the above configuration your test case should be, each concurrent 4000 requests 8000~10000 times, you cannot concurrently 8000 request 4000 such tests. Many people often make mistakes, so testers need to connect the configuration parameters of the system, can not blindly use the digital experiment.

As I said above, when the thread is turned on, it gradually increases with the request, so it is inaccurate to launch the test data for the first time, and the number of threads can be seen through the Pstree command. After the third time the thread gradually increases to 4,096, and the previously opened TCP can be reused, then the result of the test is more convincing.

Extended reading "Netkiller Web Codex" "Netkiller testing Codex" Netkiller Linux Codex

Author

Chen Jingfeng, nickname Netkiller, English name Neo "Netkiller series Codex" ebook author, readers QQ Group: 128659835, personal website: http://netkiller.github.io/

Reprint please specify the source and the author statement

2015-6-30

Problems in the stress test