How to test how many users the Web system supports

1 What kind of Performance Testing the result is effective.

1.1 Erroneous views

The Performance test tool runs a certain number of users successfully, which means that the server can support so many users. This is wrong.

Answer:

A. Because a valid test result, not only the user is running successfully, but also need to ensure that access to a page or a transaction response time in a reasonable range. The "2-5-8 principle", simply stated, is that when a user accesses a page or a transaction can get a response within 2 seconds, the system responds quickly, and when the user responds in 2-5 seconds, the response speed of the system can be felt, and when the user receives a response within 5-8 seconds, The system responds slowly, but is acceptable, and when the user is still unable to respond after more than 8 seconds, the system sucks, or thinks the system has lost its response, chooses to leave the Web site, or initiates a second request.

B. The test scenario does not necessarily mimic the real business scenario, because the browser is concurrent multi-threaded multiple TCP to complete a page, and the test tools are basically 1, 2 threads, the pressure on the server is not the same, the real environment of the TCP pressure is a performance test tool several times the virtual environment.

　2 What are the factors that affect Web server performance metrics?

Why the performance test tool needs to provide transactions (page or transaction, full script) metrics, TCP connections, throughput, server resource monitoring, number of requests/responses.

1) Hardware resources: such as CPU, memory, network card throughput, I/O capability, swap capability

2) Number of threads: Here is the JAVA Web server, the default per thread occupies 2M of memory, and 32 for system JAVA processes (such as Tomcat, JBoss) occupied only 2G (generally smaller than this), so the number of threads is limited; 64 is an unrestricted thread, But the CPU has to keep up

3) Number of TCP connections: the operating system TCP connection number is generally very large, the operating system for the TCP connection settings have default values (how to configure, can be searched online, not described here); but the actual test in TCP connection at hundreds of, the response time of the test is very long. Packet capture analysis, the original is three handshake SYN packet server is not timely response, resulting in SYN retransmission (3 seconds later, 9 seconds later).

If the SYN is lost, it will be re-sent, but the first time is 3 seconds, the 2nd time after 9 seconds, if the Syn_ack received after the re-send, resulting in a long TCP connection, resulting in extended business response time.

4) Response time: The server response time is small, the user experience is good, in the case of a large number of users concurrency, HTTP response time in the user tolerance is effective, generally using the "2-5-8 principle."

5) Software Code performance algorithm: This does not introduce, such as poor algorithm, query database for a long time and so on.

3 common questions and answers frequently encountered by testers

3.1 Why using a browser to access the page response quickly, 1-2 seconds to complete, and the use of testing tools need 10 seconds, or even a few 10 seconds to complete the script

Answer:

A. This is because the browser access page response is concurrent, concurrent with multiple threads (multiple sockets), and the performance test tool is basically a serial send request. If a page has 100 resources (CSS, HTML, JS, pictures), you need to send 100 HTTP requests, if you use 6 threads (browser), each of the approximate request 14 HTTP, if you use a thread (test tool), you need to request 100, the time is certainly much larger. Concurrency shown for the Chrome browser debugging tool:

B. In addition, the browser has a cache function, If the previous access to the www.qq.com, will be cached in the browser temporary directory, the next request to send an HTTP request will be carried on the If-match or etag, such as the header domain, the Web server to determine that the resources are not changed will be 304 response, rather than back to a number of OK, so as to reduce the transfer of resources, So the time is small. Some test tools do not carry these header fields (including LoadRunner), so the response back is OK. So when the tester defaults to the real test, it is possible to consider a partial cache, some of which are not.

3.2 How the Performance test tool simulates web virtual users

A. Recording

Using the browser for normal business operations, the performance Test tool records the HTTP request information. It is generally necessary to record URLs and header fields, content, and response codes. Although different performance test tools are recorded differently (such as LoadRunner using Hook,jmeter or Badbody,kylinpet with network card capture and proxy), it is possible to record HTTP requests for normal business.

The test tool is best able to record the cache header field, i.e. If-match or Etag,loadrunner does not seem to support recording cache header fields.

B. Impersonate a user

Send HTTP requests and receive responses based on recorded scripts, replace parameters before sending (implement different parameter values for multiple users), receive associated parameters (get parameter values from received response messages, such as cookies, Jsessionid)

Here's a simple list of how the used performance test tools are simulated (the tool runs a user and then uses the Wireshark to capture the results of the analysis):

LoadRunner: The HTTP request is sent according to the recording script, and if the HTTP request includes an inline resource (slice, CSS, JS), the second thread is launched to execute the embedded resource, that is, LoadRunner supports two TCP connections for two threads at the same time.

Kylinpet (native): Can be configured to set up one thread or multiple threads to send HTTP requests concurrently, with multiple threads concurrently and the number of TCP connections as the browser behaves.

JMeter: Only one thread, one TCP connection

Other tools: I have not used, please use the brothers and sisters can be added. Through Wireshark capture packet analysis. 3.3 How to test how many real users the Web server can support, and how the server tuning parameters are reasonable

Answer:

This requires a performance testing tool that simulates the behavior of real users, including the number of HTTP requests, the number of concurrent threads per user, and TCP connections, think times, and cache.

Why do you need to simulate the number of threads of real users and TCP connections, as mentioned above, the number of threads of the Web server and the number of TCP connections is often very low, this is not to improve the hardware can be easily resolved, this is the server tuning more complex configuration.

As a result, test scenarios where the behavior of real users (browsers or other Web clients, which may differ from each other in concurrent threads and TCP numbers) can only be simulated, and the results are the most realistic, and server tuning is the most meaningful.

4 How to test how many users the system supports

4.1 Simulating the behavior of real users

Only the same behavior as the impersonated user is valid for the number of test users supported by the system, so you need to simulate the same number of concurrent, TCP connections, and even HTTP requests. Users can be browsers, smartphones , Smart Set-top boxes, test tools simulating their behavior is the most effective test.

4.2 Test result data in a reasonable range

4.2.1 User Statistics

Number of successes, failures, online numbers per second, maximum number of online, analysis of the number of users supported by this test result, maximum number of users

4.2.2 Ctr

The average number of HTTP requests per second, and the number of responses. Analysis of the processing capacity of the system

4.2.3 Transactions

Transaction success, failure, time, the transaction is generally the entire script run time, or a page or a transaction, through the results analysis, to determine whether the time of each thing is reasonable, in line with the "2-5-8" principle, if the test results show things time is very large, it means that the system can not support the user of this test, Because of the user's response time is too large (like the train booking, too many users cause long response time, the user can not stand, then think this system rotten).

Of course, you also need to look at the percentage of transactions, and analyze whether the transaction times of 90%, 80%, 70%, 60% are within reasonable bounds.

4.2.4 TCP connection Information

The number of successful TCP connections, the number of failures, and the TCP three handshake time. Because this test results may be due to the server system or network TCP packet loss and retransmission caused by the delay is large. In the case of a server, the server receives a SYN from TCP and is not processed, and can be optimized by debugging the server's TCP configuration.

How to know is the server problem, this need performance test tool can give the TCP connection time (currently know only Kylinpet can support), if display more than 3 seconds, then need to check whether the network or server problems, You can grab packets (tcpdump or Wireshark) on the server side and then analyze the TCP SYN information (number, time)

4.2.5 Resource Usage

Server CPU, memory, bandwidth, I/O is not enough, causing the system does not go to which reason, based on the reason for tuning or upgrading.

The performance test tool's CPU usage needs to be considered when testing, and if the performance test tool occupies a high CPU, the test may be a bottleneck in the tool and the test result is invalid.

How to test how many users the Web system supports