Basic Performance Indicators of Web Performance Testing

WebPerformanceTestIn general, the processing of a Web request includes the following steps:

(1) The customer sends a request

(2) WebServerAccept the request for processing;

(3) The web server obtains data from the database;

(4) The web server generates the user's object (PAGE) and returns it to the user. The time from sending the request to the last byte is called the response time (Step 3 is not included in the processing of each request ).

1.Transaction (Transaction)

InWebPerformance TestingA transaction represents"Send request from user-> Web ServerAccept the request for processing-> Web ServerDirectionDBGet Data->Generate the user'sObject (Page), Return to the user"The general response time is for transactions.

2.Request Response Time

The request response time refers to the time taken from a request initiated by the client to the end of the response received by the client from the server. In some tools, the response is usually called"TTLB", That is"Time to last byte"It indicates the time elapsed from initiating a request to receiving the last byte response from the client. The unit of response time is generally"Seconds"Or"Millisecond". A formula can be used to indicate: Response Time = Network Response Time+ApplicationProgramResponse time.Standards can be referred to foreign3/5/10Principles:

(1) In3Within seconds, the page will give the user a response and display some information, which can be considered as "very good ";

(2) In3 ~ 5Within seconds, the page gives the user a response and displays some information, which can be considered as "good ";

(3) In5 ~ 10Within seconds, the page gives the user a response and displays some information, which can be considered "barely accepted ";

(4) Exceeds10The second is a little impatient, and the user may not continue to wait;

3, Transaction Response Time

A transaction may consist of a series of requests,The response time of the transaction is mainly for the user.,It is a macro concept and is proposed to explain the business response time to users..For example:The response time of a interbank withdrawal transaction is composed of a series of requests..Transaction Response Time is a parameter that directly measures system performance..

4.Concurrent users

Concurrency is generally divided2. One is strict concurrency, that is, all users perform the same thing or operation at the same time. This operation generally refers to the same type of business. For example, in the Credit Card Approval business, a certain number of support personnel submit completed approval services at the same time. There is also a special case that all users perform identical operations, for example, in the Credit Card Approval business, all users can apply for the business together or modify the same record.

Another concurrency is concurrency in a broad range. The difference between this concurrency and the former concurrency is that although multiple users send requests or perform operations to the system, these requests or operations can be the same or different. For the entire system, there are still many users operating on the system at the same time, so it is also in the scope of concurrency.

We can see that the latter concurrency includes the former. In addition, the latter concurrency is closer to the actual usage of users. Therefore, for most systems, only a few users are allowed."Strict concurrency". ForWebFor performance testing, this2Generally, you need to test the concurrency. Generally, You need to perform a strict concurrency test first. Strictly speaking, user concurrency generally occurs in frequently-used modules. Although the probability of occurrence is not very high, once a performance problem occurs, the consequences may be fatal. In a strict sense, concurrency testing often matchesFunction TestingAssociation, because exceptions in concurrent functions are usually a program problem, and this test is also part of the robustness and stability test.

Concurrent user count: The number of user concurrency is2Common incorrect ideas. An incorrect idea is to regard the number of concurrent users as the number of all users who use the system, on the grounds that these users may use the system at the same time; another idea is to regard the number of online users as the number of concurrent users. In fact, online users may notOthersUser concurrency occurs. For example, if a user is browsing a Web pageServerThere is no impact, but the number of online users is one of the main bases for calculating the number of concurrent users.

5.Throughput

Refers to the total amount of data transmitted over the network during one-time testing..Throughput/Transmission Time,Is the throughput..

6,TPS(Transaction per second)

Number of transactions or transactions that the system can process per second.It is an important indicator to measure the processing capability of the system..

7Click rate

Users send Web Number of HTTP requests submitted by servers -us ">. This metric is a unique indicator of the Web application : the Web application is a " request - response " mode , If the user sends an application , the server processes the application once , therefore, click the Web minimum transaction unit that the application can process . if you define each click as a transaction , click rate and TPS are two concepts: . it is easy to see . A higher click rate , more pressure on the server . click-through rate is only a performance reference indicator . it is important to analyze the impact of clicks. Note that , the click here does not mean one click operation with the mouse , because , the client may send multiple HTTP requests to the server .

8.Resource Utilization

It refers to the usage of different system resources.,For exampleCPUUtilization,Disk Utilization.Resource utilization is the main basis for analyzing system performance indicators to improve performance.,So yesWebPerformance TestingWorkKey.

Resource Utilization mainly targetsWebServer,Operating System,DatabaseServer,Networks,It is the main reference for testing and analysis bottlenecks..InWebPerformance testing in progress,Collect relevant parameters for analysis as needed.

General metricsWebApplication Server,DatabaseServer requiredTestItem)

Metrics	Description
Processortime	Server CPU usage. Generally, when the average reaches 70%, the service is close to saturation.
Memory available Mbyte	The number of available memory. If the test shows a change in the memory, you must note that the memory leakage is serious.
Physicsdisk time	Physical disk read/write time

Web Server metrics

Metrics	Description
Requests per second (avg RPS)	Average response times per second = total request time/Seconds
AVG time to last byte per terstion (mstes)	The average number of iterations of the Business script per second. Some people will confuse the above
Successful rounds	Successful requests
Failed requests	Failed requests
Successful hits	Successful clicks
Failed hits	Number of failed clicks
Hits per second	Clicks per second
Successful hits per second	Successful clicks per second
Failed hits per second	Number of failed clicks per second
Attempted connections	Number of attempted links

Database server performance indicators

Metrics	Description
User 0 connections	Number of user connections, that is, the number of connections to the database
Number of deadlocks	Database deadlock
Butter cache hit	Database cache hits

System Bottleneck Definition

Performance item	Command	Metrics
CPU limit	Vmstat	When % USER + % sys exceeds 80%
Disk I/O restrictions	Vmstat	When % iowait exceeds 40% (aix4.3.3 or later ),
Application disk restrictions	Iostat	When % tm_act exceeds 70%
Less storage space	LSPs,-	When the page space activity rate exceeds 70%
Page feed restrictions	Iostat, stat	Virtual logical volume % tm_act exceeds 30% of I/O (iostat), and the activated virtual memory rate exceeds 10 times of the number of CPUs (vmstat)
System failure	Vmstat, SAR	Page switch increase, CPU waiting and running queue

Stable system resource status

Performance item	Resources	Rating
CPU usage	70%	Good
	85%	Bad
	90% +	Poor
Disk I/0	<30%	Good
	<40%	Bad
	& Lt; 50% +	Poor
Network	<30% bandwidth	Good
Running queue	<2 * Number of CPUs	Good
Memory	No page swapping	Good
	10 pages per CPU per second	Bad
	More page exchanges	Poor

Easy to understand:

Daily access volume

Maximum concurrency of common pages

Concurrent online users

Access time

Case:

Recently, a project in the company is a portal website.Performance TestingAccording to the characteristics of the project, the main test items and test plans are formulated:

One is to test the maximum number of concurrent requests acceptable to several common pages (username parameterization, set a collection point Policy)

One is to test whether the user can operate normally under the server's long-term pressure (username parameterization, iterative running script)

One is to test whether the server can accept 0.1 million concurrent online operations. If IIS is used as an application server, the maximum number of concurrent operations per server cannot reach 0.1 million, it is necessary to use a set group to achieve Load Balancing through multiple machines. If WebSphere and other application servers are used, the maximum number of concurrent connections per server can reach 0.1 million, however, for performance consideration, a cluster must be used for load balancing among multiple machines. Generally, there is a simple computing method, and one connection generates one session, each session has a memory size setting on the server. It is 3 Mb on NT, so 0.1 million concurrent connections require GB of memory. Others The program also occupies the memory, so the number of prepared memory needs to be greater than this. Another 0.1 million users are online at the same time, which is totally different from the 0.1 million concurrency. This upstairs has already been said. But how can we convert 0.1 million concurrent online users into how many concurrent users? This requires a lot of history.Logs Information to support. System Logs must contain the number of online users at the same time and the number of user operations. The ratio of the two data records is the ratio of the number of concurrent users that you want to change. In addition, based on empirical statistics, for 1 Java Developed Web System (I haven't made any statistics, but no data can be provided ), generally, a single dual-CPU, 2 GB memory server can support up to 500 concurrent connections (in this status, most operations are time-out errors and the server is prone to downtime, in fact, there is no practical significance). The maximum number of concurrent operations that can be normally used (the waiting time for a single-step non-large data volume operation cannot exceed 20 seconds) cannot exceed 300. Assume that 0.1 million of your concurrent online users are transferred to 9000, so you need at least 18 such machines. We recommend that you have at least 30 machines. Of course, if you buy a large server with 200 CPUs, 0.1 million GB of memory, and MB of optical fiber bandwidth, even if it is concurrent users, the speed is definitely too high.

In addition, it takes nearly six hours for all the light settings to enter the running status. You can press one system to see the details. The following situations may occur:

1. server downtime;

2. Client downtime;

3. requests are rejected by the server at a certain time. All errors are displayed on the client;

4. Barely complete the test, but the network is congested or the test results are displayed for a long time. Assume that the bandwidth between the client and the server is 10000 Mbit/s/= 10 KB, then each user can only get 10 KB. This speed is close to that of one 64 kB modem for Internet access; in addition, all the above analyses did not consider the background of the system, suchDatabaseAnd middleware.

1. servers: As mentioned above, 50 PC servers are required;

2. Network: 50 K for each user, at least five hundred megabytes of bandwidth are exclusive. It is estimated that the network latency is only one second;

3. If there is a database, at least Oracle, preferably sysbase,SQLServer is definitely not supported. The database server requires at least 10 machines with 4 CPUs and 16 GB of memory;

4. If there is a CORBA, at least 10 4-CPU machines with 16 GB of memory should be prepared. In addition, with the addition of Server Load balancer, firewalls, routers and various software, there is no 10 million of the investment, certainly not.

Such a portal system does not have static pages as Jackie said because it has user permissions. However, as long as it is a multi-server cluster, we can use the test results of one machine to calculate the load capacity after the cluster of multiple machines, consider the load balancing and routing pressure at most, such as bandwidth, speed, latency, and so on. However, if they all change on one machine, we can only calculate some indicators. We can simply judge from these indicators whether they are not feasible, for example, if 0.1 million concurrent users only have one hundred megabytes of bandwidth, we can calculate that each user only has 1 kb of bandwidth, which is obviously not feasible. However, the actual results still need to be tested. After all, the system pressure and the number of users do not change linearly.

This type of system is widely used and mature, and many software can roughly estimate the performance characteristics of the system after the scheme is designed, these results in a low proportion of system tuning in terms of software performance (of course, not fully exclusive to someCodeAnd further improve the performance after the configuration is optimized), more are from the hardware aspect, such as increasing the memory, raid on the hard disk, increasing the bandwidth, or even increasing the number of machines.

In network technology, 10 M bandwidth refers to bitwise computing, that is, 10 M bit/second. The download speed is measured in bytes, therefore, the fastest theoretical download speed for converting 10 Mbps of bandwidth to byte is 1.25 Mbps!