System throughput (TPS), user concurrency, performance test concepts and formulas

PS: The following is the main concept and calculation formula of performance test, note:

A System Swallowing metric elements:

A system's swallowing metric (pressure-bearing capability) is closely associated with the request's CPU consumption, external interface, IO, and so on.

The higher the CPU consumption of a single reqeust, the slower the external system interface and IO impact, the lower the system throughput, and the higher the converse.

System throughput Several important parameters: QPS (TPS), concurrency number, response time

QPS (TPS): Number of request/transactions per second

Concurrency: Number of request/transactions processed concurrently by the system

Response time: General average response time

(Many people often confuse concurrent numbers with TPS comprehension)

After understanding the meaning of the above three elements, we can deduce the relationship between them:

QPS (TPS) = concurrency number/average response time

A system throughput typically consists of QPS (TPS), the concurrent number two factors decide, each system these two values have a relative limit value, in the application scene access pressure, as long as a certain item achieves the system highest value, the system throughput cannot go up, if the pressure continues to increase, the system throughput will drop, The reason is that the system is overloaded, context switching, memory, and other consumption causes system performance to degrade.

Determining system Response Time elements

We do projects to schedule, can many people concurrent multiple tasks, can be a person or more personal serial work, there will always be a critical path, this path is the project's duration.

The response time of a system call is the same as the project plan, there is a critical path, the key path is the system impact time;

The critical path consists of CPU operations, IO, External system response, and so on.

Two System Throughput Assessment:

When we do the system design, we need to consider the CPU operation, IO, External System response factors and the impact of the system performance of the preliminary estimate.

And the usual situation, we face the demand, we evaluated out of QPS, concurrent number, there is another dimension: Solar PV.

Observing the system's access log found that, in the case of a large number of users, the same time period within each time period of access traffic is almost the same. Like every morning of the workday. As long as we can get the daily flow chart and QPS we can calculate the daily traffic.

The usual technical methods:

1. Find out the highest TPS and solar PV in the system, these two factors have relatively stable relationship (in addition to holidays, seasonal factors outside the influence)

2. The highest daily throughput of the system is calculated by means of a stress test or an experience estimate to obtain the maximum TPS, followed by a 1 relationship. Business-to-business Chinese and Taobao face a different customer base, the two customer base network behavior is not applied, the TPS and PV relationship between them is not the same ratio.

A) Taobao

Taobao flow chart:

Taobao's relationship between TPs and PV is usually the highest TPS:PV approximately 1:11*3600 (equivalent to 11 hours by the highest TPS, this is the scene of the details of the product, different scenarios will be different)

(B) business-to-business Chinese language station

The relationship between Business-to-business TPs and PV varies widely between systems with different application scenarios, roughly estimated at 1:8-hour relationship (09 for Offerdetail Traffic analysis data). Wang Pu and offerdetail these two ratios vary greatly, probably because of the high proportion of reptiles.

Under the environment of Taobao, assuming that our stress test TPS is 100, the daily throughput of this system =100*11*3600=396 million

This is in the case of simple (single URL), some pages, a page with multiple request, the actual throughput of the system is still small.

Regardless of the time of Thinking (T_think), the following relationships (steady running) between the TPS values obtained and the number of concurrent virtual users (u_concurrent), and the transaction response time LoadRunner read (t_response):
Tps=u_concurrent/(T_response+t_think).

Relationship between concurrent number, QPS and average response time

Source: http://www.cnblogs.com/jackei/ basic concept and calculation formula of software performance test

A, the software performance concern point

You need to pay attention to the performance of a software performance test.

Let's think about what roles are involved in software design, deployment, usage, and maintenance, and then consider what performance points each of these roles are concerned about, and what we should focus on as a software performance testing engineer.

First of all, the purpose of software development is to allow users to use, we first stand in the user's perspective to analyze what the user needs to focus on what performance.

For users, when you click on a button, link or issue a command to start, to the system to the results of the user perception of the form to show up, this process consumes time is the user's intuitive impression of the software performance. That is what we call response time, when the corresponding time is relatively small, the user experience is very good, of course, the user experience response time, including personal subjective factors and objective response time, in the design of software, we need to consider how to better combine these two parts to achieve the best user experience. For example: The user in the large amount of data query, we can extract the first data presented to the user, in the process of viewing the user to continue the data retrieval, when the user does not know what we are doing backstage.

The user is concerned about the corresponding time of the user's operation.

Second, we stand in the administrator's perspective to consider the performance points to be concerned.

1, the corresponding time
2, the use of server resources is reasonable
3, Application Server and database resource use is reasonable
4, the system can achieve expansion
5, the system supports the maximum number of user access, the system's maximum business processing volume is how much
6. Where can the bottleneck of system performance exist?
7, the replacement of those equipment can improve performance
8. Can the system support 7x24 hours of business access

Again, stand in the development (design) personnel angle to consider.

1. The structure design is reasonable
2, the database design is reasonable
3, the code whether there are performance problems
4, the system is not a reasonable way to use memory
5. Is there any unreasonable thread synchronization mode in the system?
6. Is there any unreasonable resource competition in the system?

So stand in the performance Test Engineer's perspective, we need to focus on what.

In a word, we need to focus on all of the above performance points.

Second, several key terms of software performance

1. Response Time: The time required to respond to requests

Network Transmission time: N1+N2+N3+N4

Application Server processing time: A1+A3

Database server processing time: A2

Response Time =n1+n2+n3+n4+a1+a3+a2

2, the calculation formula of concurrent user number

System users: The number of system rated users, such as an OA system, the total number of users may use the system is 5,000, then this number is the number of system users.

Simultaneous number of users online: Within a certain timeframe, the largest number of simultaneous online users.
Simultaneous number of users online = number of requests per second RPS (throughput) + Concurrent connections + average user think time

Calculation of average concurrent user number: c=nl/t

Where c is the average number of concurrent users, n is the average number of users per day (login session), L is the average time in a day from login to exit (the average time of login session), T is the length of time (how long a user system is used in a day).

Number of concurrent users peak calculation: c^ is about equal to C + 3* square root c

c^ is the peak of concurrent users, C is the average number of concurrent users, the formula follows the Poisson distribution theory.

3, the calculation formula of throughput

Refers to the number of requests that the system processes users within a unit of time

From a business perspective, throughput can be measured by the number of requests per second, number of pages per second, number of people/days or the number of transactions/hours per hour.

From the network point of view, throughput can be measured in bytes per second

For an interactive application, the throughput metric reflects the pressure on the server, and he can describe the load capacity of the system

Throughput expressed in different ways can illustrate different levels of problems, for example, the number of bytes/sec can be expressed in terms of network infrastructure, server architecture, Application Server constraints, and so on; the number of requests per second represents a bottleneck that is largely constrained by Application server and application code constraints.

When there is no performance bottleneck, there is a link between throughput and the number of virtual users, which can be calculated using the following formula: F=vu * R/

Where f is throughput, VU represents the number of virtual users, R represents the number of requests per virtual user, T represents the time spent in performance testing

4. Performance counter

is to describe the performance of the server or operating system Some data indicators, such as the use of memory, process time, in the performance test to play a "monitoring and analysis" role, especially in the analysis of all scalability, a new bottleneck positioning of the key role.

Resource utilization: Refers to the use of various resources of the system, such as CPU occupancy rate of 68%, memory occupancy rate of 55%, the general use of "resource actual use/Total resources available" to form resource utilization.

5. Calculation formula of thinking time

Think time, from a business point of view, this time refers to the time interval between each request when the user is doing the operation, and in the new energy test, in order to simulate such time interval, introduced the concept of thinking times, to more realistic simulation of user action.

In the throughput this formula F=vu * R/T indicates that throughput F is the number of VU, the number of requests per user R and time T, and that R can be computed with time t and user think time TS: R = t/ts

Here's a general procedure for calculating thinking times:

A, first calculate the number of concurrent users of the system

c=nl/t F=RXC

B, the average throughput of the system is counted

F=vu * r/t rxc = VU * r/t

C, the average number of requests issued by each user

R=u*c*t/vu

D, calculate the thinking time according to the formula

Ts=t/r