Discussion on performance test load target

This post was last edited by Dionysus at 2011-5-30 23:21 First, the premise Recently, I have tracked 2 performance test projects involving the outside team, communication found that everyone in the development of test strategy on how to determine the load target, the number of concurrent users have many different methods, this article hopes to be able to explore various methods, and based on the experience of the strategy to give some of their own suggestions. In this paper, the application of the test is mainly in the banking system, and the pressure initiating tool takes loadrunner as an example. Second, terminology Unit time: This article is in 1 seconds per hour. Number of online users: access to the number of users tested applications, but the user will not be at the same time in the test server to send requests, create pressure. Number of concurrent users: part of the book in the narrow sense and the broad sense of the narrow sense of the unit time to carry out an operation of the number of users, the broad sense refers to the unit time to perform a number of different operations, the broad sense of concurrent user operations closer to the actual business environment. But the number of concurrent users in this article only refers to the narrow sense, because the broad sense is a combination of many narrow. Tps:transaction per Second, number of transactions/sec, Unit is transaction/sec. Trt:transaction Response time, transaction response times, the average transaction response time when TPS is stable, in seconds. Third, the load target 1. Load angle The development of test strategy is the focus of performance testing, including test scope, scene extraction, load target, initiating method, passing standard, etc. While the load target relation is the whole test scene design, concurrent ratio and result evaluation, so determining the load target also determines the overall direction of the test. By understanding the business requirements, the load target translates into a series of specific values, which are generally divided into two areas: Front end: Business people are more concerned about the number of concurrent users or online users, measured by number; Backend: Technicians pay more attention to the load capacity of backend application server and database server, measured by TPS; The number of front-end concurrent users in the industry has many formulas and principles, such as the 2/8 principle, 10% Online user estimates, (online user number *session time)/monitoring time, but the formulas and principles calculated by the number of concurrent users is not accurate, If there are 100,000 online users of the system can not be said to test only 100,000 *10%=1 million concurrent users can. The actual load capacity of the back-end TPS reaction applied to the test, the application of the existing specific volume of business can be calculated accurately, such as the bank system in a province on the average daily transaction volume of 100,000, you can accurately calculate the TPS mean = 100,000/(6*3600) = 4.63 Pen/sec (Shing by 6 hours), If the application is not up to the TPS requirements can not complete the day business. The same application is used to estimate the load target from different angles, the numerical results can be very difference, so how to select the load target correctly will directly affect the test method and scene design. 2. Load Index Aside from the choice of perspective, the final Test metrics, for a software and hardware environment fixed applications, only one load indicator is fixed, that is, the maximum transaction capacity-usually measured in TPS. As the load increases, the application will gradually reach the maximum transaction capacity, if the application is robust enough, the load continues to increase, the application of transaction processing capacity will not drop abruptly. SoThe goal of performance testing is to determine the maximum transaction capacity of the application under test。 The relationship between TPS, TRT, number of concurrent users, number of online users and so on is gradually smoothed out by the transaction processing capability. 1. TPS The granularity of the transaction directly affects the calculation of TPS, so the granularity of the transaction is appropriate when defined: c/S Architecture online application of a transaction will often flow through multi-layer front-end applications, the need to determine the location of the pressure-initiating tool, recommended to cross the front of the direct pressure test application, at this time a transaction represents a background transaction. b/S Architecture Management Class application of a page operation may have many interactions with the background, it is recommended that the operation on the page as a transaction, but to ensure that the interaction within the transaction in front is not to be split. When the LoadRunner initiates the pressure, the statements within the action are iterated, and the LR calculation TPS only performs several transaction within 1 seconds, and if there are multiple transaction in the action, the TPS for each transaction is the same. Does not reflect the true processing power of the various transactions, it is recommended that the action define only one or transaction as concise as possible. This TPS can accurately represent the transaction processing capability of the application under test. Using TPS as the load target is direct and accurate by acquiring applications that can obtain specific transaction (transaction) quantities by capturing production logs and referencing similar systems. But be aware thatIf you target TPS, the concurrent number of front-end configurations no longer represents the number of concurrent transactions, but the number of concurrent commits. The computational relationship between TPS and TRT will be detailed below. 2, TRT TRT refers to the average transaction response time when the TPS is stable (not necessarily the maximum), and does not focus on individual transactions, which is closely related to TPS and changes with TPS. When the load increases, the TRT increases until the transaction blocks and the transaction times out. TPSXTRT = number of concurrent commit transactions. If you target tps=20, and then trt=2 seconds, the number of concurrent commit transactions =20x2=40 pens. If 1 transactions are committed within 1 user units, the number of concurrent users can be equal to 40. When the target TPS is set up, the performance of TRT should be taken into account, and if the TRT exceeds the business requirement, even the load target is ineffective. TRT no fixed criteria, generally on the online application of OLTP, from the front-end submission to return should not be higher than 3 seconds, background applications and database processing should be in 1 seconds or so. OLAP online Analysis system or general website can follow the 3/5/8 principle, or longer. 3. Number of concurrent users It is generally understood that the number of concurrent users is the number of vuser set in LoadRunner, and the maximum concurrent users of the applied application can be found by adding vuser by gradient and comparing with TPS changes. But I thinkthe number of concurrent users is not equal to the number of VUser set in LoadRunner. Affected by such factors as transaction response time, ThinkTime, pacing and aggregation points, the VUser quantity can not directly reflect the load capacity of the application under test. Suppose the same 10 vuser concurrent once, if the response time of a program is 1 seconds, then the tps=10/1=10 of a program. and the B program response time is 5 seconds, then the B program tps=10/5=2. Also in the mixed scene with VUser proportional to reflect the load ratio of different applications is also wrong, mixed scenes due to the interaction between the transactions, single transaction load when the response is likely to be blocked now, the ratio of front-end vuser can not accurately control the back-end application of pressure. Therefore, I would prefer to hook up the number of concurrent users and the number of concurrent commit transactions to reflect the actual load measured: N users in a unit of time concurrently submitting N transaction requests to the application under test (n is the same). The number of vuser and the initiating setting are just a means of implementing the number of concurrent users. 4. Number of online users The number of online users and the number of concurrent users, TPS, TRT, there is no fixed conversion formula, I do not advocate 10% such a rough ratio, online users of on-line applications is the number of daily check-in counters, the application of the management class is at the end of the month, quarter of all the users of the system. The number of online users can be obtained from the demand or production managers, but not through the performance test to launch the number of online. Iv.. Load Target Selection 1. The application of a clear trading volume Based on the above analysis of the typical load indexes, transaction processing capability measured by TPS is the most accurate load target. TPS mean and peak value can be calculated by transaction volume of production log or similar system. Higher spikes can be estimated based on the 2/8 principles and business extensions. The online application of bank is a typical application system with definite transaction volume. LoadRunner can be controlled by setting the Run-time settings pacing for the at fixed intervals, every 1 sec, to control the execution time of each iteration by 1 seconds. If only one transaction is defined in the iteration script, and the TRT is less than 1 seconds, the number of VUser equals the number of concurrent users =tps, which can easily control the load target by adjusting the number of vuser. Note that if the iteration contains multiple transaction, or TRT becomes larger as the TPS target increases, it is necessary to adjust the number of vuser in real time and the time interval in every N sec here, based on the TPS target. 2. The application of no clear transaction volume The recommended application recommendations with no explicit trading volume are targeted to determine the maximum transaction capacity. Set pacing as as soon as the previous iteration ends, remove thinktime, deploy the pressure tool and be tested in the same network segment, no network bottleneck, so that vuser can be measured applications to generate maximum load. Weakening the amount of vuser sounds to increase until it reaches the maximum transaction capacity or other performance indicator thresholds (such as success rate or TRT) for the application being tested. New business and management Web applications belong to an application system with no explicit transaction volume. 3. The meaning of VUser Although it is recommended to weaken the meaning of vuser in determining the load target, the test should also pay attention to a situation, if the application is tested to have specific operation of the number of users, such as only the check-in or login users can submit transactions, the number of vuser can not be higher than the actual number of registered users. In accordance with the maximum number of users to pressure, to demand the TRT for the target tuning test applications, to maximize the TPS.