"Software performance testing process and case analysis" reading notes

1, the basic concept of software testing response time

Definition: The time that is required to respond to a request. Is the user perspective of the software performance of the main embodiment, with subjective color, no absolute length. "Web page reasonable response time standard": 2/5/10 seconds, 10-second maximum

Divide the response time into "Render time" and "system response time". Rendering time depends on how much time the data takes to render the page after the client receives the data, and the general performance test does not focus on "rendering time" because it depends largely on client performance. So the response time here is "system response time".

Page response time can be further decomposed into: "Network transfer Time", "Database Delay Time" and "Application Server delay Time"

Concurrency number

Number of business concurrent users definition: number of users accessing the system during the same time period (although accessing the system, but not necessarily pressure on the server, such as just browsing the system)

The maximum number of concurrent accesses that the server is subjected to is defined: from the pressure on the service side, it describes the customer who makes the request to the client at the same time, which embodies the maximum number of concurrent accesses that the server is enduring, and often combines the concurrency test to discover the problems of resource competition caused by concurrency in the system.

Example: If an OA system has 2000 users, the highest peak when there are 500 people online, of which 40% are looking at system announcements, 20% in a daze, 20% in the page jump, 20% in filling out the form. Then: the number of users of the system is 2000, the number of online users is 500, the number of business concurrent users is 500, the service side of the concurrency depends on the number of business concurrent users and user business scenarios.

Used to estimate the number of business concurrent users: where, in Equation 1, c is the average number of concurrent users, N is the number of users of login session, L is the average length of the login session, T is the length of the time period of the visit, generally 8 hours of work time. Equation 2 gives a method of calculating the peak number of concurrent users, which is estimated from Poisson distribution. If you know the average number of requests per user u, you can estimate the throughput as u*c.

Application server can be analyzed by log analytics to understand concurrency data and recommended Awstats open source tools (http://awstats.sourceforge.net/)

Throughput

• Definition: The number of customer requests processed by the system per unit of time, as measured by the number of requests/seconds or pages per second in the general web system. From the business perspective, available visitors/day, number of business/hour measurement, or network angle bytes/day

• In the absence of performance bottlenecks, throughput can be calculated as: f= (n*r)/T. where n represents the number of users, and R indicates the amount of requests per user, and T represents the time spent in the performance test.

• The graph that is commonly used to analyze the throughput of analysis is the association graph of "throughput--vu (number of virtual users)". Although the throughput metrics are seen as a reflection of the system's stress, the same throughput pressure on a system can be tested differently in the case of a different number of concurrent users. For example, for the same system, test a uses 100 vu, each with a vu interval of 1s to send a request, Test B takes 1000 concurrency, sends a request every 10 seconds, both throughput is 100, but B is more prone to performance bottlenecks. two different systems may have different user numbers and user usage patterns, but if you have basic, consistent throughput, you can say they have the same average processing power.

Performance counters: Number of memory, process time, resource utilization think Time (Think times), or sleep time

• The time interval between each request when the user is operating. In an automated testing perspective, you have to wait for a period of time between actions in your test script.

RBI test Method: Rapid bottleneck identify performance degradation curve Analysis method: Single user area, performance flat area, pressure area, performance inflection point LoadRunner performance test process 2, Performance test application domain method classification:

• Performance testing: Simulate the business pressure and use scenario combinations to verify that the system has the declared capabilities.
• Load test: By increasing the pressure on the system under test, until the performance index is saturated, the aim is to find the processing limit of the system. By means of "detect-pressurize-until performance indicators exceed expectations"
• Stress test: Test system in a certain saturated state, such as CPU, memory and so on in the saturated state of the system's conversational ability, and the system will be wrong. By increasing the access pressure (such as concurrency), the system has error -free information, System response time , and system stability over time. Generally will be * "CPU utilization of more than 75%, memory utilization of more than 70%, system error rate, response time" as a description.
• Configuration test: Through the system hardware and software adjustment, to understand the different configurations of the impact on the system performance, so as to find the optimal allocation principle. Typically used for performance tuning and planning capabilities
• Concurrency testing: test for deadlocks or other performance issues when multiple users concurrently access an app by simulating user concurrent access. Focuses on memory leaks, thread locks, resource contention , and more. Can be designed for a system or architecture, and can be used at various stages of development.
• Reliability testing: The system is tested for stability by loading a certain amount of business pressure (such as resource usage in 70%~90%) to keep the application running for a period of time.
• Fail-back testing: For system designs with redundant backups and load balancing, it can be used to verify if the system is partially faulted, whether the user can continue to use the system, and how much the user will be affected.

Application fields:

• Competency Validation: Validates the performance of a deployed system. such as performance testing, reliability testing, stress testing, failure recovery testing
• Planning capabilities: Care about how to make the system have the performance capabilities we require, such as whether the system can support user growth over time. is a kind of exploratory test. such as load testing, configuration testing and stress testing
• Performance Tuning: Configuration testing, load testing, stress testing, and fail-back testing
• Found defects: Concurrency test, stress test, fail-back test

3. Performance counter and performance analysis method

Viewing and documenting analytics at the operating system level, application server level, and database level

Operating System counters:

• Memory Analysis (Application command Vmstat):
• Processor Analysis (command top):
  1. View System\%total Processor Time performance counter count value: represents the overall processor utilization (or average utilization) of the server. When it continues to exceed 90%, it faces bottlenecks that require additional processor
  2. Look at the processor\%processor time and Processor\%user times per CPU (the CPU hours that are consumed by system non-core operations, when the server is a database server, which may be database sequencing or function operation consumption time), and Processor \%privileged time
  3. View the System\Processor Queue Length when it is larger than the total number of CPUs + 1 o'clock, indicating that processor blocking occurs. When the processor\%process time is higher, it is usually accompanied by blocking, whereas the other is not necessarily.
• Disk I/O analysis method: For the database server, or file server, streaming media server; Disk/transfer between 15~30ms is good, more than 60ms should consider replacing the hard disk, etc.
• Process Profiling methods: View the%processor time value of the process, and the processing times that the process consumes. View page invalidation.
• Network analysis Application Server counters: IIS application Server, Tomcat database counters

4. Principle of performance testing tools

Performance tests are not equivalent to performance testing tools; Performance test script recording refers to the recording of communication data between the server and the application, rather than the GUI operation, which requires the recording protocol to be selected first

Performance test Architecture (take load runner as an example)

• Virtual User Script Generator
• Pressure generator: Used to generate actual load based on school-based content. (typically a PC with 512MB of memory can run 200 or so of VU)
• User agent: A process running on a load machine (which can be seen as part of a pressure generator), typically running on a load machine in the background
• Pressure dispatch and monitoring system
• Pressure result analysis function

"Software performance testing process and case analysis" reading notes