Introduction to Software Testing--basic knowledge of software testing (vi)

Preliminary Contact performance Test!! Study book "LoadRunner Performance Test Training camp"

• For different systems, performance testing has different concerns.

The products of C/s architecture pay more attention to system resource usage, database performance, configuration requirements of operation and so on. Such as: memory, number of user connections, database deadlock, Database cache hit rate, minimum run configuration, and so on.

b/S Architecture products focus on the relevant indicators of the Web server. such as: CTR per second, throughput, number of attempts to connect, transaction success rate, and so on. b/S architecture is more complex.

• Purpose of performance testing (Know how fast & how much)

1. Evaluate the current system

2, to find bottlenecks, optimize the system (analysis → positioning → tuning)

3, predict the future performance (to consider the number of users and the increase in the volume of the system how to respond and adjust timely and so on)

• Terms and metrics for performance testing

1, concurrent number

First, 3 concepts are identified: number of system users, number of online users, and number of concurrent users.

The number of system users refers to the registered users of the system; The number of online users is the user who is logged on to the system, and the concurrent user is the user who is on-line and has pressure on the server .

The number of concurrent numbers can be determined by analyzing the server log. Common log analysis tools are Awstats, Webalizer, Analog, deep Log analyzer and so on.

Two understandings of concurrency: ① All users do the same at the same time ② multiple users to initiate multiple requests, the request may be different.

2. Response Time

　　　　Response time = Network transport (Request) time + server processing (one or more layers) time + network Transport (response) time + page front end parsing render time

After a general client initiates a request, it is pre-processed to determine if there is a cache. If so, directly read the cache, and then data processing and rendering, if not, to resolve the DNS domain name to obtain the server IP, the connection server sends the request,

After the server responds, the data is returned and then processed and rendered.

3. Number of transactions per second

　　　TPS refers to the number of transactions per second, which directly reflects the performance of the system.

Comparing it to the average transaction response time, you can analyze the effect of the number of transactions on the response time.

When the pressure increases, if the TPS curve changes slowly, it is likely that the server is starting to bottleneck. If the environment has not changed greatly, there will be a maximum processing transaction capability for the same system, which does not change with the number of concurrent users.

4. Number of hits per second

　　　　represents the number of HTTP requests that a user submits to the Web server per second. It is important to note that it is a request to the user and multiple requests to the backend. For example, clicking on a page link returns each image on the page and requires an HTTP request for each module.

The number of hits per second from the side reflects the status of the client, the number of hits per second is not normal, may be caused by network problems or scripting problems, need further specific analysis.

5. Throughput

　　　　Throughput refers to the number of requests processed per unit of time, which directly reflects the pressure on the server, and is a key concern.

To differentiate from throughput, throughput is the amount of data that a user obtains from the server within a given second, or the amount of data the server returns.

6. Think Time

Two understandings: When ① the user, the interval between each request or Operation ② meet the specific needs of the business, limiting the user's request to a certain time interval cannot be sent a second time.

7. Resource utilization

This part of the content is too miscellaneous, the main first to understand a few key indicators.

①cpu

Can reflect the busy degree of the system, divided into the system CPU and user CPU, wherein the system CPU is the processing system itself occupies resources, and the user CPU is the processing of resources occupied by the program, the object is different.

②load Average

This is the statistic that the CPU is processing and waiting for CPU processing for a period of time, that is, the amount of CPU usage queue.

③memory

Collect all kinds of information and store them. Because data is read from memory faster than it reads from disk, memory is often leaked or overflowed, and attention needs to be focused.

Less memory is available for a short period of time and does not necessarily indicate a memory leak or overflow. (May be memory corruption, detection software vulnerabilities, virus-hogging programs, etc.)

④ queue

The queue length indicates that the processing power may have reached the limit or encountered a blockage.

⑤io

Interaction with the disk, focusing on the switching frequency and disk queue length.

⑥ Network

Focus on network traffic to see if there is a bottleneck in network bandwidth.

• Performance Test classification

The classification is more ah, understand the characteristics and concepts can be, do not need to strictly distinguish.

1. Benchmark Test

The application scenarios are as follows: ① a system has never performed any performance tests and needs to perform a performance evaluation of the system as a reference for subsequent development tuning. Common

② can establish a performance benchmark under the established standards by benchmarking, so that when the system's environment and parameters change, a test under the same standard can be used to see the performance impact of the change.

③ benchmarking can identify performance issues at an earlier stage and reduce unnecessary testing.

2. Concurrent Testing

Many users have concurrency problems when they request a business or feature in a predetermined scenario, such as memory leaks, thread locks, resource contention, and so on. The concurrency test was conducted to find out the problems caused by concurrency.

method to determine the number of concurrency required: ① concurrency =PV/PV time * Page connections * HTTP response times * Factor/number of Web servers

PV (Page view): PageView. PV time is the statistics of PV times, converted into seconds, a day is 86400s.

The number of page connections include external JS, CSS, pictures, etc., generally 10. HTTP response times can typically be 1s or less. The factor is generally 5.

② Classical theory 80-20 Principles (80% of the business volume is completed in 20% of the time. ）

Throughput =80%* Traffic/(20%* time)

Note: The 28 principle evaluates to a throughput rather than a concurrency number.

③ in Dening Teacher's "software performance testing process and case analysis" mentioned in the following 3 kinds of estimation methods:

1>. C=NL/T (1)

C ' ≈c+3√c (2)

In the formula (1), C is the average number of concurrent users; n is the number of log sessions; L is the average length of the log session; t refers to the length of the time period of the investigation. For example, for a typical OA application, the length of the study should be 8 hours of working time.

The formula (2) gives a method for calculating the peak number of concurrent users, where the value of C ' is the peak of the number of concurrent users. The formula is estimated by assuming that the user's logsession produces a Poisson distribution.

Eg: Suppose there is an OA (office automation) system, the system has 3,000 users, an average of about 400 users per day to access the system, for a typical user, the average user from logging on to exit the system in a day is 4 hours, in a day, the user only within 8 hours to log on to the system. According to the formula (1) (2) can be obtained:

c=400*4/8=200

C ' ≈200+3√200=242

If you can know the average number of requests per user (assuming u), then the total throughput of the system can be estimated as u*c.

However, it is not easy to estimate C and L, and there are some deviations in this method, considering that there is a certain time concentration of business operations. Suggested improvement two points:

First, with finer time granularity, such as setting the granularity of 1 hours for investigation time, can solve the problem of time centralization;

The second is to consider the typical business model: different businesses have different business models, such as an internal system typically 30 minutes to 1 hours after the start of work in the centralized user login, and so on, based on these scenarios to estimate.

2> for Web systems used within the enterprise, a more general (and less accurate) empirical formula is:

C=N/10 (3)

C ' ≈r*c (4)

With 10% per day access to the number of users of the system as the average number of concurrent users, the maximum number of concurrent users is the number of concurrent users multiplied by the previous adjustment factor R, the value of R is generally a.

This approach can be used in performance tests that require less stringent performance tests or that have only a few data-supporting analyses.

3>. " Log Analysis "method

"Log Analysis" method refers to the application server through the analysis of the log, so as to understand the user status of the system, from the log to calculate the "maximum number of concurrent user access to the server" data.

The accuracy and reliability of the data obtained by this method are relatively high. This is most plausible for applications such as Internet applications that cannot estimate the number of users and user behavior patterns.

3. Load test

The main purpose is to verify the processing performance of a business or system under a given load condition, and also to focus on response time, number of transactions per second, and other related metrics. Load tests are designed to identify performance issues, and performance tests are designed to obtain performance metrics. In the performance test, can also not adjust the load, but in the same load situation by changing the structure of the system, algorithm, hardware configuration, etc. to obtain performance indicators.

4. Pressure test

There is no expected performance index, constantly pressurized to see when the system crashes, in order to determine the system bottleneck or unacceptable performance inflection point, in order to obtain the best concurrency of the system, the maximum number of concurrent. Stress testing allows for faster discovery of memory leaks and faster detection of the causes of system stability.

5. Stability Test

To know the stability of the system, it takes a long time to run, in this period of time to observe the system error probability, performance trends, and thus greatly reduce the system after the crash and so on. Typically, the so-called 7x24-hour stability test is performed.

Note: ① general stability testing needs to be done after the system has been molded and no serious bugs exist.

The design of the ② scene is best for simulating the actual operation of the real user.

6. Failure Recovery Test

The fail-back test focuses on whether the system can be recovered according to a predetermined policy after the problem has occurred and whether it will function properly after the recovery. However, failure recovery testing is generally carried out on a system with load balancing, mainly to test if the system has a local failure, whether it will have a large impact on the global, whether the impact is acceptable, and the user can continue to use the system.

In the actual application process, can simulate one or several load Balancer machine failure to carry out the failure recovery test, but it should be noted that not only after the failure of the user can be normal access or recovery after the system can work properly, but also pay attention to the failure, the system can also support how many concurrent users, And what alternatives are used to respond quickly.

7, the current network performance test

is in the actual network, the actual environment of testing, completely and real users, there is a certain risk, need to pay attention to:

① time period Selection: The current network performance test may affect the normal user, to avoid peak, choose midnight or early morning to carry out.

② Garbage Data processing: If the current network performance testing involves the operation of writing data, these data must be processed later, in order to facilitate the processing, the design of pre-data should have a regular follow-up.

③ Network restrictions: Current network testing If the sudden production of large amounts of data, may be limited by network bandwidth, and even the route will be considered illegal data requests to produce interception and so on. So in the current network test, the press needs and the server is deployed in the same network segment of the machine room, so that the network can avoid restrictions, and finally collect data remotely.

If there is no special situation, try not to conduct the intranet performance test, the risk is relatively large, if not to be carried out, it is important to fully assess the risks and solutions in advance, in order to respond quickly, the impact of the minimum control. So people still have to be a little bit, don't try to be brave.

Introduction to Software Testing--basic knowledge of software testing (vi)