Performance Test Results analysis

Performance Test engineers are basically able to use the test tools to load, stress testing , but most of the analysis tool to collect the results of the test is not enough, below I put personal work in the experience and collected the relevant data sorted out, I hope to be able to analyze the results of the test to help. Analysis principle:

1. Specific analysis of specific problems (this is due to different application systems, different testing purposes, different performance concerns)

2. Find bottlenecks in the following order, from easy to difficult.

Server hardware bottleneck-〉 network bottleneck (for LAN, can not be considered)-〉 server operating system bottleneck (parameter configuration)-〉 middleware bottleneck (parameter configuration, database , Web server, etc.)-〉 application bottlenecks ( SQL statements, database design, business logic, algorithms, etc.)
Note: The above process is not required in each analysis, to determine the depth of the analysis according to the purpose and requirements of the test. For some low requirements, we analyze the application system in the future of large load pressure (concurrent user number, data volume), the system hardware bottleneck is enough.

3 section exclusion is effective
Sources of information analysis:
1 error message in the course of the scene running
2 monitoring metrics data collected according to the test results

A Error prompt Analysis
Analysis Example:
1 error:failed to connect to server "10.10.10.30:8080″: [10060] Connection
Error:timed out Error:server ' 10.10.10.30″has shut down the connection prematurely
Analysis:
A, Application services die.
(Small User: a procedural problem.) Problems with the database on the program)
B, application services Not dead
(Apply service parameter setup issues)
Example: In many client connections WebLogic application servers are denied, and there is no error on the server side, it is possible that the Acceptbacklog attribute value of the server element in WebLogic is set too low. If you receive a connection refused message when you connect, you should increase the value by 25% each time
C, the connection to the database
(1, the performance parameters of the application service may be too small 2, the maximum number of connections to the database startup (related to the memory of the hardware))

2 error:page Download timeout (seconds) has expired
Analysis: May be due to the following causes
A, the application service parameter setting is too big to cause the server bottleneck
B, too many pictures on the page
C, check the field too much when the program is working on the table

Two Monitoring Index data analysis
1. Maximum number of concurrent users:
The maximum number of concurrent users that the application system can withstand under the current environment (Hardware environment, network environment, software Environment (parameter configuration)).
In a scenario run, if there are more than 3
The user's business operation failed, or the situation of the server shutdown, that in the current environment, the system can not withstand the current concurrent user load pressure, then the maximum number of concurrent users is the previous one does not appear this phenomenon of concurrent users.
If the maximum number of concurrent users reached the performance requirements, and the server resources in good condition, business operation response time has reached the user requirements, then OK. Otherwise, the reason is further analyzed according to the resource situation of each server and the response time of business operation.

2. Business Operation Response Time:
The analysis scenario run should start with an average transaction response time chart and a transactional energy summary graph. Use the transactional summary diagram to determine which transactions are responding over a long period of time during scenario execution.
Subdivide the transaction and analyze the performance of each page component. See which page components are causing the too long transaction response time. Whether the problem is related to the network or server.
If the server is taking too long, use the appropriate server diagram to determine the problem server metrics and find out why the server's performance is down. If your network is taking too long, use the Network Monitor diagram to identify network problems that are causing performance bottlenecks

3. Server Resource monitoring metrics:

Memory:
1 The Index memory page exchange rate (paging rate) in UNIX resource monitoring, if the value is occasionally higher, indicating that the thread was competing for memory. If it continues to be high, memory may be a bottleneck. It is also possible that the memory access hit rate is low.
2 in Windows Resource Monitoring, if the value of the Process/private bytes counter and the Process/working set counter continues to rise over a long period of time, the memory/available The value of the bytes counter continues to degrade, and there is a good chance of a memory leak.
Memory resources are a symptom of system performance bottlenecks:
A very high rate of page change (Gao pageout rate);
The process enters an inactive state;
All disks in the swap area have a high number of activities;
High CPU utilization of global system;
Not enough memory error (out of memory errors)

Processor:
1 The Metric CPU occupancy rate (CPU utilization) in UNIX resource monitoring (similar to the Windows operating system), indicating that the bottleneck is CPU if the value persists over 95%. Consider adding a processor or switching to a faster processor. If the server is dedicated to SQL Server, the maximum acceptable limit is 80-85%
The range of reasonable use ranges from 60% to 70%.
2 in Windows Resource monitoring, if the system/processor Queue length is greater than 2 and the processor utilization (Processor time) is low, there is a processor blockage.
CPU resources are a symptom of system performance bottlenecks:
Slow response times (slow response time)
CPU idle time is 0 (zero percent idle CPU)
Excessive user CPU time (high percent user CPU)
Excessive CPU time (high percent system CPU)
Long running process queues (large run queue size sustained over time)

Disk I/O:
1 The metric disk exchange rate (diskette rate) in UNIX resource monitoring (similar to the Windows operating system), if the parameter value has been high, indicates I/O is problematic. Consider replacing a faster hard drive system.
2 in Windows Resource Monitoring, disk bottle diameters may exist if the value of disk Time and Avg.Disk Queue length is high and page reads/sec page read operation rate is very low.
I/O resources are a symptom of system performance bottlenecks:
Excessive disk utilization (high diskette utilization)
Too long disk wait queue (large diskette queue Length)
The percentage of time waiting for disk I/O is too high (large percentage of times waiting for disk I/O)
Too high physical I/O Rate: large physical I/O rate (not sufficient in itself)
Low cache Hit rate (lower buffer cache hit ratio (not sufficient in itself))
Run the process queue too long, but the CPU is idle (large run queue with idle CPU)

4. Database server:
SQL Server database:
1 SQL Server Resource Monitoring index cache CTR (Cache Hit Ratio), the higher the value the better. If you continue below 80%, you should consider adding more memory.
2 If the full scans/sec (all Table Scan/sec) counter displays a value that is higher than 1 or 2, you should analyze your query to determine if a full table scan is really required and whether the SQL query can be optimized.
3 of Deadlocks/sec (number of deadlocks/sec): Deadlocks are very harmful to the scalability of your application and can lead to a bad user experience. The value of this counter must be 0.
4 Lock Requests/sec (Lock request/sec) to reduce the number of reads by optimizing the query, you can decrease the value of the counter.

Oracle Database:
1 If free memory is close to 0 and the hit rate of the library cache or data dictionary is less than 0.90, you need to increase the size of the shared_pool_size.
Quick Save (Shared SQL area) and data dictionary cache hit ratio:
Select (sum (pins-reloads))/sum (pins) from V$librarycache;
Select (sum (gets-getmisses))/sum (gets) from V$rowcache;
Free Memory: SELECT * from V$sgastat where name= ' freedom memory ';

2 if the cache hit rate for the data is less than 0.90, you need to increase the value of the Db_block_buffers parameter (in units: block).