Performance Test Summary (iii)--Tool selection Chapter

This article mainly summarizes the principle of performance testing tools and how to choose. Performance testing and functional testing, performance test execution is the basic function of the repetition and concurrency, need to simulate multi-user, in performance test execution need to monitor the indicator parameters, and performance test results are not so obvious, need to analyze the data. These characteristics determine that the performance test is more suitable for the tool to complete.

I. A brief discussion of why tools are needed

Let's take a look at the definition of a tool: it refers to the instrument that is required at work, and is then extended as a means of reaching, accomplishing, or promoting something . (---from Baidu's explanation)

1, from the point of view of human evolution, manufacturing and using tools is the most fundamental difference between man and ape, because tools can help us improve productivity and efficiency.

　　

2. Imagine what would happen if you did not use the tool for performance testing?

We can analyze the performance test from the perspective of Definition , performance testing refers to the simulation of a variety of normal, peak and abnormal load conditions through automated testing tools to test the system's performance indicators. If you do not use tools, manual performance testing will have the following drawbacks:

a) testing requires a significant amount of resources to be invested :

In order to simulate a variety of workloads, concurrent scenarios need to work together, often testing does not have a lot of resources, and even if there are resources artificial effect will be greatly reduced, and even some scenes alone can not be completed.

b) reproducibility is very poor :

Performance testing often requires repeated tuning and test execution, and without the help of tools, it is hard to imagine it by artificial.

c) Poor test accuracy :

Due to the need to simulate a variety of load and concurrency scenarios, if manually operated, there will inevitably be errors, and relative to the tool or program this error will be greater, the test results are very large.

d) Poor collection, collation and presentation of results :

If there are no tools, there will be a large error in the manual acquisition of data relative to the tool.

two . the relationship between performance testing and performance testing tools

1, performance testing from the test phase to classify the system testing, and the specific use of what tools do not have a direct relationship. Using tools is just one way and means to improve the efficiency and accuracy of performance testing. In essence, there is no real difference in the use of tools when doing other things.

2, performance testing is not equal to LOADRUNNER,LR is only one of the performance testing tools, and it is not omnipotent, in some cases it can not be useful. Recommended to see the " let LoadRunner go down the altar" and "Let LoadRunner go down the altar again" these two articles on performance testing and LR of the relationship between the more profound.

3, the difference between automated test tools and performance testing tools: Performance testing tools are generally based on communication protocol (the client and server exchange information to abide by the Convention), it can not care about the system's UI, and automation using object recognition technology, focus on the UI interface. Automation is impossible or difficult to load, but it is easy to pass the protocol.

Third, Performance test Tool selection reference:

Usually in a company or project, we choose any tool to do some research to choose the right tool for the company or project. The performance testing tools are no exception, and can often be considered in the following ways:

1, Cost aspects :

　　a) tool cost : Tools are usually divided into commercial closed (Shou) source (FEI) and non-commercial open (Mian) source (FEI) Two, business tools are usually more powerful, charge, because of the charge so can provide after-sales service, if there is a problem there are professionals to help deal with. and open source tools are usually free, limited functionality, maintenance tools of the organization is also spontaneous, so if you encounter problems need to solve their own, no one to provide services. The specific choice of business or open source tools, according to the company's situation, such as the size of the company, willing to bear the cost, the project comprehensive situation considerations. Generally, the cost of a tool that a big company can usually afford will be considered for buying business tools. Small companies may choose open Source tools because of financial pressure.

　　b) Learning costs : Using any tool requires learning, which results in learning costs (e.g. time), so we also need to take into account the learning costs of project team members when selecting Tools. If there are two tools A and b that can meet the requirements of the team test, if a tool is used by most people, and the B tool is used by a very small number of people, it is recommended that a tool be given priority. Typically, it is a basic requirement for testers to be familiar with the business (performance) tools of a process, an open source free (performance) tool, and familiarity with common (performance) scripting languages.

2. Supported Protocols :

Performance testing is usually very close to the protocol, such as b/s system usually use the HTTP protocol for client and server information Exchange, C/s system usually use the socket protocol for information exchange. When you select a tool, you need to consider the protocol used by the project. Whether a test tool can meet the test requirements and achieve satisfactory test results is the most basic question to consider when choosing a test tool.

3. Vitality :

There are a lot of performance testing tools now, such as lr,jmeter, such as the more popular tools on the Web related to a lot of information, but some small tools may be less online information. If the tool used in the process encountered a more extreme hand problems, in the recording solution or help, the public's tools will be relatively advantageous point, after all, the more people use, the more information, then the problem of their own may have met and resolved, immediately before the people encountered, because of the use of research more people, Through the community or forum to help believe there will always be experts to help solve.

4. Cross-platform :

Needless to say, see why Java has always been popular.

Iv. Common Performance Testing tools:

Performance testing tools, in theory, all the tools used in the performance testing process can be described as performance testing tools, usually divided into the following categories:

Description

• Server-side performance testing tools: need to support generating stress and load, recording and generating scripts, setting up and deploying scenarios, generating concurrent users, and exerting constant pressure on the system.
• Web front-end Performance testing tools: you need to work with client tools such as the Heart browser on the specific needs of the page to be presented.
• Mobile Performance testing tools: The same web-side performance testing tools also need to care about the process of the page, in addition to specific data collection functions, such as: Mobile phone CPU, memory, power, startup time and other data records.
• Resource monitoring tool: This is the ability to collect data from the performance testing process and good presentation of the results.

PS: This article mainly summarizes the server-side performance test tool LR and JMeter, the two tools will be followed by a simple alignment.

Five, common performance test tool features

• JMeter: The use of multithreaded models, the scalability is very strong, but the manufacturing pressure is not so high. It's good for some tomcat services, or some back-end interfaces. The disadvantage of JMeter is that the pressure value can not be precisely controlled, it is difficult to adapt to high concurrency, and because it is written by Java, it consumes more resources.
• LoadRunner: More like a simulator, it is more suitable for the front-end construction of more complex scenarios, such as simulating 100 user login scenarios, LoadRunner to non-technical staff to provide a good support. The LoadRunner does not apply to back-end interfaces.

The following table compares the tables for JMeter and LoadRunner:

Describe	Jmeter	LoadRunner
Architecture Principles	Monitor and collect the instructions of the concurrent clients through intermediary agents, generate scripts, and then send the application server, monitoring the application server feedback process	With JMeter
Installation	Simple, decompression, more flexible	LoadRunner installation package is relatively large, installation is cumbersome, the tool itself is relatively cumbersome
supported protocols	supports multiple protocols: http , https soap , Span class= "Font6" >ftp , database via JDBC JMS	supported protocols are very many, more comprehensive, but it seems that the tool itself is cumbersome, not flexible
Script recording	Provides a capability to record generated test scripts using a local proxyserver(proxy server), and also supports badboy recording regeneration into JMeter Script	Powerful self-recording with direct recording playback
Concurrency model	Increase concurrent users by increasing the number of thread groups, or by setting a cycle count	Supports multiple concurrency models by selecting what scenario to set in the scene and then selecting the number of virtual users
Distributed testing	Support, can set up more than one agent, through remote control to achieve multiple machine concurrency pressure	With JMeter
Resource monitoring	implemented via jmeterplugins plug-ins and serveragent	Self-brought resource monitoring function
Report Analysis	generate HTML reports by integrating with Ant	Self-supporting HTML,Word report
Virtual IP	Not supported	Support
Network simulation	Not supported	Support
Scalability	Open source, can modify the source according to demand	Implemented by extension function library
Learning costs	Mainly the information on the self-study official website.	Online materials and related training a lot, buy genuine words, there are technical support

Six, the Performance Test Tool Learning Tutorial:

1, LoadRunner: Not yet organized, follow-up will slowly organize into articles, please look forward to ...

2, Jmeter: Can see my other article Jmeter Tutorial index sticker

3, Gatling: Not used, online information is also relatively small, the introduction of the new generation of server performance testing tools Gatling

Performance Test Summary (iii)--Tool selection Chapter