Apache jmeter-Automatic website test and Performance Evaluation

 

Refer to: http://bdql.javaeye.com/blog/291987

 

Out of enthusiasm for learning, the translation summarizes part of Emily H. halili's Apache jmeter book.

 

Jmeter Introduction

It is certain that jmeter meets at least the following automatic testing standards:

• Zero purchase cost-download from http://jakarta.apache.org/site/downloads/downloads_jmeter.cgi.
• Learning curve smoothing-only basic knowledge of HTML and regular expressions is required.
• Comprehensive functions-you can only test web applications.
• Scalable Architecture-its components are designed to complete large-scale testing through consolidation. The test can run on multiple hosts simultaneously.
• API scalability-Open API and scalable programming by users.
• Good technical support-this book is also available in online user guides, user forums, and online articles.

This chapter guides you through the first jmeter trial. We will introduce you to jmeterm and give you an overall understanding of its capabilities. In addition, we will show you the jmeter operation interface, allowing you to experience several important jmeter components.

 

Let's start...
Jmeter is a desktop application designed to test the C/S structure of applications, such as web applications or FTP applications. Is one of the most widely used open-source testing tools so far. It is written in pure Java and can be extended through its API. Jmeter works on the client in C/S. It can test the server response time, CPU load, memory usage, and resource usage.

 

Most jmeter test scripts or test plans involve creating a loop, simulating requests to the server based on predefined time intervals and thread groups. These test scripts can be customized and expanded. A basic jmeter test plan should include at least three parts: thread groups, listeners, and samplervers.
A thread group is used to simulate a group of users (with at least one user ).
The sampler sends requests to the target server.
The listener captures the response data received after each request.

 

Recognize it through the interface
A typical test plan consists of one or more thread groups, logical controllers, listeners, timers, assertions, and configurations.
Thread group: one thread simulates one user. All elements of the test plan must be placed under a thread group element.
Listener: monitors the data received when jmeter runs the test case.
Controller: its logic can be customized. For example, the tester asks jmeter to send a request to the server.
Timer: Set the interval between two requests sent by a thread.
Asserted: test whether the returned results are expected values.
Configuration: Related to the examiner. You can add or modify requests.

 

The screenshot is shown below.

 

 

The jmeter interface has two panels. Once jmeter is put into operation, you can see them. Shows the test plan and workspace panel. A test plan describes the test steps after jmeter is put into operation. The workspace provides temporary data storage space. Workspace data is stored separately from the test plan.

 

 

The left-side pane shows the test plan in a tree. You can right-click the node to add or delete the node. The panel on the right shows the detailed configuration of each node.

 
 

The following is a test plan consisting of a thread group and other elements.

 
 
 

When we use jmeter for performance testing, the thread group is a required element. This element can be used to simulate multiple users to perform stress tests on the server. You can even preset the testing time, which is a detailed display of the thread group.

 
 

On the other hand, the same test plan can be extended to include functional or regression testing. Add the assertion element, as shown in.

 
 

Officially started using...

Install jmeter
You must first download the latest release of the license from the following URL.
Reminder: To perform remote testing, avoid using a directory name with spaces during compression.

Set Environment Variables
First, we recommend that you install JDK or later. Set the java_home environment variable.

 

Run jmeter
There is a bin directory under your jmeter installation directory. In Windows, run jmeter. bat. If you are running on a UNIX system, you need to run jmeter.
In this process, jmeter will automatically find the required jar package in its lib directory or lib/EXT directory. Therefore, if you develop jmeter components in the future, you also need to copy them to the LIB/exit directory of jmeter.
If you run jmeter behind the proxy or firewall, you need to provide the server name and port number, and use the command line parameters to specify.
For example
Jmeter-H 129.198.1.1-P 8000-u someusername-A someuserpassword-N localhost

 

 

Test Plan

The jmeter test plan encapsulates the test scripts you need to manually write.
What is a test plan?
The test plan defines the framework for testing. The simplest Test Plan includes the following elements:
Thread group: used to specify the number of running threads and the number of loops during the ramp. Every thread simulates a user, and the ramp period specifies the time when all threads are created. For example, if you specify five threads and a 10-second ramp, the creation time of each thread is 2 seconds. The number of cycles defines the number of test repeats. You can also specify the start time and end time.

Sampler: used to send HTTP/FTP/soap/XML/jdbc/LDAP requests to the server.

Listener: used for post-processing request data. For example, you can save data to a file or display it in a chart.

 

Start to run jmeter, and the test plan will appear in the panel window on the left of the workspace.

The detailed description is as follows:
User Defined variables: You can define static variables, which are values that can be extracted repeatedly in the test, such as the server name and port number. For example, if you want to test an application of a www.example-jmeter.net and you can define a "server" variable, the value of "$ {server}" in the test script will be replaced.

Functional Test Mode: it records the data returned from the server. If you want to perform a stress test, do not select it because it will affect the results.
Run each thread group separately: run each thread group separately. If you have two or more thread groups in your test plan. If this option is selected, jmeter runs them sequentially. Otherwise, jmeter runs them concurrently.

Add directory or jar to classpath: If you have expanded the jmeter component, you can use this item to add a jar package. Remember that it takes effect only after being restarted. In addition, you can directly copy the jar package to the jmeter/lib directory. Or edit jmeter. properties file to include additional jar files, such as "# user. classpath = .. /classes ;.. /jars/jar1.jar to avoid the problem caused by spaces in the path.

 

Composition of test plan
The test plan must be composed of at least one thread group. Each thread group can contain one or more of the following combinations:
Probe, logic controller, configuration, listener and timer. Each sampler can be associated with one or more preprocessing elements, post-processing elements, and asserted elements. Let's learn more about them.

 

Thread group
A thread group is the entrance to the test plan. It contains all other jmeter elements. A thread group controls all simulation threads created by jmeter, which simulate multi-user requests.
A thread group is a group of users who execute specific test cases. It is displayed in the control panel of the thread group. If there are multiple thread groups, they are executed independently of each other.

 

The detailed description is as follows:
Action to be taken after a sampler error: Once the probe reports an error during the test, you can make the following options: continue the test and stop the thread to stop the current test, stop test to stop the test.
Number of threads: Number of simulated users or number of Web applications.
Ramp-up period: defines the duration for jmeter to create all test threads. For example, if the number of threads is set to 10 and the duration of the ramp is 60 seconds, each thread is created at a speed of 6 seconds. After 60 seconds, all threads will be put into operation. A better strategy is to set the duration of your ramp long enough to avoid heavy workloads from the beginning of the test, but not too long, you can set the ramp-up period to equal to the number of threads to be created at the beginning, and adjust it later as needed.
Loop count: defines the number of times a test is executed. By default, a test is executed only once. Click the forever option box and the test will be executed repeatedly until you manually stop it.
Scheduler configuration: Versions later than 1.9 have this option, allowing you to set the Test start and end time. The test starts only when the start time is reached. After each test cycle, the test continues until the end time reaches the upper limit of loop count. Startup delay is used to set the jmeter start wait time before the first test thread starts, and the duration is used to set the entire test time. The former is used to overwrite the start time setting, and the latter overwrites the end time setting.

 

 

Controller
Jmeter has two types of controllers:
1. samplers: Allows sending specific types of requests to the server. Later, we will send an HTTP request to the server, so we will use the HTTP rquest collector. You can use the configuration element to configure the collector.

2. logic controllers: logic that allows you to customize when a request is sent. For example, you can use random controllers to randomly send HTTP requests to the server.

 

Collector
The jmeter collector allows you to define a request sent to the server to simulate a request sent by a user to the server through a Web page. The result generated by each collector includes the following, such as performance, latency, and scalability. By default, jmeter sends the Request command in the order that the probe appears in the test plan tree. Of course, the logic controller can be further customized to process the sampling sequence. This will be further explained in the subsequent "logical controller.

You can customize each Sampler by setting its attributes, or you can add configuration elements. In this book, because we will send many HTTP requests to the same server, we can use the default configuration element, which predefines the HTTP requests sent.

The following figure shows the sampling plane of an HTTP request:

 

 
If you want jmeter to perform verification, you can add the assertion element. For more information about assertion, I will introduce it in detail later.
The response assertion in the HTTP request sampling panel is shown as follows:

 

 

A list of all the samplervers provided by jmeter is as follows:

HTTP Request
FTP request
JDBC request
Java request
Soap/XML-RPC request
WebService (SOAP) Request
LDAP request
LDAP extended request
Access log Sampler
Beanshell Sampler
BSF Sampler
TCP Sampler
JMS publisher
JMS subscriber
JMS point-to-point
JUnit request
Mail reader Sampler
Test action

 

Logical Controller: used to define the execution sequence of a sampling machine. The sub-elements of a logical controller are: samplerger, configuration, and other logical controllers. Jmeter may randomly select (using a random Controller), repeat (using a loop controller), exchange (using a staggered Controller), and so on.

Several logical controllers can be combined to achieve different results. The following figure shows a loop controller panel: list all logical controllers provided by jmeter: simple controller
Loop Controller
Once Only Controller
Interleave Controller
Random Controller
Random order controller
Throughput Controller
Runtime Controller
If Controller
While Controller
Switch Controller
Foreach Controller
Module Controller
Include Controller
Transaction Controller
Recording Controller Listener
Displays the result of the collector as a table, graph, tree, or log text. Displays only the data of elements defined under the listener.
Each listener displays data in its own way. For example, you can use the "aggregate report" listener to display graphs. "Aggregate report" or "Summary Report" listener display table. All listeners write data to the output file with the extension jtl.
The panel of an aggregate graph listener is shown as follows:
Public attributes of all listeners:
Configure button: used to select the information to be written to the file and save it in XML or CVS format. The extension is jtl. The CVS format makes the file smaller. Once selected, save Configuration window will appear. Browser button: If you want to read or display the previously saved results.
A listener consumes a considerable amount of memory if it is associated with multiple collectors. Jmeter slows down if you choose to activate multiple listeners. Therefore, select a listener as much as possible.
One way to reduce memory usage is to save data in the default CSV format and use the simple data writer listener.
You can change the default format to XML by editing jmeter. properties. Change jmeter. Save. saveservice. output_format = CSV to XML. The following is a list of all listeners: sample result save Configuration
Graph full results
Graph results
Spline visualizer
Assertion results
View results tree
Aggregate report
View results in table
Simple data writer
Monitor results
Distribution graph (alpha)
Aggregate Graph
Mailer visualizer
Beanshell listener
Summary Report Timer
The timer component is an optional component that causes jmeter to pause, because by default, jmeter sends the next request immediately after a request ends, which can cause server overload. Adding a timer reduces the risk of server crashes. This feature is useful for functional testing.
In any case, you need to be careful to add too many timers to a test plan. Jmeter suspends the sum of all timers that can be used for the specified time between two requests. The constant Timer control panel is as follows:
 
The following is a list of all jmeter Timers:
Constant Timer
Gaussian random Timer
Uniform Random Timer
Constant throughput Timer
Synchronizing Timer
Beanshell timerassertions assertions: allow you to add verification conditions for the response. They are added as sub-elements of the collector. It is particularly useful for functional testing. In performance testing, you can use it to ensure that you have received the response data. Does not contain content errors or missing parts, as this may affect the effectiveness of the test.
You can use regular expressions like Perl to create assertions. Use this to determine whether the value returned by the application is what you expect. Let's take a look at this scenario. In an HTTP request sampler, If you request a page and return a "Login successful" string, the access is successful, you can use response assertion to determine whether the string exists in the response. As shown in: assertions-related listeners include "view result in table", "view result tree", "aggregate Report", "Summary Report" listeners, the first three listeners will display the assertions results in detail, while the latter will only briefly display the failure ratio.
The following is an assertion results panel:
The following is a list of all assertion provided by jmeter: Response assertion
Duration assertion
Size assertion
XML assertion
Beanshell assertion
Md5hex assertion
HTML assertion
XPath assertion
XML Schema assertion configuration element
The configuration element allows you to create default variables for use in the probe. Used to modify the request content sent by the sampler.
They can only be accessed by samplervers within the same range and executed prior to samplerow.
The following is the HTTP request ults panel,

Request default 1 can be accessed by the request1 and request3 samplervers because it is the parent node, while request default2 can only be accessed by request1, and request default3 can only be accessed by request3.
The following is a list of all configuration elements provided by jmeter:
CSV Data Set config
FTP request defaults
HTTP Authorization Manager
HTTP cookie Manager
HTTP Proxy Server
HTTP request ults
HTTP header Manager
Java request ults
JDBC connection Configuration
Login config Element
LDAP request defaults
LDAP extended request defaults
TCP sampler config
User Defined variables
Simple config elementpre-processor Elements
Pre-processing allows you to modify the range of sampling devices. They are often used to modify request settings before running, or to update variables that cannot be obtained from the response text.
The following are pre-processing elements provided by jmeter.
HTML link parser
Http url re-writing Modifier
HTML parameter mask
Http user parameter Modifier
USER Parameters
Counter
Beanshell preprocessorpost-processor Elements
The post-processing element is executed after the sampler sends a request. A good practice is to use it as a sub-element of the probe to ensure that it is only executed after a specific probe is run, rather than all the samplervers. This element is generally used to process response data, for example, to extract variable values that will be used later.
The following are all post-processing elements provided by jmeter.
Regular Expression Extractor
XPath Extractor
Result Status Action Handler
Save responses to a file
Generate Summary Results
Beanshell postprocessor build a test plan for testing the web site
This section describes how to create a basic website test plan. We will expand the plan later for performance testing (Chapter 1) and functional testing (chapter 2 ).
The following elements are required for a basic test plan:
Thread group
HTTP Request (sampler)
HTTP request default (configuration element)
Summary Report (listener) can be performed in the following steps:

• Add User
• Add and configure the default HTTP Request
• Add HTTP Request
• Add listeners to display and store test results
• Save and run the test plan

First, run jmeter (double-click jmeter. bat in the jmeter/bin directory). You can see the default interface, including the test plan and workspace. Add User
1. Right-click the test plan icon in the left pane and select Add | thread group.

The control panel of thread group is displayed on the right.
2. Rename the thread group to my users.
3. Set it to a connection and a user.
4. Change ramp-up period to 0, which means the user will start testing immediately. If there are other users, it means all users start testing immediately. The following figure shows the settings:
Next, perform other steps.
Add default HTTP Request
1. Right-click the My users element and select Add | config element | HTTP request ults.
2. Select and display its console interface.
3. rename it to my URL.
4. Enter www.mocksite.net or another URL in the server name or IP address bar. The rest will not move. Setting the HTTP request default element will cause all request samplervers in my users to access the same server.
Shown as follows:

Add HTTP requests
In this step, we will make two simulated requests to the service: the default page and the other page. Jmeter is executed in the order they appear in the test plan tree. We first add the response page request.
1. Right-click the My users thread element and select Add | sampler | HTTP request.
2. Select HTTP request from the left pane and make the following changes:
A. Name column: Home Page
B. Web server: this information has been set in HTTP request defaults. This does not need to be modified unless you need to redirect to another different path.
C. Path column :/
Because we have already specified the server name in the HTTP request default element, we do not have to specify other content.
3. Add other HTTP request elements under the Home Page element.
A. Name column: Sample Page
B. Path bar:/sample.html (or URL you know)
Because we have already specified the server name www.mocksite.com in the HTTP request default element, this path will be added to form the complete path.
The completion diagram is shown below:
Add listener
Finally, we add a simple listener, aggregate report. This element stores the results of all HTTP requests and outputs them in a table.
1. Select the My users element and add | listener | aggregate report.
2. Write all data to a file: output the directory name in the filename column or use the Browse button to select and enter the output file name.
The completion diagram is shown below:

Save and run the test plan
Jmeter requires us to save the test plan before running.
1. Select save test plan from the File menu. Jmeter allows you to save only a part by selecting a branch.
2. Enter the file name in save as, for example, enter my users. The extension is. JMX.
3. Select Run from the run menu.
The following are the running results of aggregate report element and control panel. Note that there is a small box in the upper right corner. The number next to it shows the number of activated threads and total number of threads.

 
The following is the interface after the test plan is run. The box in the upper-right corner is gray and only displays the total number of threads. You can also manually stop the test by selecting stop from the run menu.
The report column is described as follows:
Label-label of the example
# Sample number of samples-URL
Average-Average Time
Median-the time in the middle of a set of results
90% line-the maximum time taken for the fastest 90% of the samples
Min-the lowest time for the samples
Max-the longest time for the samples
Error %-percent of requests with errors or failures
Throughput-throughput measured in requests per unit of time
KB/sec-the throughput measured in kilobytes per second