How to save JMeter performance test data to Elasticsearch, and use Kibana for visual analysis (1)

Objective

JMeter is an open source tool for performance testing, stress testing, and is being tested by a large number of testers to test product performance, load, and more. JMeter In addition to the powerful presets of various plugins, various visual charting tools, there are some inherent flaws, such as:

• We often can only analyze the performance of the same deployment in the report, it is inconvenient to make a vertical comparison, for example, each build will run a one-time test, but the two build between the performance is not bad? We can only get the results report, and then we use other third-party tools to analyze
• The report generated by the JMeter chart plugin is not flexible enough to be a fixed number of dimensions and cannot be analyzed more flexibly.

This article will attempt to write JMeter sample data to Elasticsearch, and then through the Kibana powerful search and visualization capabilities, perform various dimensional performance analysis, help development testers to identify performance bottlenecks, monitor system performance changes, to the entire development, Testing and Operations teams publish reports

Question How to automate the writing of various raw data for performance testing to Elasticsearch

There are two possible scenarios for information collection,

• A kind of let jmeter generate a variety of reports, and then after the test run, the results of their own analysis of the script, the data into the Elasticsearch, this way does not affect the operation of JMeter, but not universal/real-time, And because the general JMeter reports do not contain comprehensive performance information, the lack of information is fatal, without the original information, the next analysis is water without
• Another is that we use, real-time acquisition of JMeter sample performance information, without affecting the performance of JMeter province, relatively real-time results written to elasticsearch. The advantage is that the information is complete, and can contain other custom parameters, in the long-run load test, you can monitor the performance of the system in real-time

For data acquisition, technically, we have developed a JMeter backend listener plug-in that handles the results of each sample of JMeter.

Information about Backendlistener can be viewed http://jmeter.apache.org/api/org/apache/jmeter/visualizers/backend/BackendListenerClient.html

We focus on the information that Sampleresult contains, and basically we can get:

• The requested URL
• Various parameters of the request
  • Cookies
  • Headers
  • Paramters
• Various parameters of the response
  • Headers
  • Cookies
  • Body
  • Response Code
• JMeter Related Information
  • Sample Label

This information is enough for us to analyze the performance, in the specific collection, we will also look at their own needs, only to save the parameters of interest, to save elasticsearch storage space, such as only the body of the response to save the error

For Sampleresult, refer to Https://jmeter.apache.org/api/org/apache/jmeter/samplers/SampleResult.html

There are several technical options for saving sample results:

• Through Elasticsearch's restclient
• Through the transportclient we use here
• Through Elasticsearch's nodeclient

Three ways performance is improved in turn, the function can be realized is enhanced. Because we just save the data, and there is no elasticsearch management requirements, but there is a strong demand for performance, so we chose Transportclient, about three ways to connect the details, you can see:

• Restclient https://www.elastic.co/guide/en/elasticsearch/client/java-rest/current/index.html
• Transportclient https://www.elastic.co/guide/en/elasticsearch/client/java-api/current/transport-client.html
• Nodeclient https://www.elastic.co/guide/en/elasticsearch/client/java-api/current/client.html

How does not affect the performance of the JMeter itself

JMeter itself needs to generate and manage multiple threads, so in general, it is not recommended that JMeter test plan contain graphical analysis plugins or minimize the use of various analysis plugins. We introduce a backendlistener, and certainly do not want to affect the performance of jmeter, or minimize the impact on the jmeter itself, our strategy is:

Using the asynchronous strategy, the results of every 50 samples are deposited elasticsearch through the Elasticsearch bulk ingest API, reducing the network overhead. Of course, the 50 here can be configured on its own, depending on the performance of the machine, the size of the sampled data, and the network condition to determine

Customizable data retention strategies, such as saving only the sampled information that you are interested in

The backendlistener of the JMeter itself is also asynchronous, and the JMeter load does not wait for the result of the storage to complete.

Of course, the specific performance impact, but also the need for rigorous testing to determine, I will not expand here, it may be followed by a number of related tests

How to support the deployment of JMeter Controller/server

Preliminary look down, JMeter is the various server sample information passed to the controller processing, so, when the JMeter deployment size is larger, the controller's sample information processing will be heavier, fortunately we generally, The controller does not have load to deal with, so it's OK ... But for the use of plug-ins, I will next deploy several JMeter server confirmation, is the individual server to handle Sampleresult or return controller, this I will in detail in the update, the company's network, various restrictions, has not had the opportunity to do this part.

How to save a variety of custom parameters (in addition to the parameters provided by JMeter) to provide more analysis possibilities for the future

For example, we want to save the version number of the currently deployed product, which is useful when comparing the performance of various versions. In the plug-in, environment variables that begin with the custom string of the current machine are stored in each jmeter record. Afterwards, the user can flexibly use these custom parameters to analyze performance (e.g., geography code, various parameters related to product deployment)

Due to the problem of time, I will introduce the approximate situation today, then I'll update

• How to create a JMeter Backendlistener, and call Elasticsearch transportclient save Sampleresult
• How to use Kibana/kibana timelion to analyze performance results from various dimensions
• How to integrate with Jenkins to create a framework for performing a set of performance tests so that owners are transparent to users

Next I'll put the relevant code on GitHub, and open up the relevant codes, and follow up on the updates.

How to save JMeter performance test data to Elasticsearch, and use Kibana for visual analysis (1)