Preliminary study on performance testing---interface performance test

(i). Performance Indicators

Pv:pageview, page views or clicks, each time the user refreshes is calculated once, the user once refreshed, to the server caused a request.

Uv:uniquevisitor, a computer client that accesses your site is a guest, and the same client in 0:00-24:00 is only remembered once.

Tps:transaction the number of transactions or things the system processes per second, and the important indicator of the system's processing power.

RT: Response time, starting from the client sending a request, to the time the client received the end of the response returned from the server, including the request send time, network transmission time and server processing time three parts.

Vu:virtual user, virtual users simulating real business logic steps, virtual user simulations are recorded in a virtual user script, often using concurrent implementations.

TPS Fluctuations: System performance relies on specific hardware, software code, application services, network resources, etc., so during performance scenario execution, TPS may behave as stable, or fluctuate, or follow a certain rise or fall trend. This indicator value is recorded using the TPS volatility factor.

CPU:CPU resource refers to the CPU resource occupancy rate of the application service system during the time period when the performance test scenario is running. CPU resource is an important parameter to judge the system processing ability and the stability of application operation.

Load: Measure of how much the system is working, queue length. The average system load, defined as the average number of processes running in a queue at a specific time interval (1m,5m,15m).

I/o: I/O can be divided into disk IO and Nic io.

JVM: The Java Virtual machine, which has its own processor, stack, register, etc., as well as its own corresponding instruction system. Java applications run on top of the JVM.

GC:GC is an automatic memory management program whose primary responsibility is to allocate memory, ensure that the object being referenced is always in memory, and that objects that are not applied are freed from memory. FGC will cause the JVM to hang.

Internet speed: Data transmission rate in the network, generally in byte/s units. The speed is reflected by the ping delay.

Flow: Performance test, generally refers to the unit time flow through the network card total traffic. Divided into inbound and outbound, generally in kilobytes.

VU (number of concurrent users) = TPS (number of executed transactions per second) XRT (response time)

In finding the right number of concurrent users, it is recommended to use the "gradient mode" of PTS (Taobao performance Automation: Https://pts.aliyun.com/lite/index.htm?spm=0.0.0.0.8t95Nt) to gradually increase the number of concurrent users, This time the pressure will be more and more, when the TPS growth rate is less than the growth rate of response time, this is the inflection point of performance, that is, the most reasonable number of concurrent users; When TPs no longer increases or decreases, the pressure at this time is the maximum, and the number of concurrent users is the maximum number of concurrent users. If the TPS at this point does not meet your requirements, then you need to look for bottlenecks to optimize.

Point A: Performance expectations

Point B: Higher than expected, system security

Point C: Above expectations, inflection points

D point: Over load, System crash (ii). LoadRunner Statistical interface performance data

LoadRunner is divided into 3 steps:
1. Create/edit Scripts;
2. Run Load Tests;
3. Analyze Test Results; First step: scripting (GET request)

New Scrips, select the Web (http/html) protocol, because it is not a page test, do not need to record operations, in the "Start recording" pop-up window, select Cancel.
To write a script manually:

Action () {Web_set_max_html_param_len ("10240");
             The value of the keyword text for the search in the result of the request returned by the string//keyword Savecount value of the text in the results found in the number of content Web_reg_find ("text={\" statuses\ ":",


    "Savecount=count", last);
    Custom transaction Unread_friends_timeline_reload, starting point lr_start_transaction ("Unread_friends_timeline_reload"); 
    Basic authentication, add Authorization value in header information, variable Authorization parametric web_add_header ("Authorization", "{Authorization}"); HTTP request Web_custom_request ("Web_custom_request", "Url=http://ip:port/2/statuses/unread_friends_timeline.json ? source=****** "," Method=get "," Reccontenttype=application/json "," Mode=http "," Enctype=ap

    Plication/x-www-form-urlencoded ", last); Determines whether the result contains the expected text, if it contains, the transaction unread_friends_timeline_reload state is Lr_pass if (Atoi (lr_eval_string ("{Count}")) = = 1) {LR

        _end_transaction ("Unread_friends_timeline_reload", Lr_pass); } else{lr_error_message ("error=%s", "errorContent ");

         Lr_end_transaction ("Unread_friends_timeline_reload", Lr_fail);

} return 0;

 }

Press "F5" key to run the script, Replay log output is as follows:

Virtual User Script started at:2015-10-07 16:58:47 starting action Vuser_init. Web Turbo Replay of LoadRunner 11.0.0 for WIN2003; Build 9409 (Jan 11:34:16) [msgid:mmsg-27143] Run mode:html [msgid:mmsg-26000] run-time Settings file: "
E:\unreadfeed\unreadfeedreload\\default.cfg "[msgid:mmsg-27141] ending action vuser_init.
Running VUser ...
Starting Iteration 1.
Starting action action. ACTION.C (3): Web_set_max_html_param_len was successful [msgid:mmsg-26392] ACTION.C (6): Registering Web_reg_find was
Successful [msgid:mmsg-26390] Action.c (): Notify:transaction "Unread_friends_timeline_reload" started. ACTION.C: warning-26593:the header being added may cause unpredictable results if applied to all ensuing URLs. It is added anyway [msgid:mwar-26593] action.c (): Web_add_header ("Authorization") highest severity level was "warning [msgid:mmsg-26391] ACTION.C (+): Registered Web_reg_find successful for "text={" statuses ":" (count=1) [MSGID:MMSG-26364] ACTION.C (+): Web_custom_request ("Web_custom_request") was successful, 91858 body bytes, 198 header Byt es [msgid:mmsg-26386] action.c (+): notify:transaction "Unread_friends_timeline_reload" ended with "Pass" status (Dura
tion:0.8905 wasted time:0.6256).
Ending action action.
Ending Iteration 1.
Ending VUser ...
Starting action Vuser_end.
Ending action vuser_end. VUser Terminated.

ACTION.C (+): notify:transaction "Unread_friends_timeline_reload" ended with "Pass" status (duration:0.8905 Wasted Time : 0.6256). Indicates that script debugging passes.

LoadRunner parameterization settings can be found in: http://www.cnblogs.com/fnng/archive/2012/06/22/2558900.html analog POST request:

Web_custom_request parameter settings: "Method=post", "body= Property name = property Value & Property Name = property value & ...", web_custom_ If a property value in the body in request contains some special characters, it must be encoded by URL or the Web server will return a 500 error.

Example:

Web_custom_request ("Web_custom_request",
        "Url=http://ip/2/statuses/update.json",
        "Method=post",
        " Reccontenttype=application/json ",
        " Mode=http ",
        " enctype=application/x-www-form-urlencoded ",
        " Body= status=123&source=******** ", last
        );

Step Two: Run Load Tests

After customizing the script, run the second step, running Load Tests, pop up the new Scenario small window, select test script on the left, and enter the scene design.

The scene settings consist mainly of 4 parts, select the global Schedule->edit Action settings scenario:

1.INITIALIZE:VU Initialization settings

2.Start vusers: Define how many vu, user growth options
(1) simultaneously at the same time; (2) Increase users at certain times

3.Duration: Run Time settings

4.Stop Vusers: The strategy of terminating VU

Once the scene is set, click Start Scenario to start the scene performance test.

Click Stop to stop the performance test, and automatically skip to the third step. Step three: Analyze Test Results

The results show that there will be 3 reports

(1) Analysis Summary:statistics Summary, Transaction summary,http responses Summary

(2) Averagee Response time:minimum, Average, Maximum, Std. Deviation (standard variance), unit: s

(3) Transactions per Second:graph minmum, Average, graph Maximum, graph Median, graph Std. Deviatior

LoadRunner does not provide the distribution of response time, only the minimum, maximum, average, standard variance, Percent, if the need to statistical 50%,60%,70%,80%,90% RT distribution, to obtain the raw data through LR, Calculated using Excel formula percentile:

Under Averagee Response Time report, select the Windows menu->raw data, the right side of RAW data, expand the option by transaction name (value is the custom transaction name in the script) Get interface response time raw data transaction Response times, paste into Excel, assuming the original data is in a column (A2-A57), for example, to get 50% RT, use the formula =percentile (a2:a57;0.5) to get the result, In turn ... (iii). Industry Performance Test Tool PTS:

Ali Performance Testing Service (HTTPS://PTS.ALIYUN.COM/LITE/INDEX.HTM?SPM=0.0.0.0.SPNISN), online Web services, short board: only support Alibaba cloud Intranet and public network, do not support intranet.
Help documents: https://home.console.aliyun.com/, powerful, real-time display of transaction success and failure TPS, number of concurrent users (number of threads), average response time data trend analysis graph, CPU utilization, Load, memory, network IO, Data trend analysis graph such as disk IO, test script is Python.

Comparison test of Pts-ab-jmeter-loadrnner pressure measuring tools:
HTTP://BBS.ALIYUN.COM/READ/236875.HTML?SPM=5176.7189909.0.0.O14YKP Open Source tools: 1.gatling:http://gatling.io/#/

Gatling is a Open-source load testing framework based on Scala, Akka and Netty
High performance
Ready-to-present HTML Reports
Scenario Recorder and developer-friendly DSL

Test script for Scala, short board: No TPS data, Pros: The results are graphically displayed clearly.
Some screenshots:

Quick Start: http://gatling.io/docs/2.1.7/
Website download too slow, has been uploaded to Csdn, download address: http://download.csdn.net/detail/neven7/9161447 2.ngrinder:http://naver.github.io/ngrinder/

Ngrinder is a platform for stress tests This enables you to execute script creation, test execution, monitoring, and Resul T report Generator simultaneously. The Open-source Ngrinder offers easy ways to conduct stress tests by eliminating inconveniences and providing integrated E Nvironments.
Ngrinder is the Naver (Korea's largest internet company NHN Search engine website) Open-source performance testing tools, provide war packages, directly to a Web service, you can use.

Default Administrator account: Admin/admin
After the account login, you need to download Agent and monitor (note that the downloaded agent will have the corresponding account information, an account corresponding to the unique agent, the other account login can not use other people's agent, can only use their own download Agent):

Installed on the test machine, monitor monitors the performance of the machine, the agent performs performance tests; The test script is Groovy/jython, and the test script is similar to the JUnit case, customizing the Runner:grinderrunner

Package org.ngrinder; Import static Net.grinder.script.Grinder.grinder import static org.junit.assert.* import static Org.hamcrest.Matchers.
* Import net.grinder.plugin.http.HTTPRequest import Net.grinder.plugin.http.HTTPPluginControl; Import net.grinder.script.GTest Import Net.grinder.script.Grinder Import Net.grinder.scriptengine.groovy.junit.GrinderRunner Import Net.grinder.scriptengine.groovy.junit.annotation.BeforeProcess Import Net.grinder.scriptengine.groovy.junit.annotation.BeforeThread//import static net.grinder.util.grinderutils.*// Can use the If you ' re using ngrinder after 3.2.3 Import org.junit.Before import org.junit.BeforeClass import Org.juni T.test Import org.junit.runner.RunWith Import httpclient.httpresponse import httpclient.nvpair/** * A Simple example U 
 Sing the HTTP plugin that shows the retrieval of a * single page via HTTP.
 * * This script was automatically generated by Ngrinder.
 * * @author User */@RunWith (Grinderrunner) class Testrunner {   public static gtest test public static HttpRequest request @BeforeProcess public static void beforeprocess
        () {httpplugincontrol.getconnectiondefaults (). Timeout = 6000 test = new Gtest (1, "www.baidu.com")
        Request = new HttpRequest () Test.record (request);
    Grinder.logger.info ("before process.");
        } @BeforeThread public void Beforethread () {grinder.statistics.delayreports=true;
    Grinder.logger.info ("before thread."); } @Test public void Test () {HttpResponse result = Request. GET ("http://www.baidu.com") if (Result.statuscode = = 301 | | result.statuscode = = 302) {Grinder.logge R.warn ("Warning. The response may is not correct. 
        The response code was {}. ", Result.statuscode);
        } else {Assertthat (Result.statuscode, is (200)); }
    }
}

Graphical setup parameters:

The results show:

Official Wiki:http://www.cubrid.org/wiki_ngrinder/entry/user-guide