Reproduced Parameter configuration of SOAPUI pressure test

Original address:soapui parameter Configuration of pressure test goooooodlife

The different load strategies available in SoapUI and SoapUI Pro allow you to simulate various types of load over time, en Abling you easily test the performance of your target services under a number of conditions. Since SoapUI also allows you to run multiple loadtests simultaneously (see an example further down), a combination of Load Tests can used to further assert the behaviour of your services. Select the desired strategy for your LoadTest from the strategy Toolbar in the LoadTest Window:

Lets has a look at the different Load strategies available and see how they can is used to do different types of load/per Formance tests.

1. Simple strategy-baseline, Load and soak testing

The simple strategy runs the specified number of threads with the specified delay between each run to simulate a "breathin G Space "for the server. For example if want to run a functional test with ten threads with ten seconds delay, set threads to ten, delay to 10000 and random to how much of the delay want to randomize (ie setting it to 0.5 would result in delays between 5 and ten sec Onds). When creating a new LoadTest, the default strategy and set at a relatively low load (5 threads with 1000ms delay).

The simple strategy are perfect for Baseline testing. Use it to assert the basic performance of your service and validate that there be no threading or resource locking is Sues. Ramp up the number of threads if you want does more elaborate load testing or use the strategy for long-running soak tests .

Since it isn ' t meant to bring your services to their knees, a setup like this can is used for continuous load-testing to E Nsure that your service performs as expected under moderate load; Set up a baseline test with no randomization of the delay, add LoadTest assertions, act as a safety net for unexpected Results and automate its execution with the command-line LoadTest runner or Maven plugins.

2. Fixed rate strategy–simple with a twist

One thing the simple strategy does does is guarantee a number of executions within a certain time, for example if Y OU might want to start your TestCase The times each second no matter what long it takes to execute. Using The simple strategy your could set up ten Threads and a delay compensating for the average gap between the end of the TestCase and the start of the next second, but this would is highly unreliable over time. The fixed-rate strategy should be used instead; Set the rate as desired (ten in We case) and off go; The strategy would automatically start the required number threads for this setting attempting to maintain the configured V Alue.

As hinted in the headline, there is some twists here:what if our TestCase takes more than one second to execute? To maintain the configured TPS value, the strategy would internally start new threads to compensate for this; After a while you might has many more than threads running due to the fact, the original ones had not finished wit Hin the set rate. And not surprisingly this could cause the target service to get even slower, resulting in + and more threads being star Ted to ' keep up ' with the configured TPS value. As you probably guessed the ' Max Threads ' setting is she to prevent SoapUI from overloading (both itself and the target SE rvices)   In this situation, specifying a value here would put a limit on the maximum number of threads the SoapUI would be allowed to start to maintain the configured TPS, if reached the existing threads would has to finish before SoapUI wil L Start any new ones.

The "Request level" setting would attempt to maintain the TPS is not on the TestCase execution level and the request level Instead, for example if you had a data-driven LoadTest or a TestCase with many requests, you want the TPS setting to APPL Y not on the execution level of the entire TestCase but on the request level.

In any case, the Fixed rate strategy are useful for baseline, load and soak-testing if you don't run into the "Thread Conge Stion "problem described above. On the other hand, you might provoke the congestion (maybe even in combination with another LoadTest) to see how your serv ICES handle this or what they recover after the congestion have been handled.

3. Variable Load Strategies

There is several strategies that can is used to vary load (the number of threads) over time, each simulating a different Kind of behavior. They can useful for recovery and stress testing, but just as well for baseline testing, either on their own or in combi Nation with other strategies. Let ' s has a quick look:

2. Variance Strategy –this varies the number of threads over time in a "sawtooth" Manor as configured; Set the Interval to the desired value and the Variance to the how much the number of threads should decrease and increase . For example if we start with a threads, set interval to $ and Variance to 0.8, the number of threads would increase from Within the first seconds, then decrease back to the continue down to 4 threads after seconds, and Finall Y go back up to the initial value after seconds. In the Statistics diagrams we can follow this variance easily:

4. Burst Strategy -  This strategy be specifically designed for Recovery testing and takes variance to its extreme; It does nothing for the configured Burst Delay and then runs the configured number of threads for the "Burst Duration"   And goes back to sleep. Here you could (and should!) set the number of threads to a high value (20+) to simulate an onslaught of traffic during a Short interval, then measure the recovery of your system with a standard base-line LoadTest containing basic performance-r Elated assertions. Lets try this with a burst delay and duration of ten seconds for seconds;
   
   
   

6. the Thread strategy lets youlinearly change the number of threads from OneLevel to another over The run of the LoadTest. It ' s main purpose Isas a means to identify at which level certain statistics change orevents occur, for example to find at Which threadcount the Maximumtps or BPS can be achieved or to find at which threadcountfunctional testing errors start OC Curring. set thestart and End thread values (for example 5 to +) and Set theduration to a relatively long duration ( i  use atleast-seconds per thread value, in this example, would be1350 seconds) to get accurate measurements ( More on Thisbelow).
   
   
   

8. Grid Strategy –this strategy allows tospecifically configure the relative change in number of thread Sover time, its main use for this was more advanced scenario andrecovery testing, where you need to see the services Behavi or undervarying loads and load changes. For the example lets say you want torun for the seconds with the ten, Ten, ten, and threads. Configure yourloadtest to start with ten threads and then enter the Followingvalues in the Grid:
   
   

   both values are stored relative tothe duration and actual threadcount of the LoadTest; If you Changethese, the corresponding Grid strategy values would berecalculated. running the test shows the following Output:

10. script Strategy , Haven script strategy is theultimate customization possibility; The script you specify iscalled regularly (the "strategy Interval" Settingin the LoadTest Options dialog) and should retur n the desirednumber of threads at. Returning a value Otherthan the current one would start or stop threads to adjust for thechange. This allows-kind of variance of the number ofthreads, for example the following script randomizes the number Ofthr EADS between 5 and 15.

    Running this and the strategy Intervalset to the number of threads would change every 5 seconds:

    The possibilities here is endless.

4. Statisticscalculation and ThreadCount changes

Many of these strategies would change the number of threads whichhas a important impact on the statistics calculation T Hat you needto is aware of; When the number of threads changes, this willusually change the response times of the target services, resultingin a Chang E in Avg, TPS, etc.  but since Theloadtest have already run at a previous number of threads theresults for those runs Would skew the result for the newthreadcount.

For example lets say you had been running at 5 threads and Gotand average to 500ms. Using The Thread strategy you increase thenumber of threads gradually; When running 6 threads the averageincreases to 600ms but since the ' old ' valuescollected for 5 threads is still there, t Hesewill in total result in a lower average. There is-ways towork around this; Select the "Reset Statistics on Threadcountchange" value in the LoadTest Options dialog, or manually Reset Thestatistics W ith the corresponding button in the LoadTest toolbar;in either case old statistics would be discarded. To see this inaction lets does a threadcount strategy test from ten to threadsover seconds (the seconds per thread), be Low you can see Resultsboth with this setting unchecked and then checked;

In the latter-the "jumps" in statistics each time theyare reset when the number of threads changes, gradually leve Lingout to a new value. The final TPS calculated at threads differsabout 10% between these and showing how the lower results "impact" the Highe R ones.

5. Running multiple loadtestssimultaneously

Ok, lets has a quick look at this; We ' ll create one baselinetest with the simple strategy and a low number of threads, and atthe same time run a burst strate Gy to see how the baseline Testperformance "recovers" after the burst;

Here's can see the simple strategy (bottom diagram) recoveringgradually after each burst of load.

6. Final Words

Hopefully you has gotten a good overview of the differentstrategies in SoapUI and how they can be used to simulate differ Entscenarios and kinds of load. As havenoticed, SoapUI focuses more on "behavioral" loadtesting (understanding how your services handler different Loads) Insteadof exact numbers, which is anyhow hard to calculate since there areso many external factors influencing it.

Reproduced Parameter configuration of SOAPUI pressure test