JUnitPerf Manual for "source translation"

JUnitPerf



Summary



Junitperf is a performance testing tool to measure the performance and execution efficiency of your code, and you can automate this process by writing unit test code for JUNITPERF. From another perspective, it is a junit extension.



If you're interested in this type of automated test, refer to my book, "Pragmatic Project automation."



You can also spend two days patronizing the site "Test-driven Development with JUnit Workshop", which provides some good JUnit learning pathways and Mike Some of Clark's handouts and practice guides help you get a deeper understanding of test-driven development.



Directory



· Brief introduction



· Purpose of Use



· Junitperf Download



· Junitperf Installation



· Junitperf Construction and testing



· How to use Junitperf



· Write an effective Junitperf test



· Limitations



· Technical support



· Donations



· Training and mentoring



· License information



· Thank



· Related Resources



Brief introduction



Junitperf is an automated test framework (tool) based on JUnit's performance and execution efficiency.



The junitperf contains the following two main classes (extended JUnit):



· Timedtest



Timedtest is used to perform a test that returns the time used to execute the test.



The Timedtest construct method needs to specify a maximum acceptable execution time. By default, when the method is executed, the test that is executed waits for execution to complete, and the test fails if the actual time spent exceeds the specified maximum time. In addition you can also specify that the test fails after the actual execution time exceeds the maximum acceptable time by specifying in the constructor method that the test does not continue.



· LoadTest



LoadTest is used to simulate multiple concurrent users to perform tests multiple iterations.



Purpose of Use



Obviously, Junitperf is an extension of the JUnit test framework. This extension of this approach allows you to dynamically add junit test cases to performance testing without affecting previous tests. This allows you to quickly and easily construct a performance test suite.



The performance test suite can automatically be performed independently of other junit test cases. In practice, it is generally necessary to avoid the use of JUnit test cases and junitperf test cases to be organized so that test suites can be executed more independently, and can be executed in different order. Longer-lasting performance tests may prolong the test time, causing you to be reluctant to perform all unit tests. Therefore, this requires that you plan to perform the test from times to time without affecting other tasks.



Junitperf tends to test for a specific performance requirement or execution efficiency requirement, and to maintain such a goal after the code is refactored. For example, you can use the Junitperf test to ensure that performance is not degraded by changing the algorithm under the same conditions. You can also use it to ensure that the refactoring of a resource pool does not result in lower execution efficiency in the case of load (this guarantee is based on the comparison of execution time and efficiency before and after the conditional change, providing only one metric).



It is very important to maintain a results-oriented test from the input-output point of view. Traditional performance metrics tools and techniques first identify potential sources of performance problems, while Junitperf is used to continuously automate testing and examine requirements and actual results.



Here is an example of a practical use scenario:



You have a well functioning program and pass the test validation functionality of the necessary JUnit test suite. From this perspective, you have achieved the desired goal of the design.



Then use a performance metrics tool to analyze which part of the program is executing the longest. Based on the design knowledge, you already have a good tool to do the actual evaluation of the program. And after refactoring the code is clear and concise, the next task is to adjust a small number of code.



You can then write Junitperf test cases, specifying acceptable performance and efficiency parameters for this part of the code. Testing without making any changes to the code will not pass, proving that the test case is correct. Then make some minor adjustments to the code.



Recompile and run the Junitperf test after each adjustment. If the actual performance reaches the desired target, the test is passed. If the actual performance does not reach the expected target, you will need to continue the adjustment process until the test passes. If the code in the future is refactored again, you can rerun the test. If the test fails and the previous performance criteria are increased, then you need to backtrack to the original and continue refactoring until the test passes.



Junitperf Download



JUnitPerf 1.9 is the most current version. Features that contain all previous versions.



This version requires Java 2 and JUnit 3.5 or above.



The release package contains a jar file, source code, sample code, API documentation, and this document.



JUnitPerf Installation



Windows



Install the following steps on Windows:



1. Extract junitperf-<version>.zip files into a directory, add%junitperf_home% in the system environment variable, the value is the directory after the file decompression.



2. Add the Junitperf to the classpath path:



Set Classpath=%classpath%;%junitperf_home%\lib\junitperf-<version>.jar



Unix (Bash)



On UNIX, install the following steps:



1. Extract the junitperf-<version>.zip to the appropriate directory. For example: $JUNITPERF _home.



2. Permissions to modify files:



Chmod-r a+x $JUNITPERF _home



3. Add the Junitperf to the classpath path:



Export classpath= $CLASSPATH: $JUNITPERF _home/lib/junitperf-<version>.jar



Build and Test



A compiled class file is already included in the $junitperf_home/lib/junitperf-<version>.jar file.



Build



$JUNITPERF _home/build.xml file is an ant build file.



You can build junitperf using the following command:



CD $JUNITPERF _home



Ant Jar



Test



The Junitperf installation package contains examples of junit test cases that are used in conjunction with Junitperf.



You can enter the following command to verify that the Junitperf installation is normal:



CD $JUNITPERF _home



Ant Test



How to use Junitperf



The best way to do this is to use the example included with Junitperf, which includes various types of tests.



The $JUNITPERF _home/samples directory contains all the sample code mentioned in this article.



Timedtest



The Timedtest construct method has two parameters, one is the existing JUnit test case, and the other is the expected maximum execution time.



For example, to create an execution time test for the Exampletestcase.testonesecondresponse () method and wait for the method to finish, if the time exceeds 1 seconds is considered failed.



Long maxelapsedtime = 1000;



Test testCase = new Exampletestcase ("Testonesecondresponse");



Test timedtest = new Timedtest (testCase, maxelapsedtime);



Similarly, if you want to end this test immediately after execution if you exceed the expected time, you can add a third parameter to the Timedtest constructor, for example:



Long maxelapsedtime = 1000;



Test testCase = new Exampletestcase ("Testonesecondresponse");



Test timedtest = new Timedtest (TestCase, Maxelapsedtime, false);



The following code creates an execution time test to test the time that is defined for the function performed by the unit Test Exampletestcase.testonesecondresponse () method.



Examples of execution efficiency tests



Import com.clarkware.junitperf.*;



Import Junit.framework.Test;







public class Exampletimedtest {







public static Test Suite () {







Long maxelapsedtime = 1000;







Test testCase = new Exampletestcase ("Testonesecondresponse");



Test timedtest = new Timedtest (testCase, maxelapsedtime);







return timedtest;



}







public static void Main (string[] args) {



Junit.textui.TestRunner.run (Suite ());



}



}



The granularity of the tests is determined by the JUnit test cases and used by the junitperf, and therefore has some limitations. The resulting execution time is the execution time of the Testxxx () method in the test case, including setup (), testxxx (), and the Teardown () method. The time to execute the test suite consists of the setup (), testxxx (), and the teardown () method execution time for all test samples in the test suite. Therefore, the expected time should also be based on the set-up and Tear-down execution time (take this part of the time also take into account).



LoadTest



LoadTest is used to emulate multiple users and perform multiple tests concurrently.



LoadTest The simplest constructor has only two parameters, test cases, and the number of users, which by default are iterated only once.



For example, create a 10 user to execute a exampletestcase.testonesecondresponse () method concurrently:



int users = 10;



Test testCase = new Exampletestcase ("Testonesecondresponse");



Test loadTest = new LoadTest (testCase, users);



The load test process can also specify an additional counter instance to specify the latency between concurrent execution of the user. The Constanttimer class constructor contains a constant parameter that specifies the delay time, and if 0 specifies that all users start at the same time. The Randomtimer class can construct a random delay time.



For example, to create a load test, 10 concurrent users perform a exampletestcase.testonesecondresponse () method each time, with a delay of 1 seconds between each user.



int users = 10;



Timer timer = new Constanttimer (1000);



Test testCase = new Exampletestcase ("Testonesecondresponse");



Test loadTest = new LoadTest (testCase, users, timer);



To emulate concurrent users to perform tests with a specified number of iterations, the LoadTest class constructor contains repeatedtest parameters. This allows you to specify the number of iterations for each test case.



For example: Create a Load test, 10 concurrent users, each user iteration executes the Exampletestcase.testonesecondresponse () method 20 times, and delays 1 seconds between each concurrent user.



int users = 10;



int iterations = 20;



Timer timer = new Constanttimer (1000);



Test testCase = new Exampletestcase ("Testonesecondresponse");



Test repeatedtest = new Repeatedtest (testCase, iterations);



Test loadTest = new LoadTest (repeatedtest, users, timer);



Or so to write:



int users = 10;



int iterations = 20;



Timer timer = new Constanttimer (1000);



Test testCase = new Exampletestcase ("Testonesecondresponse");



Test loadTest = new LoadTest (testCase, users, iterations, timer);



If your load test requires tests to contain a special test state in the Setup () method, you should use the Testfactory class to ensure that each concurrent user thread uses a local thread test instance. For example, to create a 10 user concurrent test, each user runs a local thread of the Examplestatefultest class, which can be written like this:



int users = 10;



Test factory = new Testfactory (examplestatefultest.class);



Test loadTest = new LoadTest (factory, users);



If you test one of these methods, you can do this:



int users = 10;



Test factory = new Testmethodfactory (Examplestatefultest.class, "testsomething");



Test loadTest = new LoadTest (factory, users);



The following example is a load test of the function of the test unit Test Exampletestcase.testonesecondresponse () method to test the execution efficiency of the function. There are 10 concurrent users with no delays, and each user runs only once. The LoadTest itself uses timedtest to obtain the actual operational capability of the Exampletestcase.testonesecondresponse () method under load. If the total execution time exceeds 1.5 seconds, it is considered not to pass. 10 concurrent processing is passed in 1.5 seconds.



Examples of load withstanding ability test



Import com.clarkware.junitperf.*;



Import Junit.framework.Test;







public class Examplethroughputunderloadtest {







public static Test Suite () {







int maxusers = 10;



Long maxelapsedtime = 1500;







Test testCase = new Exampletestcase ("Testonesecondresponse");



Test loadTest = new LoadTest (testCase, maxusers);



Test timedtest = new Timedtest (loadTest, maxelapsedtime);







return timedtest;



}







public static void Main (string[] args) {



Junit.textui.TestRunner.run (Suite ());



}



}



In the following example, the test is reversed, timedtest measures the execution time of the Exampletestcase.testonesecondresponse () method. The timedtest is then nested in LoadTest to emulate 10 concurrent users to execute the Exampletestcase.testonesecondresponse () method. If a user's execution time exceeds 1 seconds, it is considered not to pass.



Example of load response time test



Import com.clarkware.junitperf.*;



Import Junit.framework.Test;







public class Exampleresponsetimeunderloadtest {







public static Test Suite () {







int maxusers = 10;



Long maxelapsedtime = 1000;







Test testCase = new Exampletestcase ("Testonesecondresponse");



Test timedtest = new Timedtest (testCase, maxelapsedtime);



Test loadTest = new LoadTest (timedtest, maxusers);







return loadTest;



}







public static void Main (string[] args) {



Junit.textui.TestRunner.run (Suite ());



}



}



Performance Test Suite



The following test case example combines Exampletimedtest and exampleloadtest in a test suite so that all related performance tests are performed automatically:



Example Performance Test Suite



Import Junit.framework.Test;



Import Junit.framework.TestSuite;







public class Exampleperftestsuite {







public static Test Suite () {







TestSuite Suite = new TestSuite ();



Suite.addtest (Exampletimedtest.suite ());



Suite.addtest (Exampleloadtest.suite ());







return suite;



}







public static void Main (string[] args) {



Junit.textui.TestRunner.run (Suite ());



}



}



Write an effective Junitperf test



Timed Tests



Waiting Timed Tests



By default, the Timedtest test continues to perform junit tests if the actual test time exceeds the expected time. This waiting timed test always allows junit tests to accumulate all test results until the test is complete and all test results are checked.



If the use cases that are waiting for the test to be executed directly or indirectly derive multiple threads, this test will not return to timed test until all the threads have finished executing. On the other hand, the test will wait indefinitely. In general, unit tests should wait for all derived threads to complete, such as using the Thread.Join () method to accurately determine the result.



Non-waiting Timed Tests



In addition, Timedtest provides a construction method that immediately indicates a failure when the actual time exceeds the expected time. This type of test does not wait until the test continues execution if the execution time exceeds the maximum expected time. This type of test is more efficient than the previous one, which saves time if needed, and will no longer wait for the program to execute and immediately identify the test failed. However, unlike one of the above types, this type of test does not continue with the test if it is not passed in the middle.



Load Tests



Non-atomic Load Tests



By default, if LoadTest extends a test that derives threads directly or indirectly, it does not force such threads to execute concurrently (as defined in a transaction). This type of test assumes that its extended tests are performed interactively when the control is returned. For example, if the derived thread and control returns of the extended test do not wait for the derived process to complete, the extended test is assumed to be done once.



In general, the unit test should wait for the derived thread to execute, for example, by using the Thread.Join () method in order to accurately determine the result, but in some cases it is not necessarily the case. For example, for EJB-distributed query results, the application server might derive a new thread to handle the request. If the new thread runs the decorated test in the same thread group (by default), then a non-atomic stress test waits for a thread that is directly derived from the stress test to execute and the newly generated thread is ignored.



In short, non-atomic pressure tests simply wait for a thread that derives directly from the stress test to execute to mimic multiple concurrent users.



Atomic Load Tests



If multiple threads require a decorated test to execute successfully, this means that the decorated test is considered complete only if the threads in all decorated tests have finished executing. You can use Setenforcetestatomicity (true) to enforce this test (). This effectively prompts this test to wait for all threads of the thread group that belong to the decorated test to complete. The atomic stress test also treats any thread that prematurely exits as a failure. If a thread crashes suddenly, other threads that belong to the same thread group stop executing immediately.



If the decorated test derives threads that belong to the same thread group, by default the thread performs the decorated test, so that the atomic stress test waits indefinitely for the derived thread to finish executing.



In short, the atomic pressure test waits for all threads that belong to the same thread group to execute, and the thread that the stress test derives directly to emulate multiple user concurrency.



Limitations



Junitperf is known to have the following defects:



· The time that Timedtest returns is the time of the Testxxx () method of the test case, including the total time of the setup (), testxxx (), and teardown () three methods, which is the smallest test granularity available in any test instance. Therefore, the expected time should also take into account the running time of set-up and tear-down. Or you can use the System.currenttimemillis () method to calculate the execution time of a step in your JUnit test case.



· Junitperf is not a complete stress and performance testing tool, and it will not be used to replace other similar tools. It is only used to write local unit performance tests to help developers refactor.



· The performance of your tests can degrade significantly if too many concurrent users are cooperating in a load test. The actual threshold number is JVM specific.



· In stress tests, if too many users run concurrently, the test situation gets worse. The specification of the JVM should be referenced to specify the number of users.



Technical support



If you have any questions about Junitperf, need to improve the requirements, successful experience or bugs, or when a new version of the release is notified please email to mike@clarkware.com. Your personal information will not be disclosed.



You can also discuss the Junitperf by mailing list (http://groups.yahoo.com/group/junitperf/) and receive notification when a new version is released.



Donations



You can support Junitperf's continued development by purchasing the pragmatic Project Automation (http://www.pragmaticprogrammer.com/sk/auto/) book.



Training and mentoring



You can learn about how to quickly create test code by visiting the site http://clarkware.com/courses/TDDWithJUnit.html.



The instructions for JUnit are also available here (http://clarkware.com/mentoring.html) to help you improve your testing.



If you want more information please contact me (mailto:mike@clarkware.com).



License information



The JUNITPERF is licensed under the BSD License.



Thank



Thank you very much for the help of Ervin Varga on-line robustness and atomicity testing. In these respects he proposes the use of line Cheng to capture and process the thread's exceptions, and in Timetest and testfactory the implementation that is identified as a failure immediately if the execution time is timed out. He was very grateful for his junitperf and suggestions.



Translation



Msn:wyingquan at hotmail dot com finish time: 2005-4-19



Related Resources and reference documents



· JUnit primermike Clark, Clarkware Consulting, Inc.



This article provides a brief description of how to use the JUnit test framework to write and run simple test cases and suites.



· Continuous performance Testing with Junitperfmike Clark (Javapronews, 2003)



This article explains how to write a Junitperf test to check performance and testability from time to times.



· Test-driven development:a Practical Guide David Astels (Prentice Hall, 2003)



Includes a chapter written by the author of this article How to use Junitperf to continuously perform performance testing.



· Java Extreme Programming Cookbookeric Burke, Brian coyner (O ' Reilly & Associates, 2003)



One of them is devoted to the use of junitperf.



· Java Tools for Extreme programming:mastering Open Source Tools including Ant, JUnit, and Cactusrichard Hightower, Nichola S Lesiecki (John Wiley & Sons, 2001)



Contains a chapter that describes how to use Junitperf with HttpUnit.