Go Multi-threaded parallelism of testng

Objective

Recently in the project of automated testing work, using the TESTNG test framework, mainly involved in the test type of interface testing and based on the scene of the business scenario testing. Because most of the scenarios involved are job development and execution of big data (such as MapReduce, Spark, HQL, etc.), the execution of these tasks takes a lot of time. To cite a common example, one scenario test case is:

• Executes a mapreduce job, verifies the execution results of the job, and executes the log.

For one of the simplest Mr Tasks, if the yarn cluster has sufficient resources, its execution time will take nearly a minute. Needless to say, when yarn cluster computing resources are saturated, tasks need to wait for resource allocation and so on. When the test regression case set contains a large number of such cases, an automated regression can take a significant amount of time if the traditional single-threaded execution method is used.

multithreaded parallel execution

Based on the above scenario, we can consider that there is no coupling relationship between the automation use cases and a relatively independent use case for parallel execution. For example, I can perform different Mr Tasks, spark tasks with different threads at the same time, each of which is responsible for tracking the performance of the task.

In addition, even simple interface automation testing, if the test set contains a large number of use cases, we can also rely on testng multi-threaded way to improve execution speed.

It is important to point out that while the use cases can greatly improve the execution efficiency of use cases, we should design the use cases to consider whether they are suitable for concurrent execution and to pay attention to the common problem of multithreading: thread safety and shared variables. The recommendation is to avoid using shared variables as much as possible in the test code. If it is used carefully, use the Synchronized keyword to lock and synchronize shared variables. Otherwise, it is inevitable that your use case may appear unstable scenarios (often heard that the use case is not stable, sometimes 100% through, sometimes only 90% pass, speculation may be part of the reason is also this cause).

Multi-threaded use posture in testng different levels of concurrency

Typically, in the execution of testng, the level of testing can be divided from top to bottom in Suite, test, Class, and the left element of the arrow is a one-to-many containment relationship with the right element.

Here, test refers to the test tag in Testng.xml instead of a @Test in the testing class. A @Test in the test class actually corresponds to the method here. So when we use @BeforeSuite, @BeforeTest, @BeforeClass, @BeforeMethod These tags, the actual order of execution is based on this level.

Suite

In general, a testng.xml contains only one suite. If you think of multiple threads executing different suite, the official approach is to specify the capacity of the thread pool by command line.

3 testng1.xml testng2.xml testng3.xml

You can perform Testng1.xml, Testng2.xml, testng3.xml with three threads respectively. In fact, this situation is rarely used in practice, our test cases are often placed in a suite, if you really need to implement different suite, often also in different environments to execute, then also naturally do some other configuration (such as environment variables) changes, there will be different processes to execute. So this approach is not to be more than repeating.

Test, class, method

Test,class,method levels of concurrency can be set through the suite tag in Testng.xml, such as:

<suite name="Testng Parallel Test"Parallel="tests"Thread-count="5"><suite name="Testng Parallel Test"Parallel="Classes"Thread-count="5"><suite name="Testng Parallel Test"Parallel="Methods"Thread-count="5">

What they have in common is up to 5 threads to execute different use cases at the same time. The differences are as follows:

• Tests level: The use cases under different test tags can be executed on different threads, and the use cases under the same test tag can only be executed in the same thread.
• CLASSS level: The use cases under different class tags can be executed on different threads, and the use cases under the same class tag can only be executed in the same thread.
• Methods level: All use cases can be executed on different threads.

Figuring out the level of concurrency is important to help us properly organize use cases, such as putting a non-thread-safe test class or group into a test, so that concurrency can ensure that the use cases in these classes are single-threaded. You can also set class-level concurrency as needed, allowing use cases in the same test class to execute on the same thread.

dependencies when concurrency

In practice, many times we add dependencies to the execution of many test methods in the test class by dependonmethods/dependongroups, to achieve the desired order of execution. If the methods level concurrent execution is configured at the same time when TestNG is running, are these test methods executed in different threads, followed by a dependent order of execution? The answer is--yes. The testng is the ability to follow the established use case execution sequence in multithreaded situations.

Concurrency of different dataprovider

When you use TESTNG for automated testing, you basically use Dataprovider to manage different test data for a use case. The above method of modifying suite tags in testng.xml does not apply to concurrency between multiple sets of test data dataprovider. Execution will find that multiple sets of data in a DP are still executed sequentially.

The workaround is to add parallel=true to the @DataProvider. Such as:

ImportOrg.testng.annotations.DataProvider;Importtestdata. Scenariotestdata; Public classScenariodataprovider {@DataProvider (name= "Hadooptest", parallel=true)     Public StaticObject [] hadooptest () {return Newobject[][]{Scenariotestdata.hadoopmain, Scenariotestdata.hadooprun, Scenariotestdat    A.hadoopdeliverprops}; } @DataProvider (Name= "Sparktest", parallel=true)     Public StaticObject [] sparktest () {return Newobject[][]{Scenariotestdata.spark_java_version_default, scenariotestdata.spark_java_version_16    2, scenariotestdata.spark_java_version_200, Scenariotestdata.spark_python}; } @DataProvider (Name= "Sqooptest", parallel=true)     Public StaticObject [] sqooptest () {return Newobject[][]{scenariotestdata.sqoop_mysql2hive, Scenariotestdata.sqoop_mysql2hdfs}; }}

By default, the thread pool capacity for DP parallel execution is 10, and if you want to change the number of concurrent numbers, you can also specify the parameter Data-provider-thread-count under Suite tag:

<suite name= "Testng Parallel Test" parallel= "Methods" thread-count= "5" data-provider-thread-count= ">"

Concurrency of the same method

Sometimes, we need to perform concurrent tests on a test case, such as an HTTP interface, that is, a repeated invocation of an interface. TestNG also provides an elegant way to support the designation of ThreadPoolSize and Invocationcount in @Test tags.

@Test (enabled=true, dataprovider= "TESTDP", Threadpoolsize=5, invocationcount=10)

Where ThreadPoolSize indicates the thread pool capacity used to invoke the method, which is the same as 5 threads concurrently executing the method; Invocationcount indicates the number of times the method total needs to be executed. In this example, 5 threads execute simultaneously and stop when the total number of executions reaches 10 times.

note that the thread pool associated with DP is two separate thread pools. The thread pool here is used for multiple method, and the test data of each method is provided by the DP, if there are 3 sets of data in the DP, then actually 10 times the execution, each will be 3 times the interface, the total number of times the interface is called 10*3=30 times. A new thread pool (Dpthreadpool) is created to execute the test data concurrently when each thread ThreadPoolSize the specified 5 threads, and if the DP is used to support concurrent execution.

The sample code is as follows:

 Packagetestng.parallel.test;ImportJava.text.SimpleDateFormat;Importjava.util.Date;ImportOrg.testng.annotations.AfterClass;ImportOrg.testng.annotations.BeforeClass;ImportOrg.testng.annotations.DataProvider;Importorg.testng.annotations.Test; Public classTestClass1 {PrivateSimpleDateFormat DF =NewSimpleDateFormat ("Yyyy-mm-dd HH:mm:ss");//Set Date format@BeforeClass Public voidBeforeclass () {System.out.println ("Start Time:" + Df.format (NewDate ())); } @Test (Enabled=true, dataprovider= "TESTDP", threadpoolsize=2, invocationcount=5)     Public voidTest (String Dpnumber)throwsinterruptedexception{System.out.println ("Current Thread Id:" + thread.currentthread (). GetId () + ". Dataprovider Number: "+dpnumber); Thread.Sleep (5000); } @DataProvider (Name= "TESTDP", parallel =true)     Public StaticOBJECT[][]TESTDP () {return Newobject[][]{{"1"},            {"2"}        }; } @AfterClass Public voidAfterclass () {System.out.println ("End Time:" + Df.format (NewDate ())); }}

Test results:

Start time:2017-03-11 14:10:43[Threadutil] Starting executor timeout:0ms workers:5 Threadpoolsize:2Current Thread Id:Dataprovider Number:2Current Thread Id:Dataprovider Number:2Current Thread Id:Dataprovider number:1Current Thread Id:Dataprovider number:1Current Thread Id:Dataprovider number:1Current Thread Id:Dataprovider number:1Current Thread Id:Dataprovider Number:2Current Thread Id:Dataprovider Number:2Current Thread Id:Dataprovider Number:2Current Thread Id:Dataprovider number:1End Time:2017-03-11 14:10:58

Other TestNG Tips

As a mature and widely used testing framework, TESTNG has the rationality of its existence naturally. Here again to share some simple and useful tags, the specific use of the posture you can explore their own, the official website has a relatively full introduction, after all, their own exploration will be impressive.

1. groups/dependsongroups/dependsonmethods--setting up dependencies between use cases
2. dataproviderclass--puts Dataprovider into a dedicated class to implement test code, Dataprovider, and test data layering.
3. timeout--setting the time-out for use cases (concurrent/non-concurrency supported)
4. alwaysrun--Some dependent use cases have failed, causing the use cases to be skipped. For some test classes that "mop up" to keep the environment clean, we can use this tag if we want to enforce it.
5. The priority--sets the priority so that some test cases are given a greater probability of priority execution.
6. singlethreaded--forces a use case in a class class to execute on one thread, ignoring the method level concurrency
7. preserve-order--Specifies whether the use case is executed in accordance with the established use case sequence in Testng.xml

Summarize

The use of multi-threaded in testng in parallel execution of test cases can effectively improve the execution speed of use cases, and testng provides a good support for multithreading, even novice can easily get started with multithreading. In addition, TestNG creates threads by default using the thread pool, reducing the cost of the program.

Reference links

• TestNG Documentation

Transferred from: http://www.cnblogs.com/znicy/p/6534893.html

Go Multi-threaded parallelism of testng