. NET Core Performance Analysis: xunit.performance introduction

Xunit-performance is an extension of xunit that lets you perform performance testing on. NET core projects.

Official website: github.com/microsoft/xunit-performance

Xunit Everyone may have used it, it is used to do unit testing, it can quickly give developers the ability to function OK feedback.

As with XUnit, Xunit-performance can quickly give performance feedback.

Preparing and installing Xunit-performance

To explain, we need to prepare a project to be tested and a test project.

There were always problems after I built my project using visual Studio 2017, but then I had no problem with dotnet CLI and Vscode .

The order in which the projects are created is as follows:

1. First use the dotnet CLI to build a classlib type of test project whose target framework is. NET Standard 2.0:

There is only one class in this project, which is the class to be tested:

There are three methods for this class, each of which uses the sum of the Foreach,for and LINQ extension methods to loop and sum the collection.

2. Use the dotnet CLI to build a console project (if you use VS2017 to build the class library directly, because the VS2017 built-in test Runner), this is a test project, its version can only be 2.0 (probably because my Computer SDK version older):

It is also necessary to reference the tested project.

3. Then, follow the official documentation to install the two libraries.

Xunit-performance is currently in beta, and the two libraries need to be installed as instructed by the official website:

The latest version of the Xunit.performance.api.dll, used here is myget:dotnet.myget.org/feed/dotnet-core/package/nuget/xunit.performance.api#.

Then the latest version of the Microsoft.Diagnostics.Tracing.TraceEvent, which uses nuge:www.nuget.org/packages/ Microsoft.Diagnostics.Tracing.TraceEvent

OK, now the library is loaded.

Writing performance tests

Performance tests and unit tests are slightly different, and performance tests run many times and then average. Also take into account the impact of other factors such as memory.

In the performance test, it is not necessary to test the correctness of the function, but the program under pressure may produce different results, especially multi-threaded situation. Then you need to write a stress test.

For performance testing, we only consider speed .

Since I am using the dotnet CLI and Vscode, the test project I chose is the console project, and its main method needs to be written like this:

If you can successfully use VS2017 to build a test project, then you do not need the main method, the establishment of a class library project can be directly using the VS2017 Test Runner can be.

Performance Test Code

Below we write the performance test method.

First set up a class in the test project and then do some preparatory work:

Here I have prepared a list<keyvaluepair<int, double>> it has 100,000 data and is randomly generated.

Then there is the test method, where we use [Benchmark] instead of [Fact] in the Xunit unit test:

Xunit.performance's tests run many times, and the results are averaged .

Here we loop through the benchmark.iterations, which has a default value, which I have run 1000 times here by default.

In the recirculation, you can do some preparatory work first. Then use iteration. Startmeasurement () to begin the measurement.

Only iteration. The Startmeasurement () will be measured, and the code associated with the test is written in curly braces.

Then enter run tests at the command line:

The test results are as follows:

Console output is provided, XML,CSV,MD output (in project folder).

From the console you can see that the cycle of the test ran 1000 times, with an average result of 0.963 milliseconds .

The following are the CSV results:

The following are the MD result files:

Here is the XML result file, which has detailed data:

Internal Loop

Xunit.performance can also add an internal loop property Inneriterationcount. look at the code first and add the following methods:

[Benchmark (Inneriterationcount = 10_000)], where the inneriterationcount is the number of internal loop traversal.

After Startmeasurement (), the internal loop is performed.

In this case, the number of outer loops may be small, and the first outer loop is warmed up, not included in the test results .

And the inner loop is suitable for running faster code (Microsecond level).

Sometimes it is necessary to do a couple of loops, to do some warm-up work or to complete different levels of preparation.

Then we'll run the test.

In the results see the outer loop has 2 records, but it actually ran 3 times, for the first time as a warm-up, do not do statistics.

Its time is the sum of the inner loops, divided by 10000, and the result of the first method is not much worse.

I can print the number of output loops in the method:

The results are as follows:

You can see that it did run 3 times, but counted 2 times.

Then I'll add two additional test methods to test the other two methods separately:

To run the test:

You can see the results of these 4 test methods now.

It appears that foreach is faster than LINQ and for loops for list.

Note that the outer loop of the foreach Test ran 2 times, while the For and LINQ test cycles ran only 1 times, probably because it took too long? I'm not sure about that .

StopWatch

You can see the parameter stopwatch of the test command, which should be from the stopwatch class under the System.Diagnostics namespace.

It has the Start() and Stop() methods and some other properties used to count the elapsed time.

The Stopwatch class is cross-platform, but on other systems it can only count time, and on Windows It also uses the kernel ETW events and CPU performance counts to give you more data, please refer to the relevant information.

Conclusion

The library also has many features and command parameters, please refer to the documentation: Github.com/microsoft/xunit-performance

Note, however, that it is still a beta state and can only be obtained in myget instead of NuGet.