Use System. Diagnostics. stopwatch to accurately measure the running time of the program

Introduction

Every person who uses the stopwatch class in the system. Diagnostics namespace for performance optimization will encounter some troubles sooner or later. As you can see, tests with the same function on the same computer may differ from 25% to 30% during running time. This article describes how to use the stopwatch class to design a single-thread TestProgram0.1%-0.2% accuracy. With this precision,AlgorithmCan be tested and compared.

Background

Modern CPUs have multiple kernels, large-capacity high-speed caching, command pipelines, and many other things that affect the running time of an algorithm in a specific test scenario. White-box testing technology-such as an additional debugger or analyzer-to disable high-speed cache lines and pipelines of the CPU. The real running time is hidden, so that the execution speed of these computation methods optimized by the modern excess processor is slower than that optimized by the analyzer (due to more instructions ). Black box testing does not include a debugger or analyzer (runtime measurement), which can discover the actual performance of the algorithm and complete the performance analysis of the algorithm.
Set a test plan

The most important thing is to prevent switching between CPU cores or processors. It has a great impact on performance testing. This can be achieved by setting the processoraffinity of the process:

Process. getcurrentprocess (). processoraffinity =   New Intptr ( 2 ); // Use only the second core

To exclusively occupy the CPU kernel, we must prevent other threads from using this CPU kernel. We set the priority of processes and threads to achieve this goal:

Process. getcurrentprocess (). priorityclass = Processpriorityclass. High;
Thread. currentthread. Priority = Threadpriority. Highest;

Last, but not least, in the warm-up phase of our test needs. In my system, the results are stable after-milliseconds of warm-up. We can use stopwatch to control the warm-up (at least 1200 ms here ):

Stopwatch. Start ();
While (Stopwatch. elapsedmilliseconds <   1200 )
{
Result = Testfunction (seed, count );
}
Stopwatch. Stop ();

The following is a complete example:

Using System;
Using System. diagnostics;
Using System. Threading;

NamespacePrecisemeasure
{
ClassProgram
{
Static VoidMain (String[] ARGs)
{
Stopwatch stopwatch= NewStopwatch ();

Long Seed = Environment. tickcount; // Prevents the JIT Compiler
// From optimizing fkt cballs away
Long Result =   0 ;
Int Count =   100000000 ;

Console. writeline ( " 20 tests without correct preparation " );
Console. writeline ( " Warmup " );
For ( Int Repeat =   0 ; Repeat <   20 ; ++ Repeat)
{
Stopwatch. Reset ();
Stopwatch. Start ();
Result ^ = Testfunction (seed, count );
Stopwatch. Stop ();
Console. writeline ( " Ticks: "   + Stopwatch. elapsedticks +  
" MS: "   + Stopwatch. elapsedmilliseconds );
}
//Uses the second core or processor for the test
Process. getcurrentprocess (). processoraffinity =   New Intptr ( 2 ); //Prevents "normal" Processes //From interrupting threads
Process. getcurrentprocess (). priorityclass = Processpriorityclass. High; //Prevents "normal" threads
Thread. currentthread. Priority = Threadpriority. Highest; // From interrupting this thread
Console. writeline ();
Console. writeline ();
Console. writeline ( " 20 tests with correct preparation " );
Console. writeline ( " Warmup " );
Stopwatch. Reset ();
Stopwatch. Start ();
While (Stopwatch. elapsedmilliseconds <   1200 ) // A warmup of 1000-1500 MS
// Stabilizes the CPU cache and pipeline.
{
Result = Testfunction (seed, count ); // Warmup
}
Stopwatch. Stop ();

For ( Int Repeat =   0 ; Repeat <   20 ; ++ Repeat)
{
Stopwatch. Reset ();
Stopwatch. Start ();
Result ^ = Testfunction (seed, count );
Stopwatch. Stop ();
Console. writeline ( " Ticks: "   + Stopwatch. elapsedticks +  
" MS: "   + Stopwatch. elapsedmilliseconds );
}
Console. writeline (result ); // Prevents optimizations (current compilers are
// Too silly to analyze the dataflow that deep, but we never know)
}

Public   Static   Long Testfunction ( Long Seed, Int Count)
{
Long Result = Seed;
For ( Int I =   0 ; I < Count; ++ I)
{
Result ^ = I ^ Seed; // Some useless bit operations
}
Return Result;
}
}
}

Result:

No proper preparation

1. Ticks: 1580367 MS: 572 <-- highest value
2. Ticks: 1577003 MS: 571
3. Ticks: 1576140 MS: 571
4. Ticks: 1560964 MS: 565
5. Ticks: 1351663 MS: 489
6. Ticks: 1248383 MS: 452
7. Ticks: 1115361 MS: 404
8. Ticks: 1112813 MS: 403
9. Ticks: 1113112 MS: 403
10. Ticks: 1112012 MS: 402 <-- lowest value
11. Ticks: 1330444 MS: 482
12. Ticks: 1558493 MS: 564
13. Ticks: 1501097 MS: 543
14. Ticks: 1517796 MS: 549
15. Ticks: 1542712 MS: 558
16. Ticks: 1574959 MS: 570
17. Ticks: 1483975 MS: 537
18. Ticks: 1390578 MS: 503
19. Ticks: 1546904 MS: 560
20. Ticks: 1349507 MS: 488

The running time ranges from 402ms572ms. There is a 170 MS or 42% gap. Obviously, these results are useless.

Proper preparation:

1. Ticks: 1110518 MS: 402
2. Ticks: 1110540 MS: 402
3. Ticks: 1110543 MS: 402
4. Ticks: 1110684 MS: 402 <-- highest value
5. Ticks: 1110508 MS: 402
6. Ticks: 1110553 MS: 402
7. Ticks: 1110600 MS: 402
8. Ticks: 1110433 MS: 402 <-- lowest value
9. Ticks: 1110509 MS: 402
10. Ticks: 1110508 MS: 402
11. Ticks: 1110489 MS: 402
12. Ticks: 1110568 MS: 402
13. Ticks: 1110503 MS: 402
14. Ticks: 1110566 MS: 402
15. Ticks: 1110625 MS: 402
16. Ticks: 1110474 MS: 402
17. Ticks: 1110571 MS: 402
18. Ticks: 1110448 MS: 402
19. Ticks: 1110555 MS: 402
20. Ticks: 1110495 MS: 402

There are 20 identical results: 402 MS, which can only be distinguished by Tick (internal CPU performance counter value. There is a 251tick or 02% gap in the test results. In my system, stopwatch frequency is 2760029 tick per second. The difference between tests is only milliseconds. This is very good for measuring and comparing algorithms.

Points of interest:

One important thing should be kept in mind. The best (lowest) result value that has not been prepared is good. The CPU context and core switch have a huge impact on the application running.

Http://www.codeproject.com/KB/testing/stopwatch-measure-precise.aspx