The use and analysis of Linux real-time performance test Tool--cyclictest

About the Cyclictest tool, there are instructions on the wiki: https://wiki.linuxfoundation.org/realtime/documentation/howto/tools/cyclictest

Cyclictest is a high resolution test program, written by USER:TGLX, maintained by Clark Williams and John kacur

Documentation Installation
Get the latest sources from the Git repository, do a git clone git://git.kernel.org/pub/scm/utils/rt-tests/rt-tests.git or Fetch a released tarball from the archive, Untar to a directory of your choice and run make in the source directory. If you want to cross compile, just run make cross_compile= (for example make cross_compile=arm-v4t-linux-gnueabi-).
You can run the resulting binary from there or install it.

Run it

Make sure to is root or use the sudo to run cyclictest.
Without parameters Cyclictest creates one thread with a 1ms interval timer.
CYCLICTEST-H provides help text for the various options

[[email protected] rt-tests]#./cyclictest--helpcyclictest V 1.00usage:cyclictest <options>-a [CPUSET]--  Affinity Run thread #N on processor #N, if possible, or if cpuset given, pin threads to that  Set of processors in Round-robin order.                           e.g.-a 2 pins all threads to CPU 2, But-a 3-5,0-t 5 would run the first and fifth The threads on CPU (0), the thread #2 on CPU 3, the thread #3 on CPU 4, and the thread #5 on CPU 5. -A USEC--aligned=usec align thread wakeups to A specific offset-b USEC--breaktrace=usec sends a stop trace when the delay is greater than the value specified by USEC. USEC, the unit is the second of the Fallacy (US).                            -B--preemptirqs both preempt and Irqsoff tracing (used with-b)-C clock--clock=clock select Clocks cyclictest-c 1         0 = clock_monotonic (default) 1 = clock_realtime-c--context Context Switch tracing (used with-b)-D DIST--distance=dist DistaNCE of thread intervals in US, default=500-d--duration=time Specify a length for the test run.         Append ' m ', ' h ', or ' d ' to specify minutes, hours or days. --latency=pm_qos Write Pm_qos to/dev/cpu_dma_latency-e--event Event Tracing (used With-b)-F--ft  Race FTRACE function tracking (usually used with-B, usually with-B, not using-f)-F--fifo=<path> Create a named pipe at path and Write stats to it-h--histogram=us draw the deferred histogram on the standard output device after execution (many threads have the same permissions) us is the maximum tracking time limit, this can be used when the example is described below, combined with gnuplot can draw our The result diagram of the test. -H--histofall=us same as-h except with an additional summary column--histfile=<path> dump the LA         Tency histogram to <path> instead of stdout-i intv--INTERVAL=INTV basic thread interval, default is 1000 (in US)-I--irqsoff Irqsoff tracing (used With-b)-l LOOPS--loops=loops loop number, default is 0 (infinity), and the-I interval number to approximate the time of the entire test, such as-I 1000-l 1000000         , the total cycle time is 1000*1000000=1000000000 us =1000s, so roughly 16 minutes more. --laptop Save Battery When running cyclictest this would give you poorer realtime results Drain your battery so quickly-m--mlockall lock current and future memory allocations-M--REFR Esh_on_max delay updating the screen until a new max latency are hit.       Userful for low bandwidth.-n--nanosleep using clock_nanosleep--notrace suppress tracing-n --nsecs print results in NS instead of us (default us)-O RED--oscope=red oscilloscope mode, reduce verb        OSE output by Red-o topt--traceopt=topt trace option-p PRIO--priority=prio priority usage for highest priority thread:-P/--prio=90-p --preemptoff Preempt off tracing (used with-b)--policy=name policy of measurement thread, where         NAME may be one of:other, normal, batch, idle, FIFO or RR. --priospread spread priority levels starting at specified VALUE-Q--quiet does not print information when run with the-Q parameter, only prints the profile on exit, and the-H histnum parameter prints histnum row statistics along with a general profile on exit. -R--relative Use relative timer instead of absolute-r--resolution Check clock resolution, Callin  G Clock_gettime () many times.  List of Clock_gettime () values would be reported with-x--secaligned [USEC] Align thread Wakeups to the next full second and apply the optional offset-s--system use sys                           _nanosleep and sys_setitimer-s--SMP standard SMP testing:options-a-t-n and  Same priority of all threads--spike=<trigger> record all spikes > Trigger--spike-nodes=[num of                           Nodes] These is the maximum number of spikes we can record. The default is 1024x768 if not specified--smi Enable SMI counting-t--threads one thread PE R AvailAble processor-t [num]--threads=num number of threads:without NUM, threads = Max_cpus without-t default = 1--tracemark write a trace Mark When-b latency is exceeded-t TR      ACE--tracer=tracer Set tracing function configured tracers:unavailable (Debugfs not mounted)-U--unbuffered                           Force unbuffered output for live processing-u--numa standard NUMA testing (similar to SMP option)  Thread data structures allocated from local node-v--verbose output values on stdout For statistics format:n:c:v n=tasknum c=count v=value in us-w--wakeup task Wak  EuP tracing (used With-b)-W--wakeuprt RT Task Wakeup tracing (used with-b)--dbg_cyclictest print Info Useful for debugging Cyclictest

recommended parameters and result instances

[[email protected] rt-tests]# sudo./cyclictest-p 90-m-C 0-i 200-n-H 100-q-L 1000000 We use-P 90 to assign to Cyclictest First level 90, using the-m parameter to lock memory allocations, using the-C 0 to specify the default monotonic clock,-I 200 to specify a loop of 200US, combined with-l 1000000 for a total of 1 million loops, and-N for using nanosleep instead of simple SLE Ep,-q to not print instant messages at run time, the-H 100 is a total of 100 information in the final result. -----#/dev/cpu_dma_latency set to 0us-------------(below are the information that is printed after the end of the test/terminal test, which is the effect of-Q!). ) #Histogram000000 000000000001 000000000002 000000000003 000000000004 000000000005 000002--The delay of 5us is 1 million times in 2 cycles ( This is the meaning of each line below) 000006 000009 ..... Omitted here 000099 000005--we use-H 100, so the time delay is recorded in the results of 0US ~ 99us #total:000999914#min latencies:00005--Minimum delay 5 u S#avg latencies:00012--average delay 12us#max latencies:19920--Maximum delay 19920 us, then we specify histogram = 100 that is, only the value of 0us~99us is recorded. And the maximum delay of 19920 means there must be a lot of this delay more than the US, then where the record? The answer is that there is no specific delay value of more than 99us, only the number of delay times (recorded in overflows) is recorded below the 99us, and the first several times exceeded (recorded on thread 0). #Histogram overflows:00086-More than US #histogram Overflow at cycle number: #Thread 0:65,668 162024164458 166533 171828 174546 179471 182538 188257 198415 202689 209055211934 224529 227292 239809 267144 311992 312072 3350 66 341986 353395 355217 355295 355297 385017411492 417012 443642 453450 453463 453478 453492 453504 453505 453522 453540 4  82063 482116 482797483077 486153 515557 517062 517066 522812 538214 560636 574301 574500 598338 602175 610697 620924678231 692237 692242 692247 713557 779826 797948 851442 860635 860642 860654 860661 861147 875755880618 883622 884128 884238 885 915 887215 887457 896442 925069 928998 942590 947161 947871 955507955508 982245 982250 992192//The number of cycles recorded here is more than 99us.

$ sudo cyclictest -t 2  // 使用两个测试线程policy: other/other: loadavg: 0.00 0.01 0.05 1/346 2595T: 0 ( 2594) P: 0 I:1000 C:  14090 Min:     32 Act:  200 Avg:  177 Max:    2855 T: 1 ( 2595) P: 0 I:1500 C:   9397 Min:     23 Act:  202 Avg:  170 Max:    2863

Meaning of the output result:
T:0 a thread with a sequence number of 0
p:0 Thread priority is 0
c:9397 counter. Every time the thread is reached, the counter adds 1
i:1000 time interval is 1000 microseconds (US)
Min: Minimum delay (US)
ACT: Last time delay (US)
Avg: Average Latency (US)
Max: Maximum delay (US)

Expected Results

TGLX ' s Reference machine

All tests has been run on a Pentium III 400MHz based PC.
The tables show comparisons of vanilla Linux 2.6.16, Linux-2.6.16-hrt5 and linux-2.6.16-rt12. The tests for intervals less than the jiffy resolution has not been run on vanilla Linux 2.6.16. The test thread runs in all cases with SCHED_FIFO and priority 80. All numbers is in microseconds.

Test Case:clock_nanosleep (time_abstime), Interval 10000
microseconds,. 10000 loops, no load.

Commandline: cyclictest -t1 -p 80 -n -i 10000 -l 10000 Kernel min max avg 2.6.16 24 4043 1989 2.6.16-hrt5 12 94 20 2.6.16-rt12 6 40 10

1Test Case:clock_nanosleep (time_abstime), Interval 10000 Micro
Seconds,. 10000 loops, 100% load.

Commandline: cyclictest -t1 -p 80 -n -i 10000 -l 10000 Kernel min max avg 2.6.16 55 4280 2198 2.6.16-hrt5 11 458 55 2.6.16-rt12 6 67 29

Test case:posix interval Timer, interval 10000 micro seconds,. 10000
loops, no load.

Commandline: cyclictest -t1 -p 80 -i 10000 -l 10000 Kernel min max avg 2.6.16 21 4073 2098 2.6.16-hrt5 22 120 35 2.6.16-rt12 20 60 31

Test case:posix interval Timer, interval 10000 micro seconds,. 10000
Loops, 100% load.

Commandline: cyclictest -t1 -p 80 -i 10000 -l 10000 Kernel min max avg 2.6.16 82 4271 2089 2.6.16-hrt5 31 458 53 2.6.16-rt12 21 70 35

Test Case:clock_nanosleep (time_abstime), Interval
Seconds,. 100000 loops, no load.

Commandline: cyclictest -t1 -p 80 -i 500 -n -l 100000 Kernel min max avg 2.6.16-hrt5 5 108 24 2.6.16-rt12 5 48 7

Test Case:clock_nanosleep (time_abstime), Interval
Seconds,. 100000 loops, 100% load.

Commandline: cyclictest -t1 -p 80 -i 500 -n -l 100000 Kernel min max avg 2.6.16-hrt5 9 684 56 2.6.16-rt12 10 60 22

Test case:posix interval timer, interval micro seconds,. 100000
loops, no load.

Commandline: cyclictest -t1 -p 80 -i 500 -l 100000 Kernel min max avg 2.6.16-hrt5 8 119 22 2.6.16-rt12 12 78 16

Test case:posix interval timer, interval micro seconds,. 100000
Loops, 100% load.

Commandline: cyclictest -t1 -p 80 -i 500 -l 100000 Kernel min max avg 2.6.16-hrt5 16 489 58 2.6.16-rt12 12 95 29

FAQ

PS shows the wrong scheduling class Sched_other

Each cyclictest-task consist of one or more threads. Ps-ce shows only the main-process is not the threads of the main-process. PS-ELC | grep cyclic shows the main-process an and the containing threads with the correct scheduler class Sched_fifo.

#>./cyclictest -t5 -p 80 -n -i 10000#> ps -cLe | grep cyclic 4764  4764 TS   19 pts/1    00:00:01 cyclictest 4764  4765 FF  120 pts/1    00:00:00 cyclictest 4764  4766 FF  119 pts/1    00:00:00 cyclictest 4764  4767 FF  118 pts/1    00:00:00 cyclictest 4764  4768 FF  117 pts/1    00:00:00 cyclictest 4764  4769 FF  116 pts/1    00:00:00 cyclictest

Chrt shows the wrong scheduling class Sched_other

Don ' t use the PID of the main-process, but the PID of the threads from the main-process. The threads is shown with Ps-cle | grep cyclic.

#> chrt -p 4766pid 4766‘s current scheduling policy: SCHED_FIFOpid 4766‘s current scheduling priority: 79

The use and analysis of Linux real-time performance test Tool--cyclictest