Calculation of CPU utilization rate

Proc File system

The/proc file system is a pseudo-file system that only exists in memory and does not occupy external memory space. It provides the interface for the kernel to communicate with the process in a file system manner. Users and applications can get system information through/PROC, and can change certain parameters of the kernel. Because the information of the system, such as the process, is dynamically changed, so when a user or application reads a file from the/proc directory, the proc file system dynamically reads the required information from the system kernel and submits it.

There are some directory names in the/proc directory, which are the process directories. Each process currently running in the system corresponds to a directory/proc/pid that is the directory name of the process number under/proc, which is the interface that reads the process information. In addition, there is a task directory in the/proc/pid directory in the version above Linux2.6.0-test6, and there are some directory/proc/pid/task/tid named for the thread number of threads owned by the process/proc/pid/task directory. They are interfaces that read thread information.

/proc/cpuinfo file

This file contains information about the CPU (model number, cache size, etc.).

[Email protected] ~]$ Cat/proc/cpuinfo

processor:0

Vendor_id:genuineintel

CPU Family:15

Model:4

Model Name:intel (R) Xeon (TM) CPU 3.00GHz

Stepping:10

CPU mhz:3001.177

Cache size:2048 KB

Physical id:0

Siblings:2

Core id:0

CPU Cores:1

Fdiv_bug:no

Hlt_bug:no

F00f_bug:no

Coma_bug:no

Fpu:yes

Fpu_exception:yes

CPUID Level:5

Wp:yes

FLAGS:FPU VME de PSE TSC MSR PAE MCE cx8 APIC MTRR PGE mcacmov Pat PSE36 clflush DTS ACPI MMX FXSR SSE SSE2 SS HT TM PBE LMPNI Monitor Ds_cpl CID XTPR

bogomips:6004.52

Description: The following only explains the relevant parameters that are useful for our calculation of CPU usage.

Parameter interpretation

Processor (0) A physical identity of the CPU

Conclusion 1: This file can be used to count the logical number of CPUs (including multicore, Hyper-threading) based on the number of occurrences of processor.

/proc/stat file

The file contains information about all CPU activity, and all values in the file are accumulated from the start of the system to the current moment. The format of the file may be inconsistent in different kernel versions, and the following examples illustrate the meaning of each field in the data file.

Instance data: 2.6. On version 24-24

[Email protected]:~$ cat/proc/stat

CPU 38082 627 27594 893908 12256 581 895 0 0

Cpu0 22880 472 16855 430287 10617 576 661 0 0

CPU1 15202 154 10739 463620 1639 4 234 0 0

Intr 120053 222 2686 0 1 1 0 5 0 3 0 0 0 47302 0 0 34194 29775 05019 845 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 00 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 00 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 00 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 00 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 00 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 00 0 0 0 0

Ctxt 1434984

Btime 1252028243

Processes 8113

Procs_running 1

procs_blocked 0

The number on the first line represents the total CPU usage, so we just use the first line of numbers to calculate it. The following table resolves the meanings of the first row of values:

Parameter Resolution (unit: Jiffies)

(Jiffies is a global variable in the kernel, used to record the number of Beats generated from the start of the system, in Linux, a beat is roughly understood as the minimum time slice of the operating system process scheduling, different Linux cores may have different values, usually between 1ms to 10ms)

User (38082) accumulated from the start of the system to the current time, in the user state of the run time, does not include the nice value is a negative process.

Nice (627) Accumulated from the start of the system to the current time, the nice value is negative the CPU time occupied by the process

system (27594) from the start of the system to accumulate to the current moment, in the nuclear mentality of the running time

Idle (893908) from the start of the system to accumulate to the current moment, in addition to the IO wait time other than the wait time iowait (12256) from the start of the system to accumulate to the current moment, IO wait time (since 2.5.41)

IRQ (581) accumulated from the start of the system to the current time, hard interrupt time (since 2.6.0-TEST4)

SOFTIRQ (895) accumulated from the start of the system to the current moment, soft interrupt time (SINCE2.6.0-TEST4)Stealstolen(0) which is the Times spentin other operating systems When running in a virtualizedenvironment (since 2.6.11)

Guest (0) which is the time spent running Avirtual CPU for guest operating systems under the control of Thelinux kernel (since 2. 6.24)

Conclusion 2: Total CPU Time Totalcputime = user + nice + system+ idle + iowait + IRQ + SOFTIRQ + Stealstolen + Guest

/proc//stat file

The file contains information about all the activities of a process, and all values in that file are accumulated from the start of the system

To the current moment. The following example shows the meaning of each field in the file using the instance data.

[Email protected] ~]# Cat/proc/6873/stat

6873 (a.out) R 6723 6873 6723 34819 6873 8388608 77 0 0 0 4195831 0 0 25 0 3 0 5882654 1409024 56 4294967295 134512640 134 5137203215579040 0 2097798 0 0 0 0 0 0 0 17 0 0 0

Description: The following only explains the relevant parameters that are useful for calculating CPU usage

Parameter interpretation

pid=6873 Process Number

utime=1587 the time that the task was run in the user state, in units of jiffies

stime=41958 the mission at the time of the nuclear mentality run, the unit is jiffies

Cutime=0 the time that all dead threads are running in the user state, in units of jiffies

Cstime=0 all have died in the nuclear mentality run time, unit for Jiffies

Conclusion 3: The total CPU time of the process Processcputime =utime + stime + cutime + cstime, which includes the CPU time of all its threads.

/proc//task//stat file

The file contains information about all the activities of a process, and all values in the file are accumulated from the start of the system to the current moment. The content format of the file and the meaning of each field are the same as the/proc//stat file.

Note that the TID field in the file represents no longer a process number, but rather a lightweight process (LWP) in Linux, which is what we typically call a thread.

Conclusion 4: Thread CPU Time Threadcputime = Utime + stime

Common Command PS command for process CPU usage in the system

The PS command allows you to view information about the CPU utilization of the related processes in the system. The following is an explanation of the CPU usage in the PS command output in the Linuxman documentation:

CPU usage is currently expressed as the percentage of time spentrunning during the entire lifetime of a process. This isn't ideal,and it does not conform to the standards, that PS otherwise conformsto. CPU usage is unlikely to add up to exactly 100%.

%CPU CPU utilization of the process in "##.#" format. It is THECPU time used divided by the time the process has been running (cputime/realtime ratio), expressed as a percentage . It'll notadd up to 100% unless is lucky.

Conclusion 5:The CPU utilization calculated by the PS command is relative to the average of the process startup, and the value tends to be flat as the process runs longer.

Top command

The top command allows you to view real-time information about the processes in the system (CPU utilization, etc.). The following is an explanation of the CPU usage in the top command output in the man document for the process.

#C-Last used CPU (SMP) A number representing the last usedprocessor. In a true SMP environment this would likely changefrequently since the kernel intentionally uses weak affinity. Also,the Very act of running top may break this weak affinity and causemore processes to change CPUs more often (because O F The Extrademand for CPU time).

%CPU-CPU Usage The task ' s share of the elapsed CPU time sincethe Last screen update, expressed as a percent-age of Tota L CPUTime. In a true SMP environment, if Irix mode was Off, top willoperate in Solaris mode where a task ' s CPU usage would be divided B Ythe total number of CPUs.

Conclusion 6: the CPU used by a thread during its run may change.

Conclusion 7: The CPU utilization of the top command output in multi-core is *100% calculated by the number of CPUs.

Basic idea of calculating CPU utilization rate in single-core case

By reading/proc/stat,/proc//stat,/proc//task//stat, and/proc/cpuinfo These files get the total CPU time, the CPU time of the process, the CPU time of the thread, and the number of CPUs, It is then calculated using a certain algorithm (sampling two short enough time intervals for CPU snapshots and process snapshots to calculate the CPU utilization of the process).

Total CPU Usage calculation method:

1, sampling two short enough time interval CPU snapshot, respectively recorded as T1,T2, wherein the structure of T1, T2 are:

(User, nice, system, idle, iowait, IRQ, SOFTIRQ, Stealstolen, guest) 9-tuple;

2, calculate the total CPU time slice Totalcputime

A) sum up all CPU usage for the first time and get S1;

b) sum up all CPU usage for the second time, get S2;

c) S2-s1 get all time slices within this time interval, i.e. totalcputime = j2-j1;

3, Calculate idle time idle

Idle corresponds to the fourth column of data, with the second fourth column-the first fourth column can be

idle= Second fourth column-fourth column of the first time

6. Calculate CPU Usage

Pcpu =100* (Total-idle)/total

Calculation method for CPU usage of a process:

1. Sample two short enough time intervals for the CPU snapshot with the process snapshot,

A) Each CPU snapshot is a 9-tuple (user, nice, system, idle, iowait, IRQ, SOFTIRQ, Stealstolen, guest);

b) Each process snapshot is a 4-tuple (Utime, stime, Cutime, cstime);

2. According to Conclusion 2, Conclusion 3 calculates the total CPU time and process CPU time of two time, respectively: TotalCpuTime1, TotalCpuTime2, processCpuTime1, processCpuTime2

3. Calculates the CPU usage of the process pcpu =100* (processcputime2–processcputime1)/(TOTALCPUTIME2–TOTALCPUTIME1) (calculated by 100%, If it is multi-core, multiply the number of CPUs);

Experimental data

Experiment One: monitor the CPU usage of an empty loop process.
Description: The data on the left is the data obtained by the above algorithm, where the sampling interval is the same as the time interval of the top command to refresh the screen.
CPU usage calculated by the above method	From the top command.
99.50083 98.333336 98.0 98.83138 99.0 99.0 99.83361 98.83527 98.4975	PID USER PR NI VIRT RES SHR S %cpu %MEM Time+command 7639 Fjzag 0 206m 10m 7136 S 2.2 1:00.74java 7639 Fjzag 0 206m 10m 7136 S 2.2 1:03.71java 7639 Fjzag 0 206m 10m 7136 S 2.2 1:06.67java 7639 Fjzag 0 206m 10m 7136 S 2.2 1:09.63java 7639 Fjzag 0 206m 10m 7136 S 98 2.2 1:12.59java 7639 Fjzag 0 206m 10m 7136 S 2.2 1:15.55java 7639 Fjzag 0 206m 10m 7136 S 2.2 1:18.55java 7639 Fjzag 0 206m 10m 7136 S 2.2 1:21.54java 7639 Fjzag 0 206m 10m 7136 S 2.2 1:24.52java 7639 Fjzag 0 206m 10m 7136 S 98 2.2 1:27.46 Java

Experiment Two: monitor the CPU utilization of the jconsole process.
Description: The data on the left is the data obtained by the above algorithm, where the sampling interval is the same as the time interval of the top command to refresh the screen.
CPU usage calculated by the above method	From the top command.
8.681135 12.0 10.350584 7.6539097 7.6539097 5.0 13.021703 11.0 8.666667	PID USER PR NI VIRT RES SHR S %cpu %MEM Time+command 7753 Fjzag 0 252m 72m 22m S Ten 14.4 0:18.70jconsole 7753 Fjzag 0 252m 72m 22m S 14.4 0:19.07jconsole 7753 Fjzag 0 252m 72m 22m S One 14.4 0:19.39jconsole 7753 Fjzag 0 252m 72m 22m S 7 14.4 0:19.61jconsole 7753 Fjzag 0 252m 72m 22m S 7 14.4 0:19.83jconsole 7753 Fjzag 0 252m 72m 22m S 5 14.4 0:19.97jconsole 7753 Fjzag 0 252m 72m 22m S 14.4 0:20.38jconsole 7753 Fjzag 0 252m 72m 22m S Ten 14.4 0:20.68jconsole 7753 Fjzag 0 252m 72m 22m S 9 14.5 0:20.96 Jconsole

Calculation method for CPU usage of a thread:

1. Sample two short enough time-interval CPU snapshots with thread snapshots,

A) Each CPU snapshot is a 9-tuple (user, nice, system, idle, iowait, IRQ, SOFTIRQ, Stealstealon, guest);

b) Each thread snapshot is a 2-tuple (utime, stime);

2. According to Conclusion 2, Conclusion 4 calculates the total CPU time and thread CPU time of two time, respectively: TotalCpuTime1, TotalCpuTime2, threadCpuTime1, threadCpuTime2

3. Calculates the CPU usage of the thread pcpu =100* (threadcputime2–threadcputime1)/(TOTALCPUTIME2–TOTALCPUTIME1) (calculated by 100%, multiplied by the number of CPUs in the case of multicore) ;

Experimental data

Experiment One: monitor the CPU usage of an empty loop thread.
Description: The data on the left is the data obtained by the above algorithm, where the sampling interval is the same as the time interval of the top command to refresh the screen.
CPU usage calculated by the above method	From the top command.
  98.83138 97.00997 96.98997 97.49583 98.169716 96.8386 97.333336 93.82304 98.66667	PID USER PR NI VIRT RES SHR S %cpu %MEM Time+command 7649 Fjzag 0 206m 10m 7136 R 2.2 7:22.94java 7649 Fjzag 0 206m 10m 7136 R 2.2 7:25.86java 7649 Fjzag 0 206m 10m 7136 R 2.2 7:28.76java 7649 Fjzag 0 206m 10m 7136 R 2.2 7:31.72java 7649 Fjzag 0 206m 10m 7136 R 98 2.2 7:34.65java 7649 Fjzag 0 206m 10m 7136 R 2.2 7:37.53java 7649 Fjzag 0 206m 10m 7136 R 98 2.2 7:40.47java 7649 Fjzag 0 206m 10m 7136 R 2.2 7:43.34java 7649 Fjzag 0 206m 10m 7136 R 2.2 7:46.25 Java

Experiment Two: monitor the CPU utilization of a thread in the Jconsole program.
Description: The data on the left is the data obtained by the above algorithm, where the sampling interval is the same as the time interval of the top command to refresh the screen.
CPU usage calculated by the above method	From the top command.
1.3400335 6.644518 1.3333334 0.6677796 0.6666667 1.3333334 1.3333334	PID USER PR NI VIRT RES SHR S %cpu %MEM Time+command 7755 Fjzag 0 251m 72m 22m S 1 14.4 0:11.92jconsole 7755 Fjzag 0 251m 72m 22m S 7 14.4 0:12.12jconsole 7755 Fjzag 0 251m 72m 22m S 2 14.4 0:12.18jconsole 7755 Fjzag 0 251m 72m 22m S 0 14.4 0:12.18jconsole 7755 Fjzag 0 251m 72m 22m S 1 14.4 0:12.20jconsole 7755 Fjzag 0 251m 72m 22m S 1 14.4 0:12.24jconsole 7755 Fjzag 0 251m 72m 22m S 1 14.4 0:12.28 Jconsole

Calculation of CPU utilization in multi-core case

The following experiment data is used to illustrate the CPU utilization of a process in multicore cases is calculated by the number of CPUs *100%.

A description of the experiment:

In the case of a dual-core set of experiments, the first set of data is through the Ps-elo pid,lwp,pcpu |grep 9140 command to view the process number 9140 for each thread in the process details. The second set of data is to view the CPU utilization of the process number 9140 process through the PS command.

Data one:

PID LWP%CPU

9140 9140 0.0
9140 9141 0.0
9140 9142 0.0
9140 9143 0.0
9140 9144 0.0
9140 9149 0.0
9140 9150 0.0
9140 9151 0.0
9140 9152 0.1
9140 9153 96.6 The thread is an empty loop
9140 9154 95.9 The thread is an empty loop

In addition to the two threads highlighted in red, other threads are background threads.

Data two:

PID%CPU

9140 193

Experiment two describes:

In the case of a single-core set of experiments, the first set of data is through the Ps-elo pid,lwp,pcpu |grep 6137 command to view the process number 6137 for each thread in the process details. The second set of data is to view the CPU utilization of the process number 6137 process through the PS command.

Data one:

PID LWP%CPU

6137 6137 0.0

6137 6138 0.1

6137 6143 0.0

6137 6144 0.0

6137 6145 0.0

6137 6146 0.0

6137 6147 0.0

6137 6148 0.0

6137 6149 0.0

6137 6150 46.9 Empty loop thread

6137 6151 46.9 Empty loop thread

In addition to the two threads highlighted in red, other threads are background threads.

Data two

PID%CPU

6137 92.9

Problems:

1. Different kernel versions/proc/stat file formats are not consistent. The total CPU usage of the first behavior in the/proc/stat file.

There are 4 fields for each version: User, nice, system, idle

2.5.41 version new field: Iowait

2.6.0-test4 new fields: IRQ, SOFTIRQ

2.6.11 new field: Stealstolen:which is the time spent in otheroperating

Systems when running in a virtualized environment

2.6.24 new field: Guest:which is the time spent running a virtual Cpufor guest operating systems under the control of the LINUXK Ernel

2. The/proc/pid/task directory is a feature that is only available after Linux 2.6.0-test6.

3. With regard to the case of negative CPU usage, the current solution is to calculate the CPU usage to non-negative if a negative value is present for continuous sampling.

4. Some threads have a shorter life cycle and may have died during our sampling period.

Calculation of CPU utilization rate