Introduction to Linux System monitoring commands

Linux system monitoring has the following commands:

Uptime: Display system load condition

Mpstat: Show CPU performance

Free: Displaying memory usage

Vmstat: Displaying virtual memory usage

Iostat: Show System I/O activity

Netstat: Displays the activity of the network

DF/DU: Displaying disk usage

1. Uptime

fdipzone@ubuntu:~$ uptime  
 00:06:00 up min,  2 users,  load average:0.00, 0.03, 0.07

00:06:00 indicates the current time

32min indicates system run time

2 users represents current number

Load average:0.00, 0.03, 0.07 represent the system load in the past 1, 5, and 15 minutes respectively.

Only one parameter-V shows uptime version

fdipzone@ubuntu:~$ uptime-v  
Procps version 3.2.8

2. Mpstat

Displays various performance information for the system CPU and can be used to count individual CPUs.

Usage: mpstat [options] [< interval > [< times >]]  
option:  
[-A] [i-{SUM | CPU | All}] [-u] [p {<cpu> [,...] | All}] [-V]

Displays information about all CPUs, executed once every 1 seconds, and executed 2 times.

Mpstat-p All 1 2

-P All displays statistics for all CPUs

-P 0 Displays statistics for the first CPU, and so on.

The parameter 1 2 indicates that every 1 seconds executes 2 times, and if 2 This parameter is not entered, it will continue until the user interrupts.

fdipzone@ubuntu:~$ mpstat-p all 1 2 Linux 2.6.32-21-generic (Ubuntu) Friday, March, 2014 _x86_64_ (2 CPU) 04:09:28 HKT CPU%usr%nice%sys%iowait%irq%soft%steal%guest%idle 04:09:29 all 0    0.00 0.00 0.00 0.00 0.00 0.00 0.00 100.00 04:09:29 HKT 0 0.00 0.00 0.00 0.00  0.00 0.00 0.00 0.00 100.00 04:09:29 HKT 1 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00   100.00 04:09:29 HKT%usr%nice%sys%iowait%irq%soft%steal%guest%idle 04:09:30 HKT    All 0.42 0.00 0.00 0.00 0.00 0.00 0.00 0.00 99.58 04:09:30 HKT 0 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 100.00 04:09:30 HKT 1 0.72 0.00 0.00 0.00 0.00 0.00 0.0 0 0.00 99.28 Average time: CPU%usr%nice%sys%iowait%irq%soft%steal%guest%idle Average time: AL L 0.25 0.00 0.00 0.00 0.00 0.00 0.00 0.00 99.75 Average Time: 0 0.00 0.00 0.00 0.00 0.00 0.00 0.00 100.00 Average Time: 1 0.51 0.00 0.00 0.00 0.00 0.00 0.00 0.00 99.49

Meaning of each column:

%USR represents the percentage of CPU used to process the user process, which is a non-kernel process for the application

%nice represents the percentage of CPU when a process is demoted using the nice command

%sys represents the percentage of CPU used by the kernel process

%iowait represents the percentage of CPU time that is used to wait for I/O

%IRQ represents the percentage of CPU used to handle system interrupts

%soft represents the percentage of CPU used for software interrupts

%steal must wait for the other CPU (virtual CPU) time

%guest time to execute virtual CPU

%idle shows the CPU idle time

%INTR/S shows total number of interrupts per second CPU received

3. Free

fdipzone@ubuntu:~$ free-h  
usage:free [-b|-k|-m|-g] [-l] [-o] [-t] [-s delay] [-c Count] [-v]  
  -b,-k,-m,-g by KB, M B, or GB display  
  -L displays detailed high and low memory statistics  
  -o use the old format, no-/+ buffers/cache this item-  
  t shows the total number of Ram+swap-  
  s per X-second update  
  -C-Update x times  
  - V Display version

fdipzone@ubuntu:~$ free  
             total       used       free     shared    buffers     cached  
Mem:       1022644     904668     117976          0      96872     369652  
-/+ buffers/cache: 438144 584500  
Swap:      1046524          0    1046524

Mem: Physical Memory statistics

Total: Amount of memory

Used: Allocated Memory

Free: Idle memory

Shared: share memory, common system does not use

Buffers: System has been allocated, but the number of unused buffer

Cached: The number of cached that the system has allocated but not used

-/+ Buffers/cache: Cache statistics for physical memory

used:used-buffers-cached = Total amount of memory actually used

Free: The sum of unused buffers with cache and idle memory, that is, the system's current actual available memory

Swap: Indicates the use of swap partitions on the hard disk

4. Vmstat

Vmstat a total of two parameters, one is the interval of each collection, the other is the number of collection.

The Vmstat 1 11 indicates that each collection time is 1 seconds, and a total of 11 times are collected. Because the first collection of Vmstat records is not correct, so if you want to collect 10 times, please set the number of times 11, so that you can get 10 useful records.

fdipzone@ubuntu:~$ Vmstat 1 procs-----------memory-------------Swap-------io-----system------CPU----r   b swpd free buff cache si so bi bo in CS us sy ID WA 0 0 0 335240 52072 241320 0 0   689 42 234 460 4 13 80 3 1 0 0 335860 52072 241320 0 0 0 0 352 528 2 2 97 0-2    0 0 336364 52072 241320 0 0 0 0 564 692 4 3 94 0 0 0 0 336992 52072 241320-0    
 0 0 0 316 483 1 2 97 0 2 0 0 336432 52076 241476 0 0 156 0 332 554 2 6 91-2    0 0 0 336928 52084 241480 0 0 0 84 349 494 1 3 96 0 0 0 0 337564 52084 241476-0    
 0 0 0 263 437 0 2 98 0 0 0 0 338068 52084 241476 0 0 0 0 299 456 1 2 97-0     0 0 0 338696 52084 241476 0 0 0 0 321 473 1 3 96 0 1 0 0 339200 52084 241476-0  0 0 0 271461 0 2 98 0 0 0 0 339828 52084 241476 0 0 0 0 255 429 1 2 96-0 

Vmstat output for each parameter description:

R indicates that the running queue (that is, how many processes are actually allocated to the CPU), when this value exceeds the number of CPUs, there will be a CPU bottleneck. This also and top of the load is related to the general load more than 3 higher, more than 5 on the high, more than 10 is not normal, the state of the server is very dangerous. The top load is similar to the run queue per second. If the running queue is too large, which means your CPU is busy, it will generally cause high CPU usage.

b represents a blocked process.

SWPD virtual memory has been used in size, if greater than 0, indicates that your machine physical memory is not enough, if not the reason for the program memory leak, then you should upgrade the memory or the memory of the task to migrate to other machines.

Free of the physical memory size, my test machine memory total 1G, remaining 330M.

Buff Linux/unix system is used to store, directory contains what content, permissions, etc. cache, I this machine probably occupy more than 500 m

Cache cache is directly used to memorize the files we open, to buffer the files, (here is the smart of Linux/unix, the free physical memory part of the cache for files and directories, is to improve the performance of program execution, when the program uses memory, buffer/ Cached will soon be used. )

Si per second the size of the virtual memory from disk, if this value is greater than 0, indicates that the physical memory is not enough or memory leaks, to find the memory process to solve the loss.

So per second virtual memory is written to disk size, if this value is greater than 0, ditto.

The number of blocks received by the BI block device per second, where the block device refers to all the disks and other block devices on the system, and the default block size is 1024byte.

The number of blocks sent by the Bo block device per second, for example, when we read the file, Bo must be greater than 0. Bi and Bo are generally close to 0, or IO is too frequent and need to be adjusted.

The number of CPU interrupts per second, including time outages

CS the number of context switches per second, for example, we call the system function, we need to do context switching, thread switching, also process context switch, this value to the smaller the better, too big, to consider the number of threads or processes, such as in Apache and nginx such a Web server, We generally do performance testing will be thousands of concurrent or even tens of thousands of concurrent testing, the process of selecting a Web server can be a process or thread peak has been reduced, pressure, until the CS to a relatively small value, the process and the number of threads is more appropriate value. System call is also, each call system functions, our code will enter the kernel space, resulting in context switching, this is very resource-intensive, but also try to avoid frequent call system functions. Too many context switches means that most of your CPU waste in context switching, resulting in less CPU time to do serious things, the CPU is not fully utilized, is not desirable.

US user CPU time, higher than 70% indicates that the CPU is under pressure.

SY system CPU time, if too high, indicates a long system call time, such as IO operations frequently.

ID free CPU time, in general, ID + US + sy = 100, generally I think ID is idle CPU usage, US is the user CPU utilization, SY is the system CPU utilization rate.

WT waits for IO CPU time.

5. Iostat

Iostat is mainly used to monitor the IO load of system devices, and users can get statistics by setting the number and time of statistics.

Usage:iostat [Options] [<interval> [<count>]]  
options are:  
[-c] [d] [-n] [-n] [-h] [ -K | -M] [t] [-v] [-X] [-z]  
[<device> [...] | ALL] [-P [<device> [,...] | ALL]]

Parameter description:

-D display device (disk) Usage status

-K certain columns that use the block to force the use of kilobytes as a unit

Iostat-d-K 1 2 display once every 1 seconds, a total of 2 times

fdipzone@ubuntu:~$ iostat-d-K 1 2  
Linux 2.6.32-21-server (nmgwww34)       Friday, March, 2014  _x86_64_        (2 CPU)  
       
Device:            TPs    kb_read/s    kb_wrtn/s    kb_read    kb_wrtn        SDA 9.88 15.67       122.85  379571972 2975248356  
       
Device:            TPs    kb_read/s    kb_wrtn/s kb_read KB _wrtn  
SDA               0.00         0.00         0.00          0          0