Brief introduction
Lists the operating system level monitoring commonly used several methods, it is recommended to collect the use
Cpu
The top command can be used to monitor the overall load of the system, including CPU, memory usage, and real-time display of the resource usage of each process in the system.
Output sample
top-19:37:41 up 192 days, 9:14, 1 user, load average:0.39, 0.28, 0.27tasks:126 Total, 1 running, sleeping, 0 stopped, 0 zombiecpu (s): 0.1%us, 0.1%sy, 0.0%ni, 99.8%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%stmem:32113m Total, 11351M used, 20762M free, 266M buffersswap:4091m total, 0M used, 4091M free, 304M cached PID USER PR NI VIRT RES SHR S%cpu%MEM time+ COMMAND 1 root 0 25280 1016 968 S 0 0.0 2:37.44 Init 2 Root 20 0 0 0 0 S 0 0.0 0:02.40 Kthreadd 3 Root 0 0 0 0 S 0 0. 0 3:28.59 ksoftirqd/0 5 Root 0 0 0 0 S 0 0.0 0:00 . kworker/u:0 6 root RT 0 0 0 0 S 0 0.0 0:11.39 MIGR ation/0
Indicator description
- Load average:0.06, 0.60, 0.48
System load, which is the length of the task queue. Average in 1 minutes, 5 minutes, 15 minutes, respectively.
- Tasks
Process summary, including run-in, sleep, stop, zombie-state grooming
- Cpu (s)
Indicators |
Description |
0.3% US |
Percentage of CPU occupied by user space |
1.0% Sy |
Percentage of CPU consumed by kernel space |
0.0% ni |
User process priority scheduling CPU percentage |
98.7% ID |
Percentage of idle CPU |
0.0% WA |
Percentage of CPU time waiting for input and output |
0.0% hi |
Hardware CPU Interrupt Occupancy percentage |
0.0% si |
Soft interrupt Occupancy Percentage |
0.0% St |
Virtual machine Occupancy Percentage |
Mem
Memory statistics, including physical memory, used memory, free memory, kernel cache
- Swap
Swap area statistics, including total, used, idle, slow stock
Process List
Indicators |
Description |
Pid |
Process ID |
USER |
User name of the process owner |
PR |
Priority level |
NI |
Nice value. Negative values indicate high priority, positive values indicate low priority |
VIRT |
The total amount of virtual memory used by the process, in kilobytes. Virt=swap+res |
Res |
The size, in kilobytes, of the physical memory that the process used and was not swapped out. Res=code+data |
Shr |
Shared memory size, in kilobytes |
S |
Process status (d= non-interruptible sleep state, r= run, s= sleep, t= track/Stop, z= zombie process) |
%cpu |
Percentage of CPU used by the process |
%MEM |
Percentage of physical memory used by the process |
time+ |
Total CPU time used by the process, Unit 1/100 sec |
SWAP |
The amount of virtual memory that the process uses, the size that is swapped out, the unit kb |
CODE |
The amount of physical memory the executable code occupies, in kilobytes |
DATA |
The amount of physical memory that is used by parts other than executable code (data segment + stack), in kilobytes |
COMMAND |
Command name/command line |
Press the F key to choose to display different indicators, very detailed
Reference documents
Memory monitoring
Vmstat refers to virtual memory statistics (meomory Statistics), which is a common real-time system monitoring tool.
Output sample
~-> vmstat 2procs-----------memory-------------Swap-------io-----system-------CPU------r B swpd free buff Cache si so bi bo in CS us sy ID WA St 0 0 212 21250404 272916 320572 0 0 0 2 0 0 0 0 100 0 0 0 0 212 21250652 272916 320576 0 0 0 0 535 1061 0 0 100 0 0 0 0 212 21250652 2 72916 320576 0 0 0 0 496 1062 0 0 100 0 0 2 0 212 21250256 272916 320576 0 0 0 0 751 1107 0 0 99 0 0 0 0 212 21250404 272916 320568 0 0 0 0 496 1056 0 0 100 0 0 0 0 212 212504 04 272916 320568 0 0 0 0 593 1089 0 0 100 0 0 0 0 212 21250404 272916 320568 0 0 0 56 539 1074 0 0 100 0 0 0 0 212 21250404 272916 320568 0 0 0 0 589 1137 0 0 100 0 0 0 0 212 2 1250404 272916 320564 0 0 0 0 608 1154 0 0 100 0 0 0 0 212 21250404 272916 320564 0 0 0 0 601 1156 0 0 100 0 0 1 0 212 21250404 272916 320564 0 0 0 0 611 1155 0 0 100 0 0 0 0 212 21250404 272916 3205 64 0 0 0 0 599 1151 0 0 100 0 0
Indicator description
Indicators |
Description |
Procs.r |
Number of processes waiting for CPU time slices |
procs.b |
The number of processes waiting for a resource, such as waiting for IO or memory exchange |
Cpu.us |
User state CPU Time percentage |
Cpu.sy |
Percentage of CPU time in kernel state, reference value us+sy<=80% |
Cpu.wa |
Percentage of CPU time consumed by IO waits, reference value wa<=30 |
Cpu.id |
Idle state CPU Time percentage |
Cpu.st |
Virtual Machine CPU Elapsed time percentage, the presence of a super-tick may be higher |
System.in |
Number of device interrupts |
System.cs |
Number of context Switches |
Memory.swpd |
Amount of memory in memory swap area (KB) |
Memory.free |
Amount of memory in free pages (KB) |
Memory.buff |
The amount of memory in buffer cache, typically used in block device reading and writing |
Memory.cache |
The amount of memory as the page cache, generally as the file system cache |
Swap.si |
The number of memory swap areas entered by memory |
Swap.so |
Amount of memory entered by memory swap area |
Io.bi |
The amount of data read from the Block device (KB/S) |
Io.bo |
Amount of data written to the Block device (KB/S) |
Reference documents
About Linux OOM Killer
Free to view available memory
Network connection
The netstat command is used to display various network related information, such as network connection, routing table, interface status, etc.
Output sample
[email protected]:~> netstat -ano |headActive Internet connections (servers and established)Proto Recv-Q Send-Q Local Address Foreign Address State tcp 0 0 0.0.0.0:4444 0.0.0.0:* LISTEN tcp 0 0 0.0.0.0:19999 0.0.0.0:* LISTEN tcp 0 0 0.0.0.0:27017 0.0.0.0:* LISTEN tcp 0 0 0.0.0.0:6380 0.0.0.0:* LISTEN tcp 0 0 0.0.0.0:111 0.0.0.0:* LISTEN
Common usage
- Check out the number of time_wait
netstat -nat| awk '{print awk $NF}'|sort|uniq -c|sort -n
- Check process and number of sockets
netstat -nap| awk '{print awk $NF}'|sort|uniq -c|sort -n
- To view the number of port connections
netstat -anp |grep 27071|wc -l
Disk usage
Iostat for monitoring disk IO throughput and Resource usage
Command
iostat -d -x -k 1 10
-D to view disk health,-K for the output of more extended fields in kilobytes
Device: rrqm/s wrqm/s r/s w/s rkB/s wkB/s avgrq-sz avgqu-sz await svctm %utilxvda 0.01 8.32 0.49 4.94 6.82 53.08 22.06 0.11 19.68 1.76 0.95xvde 0.02 841.11 33.24 304.80 331.20 4583.79 29.08 1.67 4.93 1.15 38.76dm-0 0.00 0.00 33.26 1145.95 331.20 4583.79 8.34 0.30 2.61 0.33 38.81
Indicator description
Indicators |
Description |
rrqm/s |
The number of read requests per second of merge (the same block has a merge mechanism) |
rrqm/s |
The number of write requests per second of merge (the same block has a merge mechanism) |
r/s |
Number of Read requests submitted per second |
w/s |
Number of write requests committed per second |
Await |
Average response time for IO requests, reference value <=10MS |
rkb/s |
Amount of data read per second |
wkb/s |
Amount of data written per second |
Avgrq-sz |
IO average request size (sector) |
Avgqu-sz |
IO Average request Queue Size |
Svctm |
Average IO Request Execution time |
%util |
Percent CPU consumption |
Process monitoring
The PS command can give a snapshot of the process in the current system
ps -ef |grep nscl
- Sort by CPU occupancy view the top 10 processes
-> ps -aux --sort -pcpu |head -n 10USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMANDiotweb 9313 1.0 14.3 6277488 4708752 ? Sl Nov03 687:08 ./bin/mongod -f conf/mongodb.conf --authiotweb 16957 0.3 10.1 10208428 3322260 ? Sl Nov01 251:37 java -jar redis-stat-0.4.14.jar 127.0.0.1:6380 3 --server=63800root 2822 0.1 0.0 133196 1288 ? Sl Jun07 431:05 /usr/bin/vm-agentiotweb 16942 0.1 0.0 52364 13304 ? Ssl Nov01 95:57 ./bin/redis-server 0.0.0.0:6380
Indicator description
Indicators |
Description |
USER |
Process Owner |
Pid |
Process ID |
%cpu |
CPU Usage |
%MEM |
Memory usage |
Tty |
Terminal |
STAT |
Process status |
VSZ |
Virtual memory consumption |
Rss |
Physical memory consumption |
START |
Start date |
Time |
Startup duration |
COMMAND |
Bash command |
Reference documents
File occupancy
Lsof (list open files) is used to list the current system open file handles, including network sockets, device handles, and so on.
Output sample
~->lsof /optCOMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAMEmongod 9313 iotweb txt REG 252,0 36409888 761870 /opt/local/mongodb/bin/mongodmongod 9313 iotweb 4w REG 252,0 7082638 778244 /opt/local/mongodb/log/mongodb.logmongod 9313 iotweb 8uW REG 252,0 5 778246 /opt/local/mongodb/data/mongod.lockmongod 9313 iotweb 9uw REG 252,0 21 778247 /opt/local/mongodb/data/WiredTiger.lockmongod 9313 iotweb 15u REG 252,0 36864 778253 /opt/local/mongodb/data/sizeStorer.wt
Indicator description
Indicators |
Description |
COMMAND |
The name of the process |
Pid |
Process ID |
USER |
Process Owner |
Fd |
File descriptor |
TYPE |
File types, such as Dir, Reg, etc. |
DEVICE |
Specify the name of the disk |
SIZE |
Size of File |
NODE |
File Index node |
NAME |
The name of the file |
Reference documents
Common commands for Linux system monitoring