16 Linux Server monitoring commands you should know

Summary: in the process of using a Linux server, there are 16 commands to help you get a complete picture of your server's health. If you can master these commands skillfully, it is not far from being a professional Linux system administrator!

In different Linux distributions, there will be different GUI programs to display various system information, such as the SUSE Linux distribution, there is a very good graphical configuration and management Tools YAST,KDE desktop environment in the KDE system guard is also very good.

However, for a Linux system administrator, unless forced to do so, the Linux server will not run such GUI programs, mainly because the GUI will occupy a lot of system resources. So, using GUI tools is usually a simple troubleshooting, if you really want to know what really happened in the system, then please turn off your GUI, quickly into the world of Linux command line.

If you want to pursue the best performance, the Linux server should run at RunLevel 3, which is support for network and multiuser functionality, but does not support GUI functionality. If you really need a GUI, you can enter startx into the GUI at the command line.

If your Linux system is entering the GUI by default, you can adjust the configuration to let him enter RunLevel 3 by default. The specific method is:

1 open a terminal, Su to root account 2 with your favorite editor (vi/emacs/... Open/etc/inittab file 3 to find the Initdefault keyword, change "id:5:initdefault:" to "Id:3:initdefault:"

If there is no/etc/inittab file in the system at all, it does not matter, directly create this file, and add a new line "Id:3". This way, if you restart the server again, it will default to the command line state. Of course, if you only want to temporarily enter the command line state, then just enter "Init 3" in the terminal.

Now that our command line is ready, you can start to see "What's going on in the server" with powerful commands:

[    iostat       ][02/03-meminfo/free][04    -mpstat       ][05    -netstat      ][06    -Nmon         ][07    -Pmap         ] [08/09-ps/pstree    ] [Ten    -SAR          ][11    -strace       ][12    -tcpdump      ][13    -top          ][14    -uptime       ] [    vmstat       ][16    -Wireshark    ]

[01-iostat]

The Iostat command displays the detail status of your storage system. You can usually use this command to detect if your storage device is working properly,

This command can be used to detect problems with the system IO before the user complains that the server is slow.

As you can see, Iostat can display both CPU usage and the IO status of each disk.

# Iostat 1Linux 2.6.32-220.4.1.el6.i686 (roclinux) December 22, 2012 _i686_ (4 CPU) Avg-cpu:%user%nice%system%iowait%steal %idle0.55 0.00 0.03 0.02 0.00 99.40 Device:tps blk_read/s blk_wrtn/s blk_read blk_wrtnsdb 0.41 2.61 5.76 2558664 5653872s Da 0.24 0.80 4.12 784650 4038344

[02/03-meminfo/free]

The Meminfo provides a very detailed memory usage status. You can view it directly with the cat command:

Cat/proc/meminfo

Of course meminfo contains too much detail, you can use the free command directly to see a summary of the memory.

# Free-mtotal used free shared buffers cachedmem:1513 1429 0 343 836-/+ buffers/cache:249 1263swap:0 0 0

[04-mpstat]

Mpstat is used on multiprocessor servers to display the status of each CPU.

In addition, the Mpstat shows the average status of all processors.

You can set the CPU statistics that are displayed for each server, or CPU statistics for each processing.

# mpstat-p Alllinux 2.6.32-220.4.1.el6.i686 (roclinux) December 22, 2012 _i686_ (4 CPU) 17:46 35 seconds CPU%usr%nice%sys%iowait IRQ%soft%steal%guest%idle17 46 minutes 35 seconds All 0.55 0.00 0.03 0.02 0.00 0.00 0.00 0.00 99.4017 Hours 46 minutes 35 seconds 0 0.84 0.00 0.04 0.03 0. 00 0.01 0.00 0.00 99.0817 Hours 46 minutes 35 seconds 1 0.51 0.00 0.03 0.02 0.00 0.00 0.00 0.00 99.4417 Hours 46 minutes 35 seconds 2 0.45 0.00 0.02 0.01 0.00 0.00 0.00 0.00 99.5117 46 minutes 35 seconds 3 0.40 0.00 0.02 0.01 0.00 0.00 0.00 0.00 99.56 # mpstat-p 0Linux 2.6.32-220.4.1.el6.i686 (rocli Nux) December 22, 2012 _i686_ (4 CPU) 17:46 39 seconds CPU%usr%nice%sys%iowait%irq%soft%steal%guest%idle17 46 minutes 39 seconds 0 0.84 0.00 0.04 0.03 0.00 0.01 0.00 0.00 99.08

The meanings of each of these fields are summarized as follows:

1 CPU: Processor number, if all, this line represents the statistical average of all processors 2%usr: User-state CPU Utilization 3%nice: User-state CPU utilization with scheduling priority 4 SYS: kernel-state CPU utilization (this value does not include the time it takes to respond to hardware interrupts and software interrupts) 5%iowait: Percentage of time that processing IO requests causes the CPU to be idle 6%irq:cpu response hardware interrupt Time ratio 7%SOFT:CPU response software interrupt Time ratio 8 %steal: The virtual CPU's passive wait time ratio of 9%guest when the virtual machine monitor is serving the other virtualized processors: CPU time ratio consumed by running a virtual processor

[05-netstat]

The netstat command is a command that Linux administrators use almost every day (it has gradually been replaced by the SS command), and he can display a lot of information about the network, such as socket usage, routing, NIC status, protocol, network traffic statistics, and so on.

Some of the commonly used netstat options include:

-A: Show all Socke information-R: Show routing Information-I: Display NIC pretext statistics-S: Show Network protocol statistics

[06-nmon]

Nmon is the abbreviation for Nigel's Monitor, which is a well-known tool for monitoring the performance of Linux systems.

Nmon can see processor utilization, memory usage, run queue information, disk IO statistics, network IO Statistics, page break statistics, and more.

You can see the above information through a curses-based class GUI interface.

Nmon Monitoring Tools

[07-pmap]

The PMAP command displays the amount of memory that the process consumes.

You can find the process that takes up the most amount of memory through PMAP.

The following is the memory consumed by the NIGNX master process:

# Pmap 2395|head-n 102395:nginx:master process./sbin/nginx00110000 240K r-x--/lib/libgssapi_krb5.so.2.20014c000 4K-- ---/lib/libgssapi_krb5.so.2.20014d000 4K r----/lib/libgssapi_krb5.so.2.20014e000 4K RW---/lib/libgssapi_ krb5.so.2.20014f000 12K r-x--/lib/libcom_err.so.2.100152000 4K r----/lib/libcom_err.so.2.100153000 4K RW---/lib/ libcom_err.so.2.100154000 48K r-x--/lib/libnss_files-2.12.so00160000 4K r----/lib/libnss_files-2.12.so...b78e4000 20K rw---[anon]b78f3000 4K rw-s-/dev/zero (deleted) b78f4000 4K rw---[anon]bfeaa000 84K rw---[Stack]total 7280K

[08/09-ps/pstree]

PS and Pstree are a good pair of brothers in a Linux system, and they are all used to list the processes that are in the running state.

PS tells us the amount of memory used by each process and the CPU time consumed.

Pstree will tell us about the parent-child relationship between processes, and here are some of MySQL's parent-child relationship information:

# pstree-p 1829mysqld_safe (1829) ───mysqld (2307) ─┬─{mysqld} (2309) ├─{mysqld} (2310)                                 ├─{MYSQLD} (2311) ├─{mysqld} (2312)                                 ├─{MYSQLD} (2313) ├─{mysqld} (2314) ├─{mysqld} (2315)                                 ├─{MYSQLD} (2316) ├─{mysqld} (2317)                                 ├─{MYSQLD} (2318) ├─{mysqld} (2320) ├─{mysqld} (2321)                                 ├─{MYSQLD} (2322) ├─{mysqld} (2323)                                 ├─{MYSQLD} (2325) ├─{mysqld} (2544) ├─{mysqld} (2548)                                 ├─{MYSQLD} (7912) ├─{mysqld} (7914)    ├─{MYSQLD} (7916)                             ├─{MYSQLD} (24689) ├─{mysqld} (27329) └─{MYSQLD} (27331)

[10-sar]

The SAR command is a Swiss Army knife in the system monitoring tool.

The SAR command is actually comprised of three programs, the SAR (for displaying data), SA1 (for collecting data), and SA2 (for storing data).

The SAR can cover CPU utilization information, memory paging information, network IO transmission information, process creation behavior, and storage device behavior.

The biggest difference between SAR and Nmon is that the SAR is more suitable for long-term system monitoring, while Nmon is more suitable for quick viewing of information.

If you want to learn more about SAR commands, you can read the "SAR interview"-linux Command five-minute series 29.

[11-strace]

Starce is often used as a tool for tracing procedural problems, but his function is far from being so.

It can parse and record the system call behavior of the process, which makes strace a very useful diagnostic, investigative, and error correcting tool.

For example, you can apply strace to trace the configuration file information that a program needs to load at the start of the startup.

Of course, Strace also has its own flaw, which is that strace can seriously slow down the performance and speed of a survey object (a process).

By the way recommend a very good Strace article: "Strace use of detailed"

In addition, if you use the Mac,strace alternative is truss.

[12-tcpdump]

Tcpdump is a simple, easy-to-use network monitoring tool. Its network protocol analysis capability allows it to see exactly what's going on in the network, and if you want more detailed investigation, consider a more powerful Wireshark tool.

Tcpdump's series of tutorials "here".

[13-top]

The top command can display process information from the system. By default, top displays the processes in the system according to CPU usage from high to low, and refreshes the leaderboard every 5 seconds.

Of course, you can also have top sort processes by the dimensions of PID, process life, CPU time, memory consumption, and so on. (I often use P and M shortcuts, sort by CPU utilization, sort by memory usage)

With the top command, you can quickly discover those processes that are out of control or do not meet expectations.

[14-uptime]

The uptime command allows you to see how long the system has been running, to count the number of users currently logged in, and to display the current server's load status.

# uptime18:35:17 up to Days, 9:30am, 1 user, Load average:0.00, 0.00, 0.00

[15-vmstat]

In most cases, you can use the Vmstat command to see the virtual memory of your system, because Linux typically uses virtual memory for better storage performance.

If your program occupies a large amount of memory, then the system will take the memory page out of action, in order to move the program from memory to the system swap space, that is, the hard disk.

This state is called "thrashing" if the system's memory page swap-out action is more than a critical value. When the system is in a thrashing state, performance can drop sharply.

The Vmstat command can help people find such problems in a timely manner and find the culprit behind the slowing down system.

# vmstat 1procs-----------memory-------------Swap-------io------System-------CPU-----r B swpd free buff cache si so B I bo in CS us sy ID WA st2 0 0 57484 356864 861332 0 0 0 1 7 3 1 0 99 0 00 0 0 57468 356864 861360 0 0 0 0 336 145 6 1 94 0 00 0 0 57468 356864 861360 0 0 0 0 43 51 0 0 100 0 00 0 0 57468 356864 861360 0 0 0 16 51 62 0 0 100 0 0

[16-wireshark]

Wireshark's predecessor is called Ethereal, we can think Wireshark is the tcpdump command of the master brother, because Wireshark will be more professional, but also have more advanced protocol analysis and statistical capabilities.

Wireshark also has GUI interfaces and shell excuses.

If you are a veteran network administrator, you must have used ethereal. And if you're using wireshark/ethereal, then I recommend you read a very good book by Chris Sander, "Practical Packet analysis."

16 Linux Server monitoring commands you should know