16 Linux Server monitoring commands you should know

In different Linux distributions, there will be different GUI programs to display various system information, such as the SUSE Linux distribution, there is a very good graphical configuration and management Tools YAST,KDE desktop environment in the KDE system guard is also very good.

AD:

In different Linux distributions, there will be different GUI programs to display various system information, such as the SUSE Linux distribution, there is a very good graphical configuration and management Tools YAST,KDE desktop environment in the KDE system guard is also very good.

However, for a Linux system administrator, unless forced to do so, the Linux server will not run such GUI programs, mainly because the GUI will occupy a lot of system resources. So, using GUI tools is usually a simple troubleshooting, if you really want to know what really happened in the system, then please turn off your GUI, quickly into the world of Linux command line.

If you want to pursue the best performance, the Linux server should run at RunLevel 3, which is support for network and multiuser functionality, but does not support GUI functionality. If you really need a GUI, you can enter startx into the GUI at the command line.

If your Linux system is entering the GUI by default, you can adjust the configuration to let him enter RunLevel 3 by default. The specific method is:

• 1 open a terminal, Su to root account
• 2 Use your favorite editor (vi/emacs/... ) to open/etc/inittab file
• 3 Find the Initdefault keyword, change "id:5:initdefault:" to "Id:3:initdefault:"

If there is no/etc/inittab file in the system at all, it does not matter, directly create this file, and add a new line "Id:3". This way, if you restart the server again, it will default to the command line state. Of course, if you only want to temporarily enter the command line state, then just enter "Init 3" in the terminal.

Now that our command line is ready, you can start to see "What's going on in the server" with powerful commands:

1. [01-iostat]
2. [02/03-meminfo/free]
3. [04-mpstat]
4. [05-netstat]
5. [06-nmon]
6. [07-pmap]
7. [08/09-ps/pstree]
8. [10-sar]
9. [11-strace]
10. [12-tcpdump]
11. [13-top]
12. [14-uptime]
13. [15-vmstat]
14. [16-wireshark]

[01-iostat]

The Iostat command displays the detail status of your storage system. You can usually use this command to detect if your storage device is working properly,

This command can be used to detect problems with the system IO before the user complains that the server is slow.

As you can see, Iostat can display both CPU usage and the IO status of each disk.

1. # iostat 1
2. Linux 2.6.32-220.4.1.el6.i686 (roclinux) December 22, 2012 _i686_ (4 CPU)
4. AVG-CPU:%user%nice%system%iowait%steal%idle
5. 0.55 0.00 0.03 0.02 0.00 99.40
7. Device:tps blk_read/s blk_wrtn/s Blk_read Blk_wrtn
8. SDB 0.41 2.61 5.76 2558664 5653872
9. SDA 0.24 0.80 4.12 784650 4038344

[02/03-meminfo/free]

The Meminfo provides a very detailed memory usage status. You can view it directly with the cat command:

1. Cat/proc/meminfo

Of course meminfo contains too much detail, you can use the free command directly to see a summary of the memory.

1. # free-m
2. Total used free shared buffers Cached
3. mem:1513 1429 83 0 343 836
4. -/+ buffers/cache:249 1263
5. swap:0 0 0

[04-mpstat]

Mpstat is used on multiprocessor servers to display the status of each CPU.

In addition, the Mpstat shows the average status of all processors.

You can set the CPU statistics that are displayed for each server, or CPU statistics for each processing.

2. # mpstat-p All

4. Linux 2.6.32-220.4.1.el6.i686 (roclinux) December 22, 2012 _i686_ (4 CPU)

8. 17:46 35 sec CPU%usr%nice%sys%iowait%irq%soft%steal%guest%idle

10. 17:46 35 sec All 0.55 0.00 0.03 0.02 0.00 0.00 0.00 0.00 99.40

12. 17:46 35 sec 0 0.84 0.00 0.04 0.03 0.00 0.01 0.00 0.00 99.08

14. 17:46 35 sec 1 0.51 0.00 0.03 0.02 0.00 0.00 0.00 0.00 99.44

16. 17:46 35 sec 2 0.45 0.00 0.02 0.01 0.00 0.00 0.00 0.00 99.51

18. 17:46 35 sec 3 0.40 0.00 0.02 0.01 0.00 0.00 0.00 0.00 99.56

22. # Mpstat-p 0

24. Linux 2.6.32-220.4.1.el6.i686 (roclinux) December 22, 2012 _i686_ (4 CPU)

28. 17:46 39 sec CPU%usr%nice%sys%iowait%irq%soft%steal%guest%idle

30. 17:46 39 sec 0 0.84 0.00 0.04 0.03 0.00 0.01 0.00 0.00 99.08

The meanings of each of these fields are summarized as follows:

• 1 CPU: Processor number, if all, this line represents the statistical average of all processors
• 2%usr: User-State CPU utilization
• 3%nice: User-state CPU utilization with scheduling priority
• 4%sys: Kernel-State CPU utilization (this value does not include the time to respond to hardware interrupts and software outages)
• 5%iowait: Percentage of time that processing IO requests causes the CPU to be idle
• 6%irq:cpu response time ratio of hardware interrupts
• 7%soft:cpu response time ratio of software interrupts
• 8%steal: The passive wait time ratio of virtual CPUs when the virtual machine monitor is serving to other virtualized processors
• 9%guest: CPU time ratio consumed by running a virtual processor

[05-netstat]

The netstat command is a command that Linux administrators use almost every day (it has gradually been replaced by the SS command), and he can display a lot of information about the network, such as socket usage, routing, NIC status, protocol, network traffic statistics, and so on.

Some of the commonly used netstat options include:

• -A: Show all Socke information
• -R: Show routing Information
• -I: Display network card excuse statistics
• -S: Show Network protocol statistics

[06-nmon]

Nmon is the abbreviation for Nigel's Monitor, which is a well-known tool for monitoring the performance of Linux systems.

Nmon can see processor utilization, memory usage, run queue information, disk IO statistics, network IO Statistics, page break statistics, and more.

You can see the above information through a curses-based class GUI interface.

Nmon Monitoring Tools

[07-pmap]

The PMAP command displays the amount of memory that the process consumes.

You can find the process that takes up the most amount of memory through PMAP.

The following is the memory consumed by the NIGNX master process:

2. # Pmap 2395|head-n 10

4. 2395:nginx:master process./sbin/nginx

6. 00110000 240K r-x--/lib/libgssapi_krb5.so.2.2

8. 0014c000 4K-----/lib/libgssapi_krb5.so.2.2

10. 0014d000 4K R----/lib/libgssapi_krb5.so.2.2

12. 0014e000 4K RW---/lib/libgssapi_krb5.so.2.2

14. 0014f000 12K r-x--/lib/libcom_err.so.2.1

16. 00152000 4K R----/lib/libcom_err.so.2.1

18. 00153000 4K RW---/lib/libcom_err.so.2.1

20. 00154000 48K r-x--/lib/libnss_files-2.12.so

22. 00160000 4K R----/lib/libnss_files-2.12.so

24. ...

26. b78e4000 20K rw---[anon]

28. b78f3000 4K rw-s-/dev/zero (Deleted)

30. b78f4000 4K rw---[anon]

32. bfeaa000 84K rw---[Stack]

34. Total 7280K

[08/09-ps/pstree]

PS and Pstree are a good pair of brothers in a Linux system, and they are all used to list the processes that are in the running state.

PS tells us the amount of memory used by each process and the CPU time consumed.

Pstree will tell us about the parent-child relationship between processes, and here are some of MySQL's parent-child relationship information:

2. # pstree-p 1829

4. Mysqld_safe (1829) ───mysqld (2307) ─┬─{mysqld} (2309)

6. ├─{MYSQLD} (2310)

8. ├─{MYSQLD} (2311)

10. ├─{MYSQLD} (2312)

12. ├─{MYSQLD} (2313)

14. ├─{MYSQLD} (2314)

16. ├─{MYSQLD} (2315)

18. ├─{MYSQLD} (2316)

20. ├─{MYSQLD} (2317)

22. ├─{MYSQLD} (2318)

24. ├─{MYSQLD} (2320)

26. ├─{MYSQLD} (2321)

28. ├─{MYSQLD} (2322)

30. ├─{MYSQLD} (2323)

32. ├─{MYSQLD} (2325)

34. ├─{MYSQLD} (2544)

36. ├─{MYSQLD} (2548)

38. ├─{MYSQLD} (7912)

40. ├─{MYSQLD} (7914)

42. ├─{MYSQLD} (7916)

44. ├─{MYSQLD} (24689)

46. ├─{MYSQLD} (27329)

48. └─{MYSQLD} (27331)

[10-sar]

The SAR command is a Swiss Army knife in the system monitoring tool.

The SAR command is actually comprised of three programs, the SAR (for displaying data), SA1 (for collecting data), and SA2 (for storing data).

The SAR can cover CPU utilization information, memory paging information, network IO transmission information, process creation behavior, and storage device behavior.

The biggest difference between SAR and Nmon is that the SAR is more suitable for long-term system monitoring, while Nmon is more suitable for quick viewing of information.

If you want to learn more about SAR commands, you can read the "SAR interview"-linux Command five-minute series 29.

[11-strace]

Starce is often used as a tool for tracing procedural problems, but his function is far from being so.

It can parse and record the system call behavior of the process, which makes strace a very useful diagnostic, investigative, and error correcting tool.

For example, you can apply strace to trace the configuration file information that a program needs to load at the start of the startup.

Of course, Strace also has its own flaw, which is that strace can seriously slow down the performance and speed of a survey object (a process).

By the way recommend a very good Strace article: "Strace use of detailed"

In addition, if you use the Mac,strace alternative is truss.

[12-tcpdump]

Tcpdump is a simple, easy-to-use network monitoring tool. Its network protocol analysis capability allows it to see exactly what's going on in the network, and if you want more detailed investigation, consider a more powerful Wireshark tool.

Tcpdump's series of tutorials "here".

[13-top]

The top command can display process information from the system. By default, top displays the processes in the system according to CPU usage from high to low, and refreshes the leaderboard every 5 seconds.

Of course, you can also have top sort processes by the dimensions of PID, process life, CPU time, memory consumption, and so on. (I often use P and M shortcuts, sort by CPU utilization, sort by memory usage)

With the top command, you can quickly discover those processes that are out of control or do not meet expectations.

[14-uptime]

The uptime command allows you to see how long the system has been running, to count the number of users currently logged in, and to display the current server's load status.

1. # uptime
2. 18:35:17 up to Days, 9:30am, 1 user, Load average:0.00, 0.00, 0.00

[15-vmstat]

In most cases, you can use the Vmstat command to see the virtual memory of your system, because Linux typically uses virtual memory for better storage performance.

If your program occupies a large amount of memory, then the system will take the memory page out of action, in order to move the program from memory to the system swap space, that is, the hard disk.

This state is called "thrashing" if the system's memory page swap-out action is more than a critical value. When the system is in a thrashing state, performance can drop sharply.

The Vmstat command can help people find such problems in a timely manner and find the culprit behind the slowing down system.

1. # vmstat 1
2. procs-----------Memory-------------Swap-------io------System-------CPU-----
3. R b swpd free buff cache si so bi bo in CS us sy ID WA St
4. 2 0 0 57484 356864 861332 0 0 0 1 7 3 1 0 99 0 0
5. 0 0 0 57468 356864 861360 0 0 0 0 336 145 6 1 94 0 0
6. 0 0 0 57468 356864 861360 0 0 0 0 43 51 0 0 100 0 0
7. 0 0 0 57468 356864 861360 0 0 0 16 51 62 0 0 100 0 0

[16-wireshark]

Wireshark's predecessor is called Ethereal, we can think Wireshark is the tcpdump command of the master brother, because Wireshark will be more professional, but also have more advanced protocol analysis and statistical capabilities.

Wireshark also has GUI interfaces and shell excuses.

If you are a veteran network administrator, you must have used ethereal. And if you're using wireshark/ethereal, then I recommend you read a very good book by Chris Sander, "Practical Packet analysis."

Thank you!

16 Linux Server monitoring commands you should know