18 Commands & Tools to help you locate Linux performance issues

1.Top
The top command is a performance monitor that displays all the live processes that are running and active in a certain order, and periodically updates the display results. This command shows CPU usage, memory usage, swap memory usage size, cache usage size, buffer usage size, process PID, commands used, and more.
It can also show the memory and CPU usage of the running process. For the performance test engineer, the top command is very useful, I am in another article "Lao Li share: Application performance bottleneck location method and principle", which provides a case to help us perform performance positioning.

2. vmstat– Virtual Memory Statistics
The Linux VmStat command is used to display statistics for virtual memory, kernel threads, disks, system processes, I/O blocks, interrupts, CPU activity, and so on. By default, the Vmstat command is not available under the Linux system, and you need to install a Sysstat package that contains the Vmstat program.
# Vmstat
procs-----------Memory-------------Swap-------io------System-------CPU-----
R B swpd free inact active si so bi bo in CS us sy ID WA St
1 0 0 810420 97380 70628 0 0 115 4 89 79 1 6 90 3 0
3.lsof-to list open files
It is often used to display all open files and processes as a list. Open files include disk files, network sockets, pipelines, devices, and processes. One of the main scenarios for using this command is when you cannot mount a disk and display an error message that you are using or opening a file. With this command, you can easily see which file is in use
# lsof
COMMAND PID USER FD TYPE DEVICE SIZE NODE NAME
Init 1 root cwd DIR 104,2 4096 2/
Init 1 root RTD DIR 104,2 4096 2/
Init 1 root txt REG 104,2 38652 17710339/sbin/init
Init 1 root mem REG 104,2 129900 196453/lib/ld-2.5.so
Init 1 root mem REG 104,2 1693812 196454/lib/libc-2.5.so
Init 1 root mem REG 104,2 20668 196479/lib/libdl-2.5.so
Init 1 root mem REG 104,2 245376 196419/lib/libsepol.so.1
Init 1 root mem REG 104,2 93508 196431/lib/libselinux.so.1
Init 1 root 10u FIFO 0,17 953/dev/initctl

4.tcpdump-Network Packet Analyzer

Tcpdump is one of the most widely used network packet analyzers or packet monitoring programs to capture or filter TCP/IP packets that are received or transmitted on a specified interface on a network. It also has an option to save the captured package to a file for later analysis. In almost all major Linux releases, tcpdump can be used.
# tcpdump-i Eth0tcpdump:verbose output suppressed, use-v OR-VV for full protocol decode
Listening on eth0, Link-type EN10MB (Ethernet), capture size bytes
22:08:59.617628 IP tecmint.com.ssh > 115.113.134.3.static-mumbai.vsnl.net.in.28472:p 2,532,133,365:2,532,133,481 (116) Ack 3561562349 win 9648
22:09:07.653466 IP tecmint.com.ssh > 115.113.134.3.static-mumbai.vsnl.net.in.28472:p 116:232 ($) Ack 1 win 9648
22:08:59.617916 IP 115.113.134.3.static-mumbai.vsnl.net.in.28472 > Tecmint.com.ssh:. ACK-$ win 64347

5.netstat-Network Status statistics

Netstat is a command-line tool for monitoring packet and network interface statistics for incoming and outgoing networks. It is a very useful tool that system administrators can use to monitor network performance, locate and troubleshoot network-related issues.

# Netstat-a | Moreactive Internet connections (servers and established)
Proto recv-q send-q Local address Foreign address state
TCP 0 0 *:mysql *:* LISTEN
TCP 0 0 *:sunrpc *:* LISTEN
TCP 0 0 *:REALM-RUSD *:* LISTEN
TCP 0 0 *:ftp *:* LISTEN
TCP 0 0 Localhost.localdomain:ipp *:* LISTEN
TCP 0 0 localhost.localdomain:smtp *:* LISTEN
TCP 0 0 localhost.localdomain:smtp localhost.localdomain:42709 time_wait
TCP 0 0 localhost.localdomain:smtp localhost.localdomain:42710 time_wait
TCP 0 0 *:http *:* LISTEN
TCP 0 0 *:ssh *:* LISTEN
TCP 0 0 *:https *:* LISTEN
6. Htop–linux Process Monitoring
Htop is a very advanced interactive real-time Linux process monitoring tool. It is very similar to the top command, but it has richer features such as user-friendly management of processes, shortcut keys, vertical and horizontal display of processes, and so on. Htop is a third-party tool that is not included in a Linux system and you need to use the Yum Package management tool to install it

7.iotop-Monitoring Linux disk I/O

The Iotop command is also very similar to the top command and the Htop program, but it has the ability to monitor and display real-time disk I/O and process statistics. This tool is useful when you are looking for specific processes and lots of disk read and write processes.
8.iostat-Input/Output statistics

Iostat is a simple tool for collecting statistics on the input and output status of display system storage devices. This tool is often used to track performance issues with storage devices, including devices, local disks, and remote disks, such as using NFS.
# Iostat
Linux 2.6.18-238.9.1.el5 (tecmint.com) 09/13/2012

AVG-CPU:%user%nice%system%iowait%steal%idle
2.60 3.65 1.04 4.29 0.00 88.42

Device:tps blk_read/s blk_wrtn/s Blk_read Blk_wrtn
cciss/c0d0 17.79 545.80 256.52 855159769 401914750
CCISS/C0D0P1 0.00 0.00 0.00 5459 3518
CCISS/C0D0P2 16.45 533.97 245.18 836631746 384153384
CCISS/C0D0P3 0.63 5.58 3.97 8737650 6215544
CCISS/C0D0P4 0.00 0.00 0.00 8 0
CCISS/C0D0P5 0.63 3.79 5.03 5936778 7882528
CCISS/C0D0P6 0.08 2.46 2.34 3847771 3659776

9.iptraf-real-time LAN IP monitoring

Iptraf is an open source real-time network (LAN) monitoring application running on the Linux console. It collects a lot of information, such as IP traffic monitoring over the network, including TCP tokens, ICMP details, TCP/UDP traffic separation, TCP connection packets, and bytes. It also collects common information and details about the state of the interface: TCP, UDP, IP, ICMP, non-IP,IP checksum error, interface activity, and so on.

PSACCT or Acct-Monitor user Activity
PSACCT or Acct tools are used to monitor the activity status of each user in the system. These two service processes run in the background, and they are closely monitored for all activities of each user running on the system, while also monitoring the resources used by these activities.
The system administrator can use these two tools to track each user's activity, such as what the user is doing, how many resources they are using, how much time they have spent on the system, and so on.

11.monit-linux process and service monitoring tools

Monit is a free open source software and a web-based process monitoring tool. It can automatically monitor and manage system processes, programs, files, folders, permissions, sum verification codes and file systems. This software can monitor services like Apache, MySQL, Mail, FTP, ProFTP, Nginx, SSH, and so on. You can use the command line or the network excuse provided by this software to view the status of the system.
12.nethogs-monitoring the network bandwidth used by each process

Nethogs is a very small program of Open source code (similar to the top command under Linux), which closely monitors the network activity of each process on the system. It also tracks the real-time network bandwidth used by each program or application.

13.iftop-Monitoring network bandwidth

Iftop is another open source system monitoring application running in the console that shows a list of application network bandwidth usage (source host or destination host) on the system over the network interface, which is updated periodically. Iftop is used to monitor network usage, while ' top ' is used to monitor CPU usage. Iftop is a member of the ' top ' tool family that monitors the selected interface and displays the current network bandwidth usage among the two hosts.

monitorix-System and Network monitoring
Monitorix is a free lightweight application tool designed to run and monitor Linux/unix server systems and resources. It has an HTTP Web server that has a regular collection of information about the system and the network and displays it graphically. It monitors the average load and usage of the system, memory allocations, disk health, system services, network ports, message statistics (SENDMAIL,POSTFIX,DOVECOT, etc.), MySQL statistics, and so on. It is used to monitor the overall performance of the system to help detect errors, bottlenecks, and abnormal activity.
arpwatch– Ethernet Activity Monitor
The Arpwatch is designed to monitor Ethernet address resolution (changes in Mac and IP addresses) on Linux. He continuously monitors Ethernet activity for a period of time and outputs a log of changes in IP and MAC address pairs. It can also send an email notification to an administrator, alerting you to an increase in address pairing. This is useful for detecting ARP attacks on a network.

suricata– Network Security Monitoring

Suricata is an open source high-performance network security, intrusion detection and anti-monitoring tool that can run on Linux, FreeBSD, and Windows. Nonprofit organization OISF (Open Information Security Foundation) develops and owns its copyrights.

Vnstat php– Network traffic monitoring
Vnstat PHP is a web-based front-end rendering of popular web Tools "Vnstat". Vnstat PHP presents the network usage in a beautiful graphical interface. He can display upload and download traffic in hours, days, months, and output summary reports.

nagios– Network/server monitoring

Nagios is a leading and powerful open source monitoring system that allows network/system administrators to identify and resolve problems before they affect normal business. With the Nagios system, administrators can remotely detect Linux, Windows, switches, routers, and printers within a single window. It can warn of danger and indicate whether there is an exception to the system/server, which can indirectly help you to take rescue measures before the problem occurs.

18 Commands & Tools to help you locate Linux performance issues