Linux process and job management knowledge (top)

This chapter introduces some basic Linux process management basics and some common tools

The concept of Linux process, this must be distinguished from the program:

Process: A copy of a running program that is a collection of instructions that is loaded into memory. Process Id,pid numbers are used to mark each process UID, GID, and SELinux context to determine access and access to the file system, which is typically inherited from the user executing the process, and the process has a life cycle

CentOS Linux The first process is usually init or SYSTEMD, which is the parent process for all processes, with an ID of 1
Other processes are created by the parent process.
Process Type:
Daemon: daemon, process initiated during system boot, and terminal-independent process
Foreground process: terminal-related, process initiated through the terminal note: The two can be converted to each other
Process Status:
Operating state: Running
Ready state: Readiness
Sleep state:
Interruptible: interruptable
Non-disruptive: uninterruptable
Stop state: Stopped, paused in memory, but not dispatched unless manually started
Zombie State: Zombie, end process, child process does not close before parent process ends

Linux system Status viewing and management tools: Pstree, PS, Pidof, Pgrep, Top, htop, glance, Pmap, Vmstat, Dstat, Kill, Pkill, job, BG, FG, Nohup
Here are a few useful tools to introduce.

Pstree shows the left and right processes in a tree-shaped structure, especially when it shows the parent-child relationship between processes.
-P simultaneously displays the process number

[[email protected] 13:48:49 ~] #pstree-psystemd (1) ─┬─modemmanager (676) ─┬─{modemmanager} (696) │ └─{modemmanager} (711) ├─networkmanager (666) ─┬─{networkmanager} (723) │└─{net                 WorkManager} (728) ├─vgauthservice (648) ├─abrt-dbus (1762) ─┬─{abrt-dbus} (1763) │ └─{abrt-dbus} (1765) ├─abrt-watch-log (657) ├─abrt-watch-log (673) ├─ABRTD (656) ├─           Agetty (1128) ├─alsactl (677) ├─atd (1013) ├─AUDITD (623) ─┬─audispd (625) ─┬─sedispatch (627) ││└─{AUDISPD} (628) │└─{AUDITD} (624) ├─avahi-daemon (667)-- ─avahi-daemon (697) ├─BLUETOOTHD (653) ├─crond (1015) ├─CUPSD (1002) ├─dbus-daemon (649 ) ├─gssproxy (678) ─┬─{gssproxy} (689) │├─{gssproxy} (690) │├─{GSS Proxy} (691) │              ├─{gssproxy} (692) │└─{gssproxy} (693) ├─httpd (1007) ─┬─httpd (1117)             │├─HTTPD (1118) │├─httpd (1119) │├─httpd (1120) │           └─HTTPD (1121) ├─irqbalance (675) ├─ksmtuned (705) ───sleep (1811) ├─LSMD (650)           ├─lvmetad (420) ├─master (1114) ─┬─pickup (1115) │└─qmgr (1116) ├─mcelog (722)           ├─POLKITD (654) ─┬─{POLKITD} (719) │├─{POLKITD} (720) │├─{POLKITD} (721) │├─{POLKITD} (726) │└─{POLKITD} (730) ├─rngd (665) ├─RSYSL OGD (671) ─┬─{rsyslogd} (695) │└─{rsyslogd} (703) ├─SMARTD (662) ├─sshd (999) ─┬─ssh           D (1133) ───bash (1139) ─┬─glances (1239) ││└─pstree (1813) │ └─sshd (1742) ───bash (1744) ───man (1786) ───less (1797) ├─systemd-journal (397) ├─systemd-logind (664) ├─SYSTEMD-UDEVD (424)           ├─tuned (998) ─┬─{tuned} (1059) │├─{tuned} (1060) │├─{tuned} (1061) │└─{tuned} (1062) └─VMTOOLSD (669)

PS command
PS [Options]
It supports three styles of options:
? UNIX options such as-a-e
BSD options such as a
GNU options such as--help

BSD Options:
Default display of processes in the current terminal
? A option includes all processes in the terminal
? The x option includes a process that does not link the terminal
? The U option displays information about the process owner
? The F option shows the process tree, which is equivalent to--forest
? K|--sort Property Sort property, pre-attribute plus-Indicates reverse order
? O Properties ... Options display customized information pid, CMD,%cpu,%mem
? L display a list of supported properties
Common combinations:
PS Axu Show left and right processes

PS K Property O property, Properties ...
For example, the following is sorted according to the size of the memory footprint, and for example the properties specified in all terminal programs

[[email protected] 13:59:52 ~] #ps k-%mem o user,tty,time,%mem,%cpu,cmduser TT time%mem%cpu cmdroot pts/0 00:00:01 1.2 0.0/usr/bin/python/usr/bin/glances-s-B 192.168.30.104root pts/0 00:00:00 0.3 0.0- Bashroot pts/1 00:00:00 0.2 0.0-bashroot pts/1 00:00:00 0.1 0.0 man pstreeroot pts/0 00:00:00 0 .1 0.0 ps k-%mem o user,tty,time,%mem,%cpu,cmdroot pts/1 00:00:00 0.0 0.0 less-sroot tty1 00:00:00 0.     0 0.0/sbin/agetty--noclear tty1 linux[[email protected] 13:59:57 ~] #ps k%mem o user,tty,time,%mem,%cpu,cmduser TT time%MEM%cpu cmdroot tty1 00:00:00 0.0 0.0/sbin/agetty--noclear tty1 linuxroot PTS/1 0 0:00:00 0.0 0.0 less-sroot pts/0 00:00:00 0.1 0.0 ps k%mem o user,tty,time,%mem,%cpu,cmdroot pts/1 00: 00:00 0.1 0.0 man pstreeroot pts/1 00:00:00 0.2 0.0-bashroot pts/0 00:00:00 0.3 0.0-bashroot pts /0 00:00:01 1.2 0.0 /usr/bin/python/usr/bin/glances-s-B 192.168.30.104 

Some common options are the default:
?-c cmdlist Specifies commands, which are separated by multiple commands?
-L Show threads?
-e: Show all processes, equivalent to-a?
-F: Show full format program information?
-F: Show process information in a more complete format?
-H: Show process-related information in process-level format?
-u userlist Specify a valid user ID or name?
-U userlist Specify a real user ID or name?
-G gid or groupname specify a valid GID or group name?
-G gid or GroupName specify the true GID or group name?
-P PID display refers to the PID process?--ppid pid display belongs to the PID sub-process?
-M displays selinux information, equivalent to Z
The columns in the PS output message indicate the meaning:
? Vsz:virtual memory SiZe, virtual memory set, linear RAM?
Rss:resident Size, resident memory set?
STAT: Process Status
R:running
S:interruptable sleeping
D:uninterruptable sleeping
t:stopped
Z:zombie
+: Foreground process
L: Multithreaded Process
L: Memory Paging with lock
N: Low-priority process
<: high-priority process
S:session leader, session (child process) initiator

Some common examples of PS are:
? Query all the processes you have:
Ps-x
? The process that displays the specified user name (RUID) or user id:
Ps-fu Apache Ps-fu 48
? The process that displays the specified user name (EUID) or user id:
Ps-fu Wang Ps-fu 1000
? view each process running with root user rights (actual and valid ID):
Ps-u Root-u Root
List all processes owned by a group (actual group Id:rgid or name):
PS-FG Nginx
? list all processes owned by a valid group name (or session): PS-FG MySQL PS-FG 27
? The process ID to display the owning process:
PS-FP 1234
? Displays all processes under the parent process ID, such as all processes that have a parent process of 1154:
Ps-f--ppid 1234
? Displays multiple processes for the specified PID:
PS-FP 1204,1239,1263
? To display the owning process by TTY:
Ps-ft pst/0
To display all threads for a process, the LWP (lightweight process) and the NLWP (lightweight process count) columns are displayed:
Ps-fl-c Nginx
To view the Pid,ppid, user name, and command for a process:
Ps-eo Pid,ppid,user,cmd
To view the Pid,ppid, user name, and command for a process:
Ps-eo Pid,ppid,user,cmd
Use its PID to find the process name:
Ps-p 1244-o comm=
Finds all the PID of the specified process name, which is useful when writing a script that needs to read the PID from the STD output or file:
Ps-c Httpd,sshd-o pid=
? Check the execution time of a process
Ps-eo Comm,etime,user | grep nginx
To find the process that consumes the most memory and CPU:
Ps-eo PID,PPID,CMD,%MEM,%CPU--sort=-%mem | Head
Ps-eo PID,PPID,CMD,%MEM,%CPU--sort=-%cpu | Head
You can also use watch for real-time viewing of processes with the most memory and CPU consumption
Watch-n 1 ' Ps-eo pid,ppid,cmd,%mem,%cpu--sort=-%mem | Head

[[email protected] 15:26:43 ~] #pidof httpd2828 2827 2826 2825 2824 1007[[email protected] 15:48:56 ~] #ps Axu |        grep httpdroot 1007 0.0 0.4 226240 5176?        Ss 13:02 0:00/usr/sbin/httpd-dforegroundapache 2824 0.0 0.2 228324 3148?        S 14:48 0:00/usr/sbin/httpd-dforegroundapache 2825 0.0 0.2 228324 3148?        S 14:48 0:00/usr/sbin/httpd-dforegroundapache 2826 0.0 0.2 228324 3148?        S 14:48 0:00/usr/sbin/httpd-dforegroundapache 2827 0.0 0.2 228324 3148?        S 14:48 0:00/usr/sbin/httpd-dforegroundapache 2828 0.0 0.2 228324 3148? S 14:48 0:00/usr/sbin/httpd-dforegroundroot 3776 0.0 0.0 112664 972 pts/0 s+ 15:49 0:00 grep--col        Or=auto httpd[[email protected] 15:49:12 ~] #ps-C httpd PID TTY time CMD 1007?        00:00:00 httpd 2824?        00:00:00 httpd 2825?        00:00:00 httpd 2826?        00:00:00 httpd 2827? 00:00: xx httpd 2828? 00:00:00 httpd[[email protected] 15:49:21 ~] #pidof httpd |        Xargs ps-fp UID PID PPID C stime TTY STAT time cmdroot 1007 1 0 13:02?        Ss 0:00/usr/sbin/httpd-dforegroundapache 2824 1007 0 14:48?        S 0:00/usr/sbin/httpd-dforegroundapache 2825 1007 0 14:48?        S 0:00/usr/sbin/httpd-dforegroundapache 2826 1007 0 14:48?        S 0:00/usr/sbin/httpd-dforegroundapache 2827 1007 0 14:48?        S 0:00/usr/sbin/httpd-dforegroundapache 2828 1007 0 14:48? S 0:00/usr/sbin/httpd-dforeground

Sometimes we only remember a field name of a program, but we need to see its process running, and we can use pgrep to view it, which supports regular matching
pgrep [Options] Pattern
For example:

[[email protected] 14:53:41 ~]#pgrep -a .*glan.*1239 /usr/bin/python /usr/bin/glances -s -B 192.168.30.104

If you know the exact program name, you can use PIDOF to view its process:

[[email protected] 14:59:05 ~]#pidof -x glances | xargs ps -fpUID         PID   PPID  C STIME TTY          TIME CMDroot       1239   1139  0 13:06 pts/0    00:00:02 /usr/bin/python /usr/bin/glances -s -B 192.168.30.104

Pidof is best used when using-X to identify the pin name

Top command real-time monitoring of process status, system CPU memory and other information
Top has many built-in commands:
Sort:
P: As percentage of CPU occupied,%cpu
M: Occupy memory percentage,%MEM
T: Cumulative CPU Duration, time+
The first message shows:
Uptime information: l command
Tasks and CPU information: T command
CPU Display: 1 (digital)
Memory Information: M command
Exit Command: Q
Modify Refresh time interval: s
Terminates the specified process: K
Save File: W

There is also a class top style tool htop, which needs to be used Epel source installation, which provides a real-time and interactive monitoring interface:

Another class top tool glances tool also requires epel source installation, which not only provides top functionality, but also real-time view of memory, disk, network I/O, and support C/s mode, allowing clients to remotely monitor server resource status, is a comprehensive tool:
? Glances command: Epel source installation?
glances [-BDEHMNRSVYZ1] [-B bind] [-C Server] [-C conffile] [-P port] [-p password] [--password] [t refresh] [-f file] [-O Output
? Built-in commands:
A Sort processes automatically
L Show/hide Logs
C Sort processes by CPU%
b Bytes or bits for network I/O
M Sort processes by mem%
W Delete Warning logs
P Sort Processes by name
X Delete warning and critical logs
I Sort processes by I/O rate
1 Global CPU or PER-CPU stats
d show/hide disk I/O stats
H show/hide
F show/hide File System stats
T View network I/O as combination
N Show/hide Network Stats
U View Cumulative network I/O
S show/hide Sensors stats
Q Quit (ESC and ctrl-c also work)
Y show/hide hddtemp Stats
Glances can be added with options at startup:
Common options:
-B: Display nic data rate in bytes
-D: Turn off the disk I/O module
-f/path/to/somefile: Setting the input file location
-O {html| CSV}: Output format
-M: Disable Mount Module
-N: Disable network module
-T #: Delay time interval
-1: Each CPU related data is displayed separately
, the XXX fluorescence section has our commonly used items that need to be monitored for system resources:

Other tools for memory management:
Free command FREE [OPTION]
-B in bytes
-M in megabytes
-G in GB
-H easy-to-read format
-O does not display-/+buffers/cache lines
-T shows the sum of RAM + swap
-s n flush interval n seconds
-c n is exited after Flushing N times
For example, the CENTOS7 is refreshed every 5 seconds in megabytes and refreshed 3 after exiting:

[[email protected] 15:25:19 ~]#free -h -m -s 5 -c 3              total        used        free      shared  buff/cache   availableMem:           1.1G        183M        620M        8.2M        343M        764MSwap:          2.0G          0B        2.0G              total        used        free      shared  buff/cache   availableMem:           1.1G        183M        620M        8.2M        343M        764MSwap:          2.0G          0B        2.0G              total        used        free      shared  buff/cache   availableMem:           1.1G        183M        620M        8.2M        343M        764MSwap:          2.0G          0B        2.0G

For example, in CENTOS6:

[[email protected] 06:17:32 ~]#free -h -m -s 5 -c 3             total       used       free     shared    buffers     cachedMem:          980M       500M       480M       280K        55M       284M-/+ buffers/cache:       159M       820MSwap:         2.0G         0B       2.0G             total       used       free     shared    buffers     cachedMem:          980M       500M       480M       280K        55M       284M-/+ buffers/cache:       160M       820MSwap:         2.0G         0B       2.0G             total       used       free     shared    buffers     cachedMem:          980M       500M       480M       280K        55M       284M-/+ buffers/cache:       160M       820MSwap:         2.0G         0B       2.0G

The meaning of free content in CENTOS6 and before can be understood as follows:
The use of memory is divided into 4 parts:
A. Use of the program;
B. Not assigned;
C. Buffers (buffer cache)
D. Cached (page cache)

Obviously, a (program used) must be used,b (unassigned) must be free. But is C (buffers) and D (Cached) counted as used or free? On the one hand, they have been assigned, can be counted as used, on the other hand, when the program needs, can be recycled to use, can be counted as free. So, it is reasonable to count. This is the difference between the first row and the second line in the output of the free command:
First line (MEM): buffers and cached are counted as used. In other words, its free means B (unassigned); its used refers to a + C + D;
The second line (-/+ Buffers/cache): Buffers and cached are counted as free. That is, its used refers to a (used by the program); its free means B + C + D; The meaning of the line name "-/+ Buffers/cache" is "to subtract buffers and cached from used and add it to free".

In the CENTOS7:
First, C (buffers) and D (Cached) are together, i.e. buff/cache;
Second, used refers to a (program used); free means B (unassigned);
In addition, CentOS 7 added a available, what is it?
That's what the manual says:
Memavailable:an estimate of how much memory are available for starting new applications, without swapping.
As mentioned earlier, when the program needs, can be recycled C (buffers) and D (Cached), then Memavailabe is not b+c+d? When the program needs to be able to reclaim C and D, this sentence was correct before, but now is not accurate: because, now, C and D not all of the memory can be recycled. So, it is generally understood that memavailable = B (unassigned) + C (buffers) + D (Cached)-Non-recyclable parts. What is not recyclable? Shared memory segments, TMPFS,RAMFS, etc.

There is also a memory tool vmstat can also be used to view memory usage
For example, in megabytes, 2 seconds refresh, refresh 5 Then exit

  [[email protected] 06:31:20 ~] #vmstat-S M 2 5procs-----------memory-------------Swap-------io------     System-------CPU-----r B swpd free buff cache si so bi bo in CS us sy ID WA St 1 0 0 479   56 284 0 0 8 1 9 6 0 0 100 0 0 0 0 0 479 56 284 0 0 0 0     45 21 0 0 100 0 0 0 0 0 479 56 284 0 0 0 0 23 22 0 0 100 0 0 0 0 0     479 56 284 0 0 0 0 24 17 0 0 100 0 0 0 0 0 479 56 284 0 0 0 6 0 0 0 0  

PROCS:R: Number of processes that can run (running or waiting to run), and the number of cores
B: Number of processes in non-interruptible sleep state (length of blocked queue)
? Memory:
SWPD: Total amount of swap memory used
Free: Total Idle physical memory
Buffer: The total amount of memory used for buffer
Cache: The total amount of memory used for the cache
? Swap:
SI: Data rate for swapping into memory from disk (KB/S)
So: Data rate from memory swap to disk (KB/S)
Io:
BI: The rate at which data is read from the block device to the system (KB/S)
Bo: The rate at which data is saved to a block device
? System:
In:interrupts interrupt rate, including clock
Cs:context Switch Process Switching rate
? CPU:
Us:time spent running Non-kernel code
Sy:time spent running kernel code
Id:time spent idle. Before Linux 2.5.41, including io-wait time.
Wa:time spent waiting for IO. 2.5.41 before, including in Idle.
St:time stolen from a virtual machine. 2.6.11 ago, unknown.

View disk usage monitoring Iostat
? Iostat: Statistics CPU and device IO information
Example: Iostat 1 10

[[email protected] 06:32:06 ~]#iostatLinux 2.6.32-696.el6.x86_64 (centos6.magedu.com)    05/06/2018  _x86_64_    (4 CPU)avg-cpu:  %user   %nice %system %iowait  %steal   %idle           0.01    0.00    0.08    0.01    0.00   99.89Device:            tps   Blk_read/s   Blk_wrtn/s   Blk_read   Blk_wrtnscd0              0.00         0.03         0.00        384          0sda               1.49        55.91        11.06     691042     136704sdb               0.02         0.13         0.00       1624          0sdc               0.01         0.08         0.00        928          0sdd               0.02         0.19         0.00       2320          0dm-0              0.01         0.06         0.00        768          0

To view process and memory mapping relationships using the PMAP command
PMAP [options] pid [...]
-X: Display detailed format information
Example: Pmap 1
Another implementation: Cat/proc/pid/maps

Process Management In addition to the tools viewed by the above process, an appropriate process manipulation tool is also required

Kill command:
Send control signals to the process for process management, one number per signal, signal name beginning with sig (can be omitted), case insensitive
Show current system available signals: Kill–l,trap-l
Common signal: Man 7 signal, the following numbers are commonly used signal numbers
1) SIGHUP: No need to close the process and let it reread the configuration file
2) SIGINT: Abort the running process, equivalent to CTRL + C
3) Sigquit: equivalent to Ctrl+\
9) SIGKILL: Force kills a running process
SIGTERM: Terminating a running process
Sigcont: Continue Running
SIGSTOP: Background hibernation
How to specify the signal:
(1) Digital identification of the signal: 1, 2, 9
(2) Signal complete name: SIGHUP
(3) abbreviated name of signal: HUP Process Management tool

? press Pid:kill [-signal] pid ...
Kill–n SIGNAL pid;kill–s SIGNAL pid
By name: Killall [-signal] comm ...?
by mode: pkill [options] Pattern
-signal
-U uid:effective user, effective by
-U uid:real user, who actually initiates the run of the command
-T terminal: processes associated with the specified terminal
-L: Show process name (Pgrep available)
-A: Displays the full format of the process name (Pgrep available)
-P PID: Displays the child processes of the specified process

[[email protected] 16:02:55 ~]#w 16:02:56 up  3:01,  3 users,  load average: 0.00, 0.01, 0.05USER     TTY      FROM             [email protected]   IDLE   JCPU   PCPU WHATroot     pts/0    192.168.30.1     13:02    0.00s  4.51s  0.00s wroot     pts/1    192.168.30.1     13:47    1:02m  0.06s  0.06s -bashroot     pts/2    192.168.30.101   16:02   42.00s  0.04s  0.04s -bash[[email protected] 16:02:56 ~]#pkill -9 -t pts/2[[email protected] 16:03:14 ~]#w 16:03:15 up  3:01,  2 users,  load average: 0.00, 0.01, 0.05USER     TTY      FROM             [email protected]   IDLE   JCPU   PCPU WHATroot     pts/0    192.168.30.1     13:02    3.00s  4.51s  0.00s wroot     pts/1    192.168.30.1     13:47    1:03m  0.06s  0.06s -bash[[email protected] 16:03:15 ~]#

In addition, the switching of the process state is as follows:

Management of processes, including job management, continues to be introduced in the next article

Linux process and job management knowledge (top)