Linux System Monitoring and diagnostics tool: Lsof

1, Lsof Introduction

Lsof is a very practical system-level monitoring and diagnostic tool under Linux.

It means List Open Files, and it's easy for you to remember that it's a combination of "ls + of" ~

It can be used to list the file information opened by various processes, remember: Linux under "All Files",

including but not limited to pipes, sockets, directories, devices, etc.

So, with lsof, you can get a variety of information about any file that is opened.

You can generate a lot of information simply by entering lsof, because lsof needs to access core memory and various files, so it must run as root to fully function.

Example output for lsof:

2. [Email Protected]:~/lab 0# lsof
4. COMMAND PID TID USER FD TYPE DEVICE size/off NODE NAME
6. Systemd 1 root cwd DIR 8,6 4096 2/
8. Systemd 1 root RTD DIR 8,6 4096 2/
10. Systemd 1 root txt REG 8,6 2273340 1834909/usr/lib/systemd/systemd
12. Systemd 1 root mem REG 8,6 210473 1700647/lib/libnss_files-2.15.s
14. ...

2. Common usage of lsof

2.1 Monitoring open files, devices

View files, which processes are occupied by the device:

2. # lsof/dev/tty1
4. COMMAND PID USER FD TYPE DEVICE size/off NODE NAME
6. Bash 1770 Jian 0u CHR 4,1 0t0 1045/dev/tty1
8. Bash 1770 Jian 1u CHR 4,1 0t0 1045/dev/tty1
10. Bash 1770 Jian 2u CHR 4,1 0t0 1045/dev/tty1
12. Bash 1770 Jian 255u CHR 4,1 0t0 1045/dev/tty1
14. StartX 1845 Jian 0u CHR 4,1 0t0 1045/dev/tty1
16. StartX 1845 Jian 1u CHR 4,1 0t0 1045/dev/tty1
18. ...

2.2 Monitoring File System

Specify the directory, mount point, and you can see which processes are opening the files under it:

1. # lsof/data/
2. COMMAND PID USER FD TYPE DEVICE size/off NODE NAME
3. Bash 15983 Jian cwd DIR 8,5 4096 8252/data/backup
4. ...

This is useful when umount a file system failure (typically reported to the FS is busy).

List files that are opened under a directory (mount point such as/home):

1. # lsof +d/var/log/
2. COMMAND PID USER FD TYPE DEVICE size/off NODE NAME
3. RSYSLOGD 488 syslog 1w REG 8,1 1151 268940/var/log/syslog
4. RSYSLOGD 488 syslog 2w REG 8,1 2405 269616/var/log/auth.log
5. Console-k 144 root 9w REG 8,1 10871 269369/var/log/consolekit/history

Lists files opened by the specified process name:

2. # lsof-c SSH-C Init
4. COMMAND PID USER FD TYPE DEVICE size/off NODE NAME
6. Init 1 root txt REG 8,1 124704 917562/sbin/init
8. Init 1 root mem REG 8,1 1434180 1442625/lib/i386-linux-gnu/libc-2.13.so
10. Init 1 root mem REG 8,1 30684 1442694/lib/i386-linux-gnu/librt-2.13.so
12. ...
14. Ssh-agent 1528 Lakshmanan 1u CHR 1,3 0t0 4369/dev/null
16. Ssh-agent 1528 Lakshmanan 2u CHR 1,3 0t0 4369/dev/null
18. Ssh-agent 1528 Lakshmanan 3u Unix 0xdf70e240 0t0 10464/tmp/ssh-suymkxxw1495/agent.1495

2.3 Monitoring Process

Specify the process number to view the files opened by the process:

2. # lsof-p 2064
4. COMMAND PID USER FD TYPE DEVICE size/off NODE NAME
6. Firefox 2064 Jian cwd DIR 8,6 4096 1571780/home/jian
8. Firefox 2064 Jian rtd DIR 8,6 4096 2/
10. Firefox 2064 Jian txt REG 8,6 44224 1985670/usr/lib/firefox-12.0/firefox
12. Firefox 2064 Jian Mem REG 8,6 14707012 925361/usr/share/fonts/chinese/msyhbd.ttf
14. Firefox 2064 Jian Mem REG 8,6 15067744 925362/usr/share/fonts/chinese/msyh.ttf
16. Firefox 2064 Jian Mem REG 8,6 16791251 1701681/USR/SHARE/FONTS/WENQUANYI/WQY-ZENHEI.TTC
18. Firefox 2064 Jian Mem REG 0,16 67108904 10203/dev/shm/pulse-shm-3021850167
20. ...

When you want to kill all open files and devices for a user, you can do this:

1. Kill-9 ' lsof-t-u Lakshmanan '

The role of-T here is to list the process ID column separately.

For 4 ways to kill a process, please refer to:

http://www.thegeekstuff.com/2009/12/4-ways-to-kill-a-process-kill-killall-pkill-xkill/

2.4 Monitoring Network

View which processes are in use on the specified port (Lsof-i lists all open Network connections):

1. # lsof-i:22
2. COMMAND PID USER FD TYPE DEVICE size/off NODE NAME
3. sshd 1569 root 3u IPv4 10303 0t0 TCP *:ssh (LISTEN)
4. sshd 1569 Root 4u IPv6 10305 0t0 TCP *:ssh (LISTEN)
5. ...

List all network files that are opened by a process:

1. Lsof-i-a-p 234

Or:

1. Lsof-i-a-c SSH

List all TCP, UDP connections:

1. Lsof-i TCP;
2. Lsof-i UDP;

List all NFS files:

1. Lsof-n-U lakshmanan-a

To see which processes are in use for a specified network port:

2. # lsof [email protected]
4. COMMAND PID USER FD TYPE DEVICE size/off NODE NAME
6. Skype 1909 Jian 54u IPv4 9116 0t0 TCP 192.168.1.91:40640->64.4.23.153:40047 (established)
8. Pidgin 1973 Jian 7u IPv4 6599 0t0 TCP 192.168.1.91:59311->hx-in-f125.1e100.net:https (established)
10. Pidgin 1973 Jian 13u IPv4 9260 0t0 TCP 192.168.1.91:54447->BY2MSG3010511.PHX.GBL:MSNP (established)
12. ...

3, more use of skills

3.1 Monitoring user

To view the files that are open by the specified user (Lsof-u ^lakshmanan can exclude someone):

2. # lsof-u Messagebus
4. COMMAND PID USER FD TYPE DEVICE size/off NODE NAME
6. Dbus-daem 1805 Messagebus cwd DIR 8,6 4096 2/
8. Dbus-daem 1805 Messagebus RTD DIR 8,6 4096 2/
10. Dbus-daem 1805 Messagebus txt REG 8,6 1235361 1834948/usr/bin/dbus-daemon
12. Dbus-daem 1805 Messagebus mem REG 8,6 210473 1700647/lib/libnss_files-2.15.so
14. Dbus-daem 1805 Messagebus mem REG 8,6 190145 1700642/lib/libnss_nis-2.15.so
16. Dbus-daem 1805 Messagebus mem REG 8,6 490366 1700636/lib/libnsl-2.15.so
18. ...

3.2 Monitoring Applications

To view files opened by the specified program:

2. # lsof-c Firefox
4. COMMAND PID USER FD TYPE DEVICE size/off NODE NAME
6. Firefox 2064 Jian cwd DIR 8,6 4096 1571780/home/jian
8. Firefox 2064 Jian rtd DIR 8,6 4096 2/
10. Firefox 2064 Jian txt REG 8,6 44224 1985670/usr/lib/firefox-12.0/firefox
12. Firefox 2064 Jian Mem REG 8,6 14707012 925361/usr/share/fonts/chinese/msyhbd.ttf
14. Firefox 2064 Jian Mem REG 8,6 15067744 925362/usr/share/fonts/chinese/msyh.ttf
16. Firefox 2064 Jian Mem REG 8,6 16791251 1701681/USR/SHARE/FONTS/WENQUANYI/WQY-ZENHEI.TTC
18. ...

4. Command mode tricks

4.1 Combining logical query conditions

Only multiple query conditions are met, with the "-a" parameter, the default is-O.

2. # Lsof-a-C bash-u Root
4. COMMAND PID USER FD TYPE DEVICE size/off NODE NAME
6. Bash 1986 Root CWD DIR 8,6 4096 1701593/root/lab
8. Bash 1986 root RTD DIR 8,6 4096 2/
10. Bash 1986 Root txt REG 8,6 1994157 1700632/bin/bash
12. Bash 1986 root mem REG 8,6 9690800 405214/usr/lib/locale/locale-archive
14. Bash 1986 root mem REG 8,6 210473 1700647/lib/libnss_files-2.15.so

4.2 The recurrence mode of the lsof command:

Repeats the lsof based on the given parameter delay of how many seconds:

• +r says repeat mode will end itself when no file is opened.
• -R means that it will execute regardless of whether the file exists or is opened, until you interrupt it.

The output of each loop is delimited with ' ======= ', and you can also use '-R ' | ' +r ' specifies the delay time.

2. # lsof-u lakshmanan-c init-a -r5
4. =======
6. =======
8. COMMAND PID USER FD TYPE DEVICE size/off NODE NAME
10. inita.sh 2971 Lakshmanan cwd DIR 8,1 4096 393218/home/lakshmanan
12. inita.sh 2971 Lakshmanan RTD DIR 8,1 4096 2/
14. inita.sh 2971 Lakshmanan txt REG 8,1 83848 524315/bin/dash
16. inita.sh 2971 Lakshmanan mem REG 8,1 1434180 1442625/lib/i386-linux-gnu/libc-2.13.so
18. inita.sh 2971 Lakshmanan mem REG 8,1 117960 1442612/lib/i386-linux-gnu/ld-2.13.so
20. inita.sh 2971 Lakshmanan 0u CHR 136,4 0t0 7/DEV/PTS/4
22. inita.sh 2971 Lakshmanan 1u CHR 136,4 0t0 7/DEV/PTS/4
24. inita.sh 2971 Lakshmanan 2u CHR 136,4 0t0 7/DEV/PTS/4
26. inita.sh 2971 Lakshmanan 10r REG 8,1 393578/home/lakshmanan/inita.sh
28. =======

The above output is not output for the first 5 seconds, and then "inita.sh" starts, with output.

5, the last skill

About disk space alarm df-h--max=1 and DU-HX--max=1 display inconsistent issues,

The most common scenario is the following:

1. Lsof|grep-i Delete

Look at the deleted files: some deleted files, but the process is not reload, those space is still occupied, you can understand that the process is similar to Windows, the handle is not released concept bar ~ Just windows if there are files being used by the process, you are generally not deleted, and Linux Although the deletion limit is not done, but wait until the process is finished using the file to completely release, to prevent the process to collapse, this is the operating system of resources management differences IT ~

For example Nginx will have a lot of temporary files occupy the/tmp directory, deleted, still occupy space,

At this point you can:

1. pkill-9 nginx &&/etc/init.d/nginx Restart

Well, this article to the end of this, about lsof there are many, but the elder brother commonly used, know that this, elder brother can only help you here,

If you still need other content, please google it yourself, Sao years ...

6, refer:

To find open files using lsof:

Http://www.ibm.com/developerworks/cn/aix/library/au-lsof.html

Linux lsof Command Examples (Identify Open Files):

http://www.thegeekstuff.com/2012/08/lsof-command-examples/

The lsof of practical system Tools:

http://www.yLinux.org/forum/t/276

Linux System Monitoring and diagnostics tool: Lsof