Lsof searches for open files

View open files to learn more about the system. You can understand which files an application opens or which application opens a specific file as a system administrator. This allows you to make better decisions. For example, you should not Uninstall a file system with opened files. UselsofYou can check the opened file and stop the process before uninstalling it as needed. Similarly, if you find an unknown file, you can find out which application opened the file.

In UNIX environments, files are everywhere, which generates a motto: "Everything is a file ". You can access not only common data, but also network connections and hardware through files. In some caseslsWhen you request a directory list, corresponding entries are displayed. In other cases, for example, the transmission control protocol (TCP) and User Datagram Protocol (UDP) sockets, there is no corresponding directory list. However,File descriptorRegardless of the nature of the file, the file descriptor provides a common interface for the interaction between the application and the basic operating system.

Because the descriptor list of an Application Opening a file provides a lot of information about the application itself, it is very helpful to view this list. The utility used to complete this task is calledlsofIt corresponds to "list open files" (list open files ). This utility is available in almost every unix version, but it is strange that most vendors do not include it in the initial installation of the operating system. For more information aboutlsofFor more information, see references.

Lsof Introduction

Just enterlsofYou can generate a large amount of information, as shown in Listing 1. BecauselsofYou need to access the core memory and various files, so you must run it as the root user to make full use of its functions.

Listing 1. lsof sample output

bash-3.00# lsof COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAMEsched 0 root cwd VDIR 136,8 1024 2 /init 1 root cwd VDIR 136,8 1024 2 /init 1 root txt VREG 136,8 49016 1655 /sbin/initinit 1 root txt VREG 136,8 51084 3185 /lib/libuutil.so.1vi 2013 root 3u VREG 136,8 0 8501 /var/tmp/ExXDaO7d...

Each line displays an opened file. Unless otherwise specified, all files opened by all processes are displayed.Command,PIDAndUserThe process name, process identifier (PID), and owner name.Device,SIZE/OFF,NodeAndNameThe column contains information about the file, which indicates the disk name, file size, index node (File ID on the disk), and the exact name of the file. Depending on the unix version, the file size may be reported as the current position (offset) of the application to read from the file ). Listing 1 is from a Sun Solaris 10 computer that can report this information, but Linux does not.

FDAndTypeColumns have the most vague meanings. They provide more information about how to use files.FDThe column indicates the file descriptor, which is recognized by the application.TypeThe column provides more descriptions about the file format. Let's take a look at the file descriptor column. Three different values appear in Listing 1.cwdThe value indicates the current working directory of the application, which is the directory started by the application, unless it changes the directory.txtType files are program code, such as application binary files or shared libraries.initProgram. Finally, the value indicates the file descriptor of the application. This is an integer returned when the file is opened. In the last line output by listing 1, you can see that the user is usingviEdit/var/tmp/exxdao7d. The file descriptor is 3.uIndicates that the file is opened and in read/write mode, instead of read-only (r) Or write-only (w) Mode. It is not very important, but very helpful. When opening each application at first, there are three file descriptors, from 0 to 2, indicating the standard input, output, and error stream respectively. Because of this, most applications open files with FD starting from 3.

AndFDColumn comparison,TypeColumns are more intuitive. Depending on the operating system, you will find thatREGAndDIR(In SolarisVREGAndVDIR). Other values may beCHRAndBLK, Indicating characters and Block devices respectively; orUNIX,FIFOAndIPv4Indicates the Unix domain socket, first-in-first-out (FIFO) queue, and Internet Protocol (IP) socket.

Go to the/proc directory

DespitelsofThere is no direct relationship, but it is necessary to briefly introduce the/proc directory. /Proc is a directory that contains various files that reflect the kernel and process tree. These files and directories do not exist in the disk. Therefore, when you read and write these files, you actually get the relevant information from the operating system. MostlsofThe related information is stored in the directory named by the PID of the process. Therefore,/proc/1234 contains the information of the process whose PID is 1234.

There are various files in each process directory in the/proc directory, they allow applications to easily understand the memory space, file descriptor list, symbolic links to files on the disk, and other system information of the process.lsofThe utility uses this information and other information about the internal status of the kernel to generate its output. I willlsofThe output is associated with the information in the/proc directory.

Common usage

Previously, I introduced you how to simply runlsofTo display information about the files opened by each process. The rest of this article focuses on how to uselsofTo display the required information and how to correctly interpret it.

Search for files opened by the application

lsofA common usage is to find the name and number of files opened by an application. You may want to find out where a specific application Records log data, or you are tracking a problem. For example, Unix limits the number of files that a process can open. This value is usually very large, so there is no problem, and when necessary, the application can request a larger value (until a certain upper limit ). If you suspect that the application has exhausted the file descriptor, you can uselsofCount the number of opened files for verification.

To specify a single process, you can use-pParameter, followed by the PID of the process. This will not only return the files opened by the application, but also the shared library and code. Therefore, you usually need to filter the output. To complete this task, you can use-dFlag basedFDColumn to filter, use-aSign indicates that both parameters must meet (and ). If no-aBy default, the file that matches any parameter (OR) is displayed. Listing 2 showssendmailProcesses open files and use TXT to filter these files.

List 2. lsof output with PID filter and TXT file descriptor Filtering

sh-3.00# lsof -a -p 605 -d ^txtCOMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAMEsendmail 605 root cwd VDIR 136,8 1024 23554 /var/spool/mqueuesendmail 605 root 0r VCHR 13,2 6815752 /devices/pseudo/mm@0:nullsendmail 605 root 1w VCHR 13,2 6815752 /devices/pseudo/mm@0:nullsendmail 605 root 2w VCHR 13,2 6815752 /devices/pseudo/mm@0:nullsendmail 605 root 3r DOOR 0t0 58/var/run/name_service_door(door to nscd[81]) (FA:->0x30002b156c0)sendmail 605 root 4w VCHR 21,0 11010052 /devices/pseudo/log@0:conslog->LOGsendmail 605 root 5u IPv4 0x300010ea640 0t0 TCP *:smtp (LISTEN)sendmail 605 root 6u IPv6 0x3000431c180 0t0 TCP *:smtp (LISTEN)sendmail 605 root 7u IPv4 0x300046d39c0 0t0 TCP *:submission (LISTEN)sendmail 605 root 8wW VREG 281,3 32 8778600 /var/run/sendmail.pid

Listing 2 islsofThree parameters are specified. The first one is-aIt indicates that the file is displayed only when all parameters are true. The second parameter is-p 605, It limits the output of only processes with a PID of 605.psCommand to obtain this information. Last Parameter-d ^txt, Which indicates filtering TXT records (excluding the [^] symbol ).

The output in Listing 2 provides information about process behavior. For examplecwdThe working directory of the application is/var/spool/mqueue, as shown in the following figure. File descriptors 0, 1, and 2 are allocated to/dev/null (Solaris uses a large number of symbolic links, so the corresponding pseudo devices are displayed here ). FD 3 is a Solaris door (high-speed Remote Procedure Call (RPC) Interface) that is opened in read-only mode. The content in FD 4 is interesting because it is a read-only handle of a character device, essentially/dev/log. From this file, you can collect records from the application to the Unix Syslog daemon. Therefore,/etc/syslog. conf specifies the location of the log file.

As a network application,sendmailListen on the network port. File descriptors 5, 6, and 7 tell you that the application is listening for Simple Mail Transfer Protocol (SMTP) ports in IPv4 and IPv6 modes, and listening for the Submission port in IPv4 mode. The last file descriptor is written only and points to/var/run/sendmail. PID.FDUppercase in ColumnWIndicates that the application has a write lock on the entire file. This file is used to ensure that only one application instance can be opened at a time.

Find the application that opens a file

In other cases, you have a file or directory, and you need to know which application controls the file (open the file ). Listing 2 showssendmailThe process opened/var/run/sendmail. PID. If you do not know this information,lsofThis information can be provided. Listing 3 shows the corresponding output.

Listing 3. Require lsof to display information about a file

bash-3.00# lsof /var/run/sendmail.pidCOMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAMEsendmail 605 root 8wW VREG 281,3 32 8778600 /var/run/sendmail.pid

As shown in the output, Processsendmail(PID is 605) controls the file/var/run/sendmail. PID, and opens the file through the exclusive lock for writing. If you need to delete the file for some reason, the correct method is to stop the process, rather than directly delete the file. Otherwise, the daemon may fail to start normally next time, or another instance may be started later, resulting in competition.

Sometimes you only know that a file is opened somewhere in the file system. When detaching a file system, if any open file exists in the file system, the Operation will fail. You can uselsofDisplays all open files in a file system. Listing 4 shows how to uninstall/export/home and then uselsofFind out who is using the file system.

Listing 4. Use lsof to find out who is using the file system

bash-3.00# umount /export/homeumount: /export/home busybash-3.00# lsof /export/homeCOMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAMEbash 1943 root cwd VDIR 136,7 1024 4 /export/home/seanbash 2970 sean cwd VDIR 136,7 1024 4 /export/home/seanct 3030 sean cwd VDIR 136,7 1024 4 /export/home/seanct 3030 sean 1w VREG 136,7 0 25 /export/home/sean/output

In this example, Sean is performing some operations in the home directory. There are twobash(A shell) the instance is running and the current directory is set to the Home Directory of Sean. There is alsoctThe application is running in the same directory, and its standard output (file descriptor 1) is redirected to a file named output. To successfully uninstall/export/home, stop these processes after notifying the user to ensure normal conditions.

This example shows that the current working directory of the application is very important because it still maintains file resources and can prevent the file system from being detached. This is why most daemon (background processes) change their directories to the root directory or service-specific directories (suchsendmail/Var/spool/mqueue In the example) to prevent the daemon from blocking the unmounting of unrelated file systems. IfsendmailStart from the/export/home/Sean directory without changing its directory to/var/spool/mqueue. You must stop/export/home before uninstalling it.

If you are interested in files opened in the non-mount point directory, you must+dOr+DSpecify the name of the directory. Which of the following flag is used depends on the Recursion to the subdirectory (+D) Or do not need to be recursive to the subdirectory (+d). For example, to view all open files in/export/home/Sean, you can uselsof +D /export/home/sean. In the previous example, the related directory is a mount point, which is slightly different from the previous example and limitslsofAnd the kernel. This also causes potential problems, that islsof /export/homeAndlsof /export/home/(Note the slash at the end. The first method works normally because it points to the mount point. The second method does not generate any output because it points to the directory. If you use the tab key in shell to automatically complete the command, you may encounter this problem, which will help you add the ending slash. In this case, you can delete this slash or use+DSpecify a directory. The former is the preferred method, because it is faster than specifying any directory.

Uncommon usage

In the previous section, we studiedlsofDisplays the relationships between open files and processes that control them. This method is helpful when you want to perform tedious operations on the system without damaging important documents of others. You can also perform some difficult UNIX operations in the same way.

Restore deleted files

When UNIX computers are intruded, it is common to delete log files to mask attackers. Management errors may also lead to accidental deletion of important files. For example, when clearing old logs, the active transaction logs of the database are accidentally deleted. Sometimes these files can be recovered, andlsofIt can help you.

When a process opens a file, as long as the process keeps opening the file, even if it is deleted, it still exists in the disk. This means that the process does not know that the file has been deleted, and it can still read and write the file descriptor provided to it when the file is opened. In addition to this process, this file is invisible because the corresponding directory entries have been deleted.

As mentioned earlier in the/proc directory, you can access the file descriptor of a process by searching in the appropriate directory. In the subsequent content, you can seelsofDisplays the process file descriptor and related file names. Do you understand what I mean?

I hope it is so simple! When youlsofWhen transferring a file name, for examplelsof /file/I/deleted, It first usesstat()The system calls the file to obtain information about the file. Unfortunately, the file has been deleted. In different operating systems,lsofThe file name may be captured from the core memory. Listing 5 shows a Linux system, which deletes Apache logs from other places. I am usinggrepTool to check whether the file is opened.

Listing 5. Use lsof in Linux to find the deleted file

# lsof | grep error_loghttpd 2452 root 2w REG 33,2 499 3090660/var/log/httpd/error_log (deleted)httpd 2452 root 7w REG 33,2 499 3090660/var/log/httpd/error_log (deleted)... more httpd processes ...

In this example, you can see that the file descriptor of PID 2452 open file is 2 (standard error) and 7. Therefore, you can view the corresponding information in/proc/2452/FD/7, as shown in Listing 6.

Listing 6. Searching for deleted files through/proc

# cat /proc/2452/fd/7[Sun Apr 30 04:02:48 2006] [notice] Digest: generating secret for digest authentication[Sun Apr 30 04:02:48 2006] [notice] Digest: done[Sun Apr 30 04:02:48 2006] [notice] LDAP: Built with OpenLDAP LDAP SDK

The advantage of Linux is that it saves the file name and even tells us that it has been deleted. This is useful when searching for related content in a damaged system, because attackers usually delete logs to hide their traces. Solaris does not provide this information. However, we know thathttpdThe daemon uses the error_log file, so you can usepsCommand to find this PID, and then you can view all files opened by this daemon.

Listing 7. Searching for deleted files in Solaris

# lsof -a -p 8663 -d ^txtCOMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAMEhttpd 8663 nobody cwd VDIR 136,8 1024 2 /httpd 8663 nobody 0r VCHR 13,2 6815752 /devices/pseudo/mm@0:nullhttpd 8663 nobody 1w VCHR 13,2 6815752 /devices/pseudo/mm@0:nullhttpd 8663 nobody 2w VREG 136,8 185 145465 / (/dev/dsk/c0t0d0s0)httpd 8663 nobody 4r DOOR 0t0 58 /var/run/name_service_door(door to nscd[81]) (FA:->0x30002b156c0)httpd 8663 nobody 15w VREG 136,8 185 145465 / (/dev/dsk/c0t0d0s0)httpd 8663 nobody 16u IPv4 0x300046d27c0 0t0 TCP *:80 (LISTEN)httpd 8663 nobody 17w VREG 136,8 0 145466 /var/apache/logs/access_loghttpd 8663 nobody 18w VREG 281,3 0 9518013 /var/run (swap)

I use-aAnd-dThe parameter filters the output to exclude code segments, Because I know which files need to be searched.NameColumns are displayed. Two files (FD 2 and 15) use the disk name instead of the file name, and their types areVREG(Regular files ). In Solaris, the deleted file displays the name of the disk where the file is located. With this clue, you can know that the FD points to a deleted file. Actually, View/proc/8663/fd/15You can get the data to be searched.

If you can view the corresponding data through the file descriptor, you can use I/O redirection to copy it to the file, as shown incat /proc/8663/fd/15 > /tmp/error_log. In this case, you can stop the daemon (this will delete FD and delete the corresponding file), copy the temporary file to the desired location, and restart the daemon.

For many applications, especially log files and databases, this method is very useful for restoring and deleting files. As you can see, some operating systems (and different versionslsof) It is easier to find the corresponding data than other systems.

Search for Network Connections

The network connection is also a file, which means you can uselsofObtain information about them. You have seen this example in Listing 2. This example assumes that you already know the PID, but sometimes it is not. If you only know the corresponding port, you can use-iParameters are searched using Socket information. Listing 8 shows the search for TCP port 25.

Listing 8. Find the process listening to port 25

# lsof -i :25COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAMEsendmail 605 root 5u IPv4 0x300010ea640 0t0 TCP *:smtp (LISTEN)sendmail 605 root 6u IPv6 0x3000431c180 0t0 TCP *:smtp (LISTEN)

Toprotocol:@ip:portForm directionlsofThe utility transmits relevant information. The protocol is TCP or UDP (you can use 4 or 6 as the prefix, indicating the IP version), and the IP address is a Resolved Name or IP address, the port is a number or indicates the name of the Service (from/etc/services ). One or more elements (ports, IP addresses, and Protocols) are required ). In listing 8,:25Port 25. The output shows that process 605 is using IPV6 and IPv4 listening ports 25. If you are not interested in IPv4, you can change the filter6:25To listen to the IPv6 socket on port 25, or directly use6All IPv6 connections.

In addition to the objects that the daemon is listening,lsofYou can also find the connection, which is also used-iParameters. Listing 9 shows all the connections between search and 192.168.1.10.

Listing 9. Search for active connections

# lsof -i @192.168.1.10COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAMEsshd 1934 root 6u IPv6 0x300046d21c0 0t1303608 TCP sun:ssh->linux:40379 (ESTABLISHED)sshd 1937 root 4u IPv6 0x300046d21c0 0t1303608 TCP sun:ssh->linux:40379 (ESTABLISHED)

In this example,sunAndlinuxThere are two IPv6 connections between them. From a more careful study, we can see that these connections come from two different processes, but they are the same, because the two hosts are the same, and the ports are the same (SSH and 40379 ). This is because the connection that enters the main process forks a handler and passes the socket to it. You can also see thatsunThe computer is using port 22 (SSH), whilelinuxHas port 40379. This indicates,sunIs the receiver of the connection because it is associated with the known port of the service. 40379 is the source or temporary port and only makes sense for this connection.

Because, at least in UNIX, the socket is another type of filelsofYou can obtain detailed information about these connections and find out who is responsible for them.

Conclusion

UNIX uses a large number of files. As a system administrator,lsofAllows you to view the core memory to find out how the system uses these files.lsofThe simplest usage tells you which processes open which files and which processes open them. This is particularly important when you collect information about how your application works, or before you perform operations that may corrupt data to ensure that files are not in use.lsofMore advanced usage can help you find deleted files and obtain information about network connections. This is a powerful tool that can be used almost anywhere.