Relationship between file descriptors and open files in Linux

Relationship between file descriptors and open files in Linux

Relationship between file descriptors and open files in LinuxGuideThe kernel uses file descriptor to access files. The file descriptor is a non-negative integer. When an existing file is opened or a new file is created, the kernel returns a file descriptor. To read and write a file, you also need to use the file descriptor to specify the file to be read and written.1. Overview

In Linux, everything can be viewed as a file, which can be divided into common files, directory files, link files, and device files. File descriptor is an index created by the kernel to efficiently manage opened files. It is a non-negative integer (usually a small integer) used to indicate opened files, all system calls that execute the I/O operation are performed through the file descriptor. When the program was just started, 0 was the standard input, 1 was the standard output, and 2 was the standard error. If you open a new file at this time, its file descriptor will be 3.The POSIX standard requires that the current process must be used each time a file is opened (including socket ).The minimum number of available file descriptors. Therefore, if you do not pay attention to it during network communication, it may cause a string of calls.. The following figure shows the standard file descriptor:

The file description corresponds to the opened file model, for example:

2. File description restrictions

When writing file operations or network communication software, beginners may encounter the "Too program open files" problem. This is mainly because file descriptors are an important resource of the system,Although the file descriptor can be opened as much as the system memory is available, the kernel will perform corresponding processing in the actual implementation process, generally, the maximum number of opened files is 10% of the system memory (measured in KB) (system-level limit). To view the maximum number of opened files at the system level, you can use sysctl-a | grep fs. file-max command. At the same time, in order to prevent a process from consuming all the file resources, the kernel also performs the default value processing (called the user-level limit) on the maximum number of files opened by a single process. The default value is generally 1024, run the ulimit-n command. In Web servers, optimizing the server by changing the maximum file descriptor of the system default value is one of the most common methods. For more information about the optimization method, see http://blog.csdn.net/kumu_linux/article/details/7877770.

3. The file description conforms to the relationship between opened files.

Each file descriptor corresponds to an opened file, and different file descriptors also point to the same file. The same file can be opened by different processes or multiple times in the same process. The system maintains a file descriptor table for each process. The table value starts from 0. Therefore, you can see the same file descriptor in different processes, in this case, the same file descriptor may point to the same file or to different files. The specific situation should be analyzed in detail. To understand the specific situation, you need to view the three data structures maintained by the kernel.

1. Process-level file descriptor table

2. System-level open file descriptor table

3. I-node table of the file system

Each entry in the process-level Descriptor Table records information about a single file descriptor.

1. A group of symbols that control file descriptor operations. (Currently, only one such flag is defined, that is, the close-on-exec flag)

2. Reference to open file handle

The kernel maintains a system-level Descriptor table (open file description table) for all open files ). Sometimes, it is also called an open file table, and each entry in the table is called an open file handle ). An open file handle stores all information related to an open file, as shown below:

1. Current file offset (updated when read () and write () are called, or directly modified using lseek)

2. the flags parameter used to open the file (that is, the flags parameter of open)

3. File Access Mode (for example, the read-only mode, write-only mode, or read/write mode set when open () is called)

4. Signal-driven settings

5. Reference to the I-node object of the file

6. file type (for example, regular file, socket, or FIFO) and access permission

7. A pointer pointing to the lock list held by the file

8. Various attributes of the file, including the file size and timestamps related to different types of operations

The relationship between file descriptors, opened file handles, and I-node is displayed. In the figure, the two processes have many opened file descriptors.

In process A, both file descriptors 1 and 30 point to the same open file handle (number 23 ). This may be because the open () function is called multiple times by calling dup (), dup2 (), fcntl (), or the same file.

Both process A's file descriptor 2 and process B's file descriptor 2 point to the same open file handle (number 73 ). This situation may occur after fork () is called (process A and process B are parent-child processes ), or when a process passes an opened file descriptor to another process through a UNIX domain socket, it also happens. In addition, different processes independently call the open function to open the same file. At this time, the descriptor inside the process is allocated to the same descriptor as other processes open the file.

In addition, process A's descriptor 0 and process B's descriptor 3 point to different open file handles, but these handles point to the same entry (1976) of the I-node table, in other words, point to the same file. This occurs because each process initiates an open () call to the same file. A similar situation occurs when the same process opens the same file twice.

4. Summary

1. Because the process-level file descriptor table exists, the same file descriptor may appear in different processes. They may point to the same file or to different files.

2. Two different file descriptors. If they point to the same open file handle, the same file offset will be shared. Therefore, if one of the file descriptors is used to modify the file offset (caused by calling read (), write (), or lseek (), the changes will also be observed from the other descriptor, this is the case whether the two file descriptors belong to different processes or the same process.

3. to obtain and modify the opened file flags (such as O_APPEND, O_NONBLOCK, and O_ASYNC), you can perform the F_GETFL and F_SETFL operations of fcntl (). The limitations on the scope are quite similar to those of the previous one.

4. The file descriptor flag (that is, close-on-exec) is private to the process and file descriptor. Modifications to this flag will not affect other file descriptors of the same process or different processes.

From: http://blog.csdn.net/cywosp/article/details/38965239

Address: http://www.linuxprobe.com/linux-file-descriptor.html

The kernel uses file descriptor to access files. The file descriptor is a non-negative integer. When an existing file is opened or a new file is created, the kernel returns a file descriptor. To read and write a file, you also need to use the file descriptor to specify the file to be read and written.

1. Overview

In Linux, everything can be viewed as a file, which can be divided into common files, directory files, link files, and device files. File descriptor is an index created by the kernel to efficiently manage opened files. It is a non-negative integer (usually a small integer) used to indicate opened files, all system calls that execute the I/O operation are performed through the file descriptor. When the program was just started, 0 was the standard input, 1 was the standard output, and 2 was the standard error. If you open a new file at this time, its file descriptor will be 3.The POSIX standard requires that the current process must be used each time a file is opened (including socket ).The minimum number of available file descriptors. Therefore, if you do not pay attention to it during network communication, it may cause a string of calls.. The following figure shows the standard file descriptor:

The file description corresponds to the opened file model, for example:

2. File description restrictions

When writing file operations or network communication software, beginners may encounter the "Too program open files" problem. This is mainly because file descriptors are an important resource of the system,Although the file descriptor can be opened as much as the system memory is available, the kernel will perform corresponding processing in the actual implementation process, generally, the maximum number of opened files is 10% of the system memory (measured in KB) (system-level limit). To view the maximum number of opened files at the system level, you can use sysctl-a | grep fs. file-max command. At the same time, in order to prevent a process from consuming all the file resources, the kernel also performs the default value processing (called the user-level limit) on the maximum number of files opened by a single process. The default value is generally 1024, run the ulimit-n command. In Web servers, optimizing the server by changing the maximum file descriptor of the system default value is one of the most common methods. For more information about the optimization method, see http://blog.csdn.net/kumu_linux/article/details/7877770.

3. The file description conforms to the relationship between opened files.

Each file descriptor corresponds to an opened file, and different file descriptors also point to the same file. The same file can be opened by different processes or multiple times in the same process. The system maintains a file descriptor table for each process. The table value starts from 0. Therefore, you can see the same file descriptor in different processes, in this case, the same file descriptor may point to the same file or to different files. The specific situation should be analyzed in detail. To understand the specific situation, you need to view the three data structures maintained by the kernel.

1. Process-level file descriptor table

2. System-level open file descriptor table

3. I-node table of the file system

Each entry in the process-level Descriptor Table records information about a single file descriptor.

1. A group of symbols that control file descriptor operations. (Currently, only one such flag is defined, that is, the close-on-exec flag)

2. Reference to open file handle

The kernel maintains a system-level Descriptor table (open file description table) for all open files ). Sometimes, it is also called an open file table, and each entry in the table is called an open file handle ). An open file handle stores all information related to an open file, as shown below:

1. Current file offset (updated when read () and write () are called, or directly modified using lseek)

2. the flags parameter used to open the file (that is, the flags parameter of open)

3. File Access Mode (for example, the read-only mode, write-only mode, or read/write mode set when open () is called)

4. Signal-driven settings

5. Reference to the I-node object of the file

6. file type (for example, regular file, socket, or FIFO) and access permission

7. A pointer pointing to the lock list held by the file

8. Various attributes of the file, including the file size and timestamps related to different types of operations

The relationship between file descriptors, opened file handles, and I-node is displayed. In the figure, the two processes have many opened file descriptors.

In process A, both file descriptors 1 and 30 point to the same open file handle (number 23 ). This may be because the open () function is called multiple times by calling dup (), dup2 (), fcntl (), or the same file.

Both process A's file descriptor 2 and process B's file descriptor 2 point to the same open file handle (number 73 ). This situation may occur after fork () is called (process A and process B are parent-child processes ), or when a process passes an opened file descriptor to another process through a UNIX domain socket, it also happens. In addition, different processes independently call the open function to open the same file. At this time, the descriptor inside the process is allocated to the same descriptor as other processes open the file.

In addition, process A's descriptor 0 and process B's descriptor 3 point to different open file handles, but these handles point to the same entry (1976) of the I-node table, in other words, point to the same file. This occurs because each process initiates an open () call to the same file. A similar situation occurs when the same process opens the same file twice.

4. Summary

1. Because the process-level file descriptor table exists, the same file descriptor may appear in different processes. They may point to the same file or to different files.

2. Two different file descriptors. If they point to the same open file handle, the same file offset will be shared. Therefore, if one of the file descriptors is used to modify the file offset (caused by calling read (), write (), or lseek (), the changes will also be observed from the other descriptor, this is the case whether the two file descriptors belong to different processes or the same process.

3. to obtain and modify the opened file flags (such as O_APPEND, O_NONBLOCK, and O_ASYNC), you can perform the F_GETFL and F_SETFL operations of fcntl (). The limitations on the scope are quite similar to those of the previous one.

4. The file descriptor flag (that is, close-on-exec) is private to the process and file descriptor. Modifications to this flag will not affect other file descriptors of the same process or different processes.

From: http://blog.csdn.net/cywosp/article/details/38965239

Address: http://www.linuxprobe.com/linux-file-descriptor.html