FILE descriptor and FILE, FILE descriptor FILE

FILE descriptor and FILE, FILE descriptor FILE
1. file descriptor (important)

In Linux, everything can be viewed as a file, which can be divided into common files, directory files, link files, and device files. File descriptor is an index created by the kernel to efficiently manage opened files. It is a non-negative integer (usually a small integer) used to indicate opened files, all system calls that execute the I/O operation are performed through the file descriptor. When the program was just started, 0 was the standard input, 1 was the standard output, and 2 was the standard error. If you open a new file at this time, its file descriptor will be 3.

1.1 Introduction to concepts

File descriptor operations (such as open (), creat (), close (), read () return a file descriptor, which is an integer of the int type, that is, fd, in essence, it is the subscript in the file descriptor table. It acts as an index. The process finds the file pointer filp pointed to by the fd through the file descriptor table in the PCB. Each Process stores a file descriptor table in the Process Control Block (PCB). The file descriptor is the index of the table, each table item in the file description table has a pointer to an opened file.fileStruct indicates that the pointer in the file descriptor table pointsfileStruct. Each time a file is opened, fd is allocated from the smallest unused subscript by default. Disadvantages of file descriptors: it cannot be transplanted to a system other than UNIX or intuitive.

The following figure shows the relationship between them:

Each file contains the following information:

1.2 Chart Interpretation

InfileMaintain File Status Flag (fileStruct Memberf_flags) And the current read/write location (fileStruct Memberf_pos). Process 1 and process 2 open the same file, but they correspond to differentfileStructure, so it can have different File Status Flag and read/write locations.fileThe most important member in the struct isf_count, Indicates Reference Count. We will talk about it later,dup,forkSystem Call causes multiple file descriptors to point to the samefileStruct, suchfd1Andfd2Both reference the samefileStruct, so its reference count is 2. Whenclose(fd1)Will not be releasedfileStruct, instead of reducing the reference count to 1.close(fd2), The reference count will be reduced to 0 and released at the same timefileStructure, which actually closes the file.

EachfileThe structure points tofile_operationsStruct. All the members of this struct are function pointers pointing to kernel functions that implement various file operations. For example, in a user programreadA file descriptor,readEnter the kernel through the system call, and findfileStruct, findfileThefile_operationsStruct.readThe kernel function pointed to by the member to complete the user request. Call in a user programlseek,read,write,ioctl,openAnd other functions are eventually called by the kernelfile_operationsThe kernel functions that each member points to complete user requests.file_operationsStructreleaseThe Member is used to complete the user programcloseThe request is calledreleaseInsteadcloseIt does not necessarily close the file, but reduces the reference count. Only when the reference count is reduced to 0 will the file be closed. For general files opened on the same file system,read,writeThe file operation steps and methods should be the same, and the called functions should be the same, so the three open files in the figurefileThe struct points to the samefile_operationsStruct. If you open a character device fileread,writeThe operation must be different from the conventional file. It is not the data block of the read/write disk but the read/write hardware device.fileStruct should point to differentfile_operationsStruct. Various file operation functions are implemented by the driver of the device.

EachfileThe structure has a pointdentryStruct pointer. "dentry" is the abbreviation of directory entry. Sentopen,statIs a path, for example/home/akaedu/a, You Need To Find The inode of the file according to the path. To reduce the number of disk reads, the kernel caches the tree structure of the Directory, called dentry cache, where each node isdentryStruct. You only need to search for the struct from the root directory along the dentry of each part of the path./FindhomeDirectory, and then findakaeduDirectory, and then find the filea. The dentry cache only saves the recently accessed directory items. If the directory items to be found are not in the cache, they must be read from the disk to the memory.

EachdentryThe structure has a pointerinodeStruct.inodeStruct stores the Information read from the disk inode. In the example, there are two dentry, which respectively represent/home/akaedu/aAnd/home/akaedu/b, Both point to the same inode, indicating that the two files are hard links to each other.inodeThe struct stores the inode read information from the disk partition, such as the owner, file size, file type, and permission bit. EachinodeThe structure has a pointinode_operationsStruct pointer, which is also a set of function pointers pointing to some kernel functions that complete file directory operations. Andfile_operationsDifferent,inode_operationsIt is not a function that operates on a file, but a function that affects the layout of files and directories, such as adding and deleting files and directories and tracking symbolic links.inodeThe struct can point to the sameinode_operationsStruct.

inodeThe struct has a pointsuper_blockStruct pointer.super_blockStruct stores the Information read from the super block of the disk partition, such as the file system type and block size.super_blockStructs_rootA member isdentryIndicates that the root directory of the file system ismountIn this example, the partition ismountTo/homeDirectory.

file,dentry,inode,super_blockThese structs constitute the core concept of VFS (Virtual File System VFS, Virtual Filesystem.

1.3 operations on file descriptors

(1). view the Linux file descriptor

 1 [root@localhost ~]# sysctl -a | grep -i file-max --color 3 fs.file-max = 392036 5 [root@localhost ~]# cat /proc/sys/fs/file-max 7 392036 9 [root@localhost ~]# ulimit -n11 102413 [root@localhost ~]#

In Linux, the maximum file descriptor has two restrictions: one is user-level, and the other is system-level.

System-level limit: The sysctl command and proc file system view the same value, which is a system-level limit, which limits the total number of file descriptors opened by all users.

User-level limit: The ulimit command displays the maximum user-level file descriptor limit. That is to say, the total number of file descriptors occupied by programs executed after logon by each user cannot exceed this limit.

(2) modify the file descriptor Value

1 [root@localhost ~]# ulimit-SHn 102402 [root@localhost ~]# ulimit  -n3 102404 [root@localhost ~]#

The preceding modification only takes effect for the current session and is temporary. If you need to modify it permanently, You need to modify it as follows:

1 [root@localhost ~]# grep -vE'^$|^#' /etc/security/limits.conf2 *                hard nofile                  40963 [root@localhost ~]#

1 // The default configuration file contains only the hard option, and soft indicates the setting value that takes effect for the current system. hard indicates the maximum value that can be set in the system. 2 [root @ localhost ~] # Grep-vE '^ $ | ^ #'/etc/security/limits. conf3 * hard nofile 102404 * soft nofile 102405 [root @ localhost ~] #6 // soft <= hard soft limit cannot be higher than hard limit

(3). Modify System Restrictions

1 [root @ localhost ~] # Sysctl-wfs. file-max = 4000002 fs. file-max = 4000003 [root @ localhost ~] # Echo350000>/proc/sys/fs/file-max // failure after restart 4 [root @ localhost ~] # Cat/proc/sys/fs/file-max5 3500006 [root @ localhost ~] #

// The above is the temporary file descriptor Modification
// Permanently add fs. file-max = 400000 to/etc/sysctl. conf and use sysctl-p.

1.4 view the file descriptor in a program

The following program opens the/home/shenlan/hello. c file. If there is no hello. c file in this directory, the program is automatically created and the file descriptor returned by the program is 3. When a process starts, three files, standard input (0), standard output (1), and standard error handling (2), are opened. fd is allocated from the smallest unused subscript by default, therefore, the returned file descriptor is 3.

 1 #include<stdio.h> 2 #include<sys/types.h> 3 #include<sys/stat.h> 4 #include<fcntl.h> 5 #include<stdlib.h> 6 int main() 7 { 8        int fd; 9        if((fd = open("/home/shenlan/fd.c",O_CREAT|O_WRONLY|O_TRUNC,0611))<0){10               perror("openfile fd.c error!\n");11               exit(1);12        }13        else{14               printf("openfile fd.c success:%d\n",fd);15        }16        if(close(fd) < 0){17               perror("closefile fd.c error!\n");18               exit(1);19        }20        else21               printf("closefile fd.c success!\n");22        exit(0);23 }

Execution result:

1.5 process specific process for opening a file

A process calls open () to open a file. In essence, it obtains a file descriptor so that the process can perform other operations on the file through the file descriptor as the connection. When a process opens a file, it creates a file object for the file and stores the file object in the process open file table (file descriptor array) to determine the file descriptor of the opened file. The open () operation is implemented through sys_open () in the kernel. sys_open () will create the dentry, inode, and file objects of the file, in the file_struct structure process, open the file table fd_array [NR_OPEN_DEFAULT] to find an idle table item, and then return the subscript (INDEX) of this table item, that is, the file descriptor. When creating a file object, point f_op of the file object to the file system's operation function set file_operations, and the function set comes from the I node of the specific file, therefore, the virtual file system is connected with the operations of the actual file system.

2. FILE structure and FILE descriptor in C standard library

The C language uses a file pointer instead of a FILE descriptor as an I/O handle. "file pointer" points to a data structure called a FILE structure in the process user area. The FILE structure includes a buffer and a FILE descriptor value. The FILE descriptor value is an index in the FILE descriptor table. In a sense, the FILE pointer is the handle of the handle. A stream (such as fopen) returns a FILE structure pointer. The FILE structure contains a FILE descriptor. The FILE structure function can be seen as an encapsulation of system calls for direct fd operations, it has the advantage of I/O caching.

The function from FILE descriptor fd to FILE stream FILE * is
FILE * fdopen (int filedes, const char * mode );

In the early C standard library, FILE is defined in stdio. h; in Turbo C, see Tan haoqiang's C program design. The FILE struct contains the member fd, that is, the FILE descriptor. You can also install/usr/include/stdio on the Ubuntu system. in h, the struct _ IO_FILE struct is found. This struct is complex. We only care about the required part-the file descriptor, but in this struct, we did not find file descriptor-related such as fd member variables. In this case, the _ fileno struct Member of the int type attracts our attention, but it cannot be determined as a file descriptor. Therefore, writing a program test is the best method. You can test it with the following code:

 1 #include<stdio.h> 2 #include<stdlib.h> 3 #include<sys/types.h> 4 #include<sys/stat.h> 5 #include<fcntl.h> 6 int main( ) 7 { 8        char buf[50] = {"ILOVE this game!"}; 9        FILE *myfile;10 11        myfile = fopen("2.txt","w+");12        if(!myfile){13               printf("error:openfile failed!\n");14        }15        printf("The openedfile's descriptor is %d\n",myfile->_fileno);16        if(write(myfile->_fileno,buf,50)< 0){17               perror("error:writefile failed!\n");18               exit(1);19        }else{20               printf("writefile successed!\n");21        }22        exit(0);23 }

In the program, use fopen to read and write the 2.txt file. If the 2.txt file does not exist, create the file. And the returned FILE pointer is myfile. Use printf to print the value of myfile-> _ fileno to the standard terminal, pass myfile-> _ fileno as the file descriptor to the write System Call, and write the buffer data to the opened file. Then, use the cat 2.txt content. Execution result. The value of _ fileno is 3 because the standard input, output, and error values are 0, 1, and 2. The output result is as follows:
Therefore, the _ fileno member is the handle (windows System) or file descriptor returned by opening a file in the operating system. For more information, see the C standard library published by the People's post and telecommunications Publishing House. Of course you can also read the/glibc-2.9/manual/io.txt I file. In Linux, the file descriptor is allocated from small to large to check whether the file descriptor is used one by one, and then distributed, or you can write a program to test.

The file descriptor table is also called a file descriptor array, which stores all the files opened by a process. The file descriptor array is contained in the files_struct structure of the file table opened by the process. In/include/linux/fdtable. h is defined as a pointer array pointing to the file type --- fd_array [NR_OPEN_DEFAULT], where NR_OPEN_DEFAULT is also in fdtable. this is a variable related to the CPU architecture defined in h, # define NR_OPEN_DEFAULTBITS_PER_LONG.

The relationship between the FILE structure, file descriptor, and FILE structure can be used to represent: