File System in Linux Kernel

File System in Linux Kernel
File descriptor

Generally, when talking about files and file systems, you may subconsciously think that they exist on disks. However, disks can only be passively stored, but cannot actively process files. To process files, you can only copy their data to the memory for CPU processing. The processed data is first written into the memory, and then pass it back to the disk. So how does the operating system manage various files in the memory? That is, the representation of the file system in the kernel during runtime.
We know that a process is the basic unit for the operating system to allocate resources, and files are also processed in the process. For example, if you use vim to write code, the vim program becomes a process in the operating system, and the object processed by vim is a code file. The operating system uses PCB to manage processes. From the code point of view, the PCB is the task_struct structure, which has a pointer pointing to the files_struct structure. This is the so-called file descriptor table. Its code is as follows:

Struct files_struct {atomic_t count;/* Number of processes sharing the table */rwlock_t file_lock;/* protect all the following fields to avoid nesting in tsk-> alloc_lock */int max_fds; /* Maximum number of objects in the current file */int max_fdset;/* Maximum number of descriptors in the current file */int next_fd; /* Add 1 */struct file ** fd to the allocated file descriptor;/* pointer to the array of object pointers */fd_set * close_on_exec;/* points to execute exec () file descriptor */fd_set * open_fds;/* pointer to open file descriptor */fd_set close_on_exec_init;/* execute exec () */fd_set open_fds_init;/* file descriptor's initial value set */struct file * fd_array [32]; /* initialization array of file object pointers */};

This table is private to each process, and each process has one. It is also known as a user opening a file table to distinguish the system from opening a file table. We mainly focus on the last Member: struct file * fd_array [32];, which is a pointer array. Each member of the array points to a file struct, the file structure stores information about files opened by the process. Since a process can open multiple files, a pointer array is used to save their information. For each file with an entry address in the array,The index of the array is the file descriptor). Generally, the first element of the array (index 0) is the standard input file of the process, and the second element of the array (Index 1) is the standard output file of the process, the third element of the array (index 2) is the standard error file of the process. These three files are opened by default by the operating system for each process. If the process needs to open other files, the process descriptor starts from 3. File descriptors are an important resource of the system. Although file descriptors can be opened as much as the system memory is available, the kernel will process them in actual implementation, generally, the maximum number of opened files is 10% of the system memory (measured in KB) (system-level limit). To view the maximum number of opened files at the system level, you can use sysctl-a | grep fs. file-max command. At the same time, in order to prevent a process from consuming all the file resources, the kernel also performs the default value processing (called the user-level limit) on the maximum number of files opened by a single process. The default value is generally 1024, run the ulimit-n command.
In Linux, a process accesses a file through a file descriptor (fd) instead of a file name. The file descriptor is actually an integer.

Virtual File System
Let's take a look at the file structure. The Code is as follows:

Struct file {struct list_head f_list;/* all open files form a linked list */struct dentry * f_dentry;/* pointer to related directory items */struct vfsmount * f_vfsmnt; /* pointer to the VFS Installation Point */struct file_operations * f_op;/* pointer to the file operation table */mode_t f_mode;/* Open Mode of the file */loff_t f_pos; /* current location of the file */unsigned short f_flags;/* indicates the flag specified when the file is opened */unsigned short f_count; /* Number of processes using this structure */unsigned long f_reada, f_ramax, f_raend, f_ralen, f_rawin; /* pre-read Mark, maximum number of pages to be preread, file pointer after the last pre-read, number of pre-read bytes, and number of pre-read pages */int f_owner; /* transmit asynchronous I/O data through signals */unsigned int f_uid, f_gid;/* UID and GID */int f_error; /* error code for network write operations */unsigned long f_version;/* version */void * private_data;/* tty driver required */};

The file struct does not exactly correspond to the actual file. For example, when a process opens the same file multiple times, a different file struct and the corresponding file descriptor are allocated, although these file struct eventually point to the same actual physical file. It can be seen that the implementation of files in the memory is not the same as that in the disk. Files in the memory are dynamic because they need to be read and written continuously, so they are only a copy, all operations are for this copy. After the operation is completed, the result is written back to the disk file. Before the operation is executed, all the changes only exist in the memory, it does not respond to the disk in real time. You can use vim and cat to simultaneously operate a file for verification. After understanding this principle, let's look at how different file structures need to be allocated each time a file is opened: because each time a file is opened, different operations must be performed on the same file, you want the results of different operations to be written back to the disk file. If the same file is used, the subsequent operations may overwrite the previous operations.
The possibility of overwriting is from loff_t f_pos, a member of the file struct. It indicates the current read/write location of the file. Each file has a 32-bit number to indicate the location of the next read/write byte. This number is called the location of the file. Each time you open a file, unless explicitly required, the file location is set to 0, that is, the start of the file. Subsequent read or write operations will be performed from the beginning of the file, the execution system can call LSEEK (Random storage) to modify the file location. Therefore, if the two operations target different locations of the same file, using the same file struct will make them have the same read/write location, which obviously cannot be achieved. This is the same process of different operations, not to mention different processes of different operations? Therefore, each time you open a file, you need to allocate a new file structure. It can be said that the file struct mainly stores the read/write location.
If "almost" is used above, it indicates that there is an exception: when a new process is generated, the child process must share all the information of the parent process, including the file struct, and its member unsigned short f_count; it indicates how many processes are using this struct. This struct is destroyed from memory only when its value is 0.
The first member in the file structure is struct list_head f_list. It forms a double-stranded table in the file structure, which is called a system-opened file table. Its maximum length is NR_FILE, in fs. the value in h is 8192.
The relationship between the file descriptor table, the file table opened by the system, and the actual file can be as follows:

Then let's look at how the file struct points to the final actual file, which involves the second Member: struct dentry * f_dentry; the Code is as follows:

Struct dentry {atomic_t d_count;/* directory item reference counter */unsigned int d_flags;/* directory item flag */struct inode * d_inode; /* The index node associated with the file name */struct dentry * d_parent;/* directory item of the parent directory */struct list_head d_hash; /* hash table formed by directory items */struct list_head d_lru;/* unused LRU linked list */struct list_head d_child; /* List of sub-directories in the parent directory */struct list_head d_subdirs;/* List of sub-directories in this directory */struct list_head d_alias; /* linked list of the index node alias */int d_mounted;/* Installation Point of the Directory item */struct qstr d_name;/* directory Item Name (quick search) */unsigned long d_time;/* used by the d_revalidate function */struct dentry_operations * d_op;/* function set of the Directory item */struct super_block * d_sb; /* root of the Directory item tree (that is, the super block of the file) */unsigned long d_vfs_flags; void * d_fsdata;/* Data of the specific file system */unsigned char d_iname [DNAME_INLINE_LEN]; /* short file name */};

This struct stores the path information, including the file name. When I introduced the ln command, the operating system looks for a file based on the inode number, and the human needs to find the corresponding relationship between the two from the structure based on the file name. We can use these two members: struct inode * d_inode; and struct qstr d_name ;. The former points to the inode struct related to the real file, and stores the inode read information from the disk partition. At this point, the file descriptor in the process points to the final disk file. I guess the file struct will eventually be written back to the disk by the inode struct.
.