Virtual File System

Currently, our mainstream values are social harmony and world harmony. Similarly, one of the key factors for Linux's success is its ability to coexist harmoniously with other operating systems. You can transparently install disks or partitions with other operating system file formats, such as Windows, Unix of other versions, or even systems with low market share like amiga. Using the so-called Virtual File System Concept, Linux tries to support multiple file system types in the same way as other UNIX variants.

The implicit idea of a Virtual File System is to put the common information of many different types of file systems into the kernel; there is a field or function to support any operations provided by all the actual file systems supported by Linux. For each read, write, or other function called, the kernel can replace them with the actual functions that support the local Linux File System, NTFS file system, or any other file system where the file is located. 1 Virtual File System Overview

Virtual filesystem (Virtual filesystem) can also be called virtual filesystem switch (VFS). It is a kernel software layer, it is used to process all system calls related to UNIX standard file systems. Its robustness is manifested in providing a common interface for various file systems.

For example, assume that a user enters the following shell command:
$ CP/floppy/test/tmp/test

Where/Floppy is an installation point for the MS-DOS disk, And/tmp is a standard second extended File System (second extended filesystom, ext2) directory. As shown in (a), VFS is the abstraction layer between your application and file system implementation. Therefore, the CP program does not need to know what file system types/floppy/test and/tmp/test are. On the contrary, CP programs interact directly with VFS through common system calls that are familiar to UNIX program designers. The Execution Code of CP (B) is as follows:

The file systems supported by VFS can be divided into three main types:

Disk File System

These file systems manage the available storage space in the local disk partition or other devices (such as a USB flash memory) that can act as disks ). Some famous disk-based file systems supported by VFS include:

-File systems used in Linux, such as the widely used second extended File System (ext2), the latest third extended File System (third extended filesystem, ext3), and Reiser File System (reiserfs)

-UNIX file systems, such as sysv file systems (System V, coherent, XENIX), UFS (BSD, Solaris, nextstep), minix file systems, and VERITAS vxfs (SCO unixware ).

-Microsoft's file systems, such as MS-dos, vfat (Windows 95 and later versions), and NTFS (Windows NT and later versions ).

-Is09660 the CD-ROM File System (formerly the high sield file system) and the general disk format (UDF) for the DVD file system.

-Other patented file systems, such as HPFs (ibm OS/2), HFS (Apple's Macintosh), and affs (amiga's quick File System) and ADFs (acorn's disk file archiving system ).

-Other log file systems originating from non-Linux systems, such as ibm jfs and sgi xfs.

Network File System

These file systems allow easy access to the files contained in the file systems of other network computers. Some well-known network file systems supported by virtual file systems include NFS, Coda, AFS (Andrew file system), and CIFS (General Network File System for Microsoft Windows) and NCP (Novell's NetWare Core Protocol ).

Special File System
These file systems do not manage local or remote disk space. File systems such as/proc,/sys, And/dev are typical examples of special file systems.

The UNIX directory creates a tree with the root directory. The root directory is included in the root filesystem. in Linux, the root file system is usually ext2 or ext3. All other file systems can be "installed" in the root file system.

When a file system is installed in a directory, the directory content in the parent file system is no longer accessible, because any path (including the installation point ), will reference the installed file system. However, when the installed file system is uninstalled, the contents of the original directory can be reproduced.

= When a new file system is mounted

Therefore, an important feature of UNIX file systems is that they can be used by system administrators to hide files. They only need to install a file system in the directory where they want to hide files.

Disk-based file systems are usually stored in Block devices, such as hard disks, floppy disks, or CD-Rom. A useful feature of Linux VFS is the ability to process virtual block devices such as/dev/loop0, which can be used to install the file system where common files are located. As a possible application, you can protect your own private file system, because you can store the encrypted version of your file system in a common file.

The First Virtual File System was included in the SunOS operating system released by Sun in 1986. Since then, most UNIX file systems have included VFS. However, Linux VFS supports the widest range of file systems.

 

2. General file Model
The main idea of VFS is to introduce a common file model, which can represent all supported file systems. This model strictly reflects the file model provided by the traditional UNIX file system. This is not surprising, because Linux wants to run its local file system with minimal additional overhead. However, to implement a specific file system, you must convert its physical structure to a general file model of the virtual file system.

For example, in a common file model, each directory is considered as a file and may contain several files and other subdirectories. However, there are several non-Unix disk-based file systems that store each file in the directory tree using the file allocation table (fat). In these file systems, stores directories rather than files. To comply with the general file model of VFS, for the above fat-based file system implementation, Linux must be able to quickly create files corresponding to directories as necessary. Such files only exist as kernel memory objects.

Essentially, the Linux kernel cannot perform hard encoding on a specific function to perform operations such as read () or IOCTL (), but must use a pointer for each operation, the appropriate function that points to the specific file system to be accessed.

To further illustrate this concept, see the figure above, which shows how the kernel converts read () to a dedicated call to the MS-DOS file system. The call of the application to read () causes the kernel to call the corresponding sys_read () service routine, which is similar to that of other systems. We will see later in this chapter that the file is represented by a file data structure in the kernel memory. This type of data structure contains a field called f_op, which contains a function pointer pointing to a specific pair of MS-DOS files, of course, also includes the function of reading files.

Sys_read () finds the pointer to the function and calls it. In this way, the read () of the application is converted to a relatively indirect call:
File-> f_op-> Read (...);

Similarly, the write () operation also triggers the execution of an ext2 write function related to the output file. In short, the kernel is responsible for assigning a set of suitable pointers to the file variables related to each open file, and then calling the functions for each specific File System (directed by the f_op field ).

You can regard the General file model as object-oriented. Here, the object is a software structure, which defines both the data structure and the operation methods on it. For efficiency considerations, Linux coding does not adopt a object-oriented programming language (such as C ++ ). Therefore, the object is implemented as a common C data structure. The fields pointing to the function in the data structure correspond to the object method.

The generic file model consists of the following object types:

Superblock object: stores information about the installed file system. For disk-based file systems, such objects usually correspond to the file system control block stored on the disk ).

Inode object: stores general information about a specific file. For disk-based file systems, such objects usually correspond to file control blocks on disks ). Each index Node object has an index node number, which uniquely identifies files in the file system.

File object: stores information about interaction between open files and processes. This type of information only occurs in the kernel memory when the process accesses the file.

Dentry object: stores information about the link between a directory item (that is, the specific name of the file) and the corresponding file. Each disk file system stores this type of information on the disk in its own unique mode.

The following is a simple example to describe how a process interacts with a file.

Three different processes have opened the same file, and two of them use the same hard link. In this case, each process uses its own file object, but only two directory item objects are required. Each hard link corresponds to one directory item object. These two directory item objects point to the same index Node object, which identifies the super block object and subsequent common disk files.

In addition to providing a common interface for the implementation of all file systems, VFS also plays an important role in system performance, that is, high-speed cache of some file-related data structures. For example, the most recently used directory item object is stored in the so-called Directory item high-speed cache (dentry cache) disk high-speed cache, this accelerates the conversion process from the file path name to the index node of the last path component.

Generally, disk cache is a software mechanism that allows the kernel to store some information on the original disk in Ram so that further access to the data can be quickly implemented, instead of accessing the disk in a slow manner.

Note: disk cache is different from hardware cache or memory cache. The latter two are not related to disks or other devices. Hardware high-speed cache is a fast static RAM, which accelerates the direct access to memory, such as slow dynamic RAM requests. Memory high-speed cache is a software mechanism introduced to bypass the kernel memory distributor (slab distributor ).

In addition to the directory item cache and index node cache, Linux also uses other disk caches. The most important one is the so-called high-speed page cache. We will introduce it in detail in this topic.

 

3. system calls processed by VFS
The following table lists VFS system calls, which involve file systems, common files, directory files, and symbolic link files:

System Call name	Description
Mount () umount () umount2 ()	Install/uninstall a File System
Sysfs ()	Obtain File System Information
Statfs () fstatfs () statfs64 () fstatfs64 () Ustat ()	Obtain the statistics of a file system
Chroot () cmdt_root ()	Change root directory
Chdir () fchdir () getcwd ()	Operate the current directory
Mkdir () rmdir ()	Create/delete directory
Getdents () getdents64 () readdir () Link () Unlink () Rename () lookup_dcookie ()	Operate on directory items
Readlink () symlink ()	Operate on soft links
Chown () fchown () lchown () chown16 () Fchown16 () lchown16 ()	Change File Ownership
Chmod () fchmod () utime ()	Change file Properties
Stat () fstat () lstat () Access () oldstat () oldfstat () oldlstat () stat64 () lstat64 () Fstat64 ()	Read File status
Open () Close () creat () umask ()	Open/close/create a file
DUP () dup2 () fcntl () fcntl64 ()	Operate on file descriptors
Select () poll ()	Wait for an event on a group of file descriptors
Truncate () ftruncate () truncate64 () Ftruncate64 ()	Change file length
Lseek () _ llseek ()	Change file pointer
Read () write () readv () writev () sendfile () sendfile64 () readahead ()	File I/O operations
Io_setup () io_submit () io_getevents () io_cancel () io_destroy ()	Asynchronous I/O (allow multiple read and write requests)
Pread64 () pwrite64 ()	Search and access files
MMAP () mmap2 () munmap () madvise () mincore () Remap_file_pages ()	Process File Memory ing
Fdatasync () fsync () Sync () msync ()	Synchronize file data
Flock ()	Process File locks
Setxattr () lsetxattr () fsetxattr () getxattr () lgetxattr () fgetxattr () Listxattr () llistxattr () flistxattr () removexattr () lremovexattr () fremovexattr ()	Process file extension attributes

There are also a few other system calls processed by VFS, such as ioperm (), IOCTL (), pipe (), and mknod (), involving device files and MPs files, these will be discussed in subsequent blog posts. The last group of system calls processed by VFS, such as socket (), connect (), and bind () are called by the Socket System and are used to implement network functions. Some kernel service routines corresponding to the system calls listed in the table will be discussed later in the blog post.

As mentioned above, VFS is a layer between an application and a specific file system. However, in some cases, a file operation may be executed by the VFS itself, without the need to call a lower-level function. For example, when a process closes an opened file, it does not need to involve the corresponding files on the disk. Therefore, VFS only needs to release the corresponding file objects. Similarly, when the system calls lseek () to modify a file pointer, which is an attribute involved in opening the interaction between the file and the process, VFS only needs to modify the corresponding file object, instead of accessing files on the disk, you do not need to call the functions of a specific file system. In a sense, we can regard VFS as a "common" file system, which depends on a specific file system when necessary.

Virtual filesystem (Virtual filesystem) can also be called virtual filesystem switch (VFS). It is a kernel software layer, it is used to process all system calls related to UNIX standard file systems. Its robustness is manifested in providing a common interface for various file systems.

For example, assume that a user enters the following shell command:
$ CP/floppy/test/tmp/test

Where/Floppy is an installation point for the MS-DOS disk, And/tmp is a standard second extended File System (second extended filesystom, ext2) directory. As shown in (a), VFS is the abstraction layer between your application and file system implementation. Therefore, the CP program does not need to know what file system types/floppy/test and/tmp/test are. On the contrary, CP programs interact directly with VFS through common system calls that are familiar to UNIX program designers. The Execution Code of CP (B) is as follows:

The file systems supported by VFS can be divided into three main types:

Disk File System

These file systems manage the available storage space in the local disk partition or other devices (such as a USB flash memory) that can act as disks ). Some famous disk-based file systems supported by VFS include:

-File systems used in Linux, such as the widely used second extended File System (ext2), the latest third extended File System (third extended filesystem, ext3), and Reiser File System (reiserfs)

-UNIX file systems, such as sysv file systems (System V, coherent, XENIX), UFS (BSD, Solaris, nextstep), minix file systems, and VERITAS vxfs (SCO unixware ).

-Microsoft's file systems, such as MS-dos, vfat (Windows 95 and later versions), and NTFS (Windows NT and later versions ).

-Is09660 the CD-ROM File System (formerly the high sield file system) and the general disk format (UDF) for the DVD file system.

-Other patented file systems, such as HPFs (ibm OS/2), HFS (Apple's Macintosh), and affs (amiga's quick File System) and ADFs (acorn's disk file archiving system ).

-Other log file systems originating from non-Linux systems, such as ibm jfs and sgi xfs.

Network File System

These file systems allow easy access to the files contained in the file systems of other network computers. Some well-known network file systems supported by virtual file systems include NFS, Coda, AFS (Andrew file system), and CIFS (General Network File System for Microsoft Windows) and NCP (Novell's NetWare Core Protocol ).

Special File System
These file systems do not manage local or remote disk space. File systems such as/proc,/sys, And/dev are typical examples of special file systems.

The UNIX directory creates a tree with the root directory. The root directory is included in the root filesystem. in Linux, the root file system is usually ext2 or ext3. All other file systems can be "installed" in the root file system.

When a file system is installed in a directory, the directory content in the parent file system is no longer accessible, because any path (including the installation point ), will reference the installed file system. However, when the installed file system is uninstalled, the contents of the original directory can be reproduced.

= When a new file system is mounted

Therefore, an important feature of UNIX file systems is that they can be used by system administrators to hide files. They only need to install a file system in the directory where they want to hide files.

Disk-based file systems are usually stored in Block devices, such as hard disks, floppy disks, or CD-Rom. A useful feature of Linux VFS is the ability to process virtual block devices such as/dev/loop0, which can be used to install the file system where common files are located. As a possible application, you can protect your own private file system, because you can store the encrypted version of your file system in a common file.

The First Virtual File System was included in the SunOS operating system released by Sun in 1986. Since then, most UNIX file systems have included VFS. However, Linux VFS supports the widest range of file systems.

 

2. General file Model
The main idea of VFS is to introduce a common file model, which can represent all supported file systems. This model strictly reflects the file model provided by the traditional UNIX file system. This is not surprising, because Linux wants to run its local file system with minimal additional overhead. However, to implement a specific file system, you must convert its physical structure to a general file model of the virtual file system.

For example, in a common file model, each directory is considered as a file and may contain several files and other subdirectories. However, there are several non-Unix disk-based file systems that store each file in the directory tree using the file allocation table (fat). In these file systems, stores directories rather than files. To comply with the general file model of VFS, for the above fat-based file system implementation, Linux must be able to quickly create files corresponding to directories as necessary. Such files only exist as kernel memory objects.

Essentially, the Linux kernel cannot perform hard encoding on a specific function to perform operations such as read () or IOCTL (), but must use a pointer for each operation, the appropriate function that points to the specific file system to be accessed.

To further illustrate this concept, see the figure above, which shows how the kernel converts read () to a dedicated call to the MS-DOS file system. The call of the application to read () causes the kernel to call the corresponding sys_read () service routine, which is similar to that of other systems. We will see later in this chapter that the file is represented by a file data structure in the kernel memory. This type of data structure contains a field called f_op, which contains a function pointer pointing to a specific pair of MS-DOS files, of course, also includes the function of reading files.

Sys_read () finds the pointer to the function and calls it. In this way, the read () of the application is converted to a relatively indirect call:
File-> f_op-> Read (...);

Similarly, the write () operation also triggers the execution of an ext2 write function related to the output file. In short, the kernel is responsible for assigning a set of suitable pointers to the file variables related to each open file, and then calling the functions for each specific File System (directed by the f_op field ).

You can regard the General file model as object-oriented. Here, the object is a software structure, which defines both the data structure and the operation methods on it. For efficiency considerations, Linux coding does not adopt a object-oriented programming language (such as C ++ ). Therefore, the object is implemented as a common C data structure. The fields pointing to the function in the data structure correspond to the object method.

The generic file model consists of the following object types:

Superblock object: stores information about the installed file system. For disk-based file systems, such objects usually correspond to the file system control block stored on the disk ).

Inode object: stores general information about a specific file. For disk-based file systems, such objects usually correspond to file control blocks on disks ). Each index Node object has an index node number, which uniquely identifies files in the file system.

File object: stores information about interaction between open files and processes. This type of information only occurs in the kernel memory when the process accesses the file.

Dentry object: stores information about the link between a directory item (that is, the specific name of the file) and the corresponding file. Each disk file system stores this type of information on the disk in its own unique mode.

The following is a simple example to describe how a process interacts with a file.

Three different processes have opened the same file, and two of them use the same hard link. In this case, each process uses its own file object, but only two directory item objects are required. Each hard link corresponds to one directory item object. These two directory item objects point to the same index Node object, which identifies the super block object and subsequent common disk files.

In addition to providing a common interface for the implementation of all file systems, VFS also plays an important role in system performance, that is, high-speed cache of some file-related data structures. For example, the most recently used directory item object is stored in the so-called Directory item high-speed cache (dentry cache) disk high-speed cache, this accelerates the conversion process from the file path name to the index node of the last path component.

Generally, disk cache is a software mechanism that allows the kernel to store some information on the original disk in Ram so that further access to the data can be quickly implemented, instead of accessing the disk in a slow manner.

Note: disk cache is different from hardware cache or memory cache. The latter two are not related to disks or other devices. Hardware high-speed cache is a fast static RAM, which accelerates the direct access to memory, such as slow dynamic RAM requests. Memory high-speed cache is a software mechanism introduced to bypass the kernel memory distributor (slab distributor ).

In addition to the directory item cache and index node cache, Linux also uses other disk caches. The most important one is the so-called high-speed page cache. We will introduce it in detail in this topic.

 

3. system calls processed by VFS
The following table lists VFS system calls, which involve file systems, common files, directory files, and symbolic link files:

System Call name	Description
Mount () umount () umount2 ()	Install/uninstall a File System
Sysfs ()	Obtain File System Information
Statfs () fstatfs () statfs64 () fstatfs64 () Ustat ()	Obtain the statistics of a file system
Chroot () cmdt_root ()	Change root directory
Chdir () fchdir () getcwd ()	Operate the current directory
Mkdir () rmdir ()	Create/delete directory
Getdents () getdents64 () readdir () Link () Unlink () Rename () lookup_dcookie ()	Operate on directory items
Readlink () symlink ()	Operate on soft links
Chown () fchown () lchown () chown16 () Fchown16 () lchown16 ()	Change File Ownership
Chmod () fchmod () utime ()	Change file Properties
Stat () fstat () lstat () Access () oldstat () oldfstat () oldlstat () stat64 () lstat64 () Fstat64 ()	Read File status
Open () Close () creat () umask ()	Open/close/create a file
DUP () dup2 () fcntl () fcntl64 ()	Operate on file descriptors
Select () poll ()	Wait for an event on a group of file descriptors
Truncate () ftruncate () truncate64 () Ftruncate64 ()	Change file length
Lseek () _ llseek ()	Change file pointer
Read () write () readv () writev () sendfile () sendfile64 () readahead ()	File I/O operations
Io_setup () io_submit () io_getevents () io_cancel () io_destroy ()	Asynchronous I/O (allow multiple read and write requests)
Pread64 () pwrite64 ()	Search and access files
MMAP () mmap2 () munmap () madvise () mincore () Remap_file_pages ()	Process File Memory ing
Fdatasync () fsync () Sync () msync ()	Synchronize file data
Flock ()	Process File locks
Setxattr () lsetxattr () fsetxattr () getxattr () lgetxattr () fgetxattr () Listxattr () llistxattr () flistxattr () removexattr () lremovexattr () fremovexattr ()	Process file extension attributes

There are also a few other system calls processed by VFS, such as ioperm (), IOCTL (), pipe (), and mknod (), involving device files and MPs files, these will be discussed in subsequent blog posts. The last group of system calls processed by VFS, such as socket (), connect (), and bind () are called by the Socket System and are used to implement network functions. Some kernel service routines corresponding to the system calls listed in the table will be discussed later in the blog post.

As mentioned above, VFS is a layer between an application and a specific file system. However, in some cases, a file operation may be executed by the VFS itself, without the need to call a lower-level function. For example, when a process closes an opened file, it does not need to involve the corresponding files on the disk. Therefore, VFS only needs to release the corresponding file objects. Similarly, when the system calls lseek () to modify a file pointer, which is an attribute involved in opening the interaction between the file and the process, VFS only needs to modify the corresponding file object, instead of accessing files on the disk, you do not need to call the functions of a specific file system. In a sense, we can regard VFS as a "common" file system, which depends on a specific file system when necessary.