[Reprinted] Linux File System

Original article address:
Http://www.ibm.com/developerworks/cn/linux/l-linux-filesystem/

Linux File System Analysis

Discuss the Linux file system according to the layered structure

M. Tim Jones, consultant engineer, emulex Corp.

Introduction:In terms of file systems, Linux can be regarded as the "Swiss Army Knife" in the operating system ". Linux supports many file systems, from log file systems to cluster file systems and encrypted file systems. Linux is an excellent platform for using standard and fancy file systems and developing file systems. This article discusses the Virtual File System (VFS, sometimes called the Virtual File System switch) in the Linux kernel, and then introduces the main structure of connecting the file system.

 

Mark this article!

Release date:December 03, 2007
Level:Elementary
Access status13079 views
Suggestion: 0 (Add Comment)

Average score (22 in total)

Basic File System Architecture

The Linux File System architecture is an interesting example of abstracting complex systems. By using a set of common API functions, Linux supports many file systems on many storage devices. For example,ReadFunction calls can read a certain number of bytes from a specified file descriptor.ReadThe function does not know the type of the file system, such as ext3 or NFS. It does not know the storage media of the file system, such as at Attachment packet interface (atapi) disk, serial-Attached SCSI (SAS) disk, or serial Advanced Technology Attachment (SATA) disk. However, whenReadWhen a function reads a file, the data is returned normally. This article describes how to implement this mechanism and introduces the main structure of the Linux File System layer.

Back to Top

What is a file system?

First, answer the most common question "What is a file system ". A file system organizes data and metadata on a storage device. Because the definition is so broadCodeIt will be interesting. As mentioned above, there are many file systems and media. As there are so many types, we can expect that the Linux File System Interface is implemented as a hierarchical architecture, so as to implement the user interface layer, file system implementation and operation of the drive of the storage device.ProgramSeparated.

Another way to view a file system as a protocol is to regard it as a protocol. Network protocols (such as IP addresses) define the meaning of data streams transmitted over the Internet. Likewise, the file system provides the meaning of data stored on a specific media.

Mounting

In Linux, the process of associating a file system with a storage device is calledMount). UseMountCommand to attach a file system to the current file system hierarchy (Root ). During mounting, you must provide the file system type, file system, and a mount point.

To illustrate the functions of the Linux File System layer (and the mounting method), we create a file system in a file in the current file system. The implementation method is to first useDdCommand to create a file of the specified size (use/dev/zero as the source to copy the file)-in other words, a file initialized with zero, as shown in Listing 1.

Listing 1. Creating an initialized File

$Dd If =/dev/Zero of = file. img bs = 1 k count = 1000010000 + 0 records in10000 + 0 records out $

Now there is a 10 MB file. IMG file. UseLosetupCommand to associate a cyclic device with this file to make it look like a block device, rather than a regular file in the file system:

$Losetup/dev/loop0 file. img$

This file is now available as a block device (represented by/dev/loop0 ). Then useMke2fsCreate a file system on this device. This command creates a new ext2 File System of the specified size, as shown in Listing 2.

Listing 2. Create an ext2 file system with a circulating Device

$Mke2fs-C/dev/loop0 10000Mke2fs 1.35 (28-feb-2004) max_blocks 1024000, rsv_groups = 1250, rsv_gdb = 39 filesystem label = OS type: linuxblock size = 1024 (log = 0) fragment size = 1024 (log = 0) 2512 inodes, 10000 blocks500 blocks (5.00%) reserved for the super user... $

UseMountCommand (/Dev/loop0) The file. IMG file is mounted to the mount point/mnt/point1. Note: The file system type is specifiedExt2. After mounting, you can use this mount point as a new file system, suchLsCommand, see listing 3.

Listing 3. Create a mount point and mount the file system through a circulating Device

$Mkdir/mnt/point1$Mount-T ext2/dev/loop0/mnt/point1$Ls/mnt/point1Lost + found $

As shown in Listing 4, you can continue this process: Create a new file in the mounted file system, associate it with a cyclic device, and then create another file system on it.

Listing 4. Create a new cyclic File System in the cyclic File System

$Dd If =/dev/Zero of =/mnt/point1/file. img bs = 1 k count = 10001000 + 0 records in1000 + 0 records out $Losetup/dev/loop1/mnt/point1/file. img$Mke2fs-C/dev/loop1 1000Mke2fs 1.35 (28-feb-2004) max_blocks 1024000, rsv_groups = 125, rsv_gdb = 3 filesystem label =... $Mkdir/mnt/point2$Mount-T ext2/dev/loop1/mnt/point2$Ls/mnt/point2Lost + found $Ls/mnt/point1File. IMG lost + found $

Through this simple demonstration, you can easily see how powerful the Linux File System (and cyclic devices) is. You can create an encrypted file system on a cyclic device in the same way. You can use cyclic devices to temporarily Mount files as needed, which helps protect data.

Back to Top

File System Architecture

Now that we have seen the file system construction method, let's look at the Linux File System layer architecture. This article examines the Linux File System from two perspectives. First, use the high-level architecture. Then we will discuss the main structure of implementing the file system layer in depth.

Back to Top

High-level Architecture

Although most file system code is in the kernel (except for the user space file system discussed later ), however, the architecture shown in Figure 1 shows the relationship between the user space and the main components related to the file system in the kernel.

Figure 1. Architecture of Linux File System Components
 

The user space contains applications (such as file system users) and the gnu c library (glibc), which provide user interfaces for file system calls (open, read, write, and close. The function of a system call interface is like a switch, which sends system calls from the user space to an appropriate endpoint in the kernel space.

VFS is the main interface of the underlying file system. This component exports a set of interfaces and abstracts them to various file systems. The behavior of each file system may vary greatly. There are two caches for File System Objects (inode and dentry ). They cache recently used file system objects.

Each file system implementation (such as ext2 and JFS) exports a set of common interfaces for VFS. The buffer cache caches requests between the file system and related Block devices. For example, read/write requests to underlying device drivers are passed through the buffer cache. This allows you to cache requests to reduce the number of accesses to physical devices and speed up access. Manage the buffer cache in the form of a recently used (LRU) list. Note: You can useSyncCommand to send requests in the buffer cache to the storage media (forces all unwritten data to be sent to the device driver and then to the storage device ).

What is a block device? Block devices are devices that send and receive data in blocks (such as disk sectors). They support buffering and random access (you do not need to read blocks sequentially, but can access any block at any time) and other features. Block devices include hard drives, CD-ROM, and RAM disks. The character device is opposite to the block device. The character device does not have any media that can be physically addressable. Character devices include serial ports and tape devices that can only read data from these devices character by character.

This is the high-level situation of VFS and file system components. Now we will discuss the main structure for implementing this subsystem.

Main Structure

Linux treats all file systems from the perspective of a set of common objects. These objects are superblocks, inode, dentry, and files. The super block describes and maintains the state of the file system on the root of each file system. Each object (file or directory) managed in the file system is represented as an inode in Linux. Inode contains all the metadata required to manage objects in the file system (including operations that can be performed on objects ). Another group of structures is called dentry, which is used to map names and inode. A directory cache is used to save the recently used dentry. Dentry also maintains the relationship between directories and files to support moving in the file system. Finally, the VFS file represents an open file (Save the status of the opened file, such as the write offset ).

Virtual File System Layer

VFS serves as the root layer of the file system interface. VFS records the currently supported file systems and mounted file systems.

You can use a set of registration functions to dynamically add or delete file systems in Linux. The kernel stores the list of currently supported file systems. You can view this list in the user space through the/proc file system. This virtual file also displays the devices currently associated with these file systems. To add a new file system in Linux, callRegister_filesystem. The parameter of this function defines a file system structure (File_system_type). This structure defines the name of the file system, a set of attributes, and two super block functions. You can also log out of the file system.

When registering a new file system, the file system and its related information will be added to the file_systems list (see figure 2 and Linux/include/Linux/mount. h ). This List defines supported file systems. EnterCAT/proc/filesystemsTo view the list.

Figure 2. File System registered with the kernel
 

Another structure maintained in VFS is the mounted file system (see figure 3 ). This structure provides the mounted file system (see Linux/include/Linux/fs. h ). It links to the super block structure discussed below.

Figure 3. mounted file system list
 

Super Block

A super block structure represents a file system. It contains the information required to manage the file system, including the file system name (such as ext2), file system size and status, block device reference and metadata (such as idle list and so on ). A super block is usually stored in the storage media. If the super block does not exist, you can create it in real time. You can find the super block structure in./Linux/include/Linux/fs. H (see figure 4 ).

Figure 4. Super Block Structure and inode operations
 

An important element in a super block is the definition of a super block operation. This structure defines a group of functions used to manage inode in the file system. For example, you can useAlloc_inodeAllocate inodeDestroy_inodeDelete inode. AvailableRead_inodeAndWrite_inodeRead and Write inode,Sync_fsExecute file system synchronization. You can find it in./Linux/include/Linux/fs. h.Super_operationsStructure. Each file system provides its own inode methods to implement operations and provide general abstraction to the VFS layer.

Inode and dentry

Inode indicates an object in the file system, which has a unique identifier. Each file system provides a method to map a file name to a unique inode identifier and inode reference. Figure 5 shows a part of the inode structure and two related structures. Please noteInode_operationsAndFile_operations. These structures indicate the operations that can be performed on this inode.Inode_operationsDefines the operations directly executed on inode, whileFile_operationsDefine methods related to files and directories (standard system calls ).

Figure 5. inode structure and associated operations
 

Inode and the Directory cache respectively Save the recently used inode and dentry. Note that each inode in the inode cache has a corresponding dentry in the directory cache. You can find it in./Linux/include/Linux/fs. h.InodeAndDentryStructure.

Buffer cache

In addition to the implementation of each file system (which can be found in./Linux/Fs), the bottom of the file system layer is the buffer cache. This component tracks read/write requests from file system implementations and physical devices (through device drivers. To improve efficiency, Linux caches requests and avoids sending all requests to physical devices. Caches the recently used buffers (pages) in the cache, which can be quickly provided to various file systems.

Back to Top

Interesting file systems

This article does not discuss specific file systems available in Linux, but it is worth mentioning here. Linux supports many file systems, including older file systems such as minix, MS-DOS, and ext2. Linux also supports new log file systems such as ext3, JFS, and reiserfs. In addition, Linux supports encrypted file systems (such as CFs) and virtual file systems (such as/proc ).

The last noteworthy file system is filesystem in userspace (fuse ). Such a file system can send a file system request back to the user space through VFS. Therefore, if you are interested in creating your own file system, using fuse for development is a good method.

Back to Top

Conclusion

Share this article Article ......

	Submit this article to Digg
	Release to Del. icio. us
	Submit to Slashdot!

Although the implementation of a file system is not complex, it is a good example of a scalable and scalable architecture. The file system architecture has been developed for many years and has successfully supported many different types of file systems and many target storage device types. Thanks to the plug-in-based architecture and multi-layer indirect function, the recent development of Linux File Systems deserves attention.

References

Learning

• For more information, see the original article on the developerworks global site.

• The proc file system provides a new way to communicate between the user space and the kernel through the Virtual File System. "Use the/proc file system to access the content of the Linux kernel" (developerworks, November March 2006) introduced the/proc Virtual File System and demonstrated its usage.
• The Linux system can call the kernel API functions by passing control between the user space and the kernel. "Kernel commands used to call Linux systems" (developerworks, November 2007) introduced Linux System Call interfaces.
• Yolinux.com maintains a list of Linux File Systems, cluster file systems, and performance computing clusters. You can also find the complete list of Linux File Systems in file systems howto. Xenotime also describes many file systems.
• For more information about Linux user space programming, seeGNU/Linux Application Programming.
• In the developerworks Linux area, you can find more references for Linux developers and the most popular articles and tutorials.
• Read all the Linux tips and Linux tutorials on developerworks.
• Stay tuned to developerworks technical events and network broadcasts.

Obtain products and technologies

• Filesystem in userspace (fuse) is a kernel module that supports file system development in user space. The file system driver sends requests from VFS back to the user space. This is a good way to develop a file system without kernel development. If you are proficient in Python, you can also use lufs-python to write a file system in Python.

• Download the IBM product evaluation edition and try out application development tools and middleware Products from DB2, Lotus, rational, Tivoli, and websphere.

Discussion

• Use blogs, forums, podcast, andCommunityTopic. Join the developerworks community.

About the author

M. Tim Jones is an embedded software engineer. He isGNU/Linux Application Programming,AI Application ProgrammingAndBSD sockets programming from a multilanguage perspectiveAnd other books. His engineering background is very extensive, from synchronizing the kernel development of the spacecraft to the embedded architecture design, to the development of network protocols. Tim is a consultant engineer at Emulex Corp. In longmont, Colorado.