Brief Analysis of Linux Log File System

Article Title: A Brief Analysis of the Linux Log File System. Linux is a technology channel of the IT lab in China. Includes basic categories such as desktop applications, Linux system management, kernel research, embedded systems, and open source.

In modern history, the log file system is considered very strange, mainly in the research stage. Now, the log file system (ext3) has become the default file system for Linux. This article reveals some ideas behind the log file system and how to provide better integrity when the power supply fails or the system crashes. In addition, this article also introduces several existing log file systems and the next generation of log file systems.

There are many ways to define a log file system, but let's grasp the key points. The log file system is designed for fsck (checking file system consistency) when tired of staring at startup) (The log file system is also applicable to those who want the file system to have fault recovery capabilities ). If the system uses a traditional file system that does not provide the log function, the operating system will use the fsck application for consistency check when detecting that the system is shut down abnormally. The application scans the file system (which takes a long time) and fixes any security fixes. In some cases, when the file system is seriously damaged, the operating system starts to the single-user mode for further repair.

Now you should know what type of crowd the log file system is targeting, But how do they ban fsck? In general, the log file system prevents file system crashes by maintaining a log. A log is a special file. It records modifications to the file system in a circular buffer and regularly submits the changes to the file system. Once the system crashes, the log file will act as a checkpoint to restore unsaved information and prevent damage to the file system metadata.

In summary, a log file system is a file system with fault recovery capabilities. It uses logs to record modifications that have not been submitted to the file system, to prevent metadata corruption (see figure 1 ). However, like many other Linux Solutions, the log file system provides multiple solutions for you to choose from. Next let's take a brief look at the history of log file systems, and then take a look at several existing file systems to see what is the difference between them.

Figure 1. Typical Log File System

Linux Log File System history

What is the earliest Log File System? Journaled File System (JFS ). JFS was first released in 1990, and the current version supported by Linux is JFS2 developed later. In 1994, Silicon Graphics introduced a high-performance XFS for the IRIX operating system. XFS was implanted into Linux in 2001. The smart File System (SFS) developed in 1998 was initially developed for Amiga, but was subsequently released under GNU Lesser General Public License (LGPL) and received Linux support in 2005. The most common Log file system ext3fs (third extended file system) is the extension of ext2, which adds the logging function. Since 2001, ext3fs has been supported in Linux. Finally, after being introduced, the ReiserFS log file system is widely used. However, due to some legal disputes of its original developers, the ReiserFS log file system has not been further developed.

Several log variants

The log file system uses logs to buffer modifications to the file system (which can also be applied to emergency fault recovery). However, different policies can be adopted based on the recorded time and content. Three Common strategies are: writeback, ordered, and data ).

In the write-back mode, only metadata is recorded in logs, and data blocks are directly written to the disk location. In this way, the file system structure can be saved to prevent the crash, but data crash may occur (for example, after the metadata is recorded in the log, the system crashes before the data block is written to the disk ). To solve this problem, you can use the reservation mode. In the reservation mode, only the metadata is recorded to the log, but data is written to the disk before this. This ensures data consistency with the file system after system recovery. The last mode also records the data to the log. In data mode, both metadata and data are recorded in logs. This mode can prevent file system crashes and data loss to the maximum extent. However, because all data is written twice (logs are written first and then to the disk), the system performance may be reduced.

There are also many different log submission policies. For example, when the log is full or after timeout?

Current Situation of Log File System

Nowadays, several log file systems are widely used. Each type has its own advantages and disadvantages. The following describes the four most common log file systems.

JFS2

JFS2 (also known as enhanced journaled file system) is the earliest Log file system. It was applied to ibm aix before being embedded in Linux? Operating system for many years. It is a 64-bit file system. Although it was developed on the basis of the original JFS, It is improved compared with that of JFS2, it also supports a multi-processor architecture.

JFS2 supports predefined logging methods to improve performance and restore file systems in sub-seconds. JFS2 also provides partition-based file allocation (Extent-based allocation) to improve performance ). Partition-based allocation refers to the allocation of a group of continuous blocks instead of a single block. Because these blocks are continuous on the disk, the Read and Write performance will be better. Another advantage of such allocation is that metadata management can be minimized. Allocating disk space by block means updating metadata one by one. When partitions are used, metadata is updated only by partitions (which can represent multiple blocks.

JFS2 also uses the B + tree for faster Directory Search and partition descriptor management. JFS2 does not have an internal log submission policy, but is committed when the kupdate daemon times out.

XFS

XFS is one of the other early log file systems developed for the IRIX operating system by Silicon Graphicsis in 1995. It was implanted in Linux in 2001, so it is mature and reliable.

XFS supports 64-bit full address addressing and provides high performance for directory and File Allocation Based on the B + tree. XFS also uses partition-based allocation and supports variable block size (from 512 bytes to 64 KB ). In addition to partitions, XFS also uses latency allocation, that is, when a block is written to a disk, it is allocated disk space. In this way, the total number of required disk space is clear at a glance. Therefore, this function increases the possibility of allocating continuous disk blocks.

XFS also has some other interesting features that ensure rate input and output (I/O-by retaining bandwidth for file system users) and direct I/O. Data is directly copied in the buffer range of the disk and user space (instead of entering from multiple buffers ). XFS adopts the write-Back log policy.

Third extended File System (ext3fs)

Third extended file system (ext3fs) is the most popular Log file system and evolved from the ext2 file system. In fact, Ext3fs can be compatible with ext2fs, because ext3fs uses the same structure as ext2fs and only has one more log. We can even mount part of ext3fs as an ext2 file system, or convert the ext2 file system to an ext3 File System (using the tune2fs utility ).

Ext3fs allows you to record logs (write-back, reservation, and data) in three modes, but the default mode is the subscription mode. The log submission policy is configurable, but by default, the log is submitted when the log fills up 1/4 or when one of the commit timers times out.

One of the major drawbacks of ext3fs is that it was initially not designed as a log file system. It is developed based on ext2fs and lacks some advanced features (such as partitions) of other log file systems ). In terms of performance, it is inferior to ReiserFS, JFS, and XFS, but it requires less CPU and memory than similar solutions.

ReiserFS

ReiserFS is a log file system developed from the very beginning based on the log intent. ReiserFS was introduced to the mainstream 2001 kernel in 2.4 (the first log file system used in Linux ). The default logging method is subscription, and the file system can be scaled online. ReiserFS also has the tail packing function, significantly reducing disk fragments. In terms of processing small files, ReiserFS is faster than ext3f (when tail packing is available ).

ReiserFS (also called ReiserFS v3) has many advanced functions, such as B + tree. The basic format of the file system is based on a single B + tree, which improves the search efficiency and scalability. The submission policy depends on the log size, but must be based on the number of blocks to be submitted.

ReiserFS also encountered several problems-mostly recently, which is directly related to some legal disputes with developers.

The future of Log File Systems

Now that you know the current (and past) log file system, let's take a look at its development trend.

Reiser4

After successfully merging ReiserFS into the Linux kernel and adopting it in many Linux distributions, Namesys (the company that developed ReiserFS) began to develop a new log file system. Reiser4 is designed as a new log file system with many advanced features.

Resier4 is designed to achieve better Logging through wandering logs and delayed block allocation until the log is submitted (as in XFS. Reiser4 also has a flexible plug-in Architecture (to support features such as compression and encryption), but is rejected by the Linux community because these are in the virtual file system (VFS) as the best feature.

Due to the persistence of the Namesys owner, all commercial activities on Reiser4 are stopped.

Fourth, expand the File System

The fourth extended Log file system (ext4fs) evolved from ext3fs. The Ext4 file system is designed for forward and backward compatibility, but it has many new advanced features (some of which compromise compatibility ). This means that you can mount part of ext4fs as ext3fs, and vice versa.

First, ext4fs is a 64-bit file system designed to support a large capacity (1 exabyte ). It can also use partitions, but this will lose compatibility with ext3fs. Like XFS and Reiser4, ext4fs also supports allocating blocks in a delayed manner when necessary (this can reduce disk fragments ). The log content has been checked and (checksum), making the log more reliable. Ext4fs does not use the standard B + or B * tree. Instead, it is a variant of the B tree, called the H tree. It supports larger subdirectories (the ext3 ceiling is 32 KB ).

Although the delayed allocation method can reduce disk fragments, a large file system may become fragments after a long time. To solve this problem, an online disk fragmentation tool (e4defrag) was developed ). You can use this tool to organize a single file or the entire file system.

Another interesting difference between ext3fs and ext4fs lies in the date resolution of the file. In ext3, the minimum timestamp resolution is 1 second. Ext4fs is for the future: At that time, the speed of the processor and interface will continue to accelerate, requiring higher resolution. Therefore, the minimum resolution of the timestamp in ext4 is 1 nanosecond.

Ext4fs has been merged into Linux kernel since 2.6.19, but it is still not stable enough. The development of the next generation system will continue to be dedicated to this. With the advantages of the above generation, it will be the next generation of Linux Log File System.

Conclusion

The log file system provides reliability during system crash or power failure and prevents system crash. In addition, compared with traditional file system methods (such as those that depend on fsck), log file systems greatly shorten the recovery time of system crashes. The development of new logging functions should count on new algorithms and structures in the future, and rely on previous algorithms and structures to combine JFS and XFS functions. The future development of log file systems remains unknown, but it can be determined that they will be more practical and will become a new log file system standard.