The Linux File System is just a penny

Linux File System-Linux general technology-Linux programming and kernel information. The following is a detailed description. I am often asked by potential customers how they can meet high I/O requirements if they can use Linux instead of AIX or Solaris. No one has ever asked me about high performance I/O on Windows or NTFS-high IOPS (input/output per second) or high I/O Stream, because this is impossible. Since its release 10 years ago, Windows and NTFS file systems have not changed much. Given its current architecture, it cannot be expanded. The NTFS file system layout, allocation method, and structure cannot effectively support multiple TB file systems, not to mention PB-level file systems. This is not surprising because it is not Microsoft's target market.

So what was the initial target market for Linux? Of course, it is to replace Microsoft's desktop environment. Later, Linux moved from running on the desktop to running on many large SMP (symmetric multi-processor structure) servers from Sun, IBM, and SGI. But Can Linux, as an operating system, and its Linux File System, cope with the challenges from high-performance I/O?

You may think that you do not need high-performance I/O, but even for simple backup and recovery operations, each server also needs this type of I/O performance. Current LTO-4 (linear tape open protocol) tape drives can run at a rate of MB per second without compression and support up to MB per second in case of compression. If your file system does not support this streaming data rate I/O, backup and recovery will take more time than expected. For large environments with many tape drives, the inability to use these tape drives at full data rates means that more tape drives are needed to meet the backup time window requirements, the Restoration Operation is also affected. Therefore, I think everyone should be interested in the performance of the Linux File System even if it is only for backup and recovery.

So Linux File System-My definition is in ext-4, XFS, and xxx-is it possible to achieve the performance of other UNIX-based file systems for large SMP servers such as Sun and IBM? Some may also ask SGI, but SGI has a software package named ProPack, which contains a series of optimizations for Linux high-speed I/O, in addition, SGI also has its own CxFS open private domain Linux File System, which is not part of the standard Linux product. Because SGI ProPack and CxFS are not part of the standard Linux product, we will not consider them here. We stick to the standard Linux because most people use it.

We will focus on two areas:

Linux and Linux file systems.

Linux operating system problems

We will not consider what may happen in the future of Linux for the moment, but focus on the existing situation. Linux has some functions to achieve I/O performance of AIX and Solaris, such as direct I/O, but the bottom line is that Linux is not designed around high-performance multi-thread I/O.

Some fields limit the performance of Linux, such as the page size compared with other operating systems, and Linux's restrictions on direct I/O and page alignment, in addition, Linux cannot perform direct I/O Based on the Request size automatically. I have seen that the Linux kernel splits large I/O requests (larger than KB) into kb requests. This is not surprising because Linux I/O performance and file systems were originally designed to replace Windows desktops.

As far as I can see, there are other issues in Linux. For beginners, there is a lack of persons responsible. In Linux, if you find a problem, there will be people in various groups to agree to solve the problem, but the people who wrote Linux didn't necessarily respond to your problem. If a large Linux vendor agrees to your issue and provides a solution, this does not mean that this approach will soon be accepted by the -- Linux community. In addition, patches for your problems will also cause maintenance problems.

The Linux File System and Linux kernel seem to be designed to solve completely different problems than AIX or Solaris, and if you have problems, IBM and Sun are more directly responsible than the Linux community. If you run AIX or Solaris and complain to IBM or Sun and report problems, they cannot be said that we have no control.

Linux File System

Remember, most Linux file systems are developed to replace NTFS, rather than high-performance file systems such as GPFS (IBM), StorNext (kunteng), and QFS (Sun. These file systems are designed for streaming I/O. We now know that streaming I/O is important for everyone, for some high-speed IOPS, and in some cases for database access.

The Linux File System (now ext-3, and soon there will be ext-4 and xfs) that is commonly used today does not have major structural changes for a long time. Ext-4 improves some allocation on the basis of ext-3 and ext-2, but does not consider some simple things, such as RAID (Redundant Array of Independent Disks) Alignment with super locks, and the first metadata allocation.

In addition, such operations as aligning the metadata fields of the additional file system with RAID band values are not considered, and functions similar to indirect distribution are not available. Such allocation is a fixed value, therefore, if the system supports small allocation (up to 4 kb), a large number of allocation is required. Taking a 537 billion TB file system as an example, if the ext-3 uses a allocation value up to 4 kb, such a file system would require 200 distributions to represent TB. You can use an 8 Mb allocation value in an enterprise-level file system to reduce the allocation quantity to 26.2 million manageable ones. Use the allocation value of this number, bitmap or allocation table, or even smaller than or equal to memory! Xfs file systems have very similar features as ext-3. Yes, it can be allocated larger to 64 KB, but the super lock, metadata domain alignment and other problems still exist.

Linux has its own location

My previous comments do not mean that I am opposed to Linux, just as I am not inclined to AIX or Solaris. I don't even oppose Windows, because I use a Windows laptop as my main computer. However, I think the default Linux File System is not enough to replace the high-performance, high-scalability SMP file system. Computers are tools, while operating systems and file systems are also tools in the toolbox. No one uses chainsaw in the case of an electric saw. The same metaphor applies to operating systems, file systems, and the hardware they run.

Many of my daily contacts use MS Word, MS Excel, MS PowerPoint, and MS Visio. Even if not all, I can still run some Windows applications from others, but I often find problems such as font mismatch, so I decided to use Windows a long time ago, unless someone can prove it to me. My opinion here is that every computer is a tool and has its own purposes. Currently, no single computer or file system can meet the requirements of all applications. This is not surprising. Linux has a place, but for me, it does not support single instances of large file systems, and does not support good expansion of large and small file systems with high performance requirements. I don't think this situation will change very quickly.

Source: dostor