Experts help you resolve Linux file systems

Some users often ask about high-performance I/O Requirements & mdash; if they can use Linux systems instead of AIX or Solaris operating systems. But no one has ever asked Windows or NTFS

Some users often ask about high-performance I/O-if they can use Linux systems instead of AIX or Solaris operating systems. However, no one has ever asked about high-performance I/O (high IOPS or high-streaming I/O) in Windows or NTFS, because this is impossible.

Windows and NTFS file systems have not changed much since they appeared 10 years ago. they are unlikely to expand due to their own structural limitations. NTFS file systems cannot effectively support multi-TB file systems, not to mention PB-level file systems, because they are tied to the same layer, allocation policies, and architecture organization, it is not surprising that NTFS has not become Microsoft's target market.

So what is the initial target market for Linux? Of course, Microsoft desktop is replaced. Linux has been gradually transferred from running on the desktop to running on large SMP servers of Sun, IBM, and SGI. But can the Linux operating system and Linux file system meet the challenges of high-performance I/O?

You may think that you do not need high-performance I/O, but each server needs this I/O performance to meet simple requirements such as backup and recovery. Existing LTO-4 tape drives can run at a speed of 120 MB (no compression) per face, supporting a data rate of MB per face without compression. If your file system does not support the I/O required for stream data transmission, the subsequent backup and recovery may take longer than you expected. For large environments with multiple tape drives, if the tape drives do not work at full data rates, more tape drives are required to meet the backup time window, this will also affect the recovery process. Therefore, when it comes to backup or recovery, it seems that everyone is more interested in the performance of the Linux file system.

Does the Linux file system meet the file system requirements of other large Unix servers? Some may ask SGI for this question, but SGI has a software named ProPack, which provides high-speed I/O optimization for Linux, in addition, SGI also has an open and special Linux file system named CxFS, which is not part of the standard Linux suite. Because SGI ProPack and CxFS are not part of the standard Linux suite, we will not talk about them here. We will mainly talk about standard Linux systems, because most of us know this system.

We will mainly look at two points:

1. Linux operating system

2. Linux file system

Linux operating system

Instead of considering the future development of Linux, we will focus on the present. Linux has a series of features that can meet the I/O performance of AIX and Solaris systems (such as direct I/O), but Linux is not designed based on high-performance multi-thread I/O.

There are many restrictions on Linux performance, for example, the page size compared with other operating systems, Linux's restrictions on direct I/O and page adjustment, and Linux's failure to automatically directly connect to I/O through the size of I/O requests -- I once found that the Linux kernel will be larger (larger than 512 MB) the I/O request is decomposed into kB. It is not surprising that the Linux I/O performance and file system were initially designed to replace the Windows operating system.

There are other issues in Linux, and there is a lack of dedicated personnel for the new users. If you find a problem, many people say they will fix it, and those who write Linux programs may not respond to your problem. If a Linux vendor promises to fix your fault, this does not mean that the Linux department will accept their promise or accept it immediately. Patching faults may cause maintenance problems.

The Linux file system and Linux kernel should be designed to solve some faults that are completely different from those of AIX or Solaris. if you find any fault, you should contact the Linux department instead of IBM or Sun. If you are using AIX or Solaris and ask IBM or Sun for help, they will not say there is no solution.

Linux file system

Remember, most Linux file systems are designed to replace NTFS rather than GPFS (IBM), StorNext (Kunteng), or QFS (Sun. These file systems are mainly designed for streaming I/O. We now know that streaming I/O is very important for high-speed IPOS and database reading.

Currently, common Linux file systems (EXT3, EXT4, and XFS) have not undergone major structural changes for a long time. EXT2 and EXT3 are upgraded to EXT4 in some aspects, but simple features such as RAID strip's super block and the first metadata allocation have not changed.

In addition, for example, the metadata zone and indirect distribution of the attachment file system with RAID strip values, indirect distribution supports a small allocation (up to 4 kB) and a large allocation of fixed values. Taking a 53.7 billion TB system as an example, it uses the 4 KB partition supported by EXT3 to implement a TB system consisting of partitions. An 8 MB enterprise file system is 26.2 billion manageable partitions. This chart can even support the same number of partitions in the memory. The XFS file system is similar to EXT3. if The Linux page size is 64 KB, the partition size can reach 64 KB, but there are still super block adjustment, metadata zone and other problems.

Linux occupies a place

This is not to say that I am an opponent of Linux, because I am not a supporter of AIX or Solaris. I am not even opposed to Windows, because I use Windows interfaces on my computer. However, I believe that the Linux file system is not enough to replace the high-performance, high-scalability SMP file system. Computers are tools, while operating systems and file systems are also tools. No one will replace the vertical saw with a chain saw. this analogy also applies to operating systems, file systems, and hardware devices where they run.

I know that many people use Microsoft Word, Excel, PowerPoint, and Visio every day. If none of these applications can be run in a Windows-like operating system, I may use some of them, but I find some of them are incompatible, I decided to use Windows after someone proved to me that it was compatible a long time ago. Here I think every computer is a tool and has its own purpose. No computer or file system can meet the needs of all applications. This is not surprising. as far as I know, Linux does not support single instances of large file systems and expansion from small file systems to large file systems to meet high performance requirements. And I think this will not change in a short period of time.