Main storage field: Linux storage ecosystem (1)

Linux is like a Swiss Army knife for file systems. It also provides a wide range of storage technologies for desktop systems and servers. In addition to file systems, Linux integrates world-class NAS and SAN technologies, data protection, storage management, cloud support, and solid state storage. Learn more about the Linux storage ecosystem and why it can become the dominant server market.

Linux means many things. Its strength lies in its ability to flexibly support completely different use modes. However, one of the most important advantages of Linux is that it is the main force in the storage field. For Linux and storage, we often think of directly attaching storage or the latest file system, but there are much more things about storage and Linux than we can see. The elements in Linux are not only stable but also high-end.

This article discusses various storage technologies that make Linux in the storage field center. Let's start from the bottom-that is, the storage Architecture-and then gradually develop the stack into features, file systems, and plans to be executed (see figure 1 ).

Figure 1. Storage stack explored in this article

Storage Architecture

Attaching storage to the platform is the key to the overall Storage Architecture. The three common architectures cover the vast majority of models:

○ Direct-attached storage (DAS)

○ Storage area network (SAN)

○ Network-attached storage (NAS)

Of course, Linux supports all three models and develops through changes that occur along with these models.

Figure 2 illustrates these models, focusing on file systems and storage locations. The DAS model contains direct storage attachments on the platform, representing the vast majority of storage usage. SAN isolates the storage from the platform and makes it accessible through one of the block storage protocols. Finally, NAS provides a similar architecture as SAN, but performs operations at the file level.

Figure 2. Main Storage Architecture

Directly attached storage

Linux supports a wide range of DAS interfaces, including the old standard for Advanced Technology Attachment (ATA)-electronic Integrated Drive [IDE] (Integrated Drive Electronics [IDE]) /ATA-parallel SCSI and Fiber Channel and new storage interfaces, such as serially connected SCSI (Serial Attached SCSI, SAS), Serial ATA (Serial ATA, SATA) and external SATA (external SATA, eSATA ). You will also find advanced storage technologies such as USB3 (Scalable Host Controller Interface, Extensible Host Controller Interface [xHCI]) and Firewire (Institute of Electrical and Electronics Engineers 1394 ).

Storage area network

SAN provides block-level storage consolidation to share it with some servers. The storage is locally displayed to the server, and the endpoint storage device can provide additional services (such as backup and replication) for the client device ).

SAN protocols and interfaces are extensive and diverse. You can find typical SAN protocols in Linux, such as fiber channel and iFCP ). There are also newer protocols, such as SAS, fiber Channel over Ethernet (FCoE), and Internet SCSI (iSCSI, for example, it is applicable to remote Direct Memory Access (RDMA-iSER) and scsi rdma Protocol (SRP). It extends SCSI through RDMA of Infiniband.

Ethernet, as a storage protocol, has been fully implemented in Linux. It illustrates the strength and flexibility of these methods. In addition, Linux fully supports 10 Gbit/s Ethernet (10-gigabit Ethernet, 10GbE) and allows the construction of high-performance SAN. You can also find the ATA over Ethernet (ATAoE) protocol, which extends the ATA Protocol through the ubiquitous Ethernet protocol.

Network attached storage

Last but not least, NAS. NAS is a combination of network storage for different types of clients to access files. The two most popular protocols fully supported in Linux are Network File System (NFS) and Server Message Block/universal Internet File System (Server Message Block/Common Internet File System, SMB/CIFS ).

Although the original SMB implementation is proprietary, it is designed in reverse order to be supported in Linux. Later SMB revisions were publicly recorded to allow simpler development in Linux.

Linux continues to develop various enhancements and extensions for NFS. NFS is now a state protocol that optimizes data and metadata separation and parallel data access. For more information about NFS development, see the link in references. Like an Ethernet-based SAN, The 10GbE support in Linux allows high-performance NAS libraries.

Other storage Architectures

Not all storage architectures are ideal for DAS, SAN, and NAS storage. Because Linux is open, it is easier to develop new technologies internally, Which is why you can find the latest cutting-edge technologies in Linux.

An object storage architecture is worth mentioning. Although it is not a new feature, it is very interesting. The object storage architecture separates files from their metadata and stores them independently (on their respective data and metadata servers ). This separation provides some advantages, such as minimizing the bottleneck of metadata (because you only need to locate and open files to interact with this server ). You can also perform parallel access to segment data on multiple data servers to improve performance. Object Storage is implemented in Linux in various ways, including support for Object Storage Device (OSD) specifications and support for Linux clUSTER (Lustre) and Extended Object File System (exofs.

There is a similar technology named content-addressing storage (CAS), which uses a data hash value to identify its name and address. This technique, also known as fixed-content storage, is useful because it is easy to identify duplicate data: This hashed column (if strong enough) it will be the same and allow simple deduplication. The Venti architecture supports this method in Linux (except for the Plan 9 version of Bell Labs ).