Linux is the Swiss Army knife of file systems, and it also offers a wide variety of storage technologies for both desktops and servers. Beyond the file system, Linux incorporates world-class NAS and SAN technologies, data protection, storage management, supp Ort for clouds, and solid-state storage. Learn more on the Linux storage ecosystem and why it's number one in server market share.
Linux is many things, and its power lies in it ability to flexibly support vastly different usage models. But one of the Linux's most important strengths are serving as the workhorse of the storage domain. Thinking about Linux and storage commonly conjures a image of direct-attached disks or the latest file system, but there ' s much more to storage and Linux than meets the eye. Elements in the Linux is not only stable but also cutting-edge.
This article explores the various storage technologies, the keep Linux at the center of the storage universe. Let's start at the bottom-namely, storage Architectures-and then work up the stacks to features, file systems, and Futures (see Figure 1). Figure 1. Storage stack for exploration in this article Storage architecture
How the storage attaches to the platform are key to the overall storage architecture. Three general architectures cover the vast majority of models:
Of course, Linux supports all three and have evolved with the changes that is occurring with these models. Other storage media
This article focuses mainly on disk storage, but it's difficult to ignore the breadth of other devices supported in Linux. From the quickly disappearing floppy drives, CD-ROM and DVD readers and writers, and even enterprise tape systems, Linux Can ' t be beat when it comes to mass storage device support.
Figure 2 illustrates the models, with a focus on the location of the file system and storage. The DAS model covers the direct attachment of storage to the platform and represents the vast majority of storage use. The SAN separates the storage from the platform and makes it accessible over one of a number of block storage protocols. Finally, NAS provides a similar architecture to the SAN and operates at the file level. Figure 2. Major storage Architectures direct-attached storage
Linux supports a large variety of DAS interfaces, including old standards like parallel advanced technology Attachment (at A)-integrated Drive Electronics [Ide]/ata-parallel SCSI, and Fibre Channel as well as new storage interfaces like serial a Ttached SCSI (SAS), serial ATA (SATA), and external SATA (ESATA). You'll also find advanced storage technologies such as USB3 (extensible Host Controller Interface [XHCI]) and Firewire (in Stitute of Electrical and Electronics Engineers 1394). Storage Area Network
The SAN provides consolidation of block-level storage so, it can be shared among a number of servers. The storage appears local to the servers, where the endpoint storage device could implement additional services for the Clie NT devices (such as backup and replication).
Protocols and interfaces for SANs is wide and varied. You can find the typical Sans protocols in Linux such as Fibre Channel as well as their extension over IP (IFCP). Newer protocols, such as SAS, Fibre Channel over Ethernet (FCoE), and Internet SCSI (ISCSI), is also present, as is more Domain-specific protocols like ISCSI Extensions for remote Direct Memory access (Rdma-iser) and the SCSI RDMA Protocol (S RP), which extends SCSI over RDMA for Infiniband.
The emergence of Ethernet as a storage protocol have been fully realized in Linux, as it illustrates the power and Flexibil ity of these approaches. Further, 10-gigabit Ethernet (10GbE) is fully supported in Linux, permitting construction of high-performance SANs. You can also find protocols like ATA over Ethernet (ATAOE), which extends the ATA protocol over the ubiquitous Ethernet PR Otocol. network-attached Storage
Least is NAS. NAS is a consolidation of storage through a network for access by heterogeneous clients at the file level. The most popular protocols, which is fully supported in Linux, is Network File System (NFS) and Server Message Bl Ock/common Internet File System (smb/cifs).
Although the original SMB implementation was proprietary, it is reverse-engineered to being supported in Linux. The later SMB revisions were openly documented to allow simpler development in Linux.
Linux have continued to evolve with the various enhancements and extensions made to NFS. NFS is now a stateful protocol and includes optimizations for data and metadata separation as well as data access parallel Ism. You can read more about the evolution of NFS using the links in Resources. As with ethernet-based SANs, 10GbE support in Linux enables High-performance NAS repositories. Other storage architectures
Not all storage architectures fit cleanly in the DAS, SAN, and NAS buckets. Because Linux is open, it makes it easy to develop new technologies within it, which are why can find the newest Bleedi Ng-edge Technologies in Linux.
One interesting storage architecture, which is isn't new but worthwhile to mention, is the object storage architecture. O Bject storage architectures split a file from it metadata and store them independently (on their respective data and meta Data servers). This split provides certain advantages, such as minimizing the metadata bottleneck (because interactions with this server is only required to locate and open a file). Performance can also is enhanced by striping the data through multiple data servers for parallel access. Object storage is implemented in a variety of ways within Linux, including support for the object storage Device (OSD) SPE Cification as well as within the Linux ClUSTER (Lustre) and Extended Object File System (exofs).
A similar technology exists called content-addressable storage (CAS) that uses A hash of the data to identify Its name and address. This technology, also known as fixed-content storage (FCS), is advantageous, because it's easy to identify duplic Ate data:the hash (if strong enough) would be is the same and permit simple de-duplication. The Venti architecture supports this approach and exists within Linux (in addition to the Plan 9 distribution of Bell Labs ). Storage services:logical Volume Management
Storage virtualization was once a feature unique to high-end Storage systems, but it's now a standard feature of Linux. One of the most important services available in Linux is the Logical Volume Manager (LVM). The LVM is a thin layer, sits above physical storage available in the underlying storage architecture (with Accompanyi ng User-space Tools) and abstracts it to one or more logical volumes that is simpler to manage. For example, while a physical disk cannot is resized, a logical volume can be resized to add or remove space from it.
With the ability-to-abstract physical devices into logical devices, LVM creates a number of other storage capabilities, SU Ch as read-only and read-write snapshots of volumes, data striping across volumes for performance (redundant array of Inde pendent disks [raid]-0), data mirroring across volumes (RAID-1), and migration of volumes (even while online) between Phys ical devices.
For data protection beyond mirroring, Linux includes md (which stands for multipledisks) and provides a rich Set of RAID functionality. This element implements software RAID functionality, supporting RAID-4 (striped data with a parity block), RAID-5 (striped Data with a distributed parity block), RAID-6 (striped data with distributed and dual-redundant parity blocks), and raid- (Striped and mirrored data).
The LVM relies on another storage component called the Device-mapper, which provides (among other features) the A Bility to Multipath. For example, in a SAN environment, there is commonly multiple storage interfaces into the SAN fabric. Multipathing is a feature this protects against the failure of a given path, ensuring that storage remains available as Lo Ng as a path exists to communicate with the endpoint. Storage Features
In the past few years, both relatively simple features has been added to the storage stack that address the evolution of T He storage ecosystem:
The first change addresses the use of commodity drives in enterprise storage settings. Although Enterprise-class drives (such as SAS drives) is reliable, SATA drives is built with different requirements and With cost as a major factor. For the reason, SATA drives can suffer from a problem known as silent data corruption, where errors can intro Duced and not detected when the data are read from the disk. To solve this problem and support for SATA drives in enterprise settings, data integrity codes is added to blocks on the disk (where the disk uses 520-byte sectors instead of the traditional 512-byte blocks). In addition, the drive itself can validate the data being written, so that it integrity code matches the data. In this, errors can is caught as they ' re written to the disk, instead of detecting the error later if nothing can be Done about it.
This mechanism was called the data integrity field (DIF), as shown in Figure 3, and represents an 8-byte Traile R that includes a cyclic redundancy check (CRC) over the block of data, a reference tag (typically a portion of the Logica L block addressing [LBA]), and an application tag, the application defines. The reference tag is useful for catching mis-writes of data to a incorrect block, where the application tag can be used T O Catch other errors in the software stack. For example, if a PDF document was written, the application tag could be set to a value indicating a special PDF tag. When the PDF was read, each of the blocks ' s application tag can be inspected to ensure and all specify the PDF tag. DIF is supported within Linux as of kernel version 2.6.27. Figure 3. DIF structure for a 512-byte sector growing support for SSDs
The introduction of SSDs is changing, the storage ecosystem in a number of ways. These disks remove some of the large latencies found in spinning disks and therefore provide a-to-maintain data flow t O and from the CPU. But SSDs is different from hard disk drives (HDDs) in that they is consumable. The storage within an SSD can is written a finite number of times (depending on the technology); Therefore, it's important to being as efficient as possible when writing data. To make matters worse, the SSD must internally shift data to minimize the introduction of errors in a process called g Arbage collection or wear-leveling. This process results in writes to the consumable storage and should therefore is minimized.
The
Another issue with SSDs and traditional storage are that a HDD didn ' t care whether data on disk was valid. If the file system invalidated the data, the data could remain on disk without any downside. This constraint does isn't exist with SSDs because of the wear-leveling requirement. For this reason, Linux now supports the ability of the file system to communicate discarded blocks to the SSD (as of Kerne L version 2.6.29). This ability allows the SSD to remove these blocks from wear-leveling processes and helps to increase the endurance of the Drive. File Systems
What truly sets Linux apart from the other operating systems are its vast library of file systems. In Linux, you can find traditional client file systems like the third extended file system (EXT3) and the fourth extended File System (EXT4), but you'll also find the state of the art in Distributed file systems, cluster file systems, and Paral Lel file systems. You can find new, cutting-edge file systems based around new ideas and addressing new problems in the storage domain, as W Ell.
In terms of cutting-edge file systems today, Linux supports both ZFS and Butter FS (BTRFS). These-file systems compete with one another and share the distinction of copy-on-write semantics (blocks is never WRI Tten in place). In addition, both file systems support data de-duplication, internal data protection (Raid-like protection), data and meta Data checksums, and other storage features (like snapshots).
Linux is home to many distributed file systems, as well. One example is Lustre, which are a massively parallel distributed file system that supports tens of thousand of nodes and S Cales to petabytes of storage capacity. Ceph provides similar functionality and, in the past year, is introduced into the Linux kernel. Other examples in Linux include GlusterFS and the general Parallel File System (GPFS).
You can find specialized file systems on Linux, as well, including log-structured file systems like the NEW implementation Log Structure file System (NiLFS (2)) and object-based file systems like Exofs. Because Linux finds itself in many use models, you'll also find file systems for resource-constrained uses (such as Embedd Ed Systems) as well as low-latency applications such as High-performance Computing (HPC). File systems in the embedded area include the yet another Flash file System version 2 (YAFFS2), the journaling Flash file System version 2 (JFFS2), and the unsorted Block Image File System (UBIFS). File systems in the HPC space include the Parallel NFS (PNFS), Lustre, and the GPFS. Linux Storage ahead
Linux is and would continue to being the target for file systems and general storage-because of its openness and larg E Community of developers.
One of the latest changes in storage are the use of remote services for cost-efficient storage of archive data. Known today as cloud storage, numerous vendors provide efficient and transparent access to remote, centralized St Orage with varying service level agreements (covering capabilities like protection and bandwidth). The examples include Ubuntu one and Dropbox. Another service, called SpiderOak, can be used to back up your local user directories to the cloud for a small FE E.
What other features might is on the horizon for Linux? Large sector sizes (moving beyond 512-byte sectors), thin provisioning to avoid reserved but unused capacity ( Where advertised storage exceeds the physical capacity), storage de-duplication (to maximize storage availability), and an Even more efficient storage stacks to exploit new speeds and efficiencies of drives like SSDs, perhaps? Whatever is coming in storage ecosystem evolution, Linux would be there first. Resources Learn
Start building with 50+ products and up to 12 months usage for Elastic Compute Service
1 on 1 presale consultation
24/7 Technical Support 6 Free Tickets per Quarter Faster Response
Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.
A comprehensive suite of global cloud computing services to power your business
Payment Methods We Support
