Deep analysis of the future of Pnfs and file systems

With future upgrades to file system protocols, such as NFS (including concurrent NFS, or PNFS), will NFS potentially replace many of the existing proprietary file systems?

Let's take digitalfilm tree for example, a company in Hollywood that provides post-processing and virtual effects services for the entertainment industry. The company employs products that include Apple Xsan, HP storageworks arrays, QLogic switches, and other storage vendors. They also use a hybrid operating system environment that handles user work on Macs and PCs.

Ramy Katrib, founder and chief executive of Digitalfilm Tree, said: "We need to be able to deal with more than 100TB of data in a non-linear process and manage the speed of our TV programs." StorNext let us not need to increase the number of employees can greatly improve the efficiency. ”

Let's take a look at some of the other high-performance products offered by Sun and NetApp before we analyze the future trends.

Sun Lustre

Sun says lustre is "currently the world's best scalable Parallel file system," Now six of the world's top ten supercomputers and 40% of TOP100 supercomputers are using this system.

Peter Bojanic, director of Sun Lustre, said: "The lustre file system can be extended to support petabytes of data within a namespace, providing cumulative performance beyond the 100gb/s for more than 25,000 users, including the Livermore Moore National Laboratory, High-performance applications at Oak Ridge National Laboratory and the San Diego National Laboratory, these users have high requirements for large file I/O and continuous high bandwidth. ”

In addition, the system is expanding in the fields of oil and gas, rich media and content distribution networks, which require mixed workloads for large or small files. One of the differences between lustre and other systems is that it can be used as a Linux based Open-source software. That's why you'll find that it integrates with products from other high-performance computing vendors, such as SGI, Dell, HP, Cray, and Terascala.

Lustre is a target based clustered file system, but it does not support the T10 OSD, and the underlying storage allocation management is also based on blocks. Its configuration includes Lustre MetaData server and lustre Object Storage Servers. The file operation bypasses the metadata server and accesses the object server using the parallel data path in the cluster. The server is matched for aid purposes. Lustre can be run on different types of networks, including IP networks and InfiniBand networks.

NETAPP WAFL

NetApp has a file system named WAFL (Write Anywhere file Layout) that consolidates CIFS, NFS, HTTP, FTP, Fibre Channel, and iSCSI, and can be compatible with NetApp data Ontap operating system. Because WAFL integrates RAID-DP (NetApp high-performance RAID-6), it is able to keep the system running in the event of a failure of one or two disk drives.

In addition, the system uses nonvolatile memory (NVRAM) to increase speed, and NVRAM allows storage access protocol targets to respond to modification requests before modifying the write disk. Requests are written to the log via WAFL, and the file system modifications are stored in volatile memory. When the volatile memory accumulation saves multiple modifications, WAFL collects the results and submits them to a "consistency point" that is called by NetApp (actually a snapshot) and writes the consistency point to the raid system connected to the WAFL file system.

"If the consistency point is not written to disk before hardware or software failure, then the data ontap,nvram log will be Eisler back to WAFL," said Michael, senior technology director for NETAPP NFS. The consistency point is then written to the disk. Most of NetApp's competitors have snapshot technology, but NetApp uses its underlying snapshot technology to develop features such as file system-level monitoring, backup integration, cloning, duplicate data deletion, and data retention, which are allocated between networked storage devices and Flexvol. ”

Flexvol is a volume that can share a storage pool with other flexible volumes. These volumes can be enlarged or reduced on demand-the freed space is recycled into the storage pool and used by other flexvol.

The future of the file system

Of course, not everyone needs high performance. There are other more common file system protocols, such as NFS and CIFS, as well as Sun Open source ZFS file systems running on the Solaris operating system, and even the global file system of Red Hat, which is also known as the System,gfs. It provides high-performance and data-sharing capabilities for Linux platforms.

Panasas chief marketing tube Len Rosenthal says: "An NFS based solution or San file system is not designed for environments that have a large number of users or server nodes and need access paths to large file shares." Normal NAS systems are sufficient for file systems, content-oriented applications of everyday e-mail or other documents, because they require transmission through a low-bandwidth, open network, so these types of files are generally small in size. ”

He believes that ZFS is a good local file system, but it does not have any parallel data transfer capabilities and is not designed for extended applications. He believes that the local file system introduced by Linux and UNIX vendors is of great value to DAS systems. At the same time, NFS will still be a standard for network file access, but will gradually transition to PNFS in the next few years.

Rosenthal said: "The future NFS 4.1 version will support PNFS, its development prospects are bright, because the PNFS is the first time in nearly 20 years, NFS the major performance upgrade." The prospects for filing systems with proprietary client software such as GPFS, EMC MPFSI, and lustre are unclear, as Pnfs will eventually eliminate the need for these file systems. ”

Then, PNFS will likely be the future of high-performance, shared file storage. Vendors such as Panasas, Sun, NETAPP, IBM, and EMC are actively working on the PNFS Standard (NFSv4.1). Panasas even said publicly that they would migrate their Directflow protocols to pnfs through software revisions.

Pnfs can implement parallel data transfer, create a standard user Access Protocol for NFS, and provide support for PNFS for all major Linux distributions in the future, as well as proprietary UNIX versions such as Solaris and Aix.

Greg Schulz, founder and senior analyst at Storageio Group, also shares that view. "For everyday common files and data sharing in the enterprise business environment, productivity will remain at NFS V3 or higher levels," he said. If your enterprise environment is sun-centric, you will be more concerned about ZFS as they migrate to a Linux environment. But other companies will consider whether PNFS has an applicability to their environment. ”

But Sun is not expected to launch their lustre file system in the near future. Now Sun is preparing for a product that combines common file systems with High-performance file systems.

"There is no single technology that can solve all the business challenges, but by integrating these technologies we can solve most of them," says Bojanic. Lustre and Pnfs will not replace all other file systems and protocols, but will provide highly scalable systems with their exceptional scalability. ”

That explains why Sun is working on lustre.

"We are currently engaged in research on ZFS, lustre systems running in Linux and Solaris, which will provide greater scalability and data integrity protection," Bojanic said. Future Sun's open storage products will be lustre, providing exceptional scalability for different proprietary protocols, including PNFS and Windows/cifs. ”