Use ParallelNFS to enhance the File System

Article Title: Use ParallelNFS to enhance the file system. Linux is a technology channel of the IT lab in China. Includes basic categories such as desktop applications, Linux system management, kernel research, embedded systems, and open source.

Read/write speed of 1 Gbit/s

Network File System (NFS) is an important part of most LAN systems. However, NFS is not suitable for demanding input/output-intensive applications in high-performance computing, at least previously. The latest modifications to the NFS standard are incorporated into Parallel NFS (pNFS), which is a Parallel implementation of file sharing and increases the transmission rate by several orders of magnitude. This article provides an entry-level introduction.

With NFS (composed of servers, client software, and protocols between the two), a computer can share a physical file system with other computers in the same network. NFS hides the implementation and type of the file system of the server. For applications running on NFS clients, the shared file system looks the same as local storage.

Figure 1 shows how to deploy NFS in a network that contains various operating systems. Which of the following operating systems support NFS-standard Linux? Mac OS X and Windows ?. (NFS is the only file system supported by Internet Engineering Task Force ).

Figure 1. Simple NFS configuration

In Figure 1, the Linux Server is an NFS server, which shares or exports (in NFS) one or more physical attached file systems. Mac OS X and Windows are NFS clients. They use or mount the shared file system respectively. In fact, mounting an NFS file system is the same as mounting a local drive partition-during mounting, the application only reads and writes files based on access control, but does not pay attention to the skills required to persist data.

For file systems shared through NFS, Read and Write operations-represented by a blue shadow-traverse from the client (here a Windows machine) to the server. The server finally executes a request for obtaining or storing persistent data or a request for modifying file metadata, such as the permission or the last modification time.

NFS is very powerful and can be seen from the extensive use of it as Network Attached Storage (NAS. It can run on Transmission Control Protocol (TCP) and User datasync Protocol (UDP) and is relatively easy to manage. In addition, the latest licensed version of NFS is NFS version 4, which improves security and enhances Windows and UNIX-like versions? Interoperability between systems, and enhanced exclusivity through lock lease. (NFSv4 was first approved in 2003 ). NFS infrastructure is not expensive because it can run well on common Ethernet hardware. NFS can solve most of the problems.

However, NFS processing of high-performance computing (HPC) has not been ideal. High-performance computing involves a huge volume of data files, and thousands of NFS clients may exist. (Think about a computing cluster or grid with thousands of computing nodes ). Here, NFS is a burden because the limitations of NFS servers-such as bandwidth, storage capacity, and processor speed-limit the overall computing performance. NFS becomes a bottleneck here.

Or, at least previously.

The next modified version of NFS is version 4.1, which includes an extended Parallel NFS (pNFS). It combines the advantages of common NFS with the high transfer rate of Parallel input/output (I/O. When using pNFS, the client can share the file system from the server as before, but the data does not pass through the NFS server. On the contrary, the client system will be directly connected to the data storage system, providing many parallel high-speed data paths for large-scale data transmission. After a brief process of initialization and handshaking, The pNFS server begins to exit the "stage" without compromising the transmission rate.

Figure 2 shows a pNFS configuration. The top is the node of the computing cluster, such as a large number of cheap Linux-based Blade Server clusters. The left side is the NFSv4.1 server. (For convenience, we call it a pNFS server ). At the bottom is a large parallel file system.

Figure 2. Concept Structure of pNFS

Like NFS, The pNFS server also exports the file system and retains and maintains the standard metadata that describes each file in the data warehouse. The pNFS client is the same as NFS-here is a node in the cluster-mount the file system exported by the server. Similar to NFS, each node regards the file system as a local physical attached file system. Metadata changes are transmitted to the pNFS server over the network. However, unlike NFS, pNFS directly operates between nodes and storage systems when reading or writing data, as shown at the bottom of 2. Removing a pNFS server from a data transaction provides a significant performance advantage for pNFS.

Therefore, pNFS retains all the advantages of NFS and improves performance and scalability. The capacity of the extended storage system will hardly affect the client configuration, and the number of clients can be increased to improve the computing capability. You only need to synchronize the pNFS directory and storage system.

[1] [2] [3] Next page