Popular Distributed File System Parade (1)

This article briefly introduces several distributed file systems. Popular distributed file systems include Lustre, Hadoop, MogileFS, FreeNAS, FastDFS, NFS, OpenAFS, MooseFS, pNFS, and GoogleFS.

BKJIA recommended for editing: Next-generation Linux File System

LustreWww.lustre.org)

Lustre is a large-scale, secure, and reliable Cluster File System with High Availability. It is developed and maintained by SUN. The main purpose of this project is to develop the next generation cluster file system, which can support more than 10000 nodes and several PB of storage systems.

Lustre is an open-source cluster file system. It adopts the GPL license protocol. Currently, in cluster computers, the increase in data exchange between computers and disks cannot keep pace with the growth of microprocessor and memory, this also reduces the performance of applications. A new Cluster File System Software increases the I/O speed, which may reduce the cost of purchasing storage devices and change the way enterprises buy storage, the cluster file system has been used in universities, laboratories and supercomputer research centers and is about to enter the general commercial computing market. The new cluster file system uses open-source lustre technology, developed by the Department Of Energy (Department Of Energy) and provided commercial support by HP. It significantly improves the speed of input/output I/O, and has already had some impact in universities, national laboratories, and super Computing Research Centers. In the next few years, it is very likely to enter the ordinary commercial computer field.

Running in linux, the development language c/c ++

Hadoophadoop.apache.com)

Hadoop is not just a distributed file system for storage, but a framework designed to execute distributed applications on a large cluster composed of general computing devices.

Hadoop architecture:

The authorization protocol is apache and the development language is java. The resource consumption is a little high.

MogileFswww.danga.com)

Mogile Fs is an open-source distributed file system. Its main features include:

1. Application Layer Components

2. No spof

3. Automatic file copy

4. Better reliability than RAID

5. run linux without the support of RAID nigukefs.

FreeNASWww.openqrm.org)

FreeNAS is a dedicated operating system for network attached storage (NAS) services (simplified version of FreeBSD ). Based on the m0n0wall firewall, the system provides disk management and RAID software to enable the user home to convert the PC to the NAS server, support FTP/NFS/RSYNC/CIFS/AFP/UNISON/SSH sourceforge.net/pro, so that people can reuse old hardware.

FastDFScode.google.com/p/fastdfs)

FastDFS is an open-source distributed file system that manages files, including file storage, file synchronization, and file access (File Upload and file download, it solves the problems of large-capacity storage and load balancing. It is especially suitable for online services with files as the carrier, such as photo album websites and video websites. The FastDFS server has two roles: tracker and storage ). The tracker mainly performs scheduling and plays a role in load balancing for access. The storage node stores files and completes all file management functions:

Store, synchronize, and provide access interfaces. FastDFS also manages the meta data of files. The meta data of a file is related to the file. It is expressed in key value pair mode, for example, width = 1024, where key is width and value is 1024. The object meta data is a file property list that can contain multiple key-value trees.

The development language c/c ++ runs in linux.