Fastdfs an efficient Distributed file system

Colleague Happy_fish recently developed a very powerful, very fast open source Distributed File System-Fastdfs, using pure C development, execution is very efficient, able to solve the problem of large concurrency and distributed storage, simple and efficient, suitable for many do not want to use Hadoop or mogilefs Brothers are worth trying a distributed file system

Fastdfs is an open source Distributed File System, she manages the file, features include: file storage, file synchronization, file access (file upload, file download), etc., to solve the problem of mass storage and load balancing. Especially suitable for the document as the carrier of online services, such as photo albums, video sites and so on.

The FASTDFS server has two roles: Tracker (tracker) and storage node (storage). The tracker mainly does the dispatching work and plays the role of load balancing on the access.

Storage nodes store files, complete file management of all functions: storage, synchronization and provide access interface, FASTDFS at the same time to manage the meta data of files. The so-called file meta data is the file's related attributes, in the form of key value pairs (key value pair), such as: width=1024, where the key is Width,value 1024. File meta data is a list of file attributes that can contain multiple key-value pairs.

The FASTDFS system structure is shown in the following illustration:

Both the tracker and the storage node can be composed of one or more servers. The servers in the tracker and storage node can be added or offline at any time without affecting the online service. All servers in the tracker are equivalent and can be increased or decreased according to the server's pressure.

To support large capacity, storage nodes (servers) are organized by volume (or grouping). The storage system consists of one or more volumes, the files between the volumes are independent of each other, and the file capacity of all volumes is the file capacity of the entire storage system. A volume can consist of one or more storage servers, the files in a storage server under a volume are the same, and multiple storage servers in a volume play a role of redundant backups and load balancing.
When the server is added to the volume, the synchronized files are automatically completed by the system, and after synchronization is completed, the system automatically switches the new server to the online service.

Volumes can be added dynamically when storage space is low or is about to run out. Only one or more servers need to be added and configured as a new volume, which expands the capacity of the storage system.
The file identification in Fastdfs is divided into two parts: volume name and file name, both of which are indispensable.

Fastdfs File Upload

Upload file interaction process:
1. Client asked tracker upload to the storage, do not need additional parameters;
2. Tracker returns a usable storage;
3. Client direct and storage communication complete file upload.

Fastdfs File Download

To download the file interaction process:
1. Client Inquiry Tracker Download file storage, parameter for file identification (volume name and filename);
2. Tracker returns a usable storage;
3. Client direct and Storage communication complete file download.

What needs to be explained is that the client is the caller of the Fastdfs service, and the client should also be a server, and its calls to tracker and storage are calls between servers.

Click here to download the source code V1.0

Official website: http://www.csource.org
Project website: http://fastdfs.googlecode.com
Code Download: http://code.google.com/p/fastdfs/downloads/list