Fastdfs An Efficient Distributed File System

Fastdfs is an open-source lightweight distributed file system that manages files, including file storage, file synchronization, and file access (file upload and download, it solves the problems of large-capacity storage and load balancing. It is especially suitable for online services with files as the carrier, such as photo album websites and video websites.

The fastdfs server has two roles: tracker and storage ). The tracker is mainly used for scheduling and serves as a server Load balancer for access.

The storage node stores files and implements all the functions of file management: storage, synchronization, and access interfaces. fastdfs also manages the meta data of files. The meta data of a file is related to the file. It is expressed in key value pair mode, for example, width = 1024, where key is width and value is 1024. The object meta data is a file property list that can contain multiple key-value pairs.

 

Shows the fastdfs system structure:

Both the tracker and storage nodes can be composed of multiple servers. Servers in the tracker and storage nodes can be added or removed at any time without affecting online services. All servers in the tracker are equal to each other and can be increased or decreased at any time according to the pressure on the server.

To support large capacity, storage nodes (servers) are organized by means of volumes (or groups. The storage system consists of one or more volumes. The files between the volumes are independent of each other. The file capacity of all volumes is the file capacity of the entire storage system. A volume can be composed of one or more storage servers. The files on the storage servers under a volume are the same. Multiple storage servers in the volume play the role of redundant backup and load balancing.
When a server is added to a volume, the system automatically synchronizes the existing files. After the synchronization is completed, the system automatically switches the new server to the online service provider.

When the storage space is insufficient or is about to run out, you can dynamically Add a volume. You only need to add one or more servers and configure them as a new volume to expand the storage system capacity.
The file ID in fastdfs is divided into two parts: the volume name and the file name. Both are indispensable.

 

Fastdfs File Upload

Interaction Process of uploaded files:
1. The client asks the tracker about the storage to be uploaded. No parameters need to be attached;
2. Tracker returns an available storage;
3. The client directly communicates with storage to complete file upload.

 

Fastdfs File Download

Interaction process of downloading files:
1. The client asks the tracker about the storage of the downloaded file. The parameter is the file ID (volume name and file name );
2. Tracker returns an available storage;
3. The client directly communicates with storage to complete file download.

 

It should be noted that the client is the caller using the fastdfs service, and the client should also be a server. Its calls to tracker and storage are both inter-server calls.

 

Google Code: http://code.google.com/p/fastdfs/

Google Code: http://code.google.com/p/fastdfs/downloads/list