How to build Distributed File system using FASTDFS in C # and. Net

Background

Mass storage, System load migration, server throughput bottlenecks, and so on, make the file system independent of the business system to improve the scalability of the entire project and maintainability

Current mainstream program MFS Fastdfs GFS lustre Hadoop et cetera

I chose Fastdfs to say "free, fast, and find" in a slogan. Fastdfs's author is Taobao's senior architect Yuqing, very witty, very love!!! Other options have not been played for the time being without comment.

Brief introduction　

Fastdfs is an open source lightweight Distributed File system pure C implementation, support Linux, FreeBSD and other Unix systems such as Google FS, not a common file system, only through proprietary API access, currently provides the official C, Java and PHP API for Internet applications tailored to the pursuit of high-performance and high scalability, Fastdfs can be seen as a file-based key value pair storage system, called Distributed File storage services more appropriate.

Characteristics:

Packet storage, flexible and concise

Peer structure, no single point

File IDs are generated by Fastdfs as file access credentials. Fastdfs does not require a traditional name server

Seamless convergence with popular Web servers, FASTDFS has provided Apache and nginx extension modules

Large and small files can be very good support, support mass small file storage

File attached properties can be saved on the storage server

Noun Explanation:

Tracker server: Tracking servers, mainly to do scheduling work, in the access load balanced role. Recording state information for group and storage servers in the cluster in memory is a hub for connecting client and storage servers. Because all of the relevant information is in memory, the performance of Tracker server is very high, with 3 units in a larger cluster (such as hundreds of groups) sufficient.

Storage Server: Storage server, file and file attributes (meta data) are saved to the storage server.

Practice-Service side

System: Ubuntu

Development tools: Vim

Web Services: Nginx

Custom communication protocol based on socket

Service-side Installation Reference official documentation I don't know, we can contact bugs here, 2 more places, one is a libevent version problem, and the other is the storage location for Libpthread libraries in the latest Ubuntu version.

After installing FASTDFS, assume your server program is installed in the/usr/local directory

We will find the following files in the bin directory

Storage Server startup Command/usr/local/bin/fdfs_storaged/fastdfs/conf/storage.conf

Tracker Server startup command/usr/local/bin/fdfs_trackerd/fastdfs/conf/tracker.conf

We run monitor to view the configuration information

Group count:1 Group 1:group name = Test free space = 5 GB Storage Server count = 2 Active Server count = 2 Storag
     
    E_port = 23000 storage_http_port = 0 Store Path count = 1 subdir count per path= 3 Current Write Server index = 0
        Host 1:ip_addr = 192.168.234.139 (Ubuntu) ACTIVE Total storage = 9GB free storage = 5GB Total_upload_count = 2 Success_upload_count = 2 Total_set_meta_count = 0 Success_set_meta_count = 0 Total_delete_count = 0 Success_delete_count = 0 total_download_count = 0 success_downl
        Oad_count = 0 Total_get_meta_count = 0 Success_get_meta_count = 0 Total_create_link_count = 0 Success_create_link_count = 0 Total_delete_link_count = 0 success_delete_link_count = 0 last_h Eart_beat_time = 2012-01-05 18:45:50 last_source_update = 2012-01-05 01:20:28 last_sync_update = 1969-12-3 1 16:00:00 last_synced_timestamp= 1969-12-31 16:00:00 Host 2:ip_addr = 192.168.234.140 ACTIVE Total Storage = 18GB Free storage = 12GB Total_upload_count = Success_upload_count = Total_set_meta_count = 0 Success_set_meta_count = 0 Total_delete_count = 0 success_delete_count = 0 total_downl
        Oad_count = 0 Success_download_count = 0 Total_get_meta_count = 0 Success_get_meta_count = 0 Total_create_link_count = 0 Success_create_link_count = 0 total_delete_link_count = 0 success_d
        Elete_link_count = 0 Last_heart_beat_time = 2012-01-05 18:45:50 last_source_update = 2012-01-05 01:54:02 Last_sync_update = 1969-12-31 16:00:00 last_synced_timestamp= 1969-12-31 16:00:00

This article URL address: http://www.bianceng.cn/Programming/csharp/201410/45480.htm