Lightweight Distributed File system Fastdfs use installation instruction manual (beginner entry level)

lightweight Distributed File System Fastdfs Use the Installation Instructions manual (Beginner entry level)

The research group of the laboratory is based on the study of cloud computing, but all the research is in the theory of imagination, lack of distributed environment of the platform practice, cloud computing God Horse is a cloud. Therefore, I want to use Hadoop to build a basic distributed experimental environment for the lab server, which makes it easy to test God's horse in the future. Inadvertently, saw an open-source lightweight Distributed File System Fastdfs, it is implemented in pure C language, detailed information can be found in the blog: Distributed File system Fastdfs architecture anatomy. Through the collection of information, do some of their own installation and test use, and now some of the previous experience of some summary.

First, Fastdfs the installation process

( 1 ) Preparatory work - installation libevent

Fastdfs internal binding libevent as an HTTP server, the v2.x version must be installed Libevent, this article installs the V2.2 version, so you must install Libevent. It is also recommended to download files in HTTP mode.

If Libevent is already installed, verify that the installation path is/usr, because Fastdfs will need to find some dependent files in this directory when compiling the source program, otherwise the compilation Fastdfs error. If not, it is recommended that you uninstall Libevent first and then install to/usr. Ubuntu10.4 default is to install the libevent, you can uninstall it in the Software center and then follow the way described below to install. This article installs the libevent1.4.14b-stable,:http://monkey.org/~provos/libevent/

Installation steps:

Tar vxzf libevent1.4.14b-stable.tar.gz

./configure--PREFIX=/USR

Make clean;

Make

Make install

Installation libevent complete.

( 2 ) Installation Fastdfs

1, download Fastdfs source program, http://code.google.com/p/fastdfs/downloads/list, download here is fastdfs_v2.04.tar.gz, It is recommended to install the v2.x version, which is more stable and performance-enhancing than the v1.x version),

2, in the Fastdfs_v2.04.tar.gz folder under the execution:

Tar vxzf Fastdfs_v2.04.tar.gz/home/monkey/fastdfs

Tar vxzf Fastdfs_v2.04.tar.gz/home/monkey/fastdfs

Note:/home/monkey/fastdfs can also be another directory, remember that this is the decompression directory, and later in%fastdfs% to express)

3, because to use the built-in HTTP service, so edit the%fastdfs%/make.sh file, find #with_httpd=1, modify to With_httpd=1 to support HTTP.

4. Enter the%fastdfs% directory and execute:

./make.sh

./make.sh

5, in the%fastdfs% directory, the implementation of:

./make.sh Install

./make.sh Install

If you run the command-line message at the end, you see a similar

#ln-fs/usr/local/lib/libfastcommon.so.1/usr/local/lib/libfastcommon.so

#ln-fs/usr/local/lib/libfdfsclient.so.1/usr/local/lib/libfdfsclient.so

Sh./fdfs_link_library.sh

#ln-fs/usr/local/lib/libfastcommon.so.1/usr/local/lib/libfastcommon.so

#ln-fs/usr/local/lib/libfdfsclient.so.1/usr/local/lib/libfdfsclient.so

Sh./fdfs_link_library.sh

Congratulations, the installation has been successful!

Second, Fastdfs Configuration instructions for

The Fastdfs configuration file is in the%fastdfs%/conf directory, which includes:

Client.confà Client Upload configuration file

Storage.confà file Storage Server configuration file

Tracker.confà responsible for balancing dispatch server configuration files

Http.confàhttp Server configuration file

The configuration file needs to be configured differently depending on the server environment, refer to the FASTDFS official installation configuration document Http://code.google.com/p/fastdfs/wiki/Setup, and the Fastdfs principle document HTTP ://code.google.com/p/fastdfs/wiki/overview

In this example, for convenience, Tracker Server,storage server is installed on a single server, ip:211.69.205.1.

( 1 ) configuration and startup Tracker Server

1, modify the%fastdfs%/conf/tracker.conf file, modified as follows:

#可以自己指定目录位置, but the directory must exist to store information such as logs and storage server, or tracker server cannot start.

Base_path=/home/yuqing/fastdfsàbase_path=/home/monkey/fastdfs_tracker

#改成你想要的http端口, it will be the port of the HTTP download file in the future.

http.server_port=8080àhttp.server_port=8090

#http支持

# #include Http.confà#include http.conf

#默认4GB, if the space is insufficient, it is recommended to small, otherwise it will be reported to no free space anomaly, unable to start normally.

Reserved_storage_space = 4gbàreserved_storage_space = 1GB

#tracker server to the port that storage server serves, use the default, or you can customize

port=22122

2. Copy the http.conf file to the/etc/fdfs directory and execute:

CP%fastdfs%/conf/http.conf/etc/fdfs/

CP%fastdfs%/conf/http.conf/etc/fdfs/

Note: In order to support HTTP, this file must be copied to this directory, otherwise it will not start, reported Param http.xxx not exist or is empty similar error.

3, enter the/usr/local/bin/directory, start the tracker server, execute:

Fdfs_trackerd%fastdfs%/conf/tracker.conf

Dfs_trackerd%fastdfs%/conf/tracker.conf

4. Enter/home/soar/fastdfs_tracker/logs/trackerd.log to view tracker's startup log, if you see similar:

[2010-11-26 21:33:44] Info-fastdfs v2.04, Base_path=/home/monkey/fastdfs_tracker, connect_timeout=30s, network_timeout=60s, port=22122, Bind_addr=, max_connections=256, work_threads=4, store_lookup=2, store_group=, store_server=0, store_path=0, reserved _STORAGE_SPACE=1024MB, download_server=0, Allow_ip_count=-1, sync_log_buff_interval=10s, check_active_interval= 120s, thread_stack_size=64 KB, Storage_ip_changed_auto_adjust=1, storage_sync_file_max_delay=86400s, storage_sync_ file_max_time=300s

[2010-11-26 21:33:44] Info-http supported:server_port=8090, Default_content_type=application/octet-stream, anti_steal_token=0, Token_ttl =0s, Anti_steal_secret_key length=0, Token_check_fail content_type=, Token_check_fail buff Length=0, check_active_ Interval=30, Check_active_type=tcp, check_active_uri=/status.html

Congratulations, Tracker server has started successfully!

( 2 ) configuration and startup Storage Server

1 , modify %fastdfs%/conf/storage.conf file, modified as follows:

#可以自定义, but this directory must exist to store storage related log, group-related information

Base_path=/home/yuqing/fastdfsà/home/monkey/fastdfs_storge

#文件的存储位置, you can specify multiple storage locations on a single storage server

Store_path0=/home/yuqing/fastdfsàstore_path0=/home/monkey/fastdfs_storge

#必须指定

Group_name=group1

IP and port information for #修改成tracker server

tracker_server=192.168.209.121:22122àtracker_server=211.69.205.1:22122

#http支持

# #include Http.confà#include http.conf

2, enter the/usr/local/bin/directory, start the storage server, execute

Fdfs_storaged%fastdfs%/conf/storage.conf

3. Enter/home/monkey/fastdfs_tracker/logs/storage.log to view the storage server startup log, if you see similar:

[2010-11-26 21:38:59] Info-fastdfs v2.04, Base_path=/home/monkey/fastdfs_storge, Store_path_count=1, Subdir_count _per_path=256, Group_name=group1, connect_timeout=30s, network_timeout=60s, port=23000, bind_addr=, Client_bind=1, max_connections=256, work_threads=4, disk_rw_separated=1, Disk_reader_threads=1, Disk_writer_threads=1, buff_size= 256KB, heart_beat_interval=30s, stat_report_interval=60s, Tracker_server_count=1, sync_wait_msec=200ms, sync_ Interval=0ms, sync_start_time=00:00, sync_end_time=23:59, write_mark_file_freq=500, Allow_ip_count=-1, File_ Distribute_path_mode=0, file_distribute_rotate_count=100, Fsync_after_written_bytes=0, sync_log_buff_interval=10s , sync_binlog_buff_interval=60s, sync_stat_file_interval=300s, thread_stack_size=512 KB, upload_priority=10, if_ alias_prefix=, check_file_duplicate=0, FDHT group count=0, FDHT server count=0, fdht key_namespace=, FDHT keep_alive=0, HT TP server port=8888, domain name=

[2010-11-26 21:38:59] Info-http supported:server_port=8888, http_trunk_size=262144, Default_content_type=application/octet-stream, anti_ Steal_token=0, token_ttl=0s, Anti_steal_secret_key length=0, Token_check_fail content_type=, Token_check_fail Buff Length=0

[2010-11-26 21:39:29] INFO-FILE:STORAGE_PARAM_GETTER.C, line:48, storage_ip_changed_auto_adjust=1

[2010-11-26 21:39:30] INFO-FILE:TRACKER_CLIENT_THREAD.C, line:274, successfully connect to tracker server 211.69.205.1:22122, as a tracker C lient, my IP is 211.69.205.1

Congratulations, Storage server has started successfully!

Third, Fastdfs use examples to illustrate

After the previous follow, configuration, start, immediately can see the effect of Fastdfs, we first upload a file, and then download through HTTP.

( 1 ) upload Files

FASTDFS installation package, with the client program, through the program can upload files. Before using this client program, you need to configure client.conf before uploading and downloading files.

1, modify the%fastdfs%/conf/client.conf file, modified as follows:

#可自定义, but this directory must be present to hold the file upload log

Base_path=/home/yuqing/fastdfsàbase_path=/home/monkey/fastdfs_tracker

tracker_server=192.168.209.121:22122àtracker_server=211.69.205.1:22122

http.tracker_server_port=8080àhttp.tracker_server_port=8090

# #include Http.confà#include http.conf

2, enter the/usr/local/bin/directory, upload files, execute:

Fdfs_test%fastdfs%/conf/client.conf Upload Monkey.txt

Note: Monkey.txt can create one yourself under the/usr/local/bin/directory.

If the command line feedback resembles the following, the upload file succeeds.

( 2 ) Download File

In the browser, enter the URL address in, tracker server automatically redirects to the storage server where the file is stored, and the file download succeeds. At this point, you have successfully set up the Fastdfs, write your own client to access it:)

Lightweight Distributed File system Fastdfs use installation instruction manual (beginner entry level)