Installation and use of Fastdfs Distributed File System (single node)

Tracking Device: 192.168.4.121 (edu-dfs-tracker-01)

Storage server: 192.168.4.125 (edu-dfs-storage-01)

Environment: CentOS 6.6

User: Root

Data directory:/fastdfs (Note: Data directory According to your data disk mount path depends)

Install package:

Fastdfs v5.05

Libfastcommon-master.zip(a common C function library extracted from Fastdfs and FASTDHT)

fastdfs-nginx-module_v1.16.tar.gz

nginx-1.6.2.tar.gz

fastdfs_client_java._v1.25.tar.gz

Source Address: https://github.com/happyfish100/

Download Address: http://sourceforge.net/projects/fastdfs/files/

Official forum: http://bbs.chinaunix.net/forum-240-1.html

One, all tracking servers and storage servers perform the following actions

1, compile and install the required dependency pack:

# yum install make CMake gccgcc-c++

2, installation Libfastcommon:

(1) Upload or download libfastcommon-master.zip to/usr/local/src directory

(2) Decompression

# cd/usr/local/src/

# Unzip Libfastcommon-master.zip

# CD Libfastcommon-master

(3) Compiling, installing

#./make.sh

#./make.sh Install

Libfastcommon is installed by default.

/usr/lib64/libfastcommon.so

/usr/lib64/libfdfsclient.so

(4) Because the Lib directory set by the FASTDFS main program is/usr/local/lib, you need to create a soft link.

# ln-s/usr/lib64/libfastcommon.so/usr/local/lib/libfastcommon.so

# ln-s/usr/lib64/libfastcommon.so/usr/lib/libfastcommon.so

# ln-s/usr/lib64/libfdfsclient.so/usr/local/lib/libfdfsclient.so

# ln-s/usr/lib64/libfdfsclient.so/usr/lib/libfdfsclient.so

3. Installation Fastdfs

(1) Upload or download Fastdfs source package (fastdfs_v5.05.tar.gz) to/usr/local/src directory

(2) Decompression

# cd/usr/local/src/

# TAR-ZXVF Fastdfs_v5.05.tar.gz

# CD Fastdfs

(3) Compile, install (before compiling to ensure that the Libfastcommon has been successfully installed)

#./make.sh

#./make.sh Install

Installed by default installation, the corresponding files and directories after installation:

A, service scripts in:

/etc/init.d/fdfs_storaged

/etc/init.d/fdfs_tracker

B, configuration file in (sample configuration file):

/etc/fdfs/client.conf.sample

/etc/fdfs/storage.conf.sample

/etc/fdfs/tracker.conf.sample

C, the command tool in the/usr/bin/directory:

Fdfs_appender_test

Fdfs_appender_test1

Fdfs_append_file

Fdfs_crc32

Fdfs_delete_file

Fdfs_download_file

Fdfs_file_info

Fdfs_monitor

Fdfs_storaged

Fdfs_test

Fdfs_test1

Fdfs_trackerd

Fdfs_upload_appender

Fdfs_upload_file

stop.sh

restart.sh

(4) Because the Fastdfs service script set the bin directory is/usr/local/bin, but the actual command is installed in/usr/bin, you can access

The/user/bin directory uses the following command to view related commands for Fdfs:

# cd/usr/bin/

# ls | grep Fdfs

It is therefore necessary to modify the corresponding command path in the Fastdfs service script, which is to put the/etc/init.d/fdfs_storaged

And the/usr/local/bin in the/etc/init.d/fdfs_tracker two scripts are modified to/usr/bin:

# VI Fdfs_trackerd

To unify the changes using the Find substitution command:%s+/usr/local/bin+/usr/bin

# VI Fdfs_storaged

To unify the changes using the Find substitution command:%s+/usr/local/bin+/usr/bin

Second, configure Fastdfs Tracker (192.168.4.121)

1, copy the Fastdfs Tracker sample configuration file, and rename:

# cd/etc/fdfs/

# cptracker.conf.sample Tracker.conf

2. Edit Tracker configuration file:

# vi/etc/fdfs/tracker.conf

The revised content is as follows:

Disabled=false

port=22122

Base_path=/fastdfs/tracker

(Other parameters retain the default configuration, please refer to the official documentation for specific configuration explanations:

http://bbs.chinaunix.net/thread-1941456-1-1.html)

3, create the basic data directory (refer to the basic directory Base_path configuration):

# Mkdir-p/fastdfs/tracker

4, open the tracker port in the firewall (default is 22122):

# Vi/etc/sysconfig/iptables

Add the following port line:

-A input-m state--state new-m tcp-p TCP--dport 22122-j ACCEPT

Reboot the firewall:

# service Iptablesrestart

5, Start tracker:

#/etc/init.d/fdfs_trackerdstart

(The first successful startup, will be in the/fastdfs/tracker directory to create data, logs two directories)

To see if Fastdfs Tracker has started successfully:

# Ps-ef | Grepfdfs

6. Close tracker:

#/etc/init.d/fdfs_trackerdstop

7, set the Fastdfs tracker boot:

# vi/etc/rc.d/rc.local

Add the following:

# # Fastdfs Tracker

/etc/init.d/fdfs_trackerdstart

Third, configure Fastdfs storage (192.168.4.125)

1, copy FASTDFS Storage sample configuration file, and rename:

# cd/etc/fdfs/

# cpstorage.conf.sample Storage.conf

2. Edit the Memory sample configuration file:

# vi/etc/fdfs/storage.conf

The revised content is as follows:

Disabled=false

port=23000

Base_path=/fastdfs/storage

Store_path0=/fastdfs/storage

tracker_server=192.168.4.121:22122

http.server_port=8888

(Other parameters retain the default configuration, please refer to the official documentation for specific configuration explanations:

http://bbs.chinaunix.net/thread-1941456-1-1.html)

3, create the basic data directory (refer to the basic directory Base_path configuration):

# Mkdir-p/fastdfs/storage

4, open the memory port in the firewall (default is 23000):

# Vi/etc/sysconfig/iptables

Add the following port line:

-A input-m state--state new-m tcp-p TCP--dport 23000-j ACCEPT

Reboot the firewall:

# service Iptablesrestart

5, Start storage:

#/etc/init.d/fdfs_storagedstart

(The first successful startup, will be in the/fastdfs/storage directory to create data, logs two directories)

To see if Fastdfs storage has started successfully

# Ps-ef | Grepfdfs

6. Close storage:

#/etc/init.d/fdfs_storagedstop

7, set up Fastdfs memory boot:

# vi/etc/rc.d/rc.local

Add to:

# # Fastdfs Storage

/etc/init.d/fdfs_storagedstart

iv. File Upload Test (192.168.4.121)

1, modify the client configuration file in the tracker server:

# cp/etc/fdfs/client.conf.sample/etc/fdfs/client.conf

# vi/etc/fdfs/client.conf

Base_path=/fastdfs/tracker

tracker_server=192.168.4.121:22122

2, execute the following file Upload command:

#/usr/bin/fdfs_upload_file/etc/fdfs/client.conf/usr/local/src/fastdfs_v5.05.tar.gz

Return ID Number: group1/m00/00/00/wkgefvuynyeab7xfaavfol7fju4.tar.gz

(Can return the above file ID, description file Upload success)

vi. install Nginx on each storage node

1, Fastdfs-nginx-module function description

Fastdfs through the tracker server, the files are stored in the storage server, but there is a problem of synchronization latency between the same group of storage servers and the need to access file replication. Suppose the tracker server uploads the file to 192.168.4.125, and the file ID is returned to the client after the upload succeeds. At this point, the FASTDFS storage cluster mechanism synchronizes this file to the same group of storage 192.168.4.126, and if the client uses this file ID to fetch files on 192.168.4.126, the file cannot be accessed if the file is not replicated. Instead, Fastdfs-nginx-module can redirect files to the source server to fetch files, avoiding errors caused by the client due to replication latency. (Fastdfs-nginx-module is used during nginx installation)

2, upload fastdfs-nginx-module_v1.16.tar.gz to/usr/local/src

3, decompression

# cd/usr/local/src/

# tar-zxvffastdfs-nginx-module_v1.16.tar.gz

4. Modify Fastdfs-nginx-module Config profile

# CD FASTDFS-NGINX-MODULE/SRC

# VI Config

core_incs= "$CORE _incs/usr/local/include/fastdfs/usr/local/include/fastcommon/"

Modified to:

core_incs= "$CORE _incs/usr/include/fastdfs/usr/include/fastcommon/"

(Note: This path modification is very important, otherwise it will be the error when compiling the Nginx)

5, upload the current stable version nginx (nginx-1.6.2.tar.gz) to the/USR/LOCAL/SRC directory

6, the installation of Nginx required to build the dependent package

# yum Install gcc gcc-c++ make automakeautoconf libtool pcre* zlib OpenSSL openssl-devel

7. Compile and install Nginx (add Fastdfs-nginx-module module)

# cd/usr/local/src/

# TAR-ZXVF Nginx-1.6.2.tar.gz

# CD nginx-1.6.2

#./CONFIGURE--ADD-MODULE=/USR/LOCAL/SRC/FASTDFS-NGINX-MODULE/SRC

# Make && make install

8, copy the Fastdfs-nginx-module source of the configuration file to the/etc/fdfs directory, and modify

# cp/usr/local/src/fastdfs-nginx-module/src/mod_fastdfs.conf/etc/fdfs/

# vi/etc/fdfs/mod_fastdfs.conf

Modify the following configuration:

connect_timeout=10

Base_path=/tmp

tracker_server=192.168.4.121:22122

storage_server_port=23000

Group_name=group1

Url_have_group_name = True

Store_path0=/fastdfs/storage

9, copy Fastdfs part of the configuration file to the/etc/fdfs directory

# cd/usr/local/src/fastdfs/conf

# CP http.conf mime.types/etc/fdfs/

10, in the/fastdfs/storage file storage directory to create a soft connection, link it to the actual storage of data directory

# ln-s/FASTDFS/STORAGE/DATA//FASTDFS/STORAGE/DATA/M00

11. Configure Nginx

Concise version of the Nginx configuration sample:

User root;

Worker_processes 1;

Events {

Worker_connections 1024;

}

HTTP {

Include Mime.types;

Default_type Application/octet-stream;

Sendfile on;

Keepalive_timeout 65;

server {

Listen 8888;

server_name localhost;

Location ~/group ([0-9])/m00 {

#alias/fastdfs/storage/data;

Ngx_fastdfs_module;

}

Error_page 502 503 504/50x.html;

Location =/50x.html {

root HTML;

}

}

}

Attention, Description:

A, 8888 port value is to correspond with the http.server_port=8888 in/etc/fdfs/storage.conf,

Because Http.server_port defaults to 8888, if you want to change to 80, you need to change the corresponding.

B, storage corresponding to multiple group, the access path takes a group name, such as/group1/m00/00/00/xxx,

The corresponding Nginx configuration is:

Location ~/group ([0-9])/m00 {

Ngx_fastdfs_module;

}

C, if the download when the old report 404, will nginx.conf the first line Usernobody modified to user root restart.

12, open the Nginx 8888 port in the firewall

# Vi/etc/sysconfig/iptables

Add to:

-A input-m state--state new-m tcp-p TCP--dport 8888-j ACCEPT

# Service Iptables Restart

13, start Nginx

#/usr/local/nginx/sbin/nginx

Ngx_http_fastdfs_set pid=xxx

(the command to restart Nginx is:/usr/local/nginx/sbin/nginx-s reload)

14, through the browser to access the test uploaded files

Http://192.168.4.125:8888/group1/M00/00/00/wKgEfVUYNYeAb7XFAAVFOL7FJU4.tar.gz

Seven, the use of Fastdfs Demo sample explanation and Demo:

please refer to sample code and video tutorials for specific content

 

Note: Do not use the kill-9 command to kill Fastdfs process, otherwise it may cause binlog data loss.