Fastdfs Distributed File System cluster installation

Document Revision History

650) this.width=650; "src=" Http://s5.51cto.com/wyfs02/M01/85/F2/wKiom1ewaHKyABiIAAAj1fPYKaU770.png "title=" 1111. PNG "alt=" Wkiom1ewahkyabiiaaaj1fpykau770.png "/>

Tracking device:

Tracker Server1 ip:10.10.16.201 fastdfs-tracker-01

Tracker Server2 ip:10.10.16.202 fastdfs-tracker-02

Storage server:

Storage Server1 ip:10.10.16.203 fastdfs-storage-01

Storage Server2 ip:10.10.16.204 fastdfs-storage-02

Settings for each operating system environment in the cluster:

(1) Fastdfs installation directory:/usr/local/fastdfs

[Email protected] local]# Mkdir/usr/local/fastdfs

[Email protected] local]# Mkdir/usr/local/fastdfs

[Email protected] local]# Mkdir/usr/local/fastdfs

[Email protected] local]# Mkdir/usr/local/fastdfs

(2) Create Fastdfs user group and FASTDFS user (four machine simultaneous configuration):

[Email protected]]# groupadd Fastdfs

[Email protected]]# useradd-g Fastdfs Fastdfs

[Email protected]]# passwd Fastdfs

# # Set Fastdfs user password to gzzyzz.com (before the production to change)

(3) Create a data store directory:

#创建tracker目录保存运行日志 (Tracker server)

[Email protected] ~]$ mkdir-p/home/fastdfs/tracker

[Email protected] ~]$ mkdir-p/home/fastdfs/tracker

#创建Storage目录保存运行日志及其data数据 (Storage server)

[[Email protected] ~] $ mkdir-p/home/fastdfs/storage

[[Email protected] ~] $ mkdir-p/home/fastdfs/storage

Note: Because the source code compiled to install the software, so the system to be installed in advance: Make CMake gcc gcc-c++

[email protected] yum.repos.d]# yum install make cmake gcc gcc-c++

1, First install the libevent (all the servers in the cluster have the same installation):

Fastdfs when compiling the source program Fastdfs internal call libevent processing mechanism, need to use libevent some dependent files, or compile Fastdfs error.

# Uninstall the system comes with Libevent, with the version too low, or install Fastdfs error

[Email protected] fastdfs]# Rpm-qa|grep libevent

Libevent-1.4.13-4.el6.x86_64

(The system can bring the libevent-1.4.13-4.el6.x86_64 from the top)

Unloading:

[Email protected] fastdfs]# rpm-e--nodeps libevent-1.4.13-4.el6.x86_64

#下载安装libevent

Enter the/USR/LOCAL/SRC directory

# cd/usr/local/src/

[Email protected] src]#

wget http://cloud.github.com/downloads/libevent/libevent/libevent-2.0.19-stable.tar.gz

[Email protected] src]# TAR-ZXVF libevent-2.0.19-stable.tar.gz

[Email protected] src]# CD libevent-2.0.19-stable

#make clean;

[Email protected] libevent-2.0.19-stable]$/configure--prefix=/usr/local/libevent

[[email protected] libevent-2.0.19-stable]# make

[[email protected] libevent-2.0.19-stable]# make && make install

#为libevent创建软链接到/lib Library, the 64-bit system corresponds to/LIB64

[Email protected] libevent-2.0.19-stable]#

Ln-s/usr/local/libevent/lib/libevent-2.0.so.5/usr/lib/libevent-2.0.so.5

[Email protected] libevent-2.0.19-stable]#

Ln-s/usr/local/libevent/lib/libevent-2.0.so.5/usr/lib64/libevent-2.0.so.5

2, Install the FASTDFS (all the servers in the cluster have the same installation):

Enter the/USR/LOCAL/SRC directory

[Email protected] src]#

wget https://fastdfs.googlecode.com/files/FastDFS_v4.06.tar.gz

[Email protected] src]# TAR-ZXVF fastdfs_v4.06.tar.gz

[Email protected] src]# CD Fastdfs

#由于定义/usr/local/fastdfs for the Fastdfs installation directory, so you need to modify the make.sh

[Email protected] fastdfs]$ VI make.sh

#/etc/fdfs all replaced by/usr/local/fastdfs/conf

# target_prefix=/usr/local modified to/usr/local/fastdfs

# Target_conf_path=/etc/fdfs modified to/usr/local/fastdfs/conf

Attachment:

#安装

[Email protected] fastdfs]$

./make.sh C_include_path=/usr/local/libevent/include Library_path=/usr/local/libevent/lib

# Switch to Super admin

[[email protected] fastdfs]#./make.sh Install

Note: If you are not prompted to find the command at the time of installation, check to see if the Perl installation environment is not installed

wget http://www.cpan.org/src/5.0/perl-5.18.2.tar.gz

TAR-ZXVF perl-5.18.2.tar.gz

CD perl-5.18.2

./configure

Make

Make install

3. Configure tracker Server (10.10.16.201,10.10.16.202):

Enter/usr/local/fastdfs/conf

#修改tracker. conf configuration

[Email protected] conf]# vi/usr/local/fastdfs/conf/tracker.conf

Bind IP:

bind_addr=10.10.16.201 and bind_addr=10.10.16.202

#设置日志目录

Changed from Base_path=/home/yuqing/fastdfs to Base_path=/home/fastdfs/tracker

Store_group=group1

run_by_group= Change to Run_by_group=fastdfs

run_by_user= Change to Run_by_user=fastdfs

Trunk_create_file_space_threshold = 20G

#开启自trunk_create_file_space_threshold定义server ID instead of IP to facilitate internal network server replacement ip#** this way to focus on understanding, after 4.0 new features

USE_STORAGE_ID = True #使用server ID as Storage server identity

Storage_ids_filename = storage_ids.conf #<id> <group_name> <ip_or_hostname>

Id_type_in_filename = ID #文件名反解析中包含server ID, formerly IP

Copy the storage_ids.conf file

[Email protected] conf]#

cp/usr/local/src/fastdfs/conf/storage_ids.conf/usr/local/fastdfs/conf/

#编辑storage服务器ID与IP地址的对应关系

[Email protected] conf]# vi/usr/local/fastdfs/conf/storage_ids.conf

# <id> <group_name> <ip_or_hostname>

100001 group1 10.10.16.203

100002 group1 10.10.16.204

Firewalls open ports 8080 and 22122:

# Vi/etc/sysconfig/iptables

Increase:

-A input-m state--state new-m tcp-p TCP--dport 8080-j ACCEPT

-A input-m state--state new-m tcp-p TCP--dport 22122-j ACCEPT

To restart the firewall:

#/etc/init.d/iptables Restart

Custom Tracker service startup, shutdown, restart scripts:

# su Fastdfs

After switching back to the FASTDFS user

$ cd/home/fastdfs/tracker/

(1) Startup script:

[Email protected] tracker]$ VI start_tracker.sh

/usr/local/fastdfs/bin/fdfs_trackerd/usr/local/fastdfs/conf/tracker.conf

(2) Close the script (using Fastdfs's own stop.sh script):

[Email protected] tracker]$ VI stop_tracker.sh

/usr/local/fastdfs/bin/stop.sh/usr/local/fastdfs/bin/fdfs_trackerd/usr/local/fastdfs/conf/tracker.conf

(3) Restart the script (using the restart.sh script that comes with Fastdfs):

[Email protected] tracker]$ VI restart_tracker.sh

/usr/local/fastdfs/bin/restart.sh/usr/local/fastdfs/bin/fdfs_trackerd/usr/local/fastdfs/conf/tracker.conf

Note: Do not use the kill-9 parameter to kill the process, otherwise it may cause the Binlog data loss problem

To assign executable permissions to a custom script:

[Email protected] tracker]$ chmod +x *_tracker.sh

To start the tracker service:

[Email protected] tracker]$./start_tracker.sh

To see if success is enabled:

[Email protected] tracker]$ Ps-ef | grep Fastdfs

Root 27253 26511 0 10:03 pts/0 00:00:00 su Fastdfs

Fastdfs 27254 27253 0 10:03 pts/0 00:00:00 bash

Fastdfs 27580 1 0 10:25? 00:00:00/usr/local/fastdfs/bin/fdfs_trackerd/usr/local/fastdfs/conf/tracker.conf

Fastdfs 27588 27254 5 10:25 pts/0 00:00:00 ps-ef

Fastdfs 27589 27254 0 10:25 pts/0 00:00:00 grep fastdfs

[Email protected] tracker]$

(The Data directory and the logs directory will be generated in the/home/fastdfs/tracker directory after the first normal startup)

[2014-01-12 16:33:54] Info-local_host_ip_count:2, 127.0.0.1 10.10.16.201

[2014-01-12 16:34:11] INFO-FILE:TRACKER_SERVICE.C, line:920, the tracker leader is 10.10.16.202:22122

Deactivate Tracker Service:

[Email protected] tracker]$./stop_tracker.sh

Restart the tracker service:

[Email protected] tracker]$./restart_tracker.sh

4, Configure storage Server (10.10.16.203, 10.10.16.204, two machines perform the same configuration operation):

# cd/usr/local/fastdfs/conf

# VI Storage.conf

The binding IPs are:

bind_addr=10.10.16.203 and bind_addr=10.10.16.204

Base_path=/home/yuqing/fastdfs Change to Base_path=/home/fastdfs/storage

Store_path0=/home/yuqing/fastdfs Change to Store_path0=/home/fastdfs/storage

tracker_server=192.168.209.121:22122 instead

tracker_server=10.10.16.201:22122

tracker_server=10.10.16.202:22122

(We are equipped with two tracker services here)

run_by_group= Change to Run_by_group=fastdfs

run_by_user= Change to Run_by_user=fastdfs

# Same as Nginx port

Http.server_port=80

Firewalls open ports 23000 and 8888:

# Vi/etc/sysconfig/iptables

Increase:

-A input-m state--state new-m tcp-p TCP--dport 23000-j ACCEPT

-A input-m state--state new-m tcp-p TCP--dport 8888-j ACCEPT

To restart the firewall:

#/etc/init.d/iptables Restart

Custom storage service startup, shutdown, restart scripts:

# su Fastdfs

After switching back to the FASTDFS user

$ cd/home/fastdfs/storage/

(1) Startup script:

[Email protected] storage]$ VI start_storage.sh

/usr/local/fastdfs/bin/fdfs_storaged/usr/local/fastdfs/conf/storage.conf

(2) Close the script (using Fastdfs's own stop.sh script):

[Email protected] storage]$ VI stop_tracker.sh

/usr/local/fastdfs/bin/stop.sh/usr/local/fastdfs/bin/fdfs_storaged/usr/local/fastdfs/conf/storage.conf

(3) Restart the script (using the restart.sh script that comes with Fastdfs):

[Email protected] storage]$ VI restart_storage.sh

/usr/local/fastdfs/bin/restart.sh/usr/local/fastdfs/bin/fdfs_storaged/usr/local/fastdfs/conf/storage.conf

Note: Do not use the kill-9 parameter to kill the process, otherwise it may cause the Binlog data loss problem

To assign executable permissions to a custom script:

[Email protected] storage]$ chmod +x *_storage.sh

Start Storage Server:

[Email protected] storage]$./start_storage.sh

Data path:/home/fastdfs/storage/data, mkdir sub dir ...

mkdir Data path:00 ...

mkdir Data path:01 ...

mkdir Data path:02 ...

mkdir Data path:03 ...

(The data store directory is initialized for the first successful startup)

Restart the storage service:

[Email protected] storage]$./restart_storage.sh

To stop the storage service:

[Email protected] storage]$./stop_storage.sh

Storage Server (10.10.16.203 and 10.10.16.204) installs Nginx:

#安装Nginx使用fastdfs用户

#创建nginx日志目录

# mkdir-p/home/fastdfs/nginx/logs

# chmod A+w/home/fastdfs/nginx/logs

# Chown-r Fastdfs:fastdfs/home/fastdfs/nginx/logs

Install nginx1.4.4:

# CD/USR/LOCAL/SRC

#下载 Nginx

[Email protected] src]#

wget http://nginx.org/download/nginx-1.4.4.tar.gz

Extract:

# TAR-ZXVF Nginx-1.4.4.tar.gz

# CD nginx-1.4.4

#./configure--user=fastdfs--group=fastdfs--prefix=/usr/local/nginx--with-http_stub_status_module

If you have the following error message:

Checking for PCRE library ... not found

Checking for PCRE library in/usr/local/... not found

Checking for PCRE library in/usr/include/pcre/... not found

Checking for PCRE library in/usr/pkg/... not found

Checking for PCRE library in/opt/local/... not found

./configure:error:the HTTP Rewrite module requires the PCRE library.

You can either disable the module by using--without-http_rewrite_module

option, or install the PCRE library into the system, or build the PCRE library

Statically from the source with Nginx by using--with-pcre=<path> option.

(probably means: HTTP redirect Module installation needs Pcre library, online search, it is necessary to install Pcre-devel and openssl-devel)

# yum Install pcre-devel OpenSSL openssl-devel

(RHEL6.3 x64 in desktop version with the above package)

Then re-execute:

#./configure--user=fastdfs--group=fastdfs--prefix=/usr/local/nginx--with-http_stub_status_module

#--with-http_stub_status_module is used to monitor the current state of Nginx.

Configuration Summary

+ Using System PCRE Library

+ OpenSSL Library is not used

+ md5:using System Crypto Library

+ sha1:using System Crypto Library

+ Using System zlib Library

Nginx path prefix: "/usr/local/nginx"

Nginx binary file: "/usr/local/nginx/sbin/nginx"

Nginx configuration prefix: "/usr/local/nginx/conf"

Nginx configuration file: "/usr/local/nginx/conf/nginx.conf"

Nginx pid file: "/usr/local/nginx/logs/nginx.pid"

Nginx error log file: "/usr/local/nginx/logs/error.log"

Nginx HTTP access log file: "/usr/local/nginx/logs/access.log"

Nginx HTTP client request body Temporary files: "Client_body_temp"

Nginx HTTP proxy temporary files: "Proxy_temp"

Nginx http fastcgi temporary files: "Fastcgi_temp"

Nginx http Uwsgi temporary files: "Uwsgi_temp"

Nginx http scgi temporary files: "Scgi_temp"

Perform installation: # make && make install

To install the Fastdfs-nginx-module plugin:

# CD/USR/LOCAL/SRC

# wget http://fastdfs.googlecode.com/files/fastdfs-nginx-module_v1.15.tar.gz

# TAR-ZXVF Fastdfs-nginx-module_v1.15.tar.gz

#修改插件配置文件

# Vi/usr/local/src/fastdfs-nginx-module/src/config

Ngx_addon_name=ngx_http_fastdfs_module

http_modules= "$HTTP _modules ngx_http_fastdfs_module"

ngx_addon_srcs= "$NGX _addon_srcs $ngx _addon_dir/ngx_http_fastdfs_module.c"

core_incs= "$CORE _incs/usr/local/fastdfs/include/fastdfs/usr/local/fastdfs/include/fastcommon/"

core_libs= "$CORE _libs-l/usr/local/fastdfs/lib-lfastcommon-lfdfsclient"

cflags= "$CFLAGS-d_file_offset_bits=64-dfdfs_output_chunk_size= ' 256*1024 '-dfdfs_mod_conf_filename= ' \"/usr/local /fastdfs/conf/mod_fastdfs.conf\ "'"

#复制mod_fastdfs. conf to the/usr/local/fastdfs/conf/directory

# cp/usr/local/src/fastdfs-nginx-module/src/mod_fastdfs.conf/usr/local/fastdfs/conf/

The dynamic link library used at compile time:

#启动nginx报错

#./nginx:error while loading shared libraries:libfastcommon.so:cannot open Shared object File:no such file or director Y

#解决办法---> Add/usr/local/fastdfs/lib to your system file/etc/ld.so.conf

# vi/etc/ld.so.conf

Include ld.so.conf.d/*.conf

/usr/local/fastdfs/lib

#更新库文件缓存ld. So.cache

#/sbin/ldconfig-v

To compile the Fastdfs-nginx-module module:

# Recompile Install Nginx

# cd/usr/local/src/nginx-1.4.4

#./configure--ADD-MODULE=/USR/LOCAL/SRC/FASTDFS-NGINX-MODULE/SRC

# Make && make install

To modify the mod_fastdfs.conf configuration:

# vi/usr/local/fastdfs/conf/mod_fastdfs.conf

# Connect Timeout in seconds

# Default value is 30s

Connect_timeout=30

# The base path to store log files

Base_path=/tmp

# If load Fastdfs parameters from tracker server

# since V1.12

# default value is False

Load_fdfs_parameters_from_tracker=true

# if use storage ID instead of IP address

# Same as Tracker.conf

# valid only if Load_fdfs_parameters_from_tracker is False

# default value is False

# since V1.13

USE_STORAGE_ID = True

# Fastdfs Tracker_server can ocur more than once, and Tracker_server format is

# "Host:port", host can be hostname or IP address

# valid only if Load_fdfs_parameters_from_tracker is true

tracker_server=10.10.16.201:22122

tracker_server=10.10.16.202:22122

# The port of the local storage server

# The default value is 23000

storage_server_port=23000

# The group name of the local storage server

Group_name=group1

# if the Url/uri including the group name

# set to False when Uri Like/m00/00/00/xxx

# set to True when URIs like ${group_name}/m00/00/00/xxx, such as Group1/m00/xxx

# default value is False

Url_have_group_name = True

# path (disk or mount point) count, default value is 1

# must same as storage.conf

Store_path_count=1

# store_path#, based 0, if Store_path0 not exists, it's value is Base_path

# The paths must be exist

# must same as storage.conf

Store_path0=/home/fastdfs/storage

# Set the log filename, such as/usr/local/apache2/logs/mod_fastdfs.log

# Empty for output to stderr (Apache and nginx error_log file)

Log_filename=/home/fastdfs/nginx/logs/mod_fastdfs.log

Note that when downloading the old newspaper 404. The nginx.conf first row usr Nobody is modified to be resolved after the user root restarts.

Make a soft connection to storage's storage directory

# ln-s/HOME/FASTDFS/STORAGE/DATA/HOME/FASTDFS/STORAGE/DATA/M00

Access path with group name (storage corresponds to multiple group cases), such as/group1/m00/00/00/xxx:

Location ~/group ([0-9])/m00 {

Ngx_fastdfs_module;

}

Nginx Configuration Concise version:

# vi/usr/local/nginx/conf/nginx.conf

User root root;

Worker_processes 2;

Error_log/home/fastdfs/nginx/logs/error.log notice;

Pid/home/fastdfs/nginx/logs/nginx.pid;

Worker_rlimit_nofile 5120;

Events {

Use Epoll;

Worker_connections 5120;

}

HTTP {

Include Mime.types;

Default_type Application/octet-stream;

Sendfile on;

Tcp_nopush on;

Keepalive_timeout 65;

Tcp_nodelay on;

server {

Listen 80;

server_name localhost;

#charset Koi8-r;

Location ~/group ([0-9])/m00 {

#alias/home/fastdfs/storage/data;

Ngx_fastdfs_module;

}

}

}

#启动nginx:

#/usr/local/nginx/sbin/nginx

#重启nginx:

#/usr/local/nginx/sbin/nginx-s Reload

To upload a file with a Java client:

Browse

650) this.width=650; "src=" Http://s2.51cto.com/wyfs02/M02/85/F2/wKiom1ewaQHC2IWGAAJo6c5kjoQ370.png "title=" Dfs1.png "alt=" Wkiom1ewaqhc2iwgaajo6c5kjoq370.png "/>

650) this.width=650; "src=" Http://s5.51cto.com/wyfs02/M00/85/F2/wKioL1ewaZbTHgmYAAJpWVjmQU8564.png "title=" Dfs2.png "alt=" Wkiol1ewazbthgmyaajpwvjmqu8564.png "/>

File access is balanced by hardware

vip:10.10.16.209 (The virtual machine gateway for load balancing to be set to 10.10.16.8)

650) this.width=650; "src=" Http://s5.51cto.com/wyfs02/M01/85/F2/wKioL1ewabvT_o-6AAJp5ofWwBE101.png "title=" Dfs3.png "alt=" Wkiol1ewabvt_o-6aajp5ofwwbe101.png "/>

5,A

6,A

7,A

8,A

9,A

Sample Documentation Video Tutorial: Http://www.roncoo.com/details/85d6008fe77c4199b0cdd2885eaeee53

Fastdfs Distributed File System cluster installation