Fastdfs Distributed File System cluster installation

Source: Internet
Author: User
Tags http redirect openssl library


Document Revision History

650) this.width=650; "src=" Http://s5.51cto.com/wyfs02/M01/85/F2/wKiom1ewaHKyABiIAAAj1fPYKaU770.png "title=" 1111. PNG "alt=" Wkiom1ewahkyabiiaaaj1fpykau770.png "/>


Tracking device:

Tracker Server1 ip:10.10.16.201 fastdfs-tracker-01

Tracker Server2 ip:10.10.16.202 fastdfs-tracker-02


Storage server:

Storage Server1 ip:10.10.16.203 fastdfs-storage-01

Storage Server2 ip:10.10.16.204 fastdfs-storage-02


Settings for each operating system environment in the cluster:

(1) Fastdfs installation directory:/usr/local/fastdfs

[Email protected] local]# Mkdir/usr/local/fastdfs

[Email protected] local]# Mkdir/usr/local/fastdfs

[Email protected] local]# Mkdir/usr/local/fastdfs

[Email protected] local]# Mkdir/usr/local/fastdfs


(2) Create Fastdfs user group and FASTDFS user (four machine simultaneous configuration):

[Email protected]]# groupadd Fastdfs

[Email protected]]# useradd-g Fastdfs Fastdfs

[Email protected]]# passwd Fastdfs

# # Set Fastdfs user password to gzzyzz.com (before the production to change)


(3) Create a data store directory:

#创建tracker目录保存运行日志 (Tracker server)

[Email protected] ~]$ mkdir-p/home/fastdfs/tracker

[Email protected] ~]$ mkdir-p/home/fastdfs/tracker

#创建Storage目录保存运行日志及其data数据 (Storage server)

[[Email protected] ~] $ mkdir-p/home/fastdfs/storage

[[Email protected] ~] $ mkdir-p/home/fastdfs/storage


Note: Because the source code compiled to install the software, so the system to be installed in advance: Make CMake gcc gcc-c++

[email protected] yum.repos.d]# yum install make cmake gcc gcc-c++


1, First install the libevent (all the servers in the cluster have the same installation):

Fastdfs when compiling the source program Fastdfs internal call libevent processing mechanism, need to use libevent some dependent files, or compile Fastdfs error.

# Uninstall the system comes with Libevent, with the version too low, or install Fastdfs error

[Email protected] fastdfs]# Rpm-qa|grep libevent

Libevent-1.4.13-4.el6.x86_64

(The system can bring the libevent-1.4.13-4.el6.x86_64 from the top)

Unloading:

[Email protected] fastdfs]# rpm-e--nodeps libevent-1.4.13-4.el6.x86_64


#下载安装libevent

Enter the/USR/LOCAL/SRC directory

# cd/usr/local/src/

[Email protected] src]#

wget http://cloud.github.com/downloads/libevent/libevent/libevent-2.0.19-stable.tar.gz


[Email protected] src]# TAR-ZXVF libevent-2.0.19-stable.tar.gz

[Email protected] src]# CD libevent-2.0.19-stable

#make clean;

[Email protected] libevent-2.0.19-stable]$/configure--prefix=/usr/local/libevent

[[email protected] libevent-2.0.19-stable]# make

[[email protected] libevent-2.0.19-stable]# make && make install

#为libevent创建软链接到/lib Library, the 64-bit system corresponds to/LIB64

[Email protected] libevent-2.0.19-stable]#

Ln-s/usr/local/libevent/lib/libevent-2.0.so.5/usr/lib/libevent-2.0.so.5

[Email protected] libevent-2.0.19-stable]#

Ln-s/usr/local/libevent/lib/libevent-2.0.so.5/usr/lib64/libevent-2.0.so.5


2, Install the FASTDFS (all the servers in the cluster have the same installation):

Enter the/USR/LOCAL/SRC directory

[Email protected] src]#

wget https://fastdfs.googlecode.com/files/FastDFS_v4.06.tar.gz

[Email protected] src]# TAR-ZXVF fastdfs_v4.06.tar.gz

[Email protected] src]# CD Fastdfs


#由于定义/usr/local/fastdfs for the Fastdfs installation directory, so you need to modify the make.sh

[Email protected] fastdfs]$ VI make.sh

#/etc/fdfs all replaced by/usr/local/fastdfs/conf

# target_prefix=/usr/local modified to/usr/local/fastdfs

# Target_conf_path=/etc/fdfs modified to/usr/local/fastdfs/conf

Attachment:

#安装

[Email protected] fastdfs]$

./make.sh C_include_path=/usr/local/libevent/include Library_path=/usr/local/libevent/lib

# Switch to Super admin

[[email protected] fastdfs]#./make.sh Install


Note: If you are not prompted to find the command at the time of installation, check to see if the Perl installation environment is not installed

wget http://www.cpan.org/src/5.0/perl-5.18.2.tar.gz

TAR-ZXVF perl-5.18.2.tar.gz

CD perl-5.18.2

./configure

Make

Make install


3. Configure tracker Server (10.10.16.201,10.10.16.202):

Enter/usr/local/fastdfs/conf

#修改tracker. conf configuration

[Email protected] conf]# vi/usr/local/fastdfs/conf/tracker.conf

Bind IP:

bind_addr=10.10.16.201 and bind_addr=10.10.16.202


#设置日志目录

Changed from Base_path=/home/yuqing/fastdfs to Base_path=/home/fastdfs/tracker

Store_group=group1

run_by_group= Change to Run_by_group=fastdfs

run_by_user= Change to Run_by_user=fastdfs

Trunk_create_file_space_threshold = 20G

#开启自trunk_create_file_space_threshold定义server ID instead of IP to facilitate internal network server replacement ip#** this way to focus on understanding, after 4.0 new features

USE_STORAGE_ID = True #使用server ID as Storage server identity

Storage_ids_filename = storage_ids.conf #<id> <group_name> <ip_or_hostname>

Id_type_in_filename = ID #文件名反解析中包含server ID, formerly IP


Copy the storage_ids.conf file

[Email protected] conf]#

cp/usr/local/src/fastdfs/conf/storage_ids.conf/usr/local/fastdfs/conf/

#编辑storage服务器ID与IP地址的对应关系

[Email protected] conf]# vi/usr/local/fastdfs/conf/storage_ids.conf

# <id> <group_name> <ip_or_hostname>

100001 group1 10.10.16.203

100002 group1 10.10.16.204


Firewalls open ports 8080 and 22122:

# Vi/etc/sysconfig/iptables

Increase:

-A input-m state--state new-m tcp-p TCP--dport 8080-j ACCEPT

-A input-m state--state new-m tcp-p TCP--dport 22122-j ACCEPT

To restart the firewall:

#/etc/init.d/iptables Restart


Custom Tracker service startup, shutdown, restart scripts:

# su Fastdfs

After switching back to the FASTDFS user

$ cd/home/fastdfs/tracker/

(1) Startup script:

[Email protected] tracker]$ VI start_tracker.sh

/usr/local/fastdfs/bin/fdfs_trackerd/usr/local/fastdfs/conf/tracker.conf


(2) Close the script (using Fastdfs's own stop.sh script):

[Email protected] tracker]$ VI stop_tracker.sh

/usr/local/fastdfs/bin/stop.sh/usr/local/fastdfs/bin/fdfs_trackerd/usr/local/fastdfs/conf/tracker.conf


(3) Restart the script (using the restart.sh script that comes with Fastdfs):

[Email protected] tracker]$ VI restart_tracker.sh

/usr/local/fastdfs/bin/restart.sh/usr/local/fastdfs/bin/fdfs_trackerd/usr/local/fastdfs/conf/tracker.conf


Note: Do not use the kill-9 parameter to kill the process, otherwise it may cause the Binlog data loss problem


To assign executable permissions to a custom script:

[Email protected] tracker]$ chmod +x *_tracker.sh


To start the tracker service:

[Email protected] tracker]$./start_tracker.sh

To see if success is enabled:

[Email protected] tracker]$ Ps-ef | grep Fastdfs

Root 27253 26511 0 10:03 pts/0 00:00:00 su Fastdfs

Fastdfs 27254 27253 0 10:03 pts/0 00:00:00 bash

Fastdfs 27580 1 0 10:25? 00:00:00/usr/local/fastdfs/bin/fdfs_trackerd/usr/local/fastdfs/conf/tracker.conf

Fastdfs 27588 27254 5 10:25 pts/0 00:00:00 ps-ef

Fastdfs 27589 27254 0 10:25 pts/0 00:00:00 grep fastdfs

[Email protected] tracker]$

(The Data directory and the logs directory will be generated in the/home/fastdfs/tracker directory after the first normal startup)

[2014-01-12 16:33:54] Info-local_host_ip_count:2, 127.0.0.1 10.10.16.201

[2014-01-12 16:34:11] INFO-FILE:TRACKER_SERVICE.C, line:920, the tracker leader is 10.10.16.202:22122

Deactivate Tracker Service:

[Email protected] tracker]$./stop_tracker.sh


Restart the tracker service:

[Email protected] tracker]$./restart_tracker.sh


4, Configure storage Server (10.10.16.203, 10.10.16.204, two machines perform the same configuration operation):

# cd/usr/local/fastdfs/conf

# VI Storage.conf

The binding IPs are:

bind_addr=10.10.16.203 and bind_addr=10.10.16.204


Base_path=/home/yuqing/fastdfs Change to Base_path=/home/fastdfs/storage

Store_path0=/home/yuqing/fastdfs Change to Store_path0=/home/fastdfs/storage

tracker_server=192.168.209.121:22122 instead

tracker_server=10.10.16.201:22122

tracker_server=10.10.16.202:22122

(We are equipped with two tracker services here)

run_by_group= Change to Run_by_group=fastdfs

run_by_user= Change to Run_by_user=fastdfs

# Same as Nginx port

Http.server_port=80


Firewalls open ports 23000 and 8888:

# Vi/etc/sysconfig/iptables

Increase:

-A input-m state--state new-m tcp-p TCP--dport 23000-j ACCEPT

-A input-m state--state new-m tcp-p TCP--dport 8888-j ACCEPT

To restart the firewall:

#/etc/init.d/iptables Restart


Custom storage service startup, shutdown, restart scripts:

# su Fastdfs

After switching back to the FASTDFS user

$ cd/home/fastdfs/storage/

(1) Startup script:

[Email protected] storage]$ VI start_storage.sh

/usr/local/fastdfs/bin/fdfs_storaged/usr/local/fastdfs/conf/storage.conf


(2) Close the script (using Fastdfs's own stop.sh script):

[Email protected] storage]$ VI stop_tracker.sh

/usr/local/fastdfs/bin/stop.sh/usr/local/fastdfs/bin/fdfs_storaged/usr/local/fastdfs/conf/storage.conf


(3) Restart the script (using the restart.sh script that comes with Fastdfs):

[Email protected] storage]$ VI restart_storage.sh

/usr/local/fastdfs/bin/restart.sh/usr/local/fastdfs/bin/fdfs_storaged/usr/local/fastdfs/conf/storage.conf


Note: Do not use the kill-9 parameter to kill the process, otherwise it may cause the Binlog data loss problem


To assign executable permissions to a custom script:

[Email protected] storage]$ chmod +x *_storage.sh


Start Storage Server:

[Email protected] storage]$./start_storage.sh

Data path:/home/fastdfs/storage/data, mkdir sub dir ...

mkdir Data path:00 ...

mkdir Data path:01 ...

mkdir Data path:02 ...

mkdir Data path:03 ...

(The data store directory is initialized for the first successful startup)


Restart the storage service:

[Email protected] storage]$./restart_storage.sh


To stop the storage service:

[Email protected] storage]$./stop_storage.sh


Storage Server (10.10.16.203 and 10.10.16.204) installs Nginx:

#安装Nginx使用fastdfs用户

#创建nginx日志目录

# mkdir-p/home/fastdfs/nginx/logs

# chmod A+w/home/fastdfs/nginx/logs

# Chown-r Fastdfs:fastdfs/home/fastdfs/nginx/logs


Install nginx1.4.4:

# CD/USR/LOCAL/SRC

#下载 Nginx

[Email protected] src]#

wget http://nginx.org/download/nginx-1.4.4.tar.gz

Extract:

# TAR-ZXVF Nginx-1.4.4.tar.gz

# CD nginx-1.4.4


#./configure--user=fastdfs--group=fastdfs--prefix=/usr/local/nginx--with-http_stub_status_module


If you have the following error message:

Checking for PCRE library ... not found

Checking for PCRE library in/usr/local/... not found

Checking for PCRE library in/usr/include/pcre/... not found

Checking for PCRE library in/usr/pkg/... not found

Checking for PCRE library in/opt/local/... not found


./configure:error:the HTTP Rewrite module requires the PCRE library.

You can either disable the module by using--without-http_rewrite_module

option, or install the PCRE library into the system, or build the PCRE library

Statically from the source with Nginx by using--with-pcre=<path> option.


(probably means: HTTP redirect Module installation needs Pcre library, online search, it is necessary to install Pcre-devel and openssl-devel)

# yum Install pcre-devel OpenSSL openssl-devel

(RHEL6.3 x64 in desktop version with the above package)


Then re-execute:

#./configure--user=fastdfs--group=fastdfs--prefix=/usr/local/nginx--with-http_stub_status_module


#--with-http_stub_status_module is used to monitor the current state of Nginx.

Configuration Summary

+ Using System PCRE Library

+ OpenSSL Library is not used

+ md5:using System Crypto Library

+ sha1:using System Crypto Library

+ Using System zlib Library


Nginx path prefix: "/usr/local/nginx"

Nginx binary file: "/usr/local/nginx/sbin/nginx"

Nginx configuration prefix: "/usr/local/nginx/conf"

Nginx configuration file: "/usr/local/nginx/conf/nginx.conf"

Nginx pid file: "/usr/local/nginx/logs/nginx.pid"

Nginx error log file: "/usr/local/nginx/logs/error.log"

Nginx HTTP access log file: "/usr/local/nginx/logs/access.log"

Nginx HTTP client request body Temporary files: "Client_body_temp"

Nginx HTTP proxy temporary files: "Proxy_temp"

Nginx http fastcgi temporary files: "Fastcgi_temp"

Nginx http Uwsgi temporary files: "Uwsgi_temp"

Nginx http scgi temporary files: "Scgi_temp"


Perform installation: # make && make install

To install the Fastdfs-nginx-module plugin:

# CD/USR/LOCAL/SRC

# wget http://fastdfs.googlecode.com/files/fastdfs-nginx-module_v1.15.tar.gz

# TAR-ZXVF Fastdfs-nginx-module_v1.15.tar.gz


#修改插件配置文件

# Vi/usr/local/src/fastdfs-nginx-module/src/config

Ngx_addon_name=ngx_http_fastdfs_module

http_modules= "$HTTP _modules ngx_http_fastdfs_module"

ngx_addon_srcs= "$NGX _addon_srcs $ngx _addon_dir/ngx_http_fastdfs_module.c"

core_incs= "$CORE _incs/usr/local/fastdfs/include/fastdfs/usr/local/fastdfs/include/fastcommon/"

core_libs= "$CORE _libs-l/usr/local/fastdfs/lib-lfastcommon-lfdfsclient"

cflags= "$CFLAGS-d_file_offset_bits=64-dfdfs_output_chunk_size= ' 256*1024 '-dfdfs_mod_conf_filename= ' \"/usr/local /fastdfs/conf/mod_fastdfs.conf\ "'"


#复制mod_fastdfs. conf to the/usr/local/fastdfs/conf/directory

# cp/usr/local/src/fastdfs-nginx-module/src/mod_fastdfs.conf/usr/local/fastdfs/conf/


The dynamic link library used at compile time:

#启动nginx报错

#./nginx:error while loading shared libraries:libfastcommon.so:cannot open Shared object File:no such file or director Y

#解决办法---> Add/usr/local/fastdfs/lib to your system file/etc/ld.so.conf

# vi/etc/ld.so.conf

Include ld.so.conf.d/*.conf

/usr/local/fastdfs/lib


#更新库文件缓存ld. So.cache

#/sbin/ldconfig-v


To compile the Fastdfs-nginx-module module:

# Recompile Install Nginx

# cd/usr/local/src/nginx-1.4.4

#./configure--ADD-MODULE=/USR/LOCAL/SRC/FASTDFS-NGINX-MODULE/SRC


# Make && make install


To modify the mod_fastdfs.conf configuration:

# vi/usr/local/fastdfs/conf/mod_fastdfs.conf


# Connect Timeout in seconds

# Default value is 30s

Connect_timeout=30


# The base path to store log files

Base_path=/tmp


# If load Fastdfs parameters from tracker server

# since V1.12

# default value is False

Load_fdfs_parameters_from_tracker=true


# if use storage ID instead of IP address

# Same as Tracker.conf

# valid only if Load_fdfs_parameters_from_tracker is False

# default value is False

# since V1.13

USE_STORAGE_ID = True


# Fastdfs Tracker_server can ocur more than once, and Tracker_server format is

# "Host:port", host can be hostname or IP address

# valid only if Load_fdfs_parameters_from_tracker is true

tracker_server=10.10.16.201:22122

tracker_server=10.10.16.202:22122


# The port of the local storage server

# The default value is 23000

storage_server_port=23000


# The group name of the local storage server

Group_name=group1


# if the Url/uri including the group name

# set to False when Uri Like/m00/00/00/xxx

# set to True when URIs like ${group_name}/m00/00/00/xxx, such as Group1/m00/xxx

# default value is False

Url_have_group_name = True


# path (disk or mount point) count, default value is 1

# must same as storage.conf

Store_path_count=1


# store_path#, based 0, if Store_path0 not exists, it's value is Base_path

# The paths must be exist

# must same as storage.conf

Store_path0=/home/fastdfs/storage


# Set the log filename, such as/usr/local/apache2/logs/mod_fastdfs.log

# Empty for output to stderr (Apache and nginx error_log file)

Log_filename=/home/fastdfs/nginx/logs/mod_fastdfs.log



Note that when downloading the old newspaper 404. The nginx.conf first row usr Nobody is modified to be resolved after the user root restarts.


Make a soft connection to storage's storage directory

# ln-s/HOME/FASTDFS/STORAGE/DATA/HOME/FASTDFS/STORAGE/DATA/M00


Access path with group name (storage corresponds to multiple group cases), such as/group1/m00/00/00/xxx:

Location ~/group ([0-9])/m00 {

Ngx_fastdfs_module;

}



Nginx Configuration Concise version:

# vi/usr/local/nginx/conf/nginx.conf

User root root;

Worker_processes 2;

Error_log/home/fastdfs/nginx/logs/error.log notice;

Pid/home/fastdfs/nginx/logs/nginx.pid;

Worker_rlimit_nofile 5120;


Events {

Use Epoll;

Worker_connections 5120;

}


HTTP {

Include Mime.types;

Default_type Application/octet-stream;

Sendfile on;

Tcp_nopush on;

Keepalive_timeout 65;

Tcp_nodelay on;

server {

Listen 80;

server_name localhost;

#charset Koi8-r;

Location ~/group ([0-9])/m00 {

#alias/home/fastdfs/storage/data;

Ngx_fastdfs_module;

}

}

}



#启动nginx:

#/usr/local/nginx/sbin/nginx


#重启nginx:

#/usr/local/nginx/sbin/nginx-s Reload


To upload a file with a Java client:

Browse

650) this.width=650; "src=" Http://s2.51cto.com/wyfs02/M02/85/F2/wKiom1ewaQHC2IWGAAJo6c5kjoQ370.png "title=" Dfs1.png "alt=" Wkiom1ewaqhc2iwgaajo6c5kjoq370.png "/>

650) this.width=650; "src=" Http://s5.51cto.com/wyfs02/M00/85/F2/wKioL1ewaZbTHgmYAAJpWVjmQU8564.png "title=" Dfs2.png "alt=" Wkiol1ewazbthgmyaajpwvjmqu8564.png "/>

File access is balanced by hardware

vip:10.10.16.209 (The virtual machine gateway for load balancing to be set to 10.10.16.8)

650) this.width=650; "src=" Http://s5.51cto.com/wyfs02/M01/85/F2/wKioL1ewabvT_o-6AAJp5ofWwBE101.png "title=" Dfs3.png "alt=" Wkiol1ewabvt_o-6aajp5ofwwbe101.png "/>


5,A

6,A

7,A

8,A

9,A


Sample Documentation Video Tutorial: Http://www.roncoo.com/details/85d6008fe77c4199b0cdd2885eaeee53

Fastdfs Distributed File System cluster installation

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.