Fastdfs Distributed File system installation (single node)

The picture is too many, temporarily does not upload.

I. Environment and installation package
Deployment environment
Tracking device: 192.168.137.131 (kp-dfs-tracker-01)
Storage server: 192.168.137.132 (kp-dfs-storage-01)
Operating system: Centos-6.6-x86_64-bin-dvd1.iso
Action User: Root
Data directory:/fastdfs (note: Data directory According to your data disk mount path depends)


installation package
Fastdfs-5.11.tar.gz (Version 5.11)
Https://github.com/happyfish100/fastdfs
Libfastcommon-master.zip (Version 1.36)
Https://github.com/happyfish100/libfastcommon
Fastdfs-nginx-module-master.zip (Version 1.20)
Https://github.com/happyfish100/fastdfs-nginx-module
Nginx-1.13.4.tar.gz (Version 1.13.4)
Http://nginx.org/en/download.html
Fastdfs-client-java-master.zip (Version 1.26)
Https://github.com/happyfish100/fastdfs-client-java


Source Address: Https://github.com/happyfish100/
http://sourceforge.net/projects/fastdfs/files/
Fastdfs Forum: http://bbs.chinaunix.net/forum-240-1.html
Official document Configuration Note: http://bbs.chinaunix.net/thread-1941456-1-1.html




Brief introduction
Fastdfs is an open source lightweight Distributed File system that manages files, including file storage, file synchronization, file access (file uploads, file downloads), and so on, which solves the problem of large capacity storage and load balancing. Especially suitable for medium and small files (recommended range: 4KB < file_size &LT;500MB) for the carrier of online services, such as photo albums, video sites and so on.
Fastdfs for the Internet tailored to fully consider the redundant backup, load balancing, linear expansion and other mechanisms, and focus on high availability, high-performance and other indicators, the use of Fastdfs is easy to build a high-performance file server cluster to provide file upload, download and other services.
The FASTDFS consists of a tracking server (tracker server), a storage server (storage server), and a clients (client) three parts.
The Tracker (tracker) mainly makes the dispatching work, and plays the role of load balancing on the access.
The client is the caller that uses the Fastdfs service, and the client should also be a server, and its calls to tracker and storage are calls between servers.
Storage node (storage) stores files, completes all the functions of file management: storage, synchronization, and provision of access interfaces to manage meta data of files. Meta data is the file's associated attribute, expressed as a key value pair (key value pair), such as: width=1024, where the key is Width,value 1024. File meta data is a list of file attributes that can contain multiple key-value pairs.
Both the tracker and the storage node can consist of one or more servers. The servers in the tracker and storage node can be added or offline at any time without affecting the online service. All servers in the tracker are equivalent and can be increased or decreased according to the server's pressure.
To support large capacity, storage nodes (servers) are organized by volume (or grouping). The storage system consists of one or more volumes, the files between the volumes are independent of each other, and the file capacity of all volumes is the file capacity of the entire storage system. A volume can consist of one or more storage servers, the files in a storage server under a volume are the same, and multiple storage servers in a volume play a role of redundant backups and load balancing.
When the server is added to the volume, the synchronized files are automatically completed by the system, and after synchronization is completed, the system automatically switches the new server to the online service. Volumes can be added dynamically when storage space is low or is about to run out. Only one or more servers need to be added and configured as a new volume, which expands the capacity of the storage system.


The FASTDFS system structure is shown below.







Both the tracking server and the storage server perform the following installation operations
1, compile and install the required dependency pack
You need to enter Y for the installation process.
# yum install make CMake gcc gcc-c++





2. Installation Libfastcommon
(1) Upload libfastcommon-master.zip to/usr/local/src directory.
(2) Decompression
# cd/usr/local/src/
# Unzip Libfastcommon-master.zip
# CD Libfastcommon-master


(3) Compile and install
#./make.sh
#./make.sh Install


Libfastcommon is installed by default to/usr/lib64/libfastcommon.so and/usr/lib/libfastcommon.so.
(4) Because the Lib directory set by the FASTDFS main program is/usr/local/lib, you need to create a soft link.
# cd/usr/lib64
# ls | grep Libfastcommon
# ln-s/usr/lib64/libfastcommon.so/usr/local/lib/libfastcommon.so


3. Installation Fastdfs
(1) Upload fastdfs-5.11.tar.gz source package to/usr/local/src directory.
(2) Decompression
# cd/usr/local/src/
# TAR-ZXVF Fastdfs-5.11.tar.gz
# MV fastdfs-5.11 Fastdfs
# CD Fastdfs
(3) Compile and install
Make sure that you have successfully installed the Libfastcommon before compiling.
#./make.sh
#./make.sh Install




Install by default, corresponding files and directories after installation:
A, service scripts in:
/etc/init.d/fdfs_storaged
/etc/init.d/fdfs_tracker
B, configuration file in (sample configuration file):
/etc/fdfs/client.conf.sample
/etc/fdfs/storage.conf.sample
/etc/fdfs/tracker.conf.sample
C, the command tool in the/usr/bin/directory:
Fdfs_appender_test
Fdfs_appender_test1
Fdfs_append_file
Fdfs_crc32
Fdfs_delete_file
Fdfs_download_file
Fdfs_file_info
Fdfs_monitor
Fdfs_storaged
Fdfs_test
Fdfs_test1
Fdfs_trackerd
Fdfs_upload_appender
Fdfs_upload_file
View the Fastdfs Tools command.
# cd/usr/bin/
# ls | grep Fdfs



Third, configure FASTDFS tracking server
Tracking Service: 192.168.137.131 (kp-dfs-tracker-01).
1. Copy a FASTDFS Tracker sample configuration file and rename it to tracker.conf.
# cd/etc/fdfs/
# CP Tracker.conf.sample tracker.conf

2, edit the tracker configuration file.
# vi/etc/fdfs/tracker.conf
The modifications are as follows, and the other parameters retain the default configuration:
Disabled=false #启用配置文件
port=22122 #tracker的端口号, generally using the default port of 22122
Base_path=/fastdfs/tracker #tracker的数据文件和日志目录



3, create the basic data directory, and the configuration of the underlying directory Base_path path.
# mkdir-p/fastdfs/tracker


4, edit the firewall file, open the tracker port, the default port is 22122.
# Vi/etc/sysconfig/iptables
Add the following port lines to the file:
# # Fastdfs Tracker Port
-A input-m state--state new-m tcp-p TCP--dport 22122-j ACCEPT
To reboot the Firewall service:
# Service Iptables Restart


5, Start tracker tracker.
#/etc/init.d/fdfs_trackerd Start

The first boot is successful, the data, logs two directories are created in the/fastdfs/tracker directory.
See if the Fastdfs Tracker tracker has started successfully.
# Ps-ef | grep Fdfs



Check the port monitor through the netstat command to confirm that tracker is starting normally.
# Netstat-unltp|grep Fdfs



View the Tracker startup log with the following command.
# tail-f/fastdfs/tracker/logs/trackerd.log


6, close the tracker tracker.
#/etc/init.d/fdfs_trackerd Stop

7, set up Fastdfs tracker tracker boot.
Edit the Rc.local file.
# vi/etc/rc.d/rc.local
Add the following:
# # Fastdfs Tracker
/etc/init.d/fdfs_trackerd start


Iv. Configuring FASTDFS Storage Servers
Storage server: 192.168.137.132 (kp-dfs-storage-01).
1, copy FASTDFS Storage sample configuration file, and rename.
# cd/etc/fdfs/
# CP Storage.conf.sample storage.conf



2, edit the memory sample configuration file.
# vi/etc/fdfs/storage.conf
The modifications are as follows, and the other parameters retain the default configuration:
Disabled=false
port=23000
Base_path=/fastdfs/storage
Store_path0=/fastdfs/storage
tracker_server=192.168.137.131:22122
http.server_port=8888





3, create the basic data directory, and the configuration of the underlying directory Base_path path.
# mkdir-p/fastdfs/storage


4, edit the firewall file, open the memory port, the default port is 23000.
# Vi/etc/sysconfig/iptables
Add the following port lines to the file:
# # Fastdfs Storage Port
-A input-m state--state new-m tcp-p TCP--dport 23000-j ACCEPT
To reboot the Firewall service:
# Service Iptables Restart

5, start storage memory.
#/etc/init.d/fdfs_storaged Start
The first boot is successful, the data, logs two directories are created in the/fastdfs/storage directory.
See if the FASTDFS storage storage has started successfully.
# Ps-ef | grep Fdfs

Check the port monitor through the netstat command to confirm that storage is starting normally:
# Netstat-unltp|grep Fdfs



6, stop storage memory.
#/etc/init.d/fdfs_storaged Stop


7, set up FASTDFS storage memory boot.
Edit the Rc.local file.
# vi/etc/rc.d/rc.local
Add the following:
# # Fastdfs Storage
/etc/init.d/fdfs_storaged start


8, view the cluster information.
To see if the storage server has been enlisted to the tracker server, tracker and storage are successfully consolidated, run the following command.
#/usr/bin/fdfs_monitor/etc/fdfs/storage.conf

As shown above, you see the 192.168.137.132 (kp-dfs-storage-01) active typeface stating that the storage server has successfully enlisted to the tracker server.


V. File upload Test (192.168.137.131)
1. Modify the client configuration file in the tracker tracking service.
# cp/etc/fdfs/client.conf.sample/etc/fdfs/client.conf
# vi/etc/fdfs/client.conf
The revised content is as follows:
Base_path=/fastdfs/tracker
tracker_server=192.168.137.131:22122


2, execute the following file Upload command.
#/usr/bin/fdfs_upload_file/etc/fdfs/client.conf/usr/local/src/fastdfs-5.11.tar.gz
Returns the file ID number: group1/m00/00/00/wkijhfmwy4aafn6jaaukkwe5se4.tar.gz
Can return the file ID number, indicating that the file upload success, each upload returns the file ID number is different.




The filename is pieced together by group, storage directory, level two subdirectory, Fileid, file suffix name, as follows:

There are 256 level 1 directories under the/fastdfs/storage/data directory, 256 2 level subdirectories under each level, a total of 65,536 files, and newly written files are routed to one of these subdirectories in a hash manner. The file data is then stored directly as a local file into the directory.
# Cd/fastdfs/storage
# ls
# CD Data
# ls
# CD 00
# ls
# CD 00
# ls



Vi. install Nginx on each storage Storage server node
1, Fastdfs-nginx-module function.
Fastdfs files are stored on the storage server through the tracker server, but there is a problem with synchronization latency between the same group of storage servers and the need to access file replication.
Suppose the tracker server uploads the file to 192.168.137.132, and the file ID is returned to the client after the upload succeeds. At this point, the FASTDFS storage cluster mechanism synchronizes this file to the same group of storage 192.168.137.133, and if the client uses this file ID to fetch files on 192.168.137.133, the file cannot be accessed if the file is not replicated.
Fastdfs-nginx-module can redirect files to connect to the source server to fetch files, avoiding errors caused by the client due to replication latency.


2, upload fastdfs-nginx-module-master.zip to/usr/local/src directory.


3. Extract and rename files
# cd/usr/local/src/
# Unzip Fastdfs-nginx-module-master.zip
# MV Fastdfs-nginx-module-master Fastdfs-nginx-module
4, modify the Config profile in Fastdfs-nginx-module.
# Vi/usr/local/src/fastdfs-nginx-module/src/config
Check that the Core_incs configuration is as follows, and if not, modify it.
core_incs= "$CORE _incs/usr/include/fastdfs/usr/include/fastcommon/"

5, upload nginx-1.13.4.tar.gz to/usr/local/src directory.


6. Install the required dependencies for compiling the Nginx, the installation process needs to enter Yes or Y.
# yum Install gcc gcc-c++ make automake autoconf libtool pcre* zlib OpenSSL openssl-devel

7, compile and install Nginx, add Fastdfs-nginx-module module.
# cd/usr/local/src/
# TAR-ZXVF Nginx-1.13.4.tar.gz
# CD nginx-1.13.4
#./configure--ADD-MODULE=/USR/LOCAL/SRC/FASTDFS-NGINX-MODULE/SRC
# Make && make install


8, copy the configuration file in the Fastdfs-nginx-module to the/etc/fdfs directory, and modify the configuration.
# cp/usr/local/src/fastdfs-nginx-module/src/mod_fastdfs.conf/etc/fdfs/
# vi/etc/fdfs/mod_fastdfs.conf
Modify the following configuration:
connect_timeout=10
Base_path=/tmp
tracker_server=192.168.137.131:22122
storage_server_port=23000
Group_name=group1
Url_have_group_name = True
Store_path0=/fastdfs/storage


9, copy the Fastdfs two configuration files to the/etc/fdfs directory.
# cd/usr/local/src/fastdfs/conf
# CP http.conf mime.types/etc/fdfs/


10, in the/fastdfs/storage file storage directory to create a soft connection, link it to the actual data storage directory.
# ln-s/fastdfs/storage/data//fastdfs/storage/data/m00


11. Configure Nginx
# Cd/usr/local/nginx
# CD Conf
# VI Nginx.conf
Concise version of the Nginx configuration sample:
User root;
Worker_processes 1;
Events {
Worker_connections 1024;
}
HTTP {
Include Mime.types;
Default_type Application/octet-stream;
Sendfile on;
Keepalive_timeout 65;
server {
Listen 8888;
server_name localhost;
Location ~/group ([0-9])/m00 {
#alias/fastdfs/storage/data;
Ngx_fastdfs_module;
}
Error_page 502 503 504/50x.html;
Location =/50x.html {
root HTML;
}
}
}

Attention:
(1), the listen 8888 port value should correspond to the http.server_port=8888 in the file/etc/fdfs/storage.conf, because the Http.server_port defaults to 8888, if you want to change to 80, You have to change it.
(2), storage corresponding to have multiple group groupings, access path with group name, such as/group1/m00/00/00/xxx,
The corresponding Nginx configuration is:
Location ~/group ([0-9])/m00 {
Ngx_fastdfs_module;
}
(3), if the download is always reported error 404, will nginx.conf the first line of user nobody modified to user root, restart the Nginx service.


12, open the Nginx 8888 port in the firewall.
# Vi/etc/sysconfig/iptables
Add the following:
# # Nginx Port
-A input-m state--state new-m tcp-p TCP--dport 8888-j ACCEPT
To reboot the Firewall service:
# Service Iptables Restart
13, start Nginx.
#/usr/local/nginx/sbin/nginx

Reboot Nginx Command:
/usr/local/nginx/sbin/nginx-s Reload
Stop Nginx Command:
/usr/local/nginx/sbin/nginx-s stop


14, set up Nginx boot.
Use Chkconfig to manage nginx services. Create a Nginx script file under the/etc/init.d/directory.
# Vi/etc/init.d/nginx
Add the following in the Nginx script file:
#!/bin/sh
#
# Nginx-this script starts and stops the Nginx daemon
#
# Chkconfig:-85 15
# Description:nginx is a HTTP (s) server, HTTP (s) reverse \
# Proxy and IMAP/POP3 proxy server
# Processname:nginx
# config:/etc/nginx/nginx.conf
# config:/etc/sysconfig/nginx
# Pidfile:/var/run/nginx.pid


# Source function library.
. /etc/rc.d/init.d/functions


# Source Networking configuration.
. /etc/sysconfig/network


# Check that networking are up.
["$NETWORKING" = "no"] && exit 0


nginx= "/usr/local/nginx/sbin/nginx"
prog=$ (basename $nginx)


Nginx_conf_file= "/usr/local/nginx/conf/nginx.conf"


[-f/etc/sysconfig/nginx] &&. /etc/sysconfig/nginx


Lockfile=/var/lock/subsys/nginx


Make_dirs () {
# Make required Directories
User= ' $nginx-v 2>&1 | grep "Configure Arguments:.*--user=" | Sed ' s/[^*]*--user=\ ([^]*\). */\1/g '-'
If [-N "$user"]; Then
If [-Z "' grep $user/etc/passwd ']"; Then
Useradd-m-s/bin/nologin $user
Fi
Options= ' $nginx-v 2>&1 | grep ' Configure arguments: '
for opt in $options; Todo
If [' Echo $opt | grep '. *-temp-path ']; Then
Value= ' echo $opt | Cut-d "=" F 2 '
if [!-D "$value"]; Then
# echo "Creating" $value
Mkdir-p $value && chown-r $user $value
Fi
Fi
Done
Fi
}


Start () {
[x $nginx] | | Exit 5
[f $NGINX _conf_file] | | Exit 6
Make_dirs
Echo-n $ "Starting $prog:"
Daemon $nginx-C $NGINX _conf_file
Retval=$?
Echo
[$retval-eq 0] && Touch $lockfile
Return $retval
}


Stop () {
Echo-n $ "Stopping $prog:"
Killproc $prog-quit
Retval=$?
Echo
[$retval-eq 0] && rm-f $lockfile
Return $retval
}


Restart () {
Configtest | | Return $?
Stop
Sleep 1
Start
}


Reload () {
Configtest | | Return $?
Echo-n $ "Reloading $prog:"
Killproc $nginx-hup
Retval=$?
Echo
}


Force_reload () {
Restart
}


Configtest () {
$nginx-T-C $NGINX _conf_file
}


Rh_status () {
Status $prog
}


Rh_status_q () {
Rh_status >/dev/null 2>&1
}


Case "$" in
Start
Rh_status_q && Exit 0
$
;;
Stop
Rh_status_q | | Exit 0
$
;;
Restart|configtest)
$
;;
Reload
Rh_status_q | | Exit 7
$
;;
Force-reload)
Force_reload
;;
Status
Rh_status
;;
Condrestart|try-restart)
Rh_status_q | | Exit 0
;;
*)
echo $ "Usage: $ {start|stop|status|restart|condrestart|try-restart|reload|force-reload|configtest}"
Exit 2
Esac
Note that the following must be consistent with the installation settings path:
Nginx the path to execute the program.
nginx= "/usr/local/nginx/sbin/nginx"
The path to the Nginx configuration file.
Nginx_conf_file= "/usr/local/nginx/conf/nginx.conf"
The script comes from the official website of Nginx. https://www.nginx.com/resources/wiki/start/topics/examples/redhatnginxinit/

To set file execution permissions after saving the script file:
# chmod A+x/etc/init.d/nginx
First add the Nginx service to the Chkconfig management list:
# chkconfig--add/etc/init.d/nginx
Use the service to initiate the nginx operation.
# service Nginx Start
Stop the Nginx command.
# Service Nginx Stop
Restart the Nginx command.
# Service Nginx Restart
Setup Nginx Boot:
# chkconfig Nginx on


15, through the browser to access the files uploaded during the test.
Http://192.168.137.132:8888/group1/M00/00/00/wKiJhFmWY4aAfn6JAAUkKwe5sE4.tar.gz
Http://192.168.137.132:8888/group1/M00/00/00/wKiJhFmWqo6AW1JBAAKz9NDPGdk780.jpg





Fastdfs upload/download process:

Upload process:
1. Client asks Tracker to connect to a port specified by a Tracker server.
2. Tracker returns an available storage.
3, client direct and storage communication to complete file upload.

Download process:
1, Client Inquiry Tracker, connect to a certain tracker server specified port, the parameter is the file ID (volume name and file name).
2, tracker return a usable storage.
3, client direct and storage communication to complete the file download.



Close command: Service iptables stop
Permanently shutdown firewall: chkconfig iptables off
View Firewall shutdown Status: Service iptables status