Install FastDFS + nginx module on Centos7

Install FastDFS + nginx module on Centos7
1. About FastDFS

From: http://www.oschina.net/p/fastdfs
FastDFS is an open-source distributed file system that manages files, including file storage, file synchronization, and file access (File Upload and file download, it solves the problems of large-capacity storage and load balancing. It is especially suitable for online services with files as the carrier, such as photo album websites and video websites.

The FastDFS server has two roles: tracker and storage ). The tracker is mainly used for scheduling and serves as a server Load balancer for access.

The storage node stores files and implements all the functions of file management: storage, synchronization, and access interfaces. FastDFS also manages the meta data of files. The meta data of a file is related to the file. It is expressed in key value pair mode, for example, width = 1024, where key is width and value is 1024. The object meta data is a file property list that can contain multiple key-value pairs.

GitHub: https://github.com/happyfish100/fastdfs
A good blog written: http://tech.uc.cn /? P = 221

2. Install FastDFS

Reference:
Https://github.com/happyfish100/fastdfs/blob/master/INSTALL

2.1 install the compilation tool

yum -y groupinstall 'Development Tools' yum -y install wget 

• 1
• 2

2.2 install the libfastcommon class library

Follow this command first, otherwise an error is returned:

cc -Wall -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -g -O -DDEBUG_FLAG -DOS_LINUX -DIOEVENT_USE_EPOLL -c -o ../common/fdfs_global.o ../common/fdfs_global.c  -I../common -I/usr/include/fastcommon../common/fdfs_global.c:20:20: fatal error: logger.h: No such file or directory #include "logger.h"                    ^compilation terminated.........

• 1
• 2
• 3
• 4
• 5
• 6

Download and install it directly:

wget https://github.com/happyfish100/libfastcommon/archive/master.zipunzip master.zipcd libfastcommon-master./make.sh./make.sh install

• 1
• 2
• 3
• 4
• 5

2.3 install FastDFS

wget  https://github.com/happyfish100/fastdfs/archive/V5.05.tar.gztar -zxvf V5.05.tar.gz cd fastdfs-5.05/./make.sh./make.sh install

• 1
• 2
• 3
• 4
• 5
• 6

2.4, start FastDFS

After installation, the configuration file is in the/etc/fdfs directory.

mv storage.conf.sample  storage.confmv tracker.conf.sample tracker.conf

• 1
• 2

First, modify the configuration file:/etc/fdfs/tracker. conf and change the path to the/data/fdfs directory.

# the base path to store data and log filesbase_path=/data/fdfs

• 1
• 2

Start:/usr/bin/fdfs_trackerd/etc/fdfs/tracker. conf start

Modify the configuration file:/etc/fdfs/storage. conf, modify the path to the/data/fdfs directory, and configure the tracker_server address.

# the base path to store data and log filesbase_path=/data/fdfs# tracker_server can ocur more than once, and tracker_server format is#  "host:port", host can be hostname or ip addresstracker_server=192.168.1.36:22122# store_path#, based 0, if store_path0 not exists, it's value is base_path# the paths must be existstore_path0=/data/fdfs#store_path1=/home/yuqing/fastdfs2

• 1
• 2
• 3
• 4
• 5
• 6
• 7
• 8
• 9

/Usr/bin/fdfs_storaged/etc/fdfs/storage. conf start

When the log is started, the startup is successful if there is no error in the log. View fdfs processes:

#ps -ef | grep fdfsroot     19880     1  0 21:42 ?        00:00:00 /usr/bin/fdfs_trackerd /etc/fdfs/tracker.conf startroot     20050     1  0 22:08 ?        00:00:00 /usr/bin/fdfs_storaged /etc/fdfs/storage.conf startroot     20060 19915  0 22:08 pts/1    00:00:00 grep --color=auto fdfs

• 1
• 2
• 3
• 4

2.5 install nginx + lua + fastdfs module download fastdfs-nginx-module https://github.com/happyfish100/fastdfs-nginx-module download openresty nginx + lua package

wget https://github.com/happyfish100/fastdfs-nginx-module/archive/master.zipunzip fastdfs-nginx-module-master.zipwget http://openresty.org/download/ngx_openresty-1.7.10.1.tar.gztar -zxvf ngx_openresty-1.7.10.1.tar.gz

• 1
• 2
• 3
• 4

Compile and install: note that you need to add parameters. One is the stub_status statistics module and the other is the realip module. Http://blog.danielss.com /? P = 80 http://wiki.nginx.org/HttpRealipModuleChs

yum -y install pcre-devel openssl openssl-develcd ngx_openresty-1.7.10.1./configure --with-luajit --with-http_stub_status_module --with-http_ssl_module --with-http_realip_module --add-module=/root/fastdfs-nginx-module-master/src gmakegmake install

• 1
• 2
• 3
• 4
• 5

Nginx + lua installation path:

  nginx path prefix: "/usr/local/openresty/nginx"  nginx binary file: "/usr/local/openresty/nginx/sbin/nginx"  nginx configuration prefix: "/usr/local/openresty/nginx/conf"  nginx configuration file: "/usr/local/openresty/nginx/conf/nginx.conf"  nginx pid file: "/usr/local/openresty/nginx/logs/nginx.pid"  nginx error log file: "/usr/local/openresty/nginx/logs/error.log"  nginx http access log file: "/usr/local/openresty/nginx/logs/access.log"

• 1
• 2
• 3
• 4
• 5
• 6
• 7

Start and restart nginx

# Start/usr/local/openresty/nginx/sbin/nginx # restart/usr/local/openresty/nginx/sbin/nginx-s restart

• 1
• 2
• 3
• 4

3. Test FastDFS.

Configuration Service:

cd /etc/fdfsmv client.conf.sample client.conf

• 1
• 2

Modify Configuration:

# the base path to store log filesbase_path=/data/fdfs# tracker_server can ocur more than once, and tracker_server format is#  "host:port", host can be hostname or ip addresstracker_server=192.168.1.36:22122

• 1
• 2
• 3
• 4
• 5
• 6

Configure nginx:

cp /root/fastdfs-nginx-module-master/src/mod_fastdfs.conf /etc/fdfs/mkdir /data/ngx

• 1
• 2

Modify the mod_fastdfs.conf configuration file: Modify the tracker port; otherwise, nginx cannot be started. The path must be modified. Otherwise, the file cannot be found.

# the base path to store log filesbase_path=/data/ngx# FastDFS tracker_server can ocur more than once, and tracker_server format is#  "host:port", host can be hostname or ip address# valid only when load_fdfs_parameters_from_tracker is truetracker_server=192.168.1.36:22122# store_path#, based 0, if store_path0 not exists, it's value is base_path# the paths must be exist# must same as storage.confstore_path0=/data/fdfs

• 1
• 2
• 3
• 4
• 5
• 6
• 7
• 8
• 9

Copy the configuration file. Otherwise, nginx cannot be started and an error is returned:

[2015-04-28 11:30:14] ERROR - file: ini_file_reader.c, line: 315, include file "http.conf" not exists, line: "#include http.conf"[2015-04-28 11:30:14] ERROR - file: /root/fastdfs-nginx-module-master/src/common.c, line: 155, load conf file "/etc/fdfs/mod_fastdfs.conf" fail, ret code: 22015/04/28 11:30:14 [alert] 19969#0: worker process 2753 exited with fatal code 2 and cannot be respawned

• 1
• 2
• 3
• 4

Solution:

cp /root/fastdfs-master/conf/http.conf /root/fastdfs-master/conf/mime.types  /etc/fdfs

• 1

Nginx Configuration Modification vi/usr/local/openresty/nginx/conf/nginx. conf

    server {        listen       80;        server_name  localhost;        charset utf8;        location /M00 {                alias /data/fdfs/data;                ngx_fastdfs_module;        }   ......

• 1
• 2
• 3
• 4
• 5

Create a connection

ln -s /data/fdfs/data /data/fdfs/data/M00

• 1

Restart nginx. Test command:

echo "test fastdfs" > test.html /usr/bin/fdfs_test /etc/fdfs/client.conf upload test.html

• 1
• 2

Upload successful. Execution result:

This is FastDFS client test program v5.05Copyright (C) 2008, Happy Fish / YuQingFastDFS may be copied only under the terms of the GNU GeneralPublic License V3, which may be found in the FastDFS source kit.Please visit the FastDFS Home Page http://www.csource.org/for more detail.[2015-04-25 18:06:09] DEBUG - base_path=/data/fdfs, connect_timeout=30, network_timeout=60, tracker_server_count=1, anti_steal_token=0, anti_steal_secret_key length=0, use_connection_pool=0, g_connection_pool_max_idle_time=3600s, use_storage_id=0, storage server id count: 0tracker_query_storage_store_list_without_group:        server 1. group_name=, ip_addr=192.168.1.36, port=23000group_name=group1, ip_addr=192.168.1.36, port=23000storage_upload_by_filenamegroup_name=group1, remote_filename=M00/00/00/wKgC1FU7ZxGAF19vAAAADcvBslg01.htmlsource ip address: 192.168.1.36file timestamp=2015-04-25 18:06:09file size=13file crc32=3418468952example file url: http://192.168.1.36/group1/M00/00/00/wKgC1FU7ZxGAF19vAAAADcvBslg01.htmlstorage_upload_slave_by_filenamegroup_name=group1, remote_filename=M00/00/00/wKgC1FU7ZxGAF19vAAAADcvBslg01_big.htmlsource ip address: 192.168.1.36file timestamp=2015-04-25 18:06:09file size=13file crc32=3418468952example file url: http://192.168.1.36/group1/M00/00/00/wKgC1FU7ZxGAF19vAAAADcvBslg01_big.html

Test Access:

curl http://192.168.1.36/M00/00/00/wKgC1FU7ZxGAF19vAAAADcvBslg01_big.html

• 1

If "test fastdfs" is returned, the configuration is successful. The above steps must be correct.

View nginx error logs: tail-f/usr/local/openresty/nginx/logs/error. log

4. Summary

FastDFS is a very good distributed file system. The configuration is simple and can be deployed on multiple machines. At the same time, an official Nginx module is provided for direct access using nginx, which is very convenient. This reduces the middle tomcat layer and increases efficiency. Next, we will continue to study lua's image compression. Compress images directly on the nginx layer. Higher efficiency, while reading FastDFS files.