Nginx+fastdfs-nginx-module+fastdfs Configuring a Distributed File system

Introduction

Fastdfs is an open source, lightweight, distributed file system that manages files, including file storage, file synchronization, file access (file upload, file download), and so on, which solves the problem of large capacity storage and load balancing. Especially suitable for file-based online services, such as photo album sites, video sites and so on.

The FASTDFS server has two roles: Tracker (tracker) and storage node (storage). The tracker mainly does the dispatching work, and the function of load balancing on the access.

Tracker Server: tracking server, mainly to do scheduling work, play a balanced role, responsible for the management of all storage server and group, each storage after launch will connect tracker, inform oneself belong to group and other information , and maintain a periodic heartbeat, tracker based on the storage heartbeat information, set up a group--->[storage server List] mapping table, tracker management of the metadata is very small, will be stored directly in memory; the meta-information on the tracker is Storage reported information generated, itself does not need to persist any data, tracker is a peer relationship, so it is very easy to extend the tracker service, directly increase the tracker server, all tracker accept stroage heartbeat information, Generate metadata information to provide read/write services (with the benefit of other master-slave architectures there is no single point, tracker is not a bottleneck, and the final data is transferred with an available storage Server) Note: The group data and server data managed by tracker can be found Storage_groups_new.dat, Storage_servers_new.dat, in the tracker path.

Storage server: Storage server, mainly provides capacity and backup services, in group, each group can contain multiple storage servers, data is backed up, storage capacity space in the group content of the smallest storage; The storage server configuration within the group is the same; the organization of storage in group is convenient for application isolation, load balancing and replica number customization; The disadvantage is that the capacity of group is limited by the capacity of single-machine storage, and the number of machines in group is broken. Recovery can only rely on other machines in the group resynchronization (bad disk replacement, re-hang load fdfs_storaged can)
First , pre-installation preparation 1. System Description

Operating system: CentOS release 6.5 (Final)

ip:20.10.80.17

Host Name: Localhost.localdomain

Deployment role: Tracker storage Nginx 2. Preparing the compilation Environment

Yum-y install gcc gcc+ gcc-c++ OpenSSL openssl-devel pcre pcre-devel

Note: Refer to the file "Fastdfs prepare component installation before installation." Docx " 3. Detecting Firewalls

Detect if Firewall is on command: Service iptables status

Firewall shutdown command: Service iptables stop

Note: To facilitate testing, turn off Firewall service iptables stop

After the installation, the corresponding ports and rules can be added to the firewall (port fill in the actual port of its own configuration)

Tracker added to firewall rules

Iptables-i input-p tcp-m State--state new-m TCP--dport22122-j ACCEPT storage iptables-i input-p tcp-m State--st Ate new-m TCP--dport23000-j accept Nginx iptables-i input-p tcp-m State--state new-m TCP--dport80-j ACCEPT port to be modified into your own configuration in the actual port, modified after the following command to save/etc/init.d/iptables Save

Second, source preparation

source Download, download the unified copy to the directory/usr/local/src/

 

2.1.1 Download Fastdfs 5.01 address:

Http://jaist.dl.sourceforge.net/project/fastdfs/FastDFS%20Server%20Source%20Code/FastDFS%20Server%20with%20PHP %20extension%20source%20code%20v5.01/fastdfs_v5.01.tar

2.1.2 Download Nginx 1.7.0:

Http://nginx.org/download/nginx-1.7.0.tar.gz

2.1.3 fastdfs-nginx-module_v1.16:

Http://jaist.dl.sourceforge.net/project/fastdfs/FastDFS%20Nginx%20Module%20Source%20Code/fastdfs-nginx-module_v1.16.tar.gz

Iii. Installation and compilation 3.1 Installing Fastdfs

cd/usr/local/src/

Tar XF fastdfs_v5.01.tar.gz

CD Fastdfs

./make.sh &&./make.sh Install 3.2 Decompression Fastdfs-nginx-module

cd/usr/local/src/

TAR-ZXVF fastdfs-nginx-module_v1.16.tar.gz

 

Note: ZXVF

X: Extract the files from the tar package

Z: Indicates that the TAR package is compressed by gzip, so it needs to be decompressed with Gunzip

V: Show more information

F xxx.tar.gz: Specifies that the file being processed is xxx.tar.gz,f followed by the file name to be processed, which means you cannot write this tar-zxfvfastdfs-nginx-module_v1.16.tar.gz 3.3 Installing Nginx and attaching Fastdfs-nginx-module module

cd/usr/local/src/

Tar XF nginx-1.7.0.tar.gz

CD nginx-1.7.0

Compile and install Nginx with Fastdfs-nginx-module module

./configure--user=nginx--group=nginx--prefix=/usr/local/nginx--add-module=. /FASTDFS-NGINX-MODULE/SRC

Make

Make Install Iv. Configuration configuration of the 4.1 tracker 4.1.1 Create tracker data and log storage directory

mkdir-p/usr/local/src/fastdfs/tracker 4.1.2 Modify the tracker.conf profile of the Fastdfs (when the Fastdfs is installed,/etc/fdfs tracker.conf Two configuration files appear in the storage.conf directory)

vim/etc/fdfs/tracker.conf

Major modifications:

base_path=/usr/local/src/fastdfs/tracker

max_connections=100

work_threads=8

store_lookup=0

store_path=0

reserved_storage_space=10%

Run_by_group=root

Run_by_user=root

rotate_error_log=true

for the convenience of migrating data later, it is recommended to use the server's ID to generate the URL name of the file instead of using the default IP, if you use the ID to generate the filename URL, modify the following parameters in the tracker.conf file

# if use storage ID instead of IP address # Default value is False # since V4.00 use_storage_id = True
# Specify storage IDs filename, can use relative or absolute path # since V4.00 storage_ids_filename =/usr/local/src/fast Dfs/conf/storage_ids.conf
# ID type of the storage server in the filename, values is: # # ip:the IP address of the storage server # # Id:the Server ID of the storage server # This paramter was valid only if use_storage_id set to True # default value is IP # since V4.0 3 Id_type_in_filename = ID

Again in mod_fastdfs.conf modify the file as follows parameter according to the comment description of these two parameters can be known, can not be set only when the mod_fastdfs.conf configuration file load_fdfs_parameters_from_tracker= False to take these two parameter values in the mod_fastdfs.conf configuration file # If use storage ID instead of IP address # Same as tracker.conf # valid if Load_fdfs_parameters_from_tracker is False # default value is False # since V1.13 use_storage_id = true # specify Stora GE IDs filename, can use relative or absolute path # same as tracker.conf # valid if Load_fdfs_parameters_from_trac Ker is false # since V1.13 storage_ids_filename = /usr/local/src/fastdfs/conf/storage_ids.conf

4.1.3 Tracker to service management and set boot up

cp/usr/local/src/fastdfs/init.d/fdfs_trackerd/etc/init.d///service scripts must be stored in the/etc/ini.d/directory

chkconfig--add fdfs_trackerd//Adds the specified system service, allows the chkconfig instruction to manage it, and simultaneously adds relevant data to the system-initiated narrative file

chkconfig Fdfs_trackerd on//set boot up

Service Fdfs_trackerd Start//Even if services are started 4.2 Modifying the storage.conf configuration file

Create a data store directory

mkdir-p/usr/local/src/fastdfs/storage/data

Modifying the storage.conf configuration file for Fastdfs

vim/etc/fdfs/storage.conf

Modify the main items as follows:

work_threads=8

store_path_count=1

Base_path=/usr/local/src/fastdfs/storage

Store_path0=/usr/local/src/fastdfs/storage

# #tracker_server Address

tracker_server=20.10.80.17:22122

if it is multiple tracker_server, write more directly, as shown below

# #运行 Fastdfs user Group

Run_by_group=root

# #运行 users of Fastdfs

Run_by_user=root

file_distribute_path_mode=1

rotate_error_log=true

4.3 integrating the Fastdfs-nginx-module module mod_fastdfs.conf configuration file

Copy the Nginx module's configuration file to/etc/fdfs for modification.

cp/usr/local/src/fastdfs-nginx-module/src/mod_fastdfs.conf/etc/fdfs/

Major modification items

connect_timeout=30

tracker_server=20.10.80.17:22122

Group_name=group1

Url_have_group_name = True

store_path_count=1

Store_path0=/usr/local/src/fastdfs/storage

4.1 nginx configuration 4.4.1nginx.conf configuration file

vim/usr/local/nginx/conf/nginx.conf

The following changes are included:

User root root; ---user user groups

Worker_processes 3;

Error_log/usr/local/nginx/logs/error.log;

Error_log/usr/local/nginx/logs/error.log notice;

Error_log/usr/local/nginx/logs/error.log info;

Pid/usr/local/nginx/logs/nginx.pid;

Events {

Worker_connections 1024;

}

HTTP {

Include Mime.types;

Default_type Application/octet-stream;

Server_names_hash_bucket_size 128;

Client_header_buffer_size 32k;

Large_client_header_buffers 4 32k;

Client_max_body_size 50m;

Log_format Main ' $remote _addr-$remote _user [$time _local] "$request"

' $status $body _bytes_sent ' $http _referer '

' "$http _user_agent" "$http _x_forwarded_for";

Access_log/usr/local/nginx/logs/access.log main;

Upstream server_group1{

server20.10.80.17:80;

}

server {

Listen80;

server_name Localhost.localdomain;

Access_log off;

Integration of #这里是对nginx和fastdfs-nginx-module

location/group1/m00{
Root/usr/local/src/fastdfs/storage/data;
Ngx_fastdfs_module;
}

}

}

Note: If you want to do a storage cluster, then put all the servers in upstream server_group1{}, as shown below

4.4.2 Creating a soft connection

ln-s/usr/local/src/fastdfs/storage/data/usr/local/src/fastdfs/storage/data/m00

This soft connection can also not be created, if not created, the root directory of data is as follows:

Take Group1/m00/14/f3/oyybafapb7meteqjaaaaaagc1og742.png as an example, the picture goes directly into 14 here, and then enters the F3 to find

If you create a soft connection, the following figure

There is more than one M00 path, you can enter the M00, then enter 14, then enter the F3 to find the picture, of course, directly into the 14, into the F3 can also be found, this is just a connection.

Because we again in the Nginx location under the group1 with a M00 path. 4.4.3 give storage to service management and set boot up

cp/usr/local/src/fastdfs/init.d/fdfs_storaged/etc/init.d/

chkconfig--add fdfs_storaged

chkconfig fdfs_storaged on

Service fdfs_storaged Start

starting storage, a multilevel storage directory is automatically created based on the settings of the configuration file, as shown below:

v. Problems and Solutions

A) Nginx extension module, does not display the problem of the picture correctly

In the configuration file/etc/fdfs/mod_fastdfs.conf, the default setting is this: Http.need_find_content_type=false This parameter needs to be set to true in Nginx, should be set to False in Apache

Note: Http.need_find_content_type=false This configuration is in/etc/fdfs/mod_ Fastdfs.conf in the configuration, if the configuration file is not changed directly add, after saving to restart Nginx.

b) The request

[2011-12-1215:24:21] Error-file:/tmp/fastdfs-nginx-module/src/common.c, Line:561,logic file:m00/00/00/ Wkgbnu7lqyjzjz4maa4crxl5scq670.jpg NOT EXISTS
2011/12/12 15:24:21 [ERROR] 14147#0: * Open () "/home/nginx/nginx/html/favicon.ico" failed (2:no such file ordirectory), C lient:192.168.1.123, Server:localhost, Request: "Get/favicon.ico http/1.1", Host: "20.10.80.17:80"

Modify/fastdfs/conf/mod_fastdfs.conf inside Url_have_group_name = True

VI. Testing

1. start the Tracker,storage,nginx on 20.10.80.17

Start Tracker Service Fdfs_trackerd start
Start Storage Service fdfs_storaged start
Start Nginx/usr/local/nginx/sbin/nginx

If you start the nginx problem, you can see whether this blog refers to the problem and the solution

http://blog.csdn.net/ystyaoshengting/article/details/48681193

2. Configure a client to perform on the tracker

Vim/etc/fdfs/client.conf

Base_path=/usr/local/src/fastdfs/storage

tracker_server=20.10.80.17:22122

If there are multiple tracker, it is also necessary to write as many tracker_server as Storage.conf says.

3. View cluster details

fdfs_monitor/etc/fdfs/client.conf

The result is the following figure

4. Test Upload

Fdfs_upload_file/etc/fdfs/client.conf/usr/fastdfs/hibernate.jpg

return:/group1/m00/00/00/wkhqffwyq1iaxucbaaf9-d2hupa735.jpg

5. Browser Access Http://20.10.80.17/group1/M00/00/00/wKhQFFWYx6WAaeCIAALJTvfxx5I331.jpg detects if the upload was successful