Distributed File System FastDFS deployment

Distributed File System FastDFS deployment

I. Introduction

FastDFS is an open-source distributed file system similar to Google FS. It is implemented in pure C language and supports Linux, FreeBSD, AIX, and other UNIX systems. It can only access files through private APIs. It does not support POSIX interface and cannot be used as mount. To be precise, Google FS, FastDFS, mogileFS, HDFS, and TFS are not system-level distributed file systems, but application-level Distributed File storage services.

Ii. FastDFS Architecture

Figure 1 shows the system architecture of FastDFS.

Figure 1 system architecture of FastDFS

As shown in figure 1, Tracker servers are independent of each other and there is no direct connection between them.

The client and Storage server actively connect to the Tracker server. The Storage server reports its status information to the Tracker server, including the remaining disk space, file synchronization status, and the number of file uploads and downloads. The Storage server connects to all the Tracker servers in the cluster and reports their statuses to them. The Storage server starts a separate thread to complete the connection and timing report to a Tracker server. It should be noted that the Storage server in a group is not set through the configuration file, but obtained through the Tracker server.

Different groups of Storage servers do not communicate with each other, and Storage servers in the same group are connected to each other for file synchronization.

The Storage server uses binlog files to record file upload, deletion, and other update operations. Only the file name is recorded in binlog, but the file content is not recorded.

File synchronization is only performed between the Storage servers in the same group. The push method is used, that is, the source server is synchronized to the target server. Only the source data needs to be synchronized, And the backup data does not need to be synchronized again. Otherwise, a loop is formed. An exception is that when a new Storage server is added, an existing Storage server synchronizes all existing data (including source data and backup data) to the new server.

In the Storage server, a dedicated thread synchronizes files based on binlog. To minimize mutual influence and minimize system conciseness, the Storage server starts a thread for each server except itself in the group for file synchronization.

File synchronization adopts incremental synchronization. The system records the synchronized location (binlog file offset) to the identification file. ID file name format: {dest storage IP }_{ port}. mark, for example, 192.168.1.14 _ 23000. mark.

3. Interaction Process of file upload and download

Next, let's take a look at the interaction process of file upload and download. The file upload and download processes are shown in figure 2 and Figure 3 respectively. The file upload process is as follows:

Figure 2 File Upload Process

Figure 3 File Download Process

1. The Client queries the Storage server uploaded by the Tracker server;

2. The Tracker server returns an available Storage server. The returned data is the IP address and port of the Storage server;

3. The Client directly establishes a connection with the Storage server to upload files. The Storage server returns the new file ID, and the file upload is complete.

The steps for downloading an object are as follows:

1. The Client asks the Tracker server to download the Storage server of the specified file. The parameter is the file ID (including the group name and file name );

2. The Tracker server returns an available Storage server;

3. The Client directly establishes a connection with the Storage server to complete file download.

4. IP Address Allocation (firewall disabled by default in this article)

1.172.16.8.8 nginx tracker

2.172.16.8.9 storage1

3.172.168.10 storage2

Note: all are http://sourceforge.net/projects/fastdfs/files? Source = navbar

5. Install tracker

1. Install

# Wget

# Tar xf download

# Cd FastDFS/

#./Make. sh run the file

Error Message

Compilation terminated.

Make: *** [../common/fdfs_global.o] Error 1

2. Check the official documentation and install libfastcommon.

# Wget https://github.com/happyfish100/libfastcommon/archive/master.zip

# Unzip master.zip

# Cd libfastcommon-master/

#./Make. sh

#./Make. sh install

3. Continue to install FastDFS

# Cd FastDFS/

#./Make. sh

#./Make. sh install

4. Configuration

# Cd/etc/fdfs/

# Cp tracker. conf. sample tracker. conf

Modify configuration file

Disabled = false # enable the configuration file

Port = 22122 # Set the tracker port number

Base_path =/data/fastdfs/trackerd # Set the tracker data file and log directory (which must be created in advance)

Http. server_port = 18080 # Set the http port number

5. Start

# Mkdir-p/data/fastdfs/trackerd

#/Usr/bin/fdfs_trackerd/etc/fdfs/tracker. conf restart

Set startup

# Echo '/usr/bin/fdfs_trackerd/etc/fdfs/tracker. conf restart'>/etc/rc. d/rc. local

6. Install storage (both are the same, and one configuration is provided in this document)

1. The installation steps are the same as those of tracker.

2. Configuration

Disabled = false # enable the configuration file

Group_name = group1 # group name, which can be modified as needed

Port = 23000 # Set the storage port number

Base_path =/data/fastdfs/storage # Set the storage log directory (which must be created in advance)

Store_path_count = 1 # Number of storage paths, which must match the number of store_path

Store_path0 =/data/fastdfs/storage # storage path

Tracker_server = 172.16.8.8: 22122 # IP address and port number of the tracker server

Http. server_port = 18888 # Set the http port number

3. Start

# Mkdir-p/data/fastdfs/storage

#/Usr/bin/fdfs_storaged/etc/fdfs/storage. conf restart

Set startup

# Echo '/usr/bin/fdfs_storaged/etc/fdfs/storage. conf restart'>/etc/rc. d/rc. local

4. view the status

/Usr/bin/fdfs_monitor/etc/fdfs/storage. conf

7. Install nginx in storage

Installing nginx on storage mainly aims to improve the http access service and solve the synchronization Delay Problem of the storage service in the group.

1. nginx installation. You can use the source code for compilation. Please follow the steps below to add -- add-module =/root/fastdfs-nginx-module/src/during compilation and installation/

2. Configuration

# Wget

# Tar xf download.1

3. Solve the error

Make will report an error,

Root/fastdfs-nginx-module/src // common. c: 21: 25: fatal error: fdfs_define.h: No such file or directory

# Include "fdfs_define.h"

^

Compilation terminated.

Make [1]: *** [objs/addon/src/ngx_http_fastdfs_module.o] Error 1

Solution:

Vim/root/fastdfs-nginx-module/src/config

 

CORE_INCS = "$ CORE_INCS/usr/local/include/fastdfs/usr/local/include/fastcommon /"

CORE_LIBS = "$ CORE_LIBS-L/usr/lib-lfastcommon-lfdfsclient"

Recompile and install

4. Configuration

# Cp/root/fastdfs-nginx-module/src/mod_fastdfs.conf/etc/fdfs /.

# Cp/root/FastDFS/conf/http. conf/etc/fdfs /.

/Root/FastDFS/conf/mime. types/etc/fdfs /.

Modify the nginx configuration file

Server {

Listen 18888;

Server_name localhost;

Location ~ /Group1/M00 {

Root/fdfs/storage/data;

Ngx_fastdfs_module;

}

 

Error_page 500 502 503 x.html;

Location =/50x.html {

Root/usr/share/nginx/html;

}

 

}

6. Start nginx

8. Install nginx in tracker

Nginx installed on tracker is mainly used to provide reverse proxy for http access and cache services for Server Load balancer.

1. The installation configuration is the same as that on storage.

2. Configure the load function

Upstream fdfs_group3 {

Server 172.16.1.207: 8080 weight = 1 max_fails = 2 fail_timeout = 30 s;

Server 172.16.1.208: 8080 weight = 1 max_fails = 2 fail_timeout = 30 s;

}

Server {

#

Set server port

Listen 8080;

#

Set

Group1

Server Load balancer Parameters

Location/group1/M00 {

Proxy_next_upstream http_502 http_504 error timeout invalid_header;

Proxy_cache http-cache;

Proxy_cache_valid 200 304 12 h;

Proxy_cache_key $ uri $ is_args $ args;

Proxy_pass http: // fdfs_group1;

Expires 30d;

9. Test

Cd/etc/fdfs/

Cp client. conf. sample client. conf

Base_path =/data/fdfs/tracker #

Log storage path

Tracker_server = 172.16.8.8: 22122 # tracker

Server

IP

Address and port number

Http. tracker_server_port = 8080 # tracker

Server

Http

Port Number

Upload files

/Usr/bin/fdfs_upload_file/etc/fdfs/client. conf/root/install3.sh

Group1/M00/00/00/rBAIClZB6ySAYFbtAAAGpOjLUVA6601. sh

Log error

[21:04:04] ERROR-file: ../common/fdfs_global.c, line: 52, the format of filename "group1/M00/00/00/rBAIClZB6ySAYFbtAAAGpOjLUVA6601. sh" is invalid

Vi/etc/fdfs/mod_fastdfs.conf

Url_have_group_name = true

Test upload again

[Root @ vrvap2 fdfs] #/usr/bin/fdfs_upload_file/etc/fdfs/client. conf/root/new.jpg

Group1/M00/00/00/rBAIClZB7fWAF2BmAADCkWSm8AA128.jpg

The test is successful.

Complete installation and configuration steps for fastDFS in CentOS 6.2

FastDFS installation in Ubuntu, PHP Client

Install, configure, and test the FastDFS Distributed File Server

FastDFS integration Nginx problem sorting

Build FastDFS in CentOS

Full record of FastDFS installation in Ubuntu

FastDFS details: click here
FastDFS: click here

This article permanently updates the link address: