centos6.5 build Fastdfs Distributed storage picture

What is Fastdfs.

Fastdfs is an open-source, lightweight, Distributed File system. It solves the problem of large amount of data storage and load balancing. Especially suitable for medium and small files (recommended range: 4KB < file_size <500MB) for the carrier of online services, such as photo albums, video sites and so on. The UC is based on FASTDFS development to provide users with: Network disk, community, advertising and application downloads and other business storage services.

FASTDFS Architecture:

The FASTDFS server has three roles: a Tracking Service (tracker server), storage servers (storage server), and clients (client). Tracker server: Tracking servers, mainly to do scheduling work, the role of load balancing. The state information of all storage groups and storage servers in the cluster is recorded in memory, which is the hub of interaction between client and data server. More streamlined than the master in GFs, which does not record file index information and consumes very little memory. Storage server: Storage server (also known as storage node or data server), file and file attributes (meta data) are saved to the storage server. Storage server directly utilizes the OS's file system call to manage files. Client: Clients, as initiators of a business request, use the TCP/IP protocol to interact with the tracker server or storage node through proprietary interfaces.

Tracker server: Tracking servers, mainly to do scheduling work, in the access load balanced role.
Storage server: A storage server (also known as a data server).

Storage nodes take the form of Grouping (group). The storage system consists of one or more group, and the files between group and group are independent of each other.

The cumulative file capacity of all group is the file capacity of the entire storage system. A group can consist of one or more storage servers, the files in a storage server under a group are the same, and multiple storage servers in group play the role of redundant backup and load balancing (one group has the smallest storage server capacity in that group, Storage servers in different groups do not communicate with each other, and the storage servers within the group are interconnected
Connect to file synchronization).

When the server is added to the group, the synchronized files are automatically completed by the system, and after synchronization is completed, the system automatically switches the new server to the online service.

You can add a group dynamically when storage space is low or is about to run out. Only one or more servers need to be added and configured as a new group,
This expands the capacity of the storage system.

Fastdfs Two roles: Tracker Server and Storage server. Tracker server as a central node, its main role is load balancing and scheduling. Tracker server records the status of groups and storage servers in memory, does not record file index information, and consumes a small amount of memory.
In addition, when the client (application) and storage server access the Tracker server, the Tracker server scans the memory for grouping and storage server information, and then responds. It can be seen from this that tracker server is very lightweight and does not become a system bottleneck.

Storage server in Fastdfs is often referred to as trunk server or data server in other file systems. Storage server directly uses the OS's file system to store files. Fastdfs does not block the file storage, the client uploads the file and storage server file one by one correspondence (Fastdfs in the file identity is divided into two parts: group name and file name, both indispensable).

Fastdfs Download Address: http://code.google.com/p/fastdfs/downloads/list
Libevent Official website: http://libevent.org/

Current Latest edition: Https://github.com/downloads/libevent/libevent/libevent-2.0.21-stable.tar.gz
Current Latest edition: Http://fastdfs.googlecode.com/files/FastDFS_v4.06.tar.gz

System environment:
Fastdfs_tracker centos6.5_64 ip:192.168.1.50
FASTDFS_STORAGE_S1 centos6.5_64 ip:192.168.1.51
Fastdfs_storage_s2 centos6.5_64 ip:192.168.1.52
Fastdfs_client centos6.5_64 ip:192.168.1.54

SELinux closed, iptables closed.

Configure tracker:
First install libevent, after Fastdfs

1 2 3 4 5 6 7 8

[root@centos-6.5-x64 ~] #yum install-y gcc gcc-c++ [root@centos-6.5-x64 ~] #tar zxvf libevent-2.0.21-stable [ root@centos-6.5-x64 ~] #tar zxvf fastdfs_v4.06.tar.gz [root@centos-6.5-x64 ~] #cd libevent-2.0.21-stable [ root@centos-6.5-x64 libevent-2.0.21-stable]#./configure && make && make install [root@centos-6.5-x64 libevent-2.0.21-stable]# CD [root@centos-6.5-x64 ~] #cd Fastdfs [root@centos-6.5-x64 Fastdfs] #sh make.sh && sh make.sh Install

The Fastdfs configuration file path defaults to the/etc/fdfs/
[root@centos-6.5-x64 ~] #cd/etc/fdfs/
[Root@centos-6.5-x64 Fdfs] #ll
Total Dosage 56
-rw-r–r–1 root root 1463 April 04:10 client.conf
-rw-r–r–1 root root 858 April 04:10 http.conf
-rw-r–r–1 root root 31172 April 04:10 mime.types
-rw-r–r–1 root root 7460 April 04:10 storage.conf
-rw-r–r–1 root root 6621 April 04:10 tracker.conf

client.conf Client Upload configuration file
storage.conf file Storage Server configuration file
Tracker.conf is responsible for balancing the dispatch server configuration file
http.conf HTTP Server configuration file (can be ignored)

The orders related to Fastdfs are under/usr/local/bin/.

To edit the tracker configuration file:
[Root@centos-6.5-x64 Fdfs] #grep-V "#" tracker.conf | Grep-v "^$"

#tracker. conf configuration file is valid, False is in effect, true is masked.
Disabled=false

#程序的监听地址, if not set, listen for all addresses.
Bind_addr=

#tracker监听的端口
port=22122

#连接超时时间
Connect_timeout=30

#tracker在通过网络发送接收数据的超时时间
Network_timeout=60

#数据和日志的存放地点
Base_path=/home/yuqing/fastdfs

#最大连接数
max_connections=256

#工作线程数一般为cpu个数
Work_threads=4

#在存储文件时选择group的策略, 0: Rotation strategy 1: Specify a group of 2: Load Balancing, select the largest group of free space.
store_lookup=2