Use of Distributed File System MogileFS

Use of Distributed File System MogileFS

Introduction to MogileFS

MogileFS is an open-source distributed storage component developed by Danga Interactive under LiveJournal. It is mainly used to store massive small files.

MogileFS has three components:

1) Tracker Nodes: it is the core node of the file system cluster. It stores metadata with database Nodes. Its main functions include monitoring the backend Storage Nodes and instructing the Storage Nodes to add, delete, and query data, and data replication between nodes. The process running on the node is mogilefsd. There are often multiple Tracker nodes to achieve load balancing and high availability;

2) Storage Nodes: the location where the data is actually stored. It works in the WebDAV mode to create, delete, rename, and perform other operations. The running process is mogstored;

3) database Nodes: Helps Tracker Nodes store metadata and global information, including host, domain, and class. Highly available clusters are recommended for databases to prevent spof.

Implementation Mechanism

In this file system cluster, each data block has multiple copies and is stored on multiple Storage nodes, the number of copies of different file systems varies (generally three copies are recommended). The Tracker Node separately records the Storage location of each data block (in which Storage and Path ). If a Storage node is offline and the client requests data from the Tracker node, the Tracker node returns the Storage location of the redundant copy to the client. This ensures the high availability of data in the entire file system. When data is stored on a storage node, each storage node sends copies to other nodes through inter-node communication based on Tracker instructions. If a storage is deprecated, the replicas on the node are re-provided by other nodes to ensure that the number of replicas is not less than the specified value.

MogileFS Structure

Working principle: when the client needs to access data, it first interacts with the Tracker. The Tracker queries the database to obtain the metadata that the client needs to access when accessing data, and then returns it to the client, the client uses the results returned by Tracker to interact with the Storage Nodes node for data access (one or more Nodes may be connected at a time ).

MogileFS sets Memcached

Use Nginx in MogileFS

Use Nginx In the Open-Source Distributed File System MogileFS

Custom Nagios monitoring MogileFS storage node script

Deploy MogileFS

Deploy Tracker Nodes

1) install the package

Use cpan for online installation. First install the necessary perl library.

[Root @ node1 ~] # Cpan Sys: Syscall

[Root @ node1 ~] # Cpan IO: WrapTie

[Root @ node1 ~] # Cpan Danga: Socket

[Root @ node1 ~] # Cpan IO: AIO

[Root @ node1 ~] # Cpan MogileFS: Client

[Root @ node1 ~] # Cpan Net: Netmask

[Root @ node1 ~] # Cpan Perlbal

Install MogileFS Server:

[Root @ node1 ~] # Cpan MogileFS: Server

Or download the source code package for compilation and installation:

[Root @ node1 ~] # Wget http://mirrors.ustc.edu.cn/CPAN/authors/id/D/DO/DORMANDO/MogileFS-Server-2.72.tar.gz

[Root @ node1 ~] # Tar xf MogileFS-Server-2.72.tar.gz

[Root @ node1 ~] # Cd MogileFS-Server-2.72

[Root @ node1 MogileFS-Server-2.72] # perl Makefile. PL

[Root @ node1 MogileFS-Server-2.72] # make & make install

2) complete configuration

Create a configuration file directory and add a configuration file:

[Root @ www ~] # Mkdir/etc/mogilefs

[Root @ node1 ~] # Vim/etc/mogilefs/mogilefsd. conf

Daemonize = 1

Pidfile =/var/run/mogilefsd. pid

Db_dsn = DBI: mysql: mogilefs: host = 192.168.1.134; port = 3306; mysql_connect_timeout = 5 # database connection information

Db_user = mogilefs # user name on the database

Db_pass = mogilefs # password for the user name

Listen = 0.0.0.0: 7001 # Listening address and port

Conf_port = 1, 7001

Query_jobs = 10 # Number of processes used for query

Delete_jobs = 1 # Number of processes used for the delete operation

Replicate_jobs = 5 # Number of processes used for replication

Reaper_jobs = 1 # Number of processes used to recycle Resources

Create a database on the database node and authorize the user:

MariaDB [(none)]> create database mogilefs;

Query OK, 1 row affected (0.04 sec)

MariaDB [(none)]> grant all on mogilefs. * to 'mogilefs' @ '2017. 192. %. % 'identified by 'mogilefs ';

Query OK, 0 rows affected (0.00 sec)

MariaDB [(none)]> flush privileges;

Query OK, 0 rows affected (0.00 sec)

Initialize the database:

[Root @ node1 ~] # Mogdbsetup -- dbhost = 192.168.1.134 -- dbname = mogilefs -- dbuser = mogilefs -- dbpass = mogilefs -- dbrootpass = mogilefs

If the following error occurs, the perl driver for connecting to mysql is not installed. Install perl-DBD-MySQL.

Failed to load MogileFS: Store: MySQL: Can't locate DBD/mysql. pm in @ INC (@ INC contains: lib/usr/local/lib64/perl5/usr/local/share/perl5/usr/lib64/perl5/vendor_perl/usr/share/perl5/vendor_perl/usr/lib64/perl5/ usr/share/perl5 .) at/usr/local/share/perl5/MogileFS/Store/MySQL. pm line 5 .........

........

######################################## ###

[Root @ node1 ~] # Yum install perl-DBD-MySQL

3) start the service

Mogilefsd cannot run as root. It must be run by another user. To do this, add the mogilefs user and start the service:

[Root @ node1 ~] # Useradd-r mogilefs

[Root @ node1 ~] # Su-mogilefs-c "mogilefsd-c/etc/mogilefs/mogilefsd. conf -- daemon"

A better way is to add a service script and start the service through the script.

Note that the Sys: Syscall version 0.25 has a bug. When the file system is running, adding data to the file system does not retain two copies according to the Default policy, the default policy is as follows:

Information of the uploaded file:

We can see that there is only one copy of the data, which is saved on the 192.168.1.127 node. Sys: version 0.25 of Syscall is the latest version. You can downgrade to version 0.23 by default when using cpan, or directly download and install version 0.23.

Wget http://mirrors.ustc.edu.cn/CPAN/authors/id/ B /BR/BRADFITZ/Sys-Syscall-0.23.tar.gz

###############################

Perl Makefile. PL

Make & make install

Deploy Storage Nodes

1) Installation Package (consistent with the above)

[Root @ node2 ~] # Cpan Sys: Syscall

[Root @ node2 ~] # Cpan IO: WrapTie

[Root @ node2 ~] # Cpan Danga: Socket

[Root @ node2 ~] # Cpan IO: AIO

[Root @ node2 ~] # Cpan MogileFS: Client

[Root @ node2 ~] # Cpan Net: Netmask

[Root @ node2 ~] # Cpan Perlbal

[Root @ node2 ~] # Cpan MogileFS: Server

2) complete configuration

Create a directory and add a configuration file:

[Root @ node2 ~] # Mkdir/etc/mogilefs

[Root @ node2 ~] # Vim/etc/mogilefs/mogstored. conf

Maxconns = 10000 # maximum number of concurrent connections

Httplisten = 0.0.0.0: 7500 # http listener address

Mgmtlisten = 0.0.0.0: 7501 # management interface listening address

Docroot =/mogdata/data # directory used for storage

Add and mount the partition. Add the directory according to the docroot configuration information in/etc/mogilefs/mogstored. conf and modify the owner group:

[Root @ node2 ~] # Fdisk/dev/sdb

[Root @ node2 ~] # Mkfs-t ext4-B 4096/dev/sdb1

[Root @ node2 ~] # Mkdir-pv/mogdata/data

Mkdir: created directory '/mogdata'

Mkdir: created directory '/mogdata/data'

[Root @ node2 ~] # Mount/dev/sdb1/mogdata/data/

[Root @ node2 ~] # Chown-R mogilefs. mogilefs/mogdata/

3) start the service

Add a user and start the service (like above, it is best to use a script ):

[Root @ node2 ~] # Useradd-r mogilefs

[Root @ node2 ~] # Su-mogilefs-c "mogstored-c/etc/mogilefs/mogstored. conf -- daemon"

Install MogileFS-Utils to manage MogileFS

Install MogileFS-Utils on the control node to add management operations such as host, device, domain, and class.

[Root @ node1 ~] # Cpan MogileFS: Utils

Or

[Root @ node1 ~] # Wget http://mirrors.ustc.edu.cn/CPAN/authors/id/D/DO/DORMANDO/MogileFS-Utils-2.29.tar.gz

[Root @ node1 ~] # Tar xf MogileFS-Utils-2.29.tar.gz

[Root @ node1 ~] # Cd MogileFS-Utils-2.29

[Root @ node1 MogileFS-Utils-2.29] # perl Makefile. PL

[Root @ node1 MogileFS-Utils-2.29] # make & make install

We recommend that you directly use cpan to install the dependent packages.

For more details, please continue to read the highlights on the next page:

1 2 Next Page