NDB Cluster (distributed storage engine) storage Engine brief introduction and implementation of MySQL Cluster

One, NDB Cluster (distributed storage engine) storage Engine Brief Introduction

NDB storage Engine is also called NDB Cluster storage engine, mainly used for MySQL Cluster distributed cluster environment,

Cluster is a new feature that MySQL has only begun to provide from version 5.0. This part we may not just introduce NDB
Storage engine, because the NDB storage engine will lose too much meaning as it leaves the MySQL CLuster entire environment. So

The next one will be a description of the MySQL cluster setup.

Mysql Cluster is a technology whose main function is to deploy the Cluster of in-memory databases in a non-shared related system, mainly through the NDB Cluster (abbreviated NDB) storage engine.

With a non-shared architecture, the system is able to use inexpensive hardware and has no special requirements for hardware and software.

In general, the environment structure of a MySQL Cluster is mainly as follows:

650) this.width=650; "src=" Http://s3.51cto.com/wyfs02/M02/43/D1/wKioL1PdBLWSVGssAAGBsv50qV4277.jpg "title=" Screenshot11.png "alt=" Wkiol1pdblwsvgssaagbsv50qv4277.jpg "/>

Mainly consists of the following three parts:

1). The Manage node host that is responsible for managing each node:

The Management node is responsible for the management of each node in the entire cluster cluster, including the configuration of the cluster, startup shutdown
Each node, as well as the backup and recovery of the implementation data. The management node acquires the state of the nodes in the entire cluster environment and
Error messages, and each node in the cluster cluster is fed information to all other nodes in the cluster. By
The configuration of the entire cluster environment on the management node, while serving as the basic communication work of the nodes in the cluster,

So he must be the first node to be started.

2). SQL Server node (hereafter referred to as SQL node), which is what we often call MySQL server:
It is primarily responsible for implementing everything that a database is above the storage layer, such as connection management, query optimization and
should, cache management, and so on, only the work of the storage layer is given to the NDB data node to deal with. In other words, in purely
The SQL node in the MySQL Cluster environment can be considered a MySQL that does not need to provide any storage engine
Server because his storage engine has a cluster environment in the NDB node to serve. So, the SQL layer MySQL service
There is a certain difference between the start of the service and the normal MySQL boot, you must add the Ndbcluster item, you can add the

The MY.CNF configuration file can also be specified by starting the command line.

3). Storage layer of NDB data node, which is said above NDB Cluster:
NDB is an in-memory storage engine in other words, he will load all the data and index data into memory.
However, the data is persisted to the storage device. However, the latest version, has supported users to choose their own data can not be completely
Load into memory, which is too large for some data or cost-based and does not have enough memory space to store

It's really good news for all the users of the data.

NDB node is mainly to realize the function of the underlying data storage, and save the cluster data. Each of the NDB nodes is saved
Part of the complete data (or a complete piece of data, depending on the number of nodes and configuration), in MySQL CLuster
The face is called a fragment. And every fragment, in normal circumstances, will have a copy on the other host (or
The exact same image exists. This is done by configuration, so as long as it's properly configured, Mysql

Cluster there is no single point of issue at the storage layer.

In general, NDB nodes are organized into a single NDB group, and a NDB group is actually a group of NDB nodes with exactly the same physical data.

The above mentioned NDB each node to the data organization, perhaps each node has all the data may also only save
Part of the data, mainly by the number of nodes and parameters to control. First in MySQL Cluster Master config file (in the Tube
Config.ini), there is a very important parameter called Noofreplicas, this parameter
Specifies the number of copies of each piece of data that is redundantly stored on different nodes, which should generally be set at least 2, or
Just set it to 2. As a normal case, the probability of failure of two redundant nodes is not
Often small, of course, if the machine and memory enough, you can continue to increase. One node above is to save all the numbers
Data is also limited by the number of storage nodes. The NDB storage engine first guarantees that the Noofreplicas parameter
Data redundancy, use storage nodes, and then segment data according to the number of nodes to continue using multiple

The NDB node of the remainder, the number of segments divided by the total number of nodes by Noofreplicas.

Second, the implementation of MySQL cluster cluster architecture

on the Management node:
Rpm-ivh mysql-cluster-gpl-management-7.1.18-1.el6.x86_64.rpm
Rpm-ivh mysql-cluster-gpl-tools-7.1.18-1.el6.x86_64.rpm

Mkdir/usr/mysql-cluster

vi /usr/mysql-cluster/config.ini      [ndbd default]     noofreplicas=1                           //running NDB storage Engine      Datamemory=30m    indexmemory=10m        [ndb_mgmd ]    nodeid=1    hostname=192.168.1.21             //Management Node ip    datadir=/usr/mysql-cluster     [ndbd]    nodeid=2    hostname=192.168.1.22             //Storage Node Ip    datadir=/usr /mysql-cluster    [mysqld]                          //Client     nodeid=3     hostname=192.168.1.23

Ndb_mgmd-f/usr/mysql-cluster/config.ini--initial//Specify configuration file, initialize
NDB_MGM//You can view the information for each node

On the storage node:
RPM-IVH mysql-cluster-gpl-storage-7.1.18-1.el6.x86_64.rpm

VI/ETC/MY.CNF [mysqld] Ndbcluster//Run NDB cluster engine ndb-connectstring=192.168.1.21 Specify the management node [Mysql_cluster] ndb-connectstring=192.168.1.21

NDBD--initial//Initialize, on the manager node NDB_MGM view storage node is connected

SQL node:
mysql-cluster-gpl-client-7.1.18-1.el6.x86_64.rpm
mysql-cluster-gpl-server-7.1.18-1.el6.x86_64.rpm

If MySQL, Mysql-server, mysql-libs are installed on the SQL node, You need to uninstall it to install Mysql-cluster, or you will get an error
error:failed dependencies:
    mysql conflicts with mysql-cluster-gpl-server-7.1.18-1.el6.x86_64
    mysql-server conflicts with mysql-cluster-gpl-server-7.1.18-1.el6.x86_64

Rpm-e mysql-libs--nodeps         //Force uninstall
Rpm-ivh mysql-cluster-gpl-*      

VI/ETC/MY.CNF [Mysqld] .... ndbcluster (The following data is added at the bottom of the [mysqld] area) ndb-connectst ring=192.168.1.21 [Mysql_cluster] ndb-connectstring=192.168.1.21

/etc/init.d/mysql start//starts MySQL, and then NDB_MGM view on the management node

There are many files in the SQL node that are generated in the directory/var/lib/mysql/mysql when initialized, such as failure to initialize at startup, and can be executed manually mysql_install_db--user=mysql generation.

--------------------

To add a new storage node and SQL node:

Before adding a new storage node and SQL node, you must first close the existing management node, storage node, SQL node
on the management node NDB_MGM-E shutdown can only shut down the storage node and the management node's own services, SQLT node still need to manually stop, And then add it later.
The node to add is first added in the configuration file of the management node:

vi /usr/mysql-cluster/config.ini     [ndbd default]     noofreplicas=2          //defines two nodes as a single node group,     datamemory=30M    indexmemory=10M    [ndb_mgmd]     nodeid=1    hostname=192.168.1.21    datadir=/usr/ Mysql-cluster    [ndbd]    nodeid=2    hostname= 192.168.1.22    datadir=/usr/mysql-cluster         [ndbd]                              //Added storage nodes      nodeid=4    hostname=192.168.1.10    datadir=/usr/mysql-cluster             [mysqld]    nodeid=3    hostname= 192.168.1.23    [mysqld]                            //joined SQL node     nodeid=5    hostname=192.168.1.24

  Then configure the newly added node (consistent with the original configuration)

ndb_mgmd-f/usr/mysql-cluster/config.ini--initial   // Reinitialize the management side
Start each node, NDB_MGM show on the management node, you can see that each node is working properly

--------------

Inserting data on any SQL node can be seen on another node,
The
executes the backup Data command on the management node, the data is backed up separately to each storage node, and the Restore command is executed on each storage node at restore time;
ndb_mgm> start Backup
Waiting for completed, this Several minutes
Node 2:backup 1 started from Node 1
Node 2:backup 1 started from Node 1 completed
&nbs P startgcp:1125 stopgcp:1128
  #Records: 2059 #LogRecords: 0
 data:51264 bytes log:0 bytes

on storage node/ The backup data can be seen in the Usr/mysql-cluster/backup directory, and when data is deleted on the SQL node, it can be recovered using this backup (only data can be recovered, such as deleting a table or library directly, requiring manual addition, to recover);
Restore data requires a command ndb_ Restore (provided by package mysql-cluster-gpl-tools-7.1.18-1.el6.x86_64.rpm), if this command is not available on the storage node, it can be copied directly from the management node or the tools package can be installed.

Note: When recovering data, there must be an idle connection on the management node, otherwise it cannot be recovered.

ndb_restore-n 2-b 1-r/usr/mysql-cluster/backup/backup-1/

-n means node id-b means backup id-r refers to backup directory

This article is from the "Dance Fish" blog, so be sure to keep this source http://ty1992.blog.51cto.com/7098269/1534313