Build a high-availability MongoDB cluster

Build a high-availability MongoDB cluster (i)--Configure MongoDBPosted on 171 months, by Lanceyan | 9 Reviews

In the age of big data, traditional relational databases need to be able to have higher services to solve the challenges of high concurrency and read-write, efficient storage of massive data, high scalability and high availability. But it is because of these problems that NoSQL was born.

NoSQL has these advantages:

large data volumes can be easily freed from traditional MySQL single-table storage-scale limits by storing large amounts of data from inexpensive servers.

High Scalability , NoSQL removed the relational database relational features, it is easy to scale horizontally, out of the past always vertical expansion of the criticism.

High Performance , NoSQL gets the data in a simple key-value way, very fast. And NoSQL caches are record-level, a fine-grained cache, so NoSQL has a lot of performance at this level.

Flexible Data Model , NoSQL can store custom data formats at any time without having to create fields in advance for the data to be stored. In the relational database, adding or deleting fields is a very troublesome thing. If it is a table with very large amounts of data, adding fields is simply a nightmare.

highly Available , NoSQL can easily implement a highly available architecture without compromising performance. For example, MongoDB can quickly configure a highly available configuration with MONGOs, MONGO shards.

In a NoSQL database, most queries are the way key-value pairs (key, value) are. MongoDB is a product between relational and non-relational databases, most like relational databases in non-relational databases. Supports an object-oriented query language that can achieve almost the most functionality of a relational database single-table query, and also supports indexing of data. So this is very convenient, we can use the SQL Operation MongoDB, from the relational database migration, developer learning costs will be greatly reduced. If you do a layer of encapsulation of the underlying SQL API, the development can basically not feel the difference between MongoDB and relational databases. The same mongodb also claims to be able to quickly build a highly available and scalable distributed cluster, there are many articles on the Internet, in the time we build also need to find a lot of things to modify, so put their actual steps to record down for forgetting. Let's see how we can build this stuff step-by-step.

a single instance of MongoDB . This configuration is only suitable for simple development use, production use not, because the single node hangs out the entire data business, such as.

Although not available for production, this mode can be set up quickly and can be used to manipulate the database with MongoDB commands. Below is a list of the steps to install a single node MongoDB under Linux

1. Setting Up MongoDB Test folder

#存放整个mongodb文件
mkdir- P /data/mongodbtest/single

#存放mongodb数据文件
mkdir- P /data/mongodbtest/single/data

#进入mongodb文件夹
CD /data/mongodbtest/single

2. Download the MongoDB install package

wget http://fastdl.mongodb.org/linux/mongodb-linux-x86_64-2.4.6.tgz

#解压下载的压缩包
Tar xvzf mongodb-linux-x86_64-2.4.6.tgz

#进入mongodb程序执行文件夹
CD mongodb-linux-x86_64-2.4.6/bin/

3. Start a single instance MongoDB

Mongod--dbpath /data/mongodbtest/single/data

Output log as follows, Success!

[Initandlisten] DB version v2.4.6
........
[Initandlisten] Waiting for connections on port 27017
[WEBSVR] Admin Web console waiting for connections on port 28017

The MongoDB default comes with a Web access interface that can be accessed in the form of IP + ports.

http://192.168.0.1:28017/

second, master-slave mode . The use of MySQL database is widely used, the use of dual-machine backup of the main node hung off after the node can take over the host to continue service. So this mode is much better than the high availability of a single node.

Let's take a look at how to build a MongoDB master-slave replication node in one step:

• 1. Prepare two machines 192.168.0.1 and 192.168.0.2. 192.168.0.1 as the primary node , 192.168.0.2 as the slave node .
• 2. Download the MongoDB installation package separately. Create folder/data/mongodbtest/Masteron 192.168.0.1, 192.168.0.2 create folder/data/mongodbtest/slave.
• 3, start the MongoDB Master node program in 192.168.0.1. Note the following " –master " parameter, which indicates the master node.

  Mongod–dbpath/data/mongodbtest/master–master

Output log as follows, Success!

[Initandlisten] MongoDB starting:pid=18285 port=27017 Dbpath=/data/mongodbtest/master master=1
#日志显示主节点参数
[Initandlisten] Options: {dbpath: "/data/mongodbtest/master", master:true }
........
[Initandlisten] Waiting for connections on port 27017

4. Start MongoDB from node program in 192.168.0.2. A key configuration that specifies the primary node IP address and port –source 192.168.0.1:27017 and identifies the –source parameter from the node.

Mongod–dbpath/data/mongodbtest/slave –slave–source 192.168.0.1:27017

Output log as follows, Success!

[Initandlisten] MongoDB starting:pid=17888 port=27017 Dbpath=/data/mongodbtest/slave slave=1
........
#日志显示从节点参数
[Initandlisten] Options: {dbpath: "/data/mongodbtest/slave", slave:true, source: "192.168.0.1:27017″}
........
[Initandlisten] Waiting for connections on port 27017
#日志显示从节点 replicate data synchronously from the master node
[Replslave] Repl:from host:192.168.0.1:27017

5, test master-slave replication.

Connect to the terminal on the primary node:

MONGO 127.0.0.1

#建立test database.
Use test;

Insert data toward the TestDB table.
> Db.testdb.insert({"test1":"Testval1"})

Query TestDB data to see if it is successful.
> Db.testdb.find();
{ "_id": ObjectId("5284e5cb1f4eb215b2ecc463"), "test1": "Testval1" } /c18>

You can see the synchronization log for the host

[Initandlisten] Connection accepted from 192.168.0.2:37285 #3 (2 connections now OPEN)
[slavetracking] Update local.slaves query: {_id:objectid (' 5284e6268ed115d6238bdb39′), config: {host: " 192.168.0.2:35271″, upgradeneeded:true}, NS: "Local.oplog. $main"} update: {$set: {syncedto:timestamp 1384441570000|1 }} nscanned:1 nupdated:1 fastmod:1 keyupdates:0 Locks (micros) w:132015 132ms

Check the data from the host.

MONGO 127.0.0.1

View the current database.

> Show DBS;
Local 0.203125GB
Test 0.203125GB

Use test;
Db.testdb.find();
{ "_id": ObjectId("5284e5cb1f4eb215b2ecc463"), "test1": "Testval1" }

The data has been synchronized after the query. Look at the log and find that the data from the host is actually synchronized from the host.

Thu Nov23:05: [Replslave] Repl:checkpoint applied operations
Thu Nov 23:05: [Replslave] Repl:syncedTo:Nov 23:08:5284e75a:1

View service Status

> Db.printreplicationinfo();
This is a slave, printing slave replication info.
source:192.168.0.1:27017
Syncedto:sun Nov 16:04:02 gmt+0800 (CST)
= -54 secs ago ( -0.01hrs)

The master-slave structure of MongoDB was built.

failover Test , now if the primary server is hung up, can the server function properly?

• A, first test from the server can be regarded as the primary server, that is, to write from the server to synchronize the master server?

  Connect MongoDB on the 192.168.0.2.

  MONGO 127.0.0.1:27017
  > Db.testdb.insert({"test3":"Testval3"});
  Not master

  You can see that the slave node of MongoDB is not able to provide write operations and can only provide read operations.

b, if the server is hung off, the primary server can also provide services. If the primary server hangs out, can the slave server automatically become writable.
Test it!

First, kill the original MongoDB master server.

Kill -3 'ps -ef| grep Mongod| grep- v grep| awk ' {print $} '

Tests whether the server can be writable. Connect the MongoDB test on the 192.168.0.2.

> Db.testdb.insert({"test3":"Testval3"});
Not master

It seems that from the server does not automatically take over the function of the primary server, only manually processed!

Stop from the server, start in the original data file and add the primary server flag.

Mongod--dbpath /data/mongodbtest/slave --master

Wait until the boot succeeds (the time is a bit long). Connecting on the 192.168.0.2

MONGO 192.168.0.2:27017

。

> Db.testdb.find();
{ "_id": ObjectId("5288629e9b0318be4b20bd4c"), "test1": "Testval1" }
{ "_id": ObjectId("528862D69B0318BE4B20BD4D"), "test2": "Testval2" } /c17>

Success!

multiple slave nodes . Now just a database server and provide write and provide read, machine hosting will appear bottleneck. Do you remember the read and write separation in MySQL? Write 20% to the master node, and 80% of the read-and-drop to the slave node to reduce the load on the server. But most of the application is the pressure of the read operation, one from the node pressure load, can be a slave node into multiple nodes. That MongoDB of a master more from can support it? The answer is yes.

To facilitate testing, set up a folder on 192.168.0.2/data/mongodbtest/slave1 as another slave server.
Start the Slave2 service,

Mongod--dbpath /data/mongodbtest/slave1 --slave --port 27017--source 192.168.0.1: 27017.

After successful start-up via MongoDB connectivity test:

> Db.testdb.find();
{ "_id": ObjectId("5288629e9b0318be4b20bd4c"), "test1": "Testval1" }
{ "_id": ObjectId("528862D69B0318BE4B20BD4D"), "test2": "Testval2" } /c17>

Build this set of master-slave replication system is not very robust, but it is not ... Look at these questions?

• is the primary node hanging up can I switch connections automatically? Manual switching is required.
• How to solve the main node's write pressure too much?
• From the node each of the above data is a full copy of the database, from the node pressure will not be too large?
• Even if routing access policy is implemented for routing from nodes, can it be extended automatically?

There are so many questions, are there other solutions? The next article goes on.

Reference:
NoSQL starts-why use NoSQL http://www.infoq.com/cn/news/2011/01/nosql-why/
MongoDB Manual http://cn.docs.mongodb.org/manual/single/

Build a high-availability MongoDB cluster