Build a high-availability MongoDB cluster (i)--Configure MongoDB

Build a highly available mongodb cluster (1)-configure mongodb
In the era of big data, traditional relational databases must solve the problems of high concurrent reads and writes, efficient storage of massive data, high scalability, and high availability to provide higher services. But it was because of these problems that Nosql was born.

NOSQL has these advantages:

Large amount of data, you can store a large amount of data through a cheap server, easily get rid of the limit of the traditional MySQL single table storage magnitude.

High scalability, Nosql removes the relational characteristics of relational databases, it is easy to scale horizontally, and it has got rid of the old criticism of vertical expansion.

High performance. Nosql obtains data through a simple key-value method, which is very fast. There is also a NoSQL Cache, which is a record-level cache. It is a fine-grained cache, so NoSQL has a much higher performance at this level.

Flexible data model. NoSQL does not need to establish fields for the data to be stored in advance. It can store custom data formats at any time. In a relational database, adding and deleting fields is a very troublesome thing. For very large data tables, adding fields is a nightmare.

High availability. NoSQL can easily implement a highly available architecture without affecting performance. For example, mongodb can quickly configure high-availability configurations through mongos and mongo shards.

In nosql database, most of the queries are in the form of key-value pairs (key, value). MongoDB is a product between a relational database and a non-relational database. It is most like a relational database among non-relational databases. Supports similar to the object-oriented query language, can almost achieve most of the functions of a single-table query similar to a relational database, and also supports data indexing. So this is very convenient. We can use sql to operate MongoDB and migrate from a relational database. Developer learning costs will be greatly reduced. If you encapsulate the underlying sql API again, you can't feel the difference between mongodb and relational databases. Similarly, MongoDB is also known to be able to quickly build a highly available and scalable distributed cluster. There are many articles on the Internet. When we are building, we need to find and modify many things, so record the actual steps of yourself to memorize. Let's see how to build this stuff step by step.

First, mongodb single instance. This configuration is only suitable for simple development, not for production use, because a single node hangs up the entire data service, such as.

Although it cannot be used in production, this mode can be quickly set up and started, and the database can be operated with mongodb commands. The steps to install single node mongodb under Linux are listed below.

1.Create mongodb test folder

#Store the entire mongodb file
mkdir -p / data / mongodbtest / single

#Store mongodb data files
mkdir -p / data / mongodbtest / single / data

#Enter the mongodb folder
cd / data / mongodbtest / single
2.Download the mongodb installation package

wget http://fastdl.mongodb.org/linux/mongodb-linux-x86_64-2.4.6.tgz

#Unzip the downloaded archive
tar xvzf mongodb-linux-x86_64-2.4.6.tgz

#Enter mongodb program execution folder
cd mongodb-linux-x86_64-2.4.6 / bin /
3.Start single instance mongodb

mongod --dbpath / data / mongodbtest / single / data
The output log is as follows, success!

[initandlisten] db version v2.4.6
........
[initandlisten] waiting for connections on port 27017
[websvr] admin web console waiting for connections on port 28017

mongodb comes with a web access interface by default, which can be accessed through the form of IP + port.

http://192.168.0.1:28017/

Second, the master-slave mode. When using the mysql database, it is widely used. After the dual-node backup is used, the master node can be replaced by the slave node to continue the service. So this model is much better than single node high availability.

Let's see how to build a mongodb master-slave replication node step by step:

1. Prepare two machines 192.168.0.1 and 192.168.0.2. 192.168.0.1 is used as the master node and 192.168.0.2 is used as the slave node.
2. Download the mongodb installation package separately. Create a folder / data / mongodbtest / master on 192.168.0.1, and create a folder / data / mongodbtest / slave on 192.168.0.2.
3. Start the mongodb master node program at 192.168.0.1. Note the following "-master" parameter, which indicates the master node.
mongod --dbpath / data / mongodbtest / master --master

The output log is as follows, success!

[initandlisten] MongoDB starting: pid = 18285 port = 27017 dbpath = / data / mongodbtest / master master = 1
#Log show master node parameters
[initandlisten] options: {dbpath: “/ data / mongodbtest / master”, master: true}
........
[initandlisten] waiting for connections on port 27017

4. Start the mongodb slave program at 192.168.0.2. Key configuration, specify the master node IP address and port –source 192.168.0.1:27017 and identify the slave node –source parameters.

mongod --dbpath / data / mongodbtest / slave --slave --source 192.168.0.1:27017

The output log is as follows, success!

[initandlisten] MongoDB starting: pid = 17888 port = 27017 dbpath = / data / mongodbtest / slave slave = 1
........
#Log display slave node parameters
[initandlisten] options: {dbpath: “/ data / mongodbtest / slave”, slave: true, source: “192.168.0.1:27017 ″}
........
[initandlisten] waiting for connections on port 27017
#Log shows the slave node synchronously replicates data from the master node
[replslave] repl: from host: 192.168.0.1: 27017

5. Test master-slave replication.

Connect to the terminal on the master node:

mongo 127.0.0.1

#Create a test database.
use test;

Insert data into the testdb table.
> db.testdb.insert ({"test1": "testval1"})

Query testdb data to see if it succeeds.
> db.testdb.find ();
{"_id": ObjectId ("5284e5cb1f4eb215b2ecc463"), "test1": "testval1"}
You can see the synchronization log of the host

[initandlisten] connection accepted from 192.168.0.2:37285 # 3 (2 connections now open)
[slaveTracking] update local.slaves query: {_id: ObjectId ('5284e6268ed115d6238bdb39 ′), config: {host: “192.168.0.2:35271 ″, upgradeNeeded: true}, ns:“ local.oplog. $ main ”} update: {$ set: {syncedTo: Timestamp 1384441570000 | 1}} nscanned: 1 nupdated: 1 fastmod: 1 keyUpdates: 0 locks (micros) w: 132015 132ms

Check the data from the host.

mongo 127.0.0.1

View the current database.

> show dbs;
  local 0.203125GB
  test 0.203125GB

use test;
db.testdb.find ();
{"_id": ObjectId ("5284e5cb1f4eb215b2ecc463"), "test1": "testval1"}
The data has been synchronized after the query. Looking at the logs again, it was found that the slave did indeed synchronize data from the master.

Thu Nov 14 23:05:13 [replslave] repl: checkpoint applied 15 operations
Thu Nov 14 23:05:13 [replslave] repl: syncedTo: Nov 14 23:08:10 5284e75a: 1
View service status

> db.printReplicationInfo ();
          this is a slave, printing slave replication info.
          source: 192.168.0.1:27017
              syncedTo: Sun Nov 17 2013 16:04:02 GMT + 0800 (CST)
                      = -54 secs ago (-0.01hrs)
At this point, the master-slave structure of mongodb is set up.

Failover test, now if the two servers are down, can the slaves run normally?

a, first test whether the slave server can be regarded as the master server, that is, writing to the slave server can synchronize the master server?
 

Connect to mongodb on 192.168.0.2.

mongo 127.0.0.1:27017
> db.testdb.insert ({"test3": "testval3"});
not master
It can be seen that the slave node of mongodb cannot provide write operations, only read operations.

b. If the slave server hangs up, the master server can also provide services. Can the slave automatically become writable if the master hangs?
have a test!

First kill the original mongodb master server.

kill -3 `ps -ef | grep mongod | grep -v grep | awk‘ {print $ 2} ‘`
Tests if the slave is writable. Connect to mongodb test on 192.168.0.2.

> db.testdb.insert ({"test3": "testval3"});
not master
It seems that the slave server does not automatically take over the function of the master server, only manual processing!

Stop the slave server, start it in the original data file and add the master server flag.

mongod --dbpath / data / mongodbtest / slave --master
Wait until the startup is successful (it takes a little longer). Connect on 192.168.0.2

mongo 192.168.0.2:27017
.

> db.testdb.find ();
{"_id": ObjectId ("5288629e9b0318be4b20bd4c"), "test1": "testval1"}
{"_id": ObjectId ("528862d69b0318be4b20bd4d"), "test2": "testval2"}
success!

Multiple slave nodes. Now it's just a database server that provides both writes and reads, and there will be bottlenecks in machine hosting. Do you remember the read and write separation in mysql? Placing 20% of the writes to the master node and 80% of the reads to the slave nodes reduces the server load. But most applications are the pressure brought by the read operation. One slave node cannot load the pressure, and one slave node can be turned into multiple nodes. Can mongodb support more than one master? answer The case is yes.

To facilitate testing, create another folder / data / mongodbtest / slave1 on 192.168.0.2 as another slave server.
Start slave2 service,

mongod --dbpath / data / mongodbtest / slave1 --slave --port 27017 --source 192.168.0.1:27017.
Tested by mongodb connection after successful startup:

> db.testdb.find ();
{"_id": ObjectId ("5288629e9b0318be4b20bd4c"), "test1": "testval1"}
{"_id": ObjectId ("528862d69b0318be4b20bd4d"), "test2": "testval2"}
Is this master-slave replication system very stable? In fact, it is not. . . Look at these questions?

Can the master node automatically switch connections? Manual switching is currently required.
How to solve the excessive write pressure on the master node?
The data on each of the slave nodes is a full copy of the database. Will the stress on the slave nodes be too great?
Is it possible to auto-expand even if the routing access policy is implemented on the slave nodes?
There are so many questions, are there any other solutions? The next article goes on.

Build a highly available mongodb cluster (1)-configure mongodb