Mongodb replica set shard sharded high-availability cluster

I. mongodb sharding, two common architectures

1. Each client system contains a router mongos,

The number of servers has increased from a dozen to several hundred. The mongos route and the mongod Shard server have established several hundred or even thousands of connections, and the load is very heavy. This means that each chunk is balanced (the balancing measures required by the MongoDB sharding cluster to maintain even data distribution) it takes a long time to transmit the chunk location information stored in the configuration database. This is because every mongos route must clearly know where each chunk exists in the cluster.

2. Vro Independent

The routing independently solves the problem mentioned above. The fewer routes, the simpler the topology.
The router here includes the connection pool function. The above two pictures are from the network
II. Server description
1. mongodb multipart storage
Server1 dual: Shard1/Shard2 ip: 192.168.10.103
Server 2: Shard1/Shard2 ip: 192.168.10.219
2. Routing
Server3 ip: 192.168.10.209
From the configuration of the vro, we can see that only independent routing is used.
III. Install and configure mongodb replica sets shard
1. mongodb installation
Download Address: http://www.mongodb.org/downloads
After decompression, you can use
2. Create directories on the three servers respectively.

The code is as follows:	Copy code
# Mkdir/usr/local/mongodb26/configsvr/-p # Mkdir/usr/local/mongodb26/log/-p # Mkdir/usr/local/mongodb26/shard1/-p # Mkdir/usr/local/mongodb26/shard2/-p # Mkdir/usr/local/mongodb26/key/-p # Mkdir/usr/local/mongodb26/conf/-p

Why create a directory? Because the decompressed mongodb has a directory bin directory, which is too small.
3. Create a partition configuration file for 192.168.10.103 and 192.168.10.219.

The code is as follows:

Copy code

# Cat/usr/local/mongodb26/conf/shard1.conf Dbpath =/usr/local/mongodb26/shard1 // database path Directoryperdb = true // set that each database will be saved in a separate directory Shardsvr = true // start the Shard ReplSet = shard1 // set the full set name to shard1. replSet indicates that the server knows that there are other machines in this "shard1" replica set. Port = 20001 // port number OplogSize = 100 // Copy the log size in MB Pidfilepath =/usr/local/mongodb26/shard1/mongodb. pid // pid file path Logpath =/usr/local/mongodb26/log/shard1.log // log path Logappend = true // write data in append mode Profile = 1 // database analysis, 1 indicates that only slow operations are recorded Slowms = 5 // Set the time for identifying slow queries. The Root mysql slow query is a bit like Fork = true // run as a daemon to create a server process Shard2 root shard1 is basically the same # Cat/usr/local/mongodb26/conf/shard2.conf Dbpath =/usr/local/mongodb26/shard2 Directoryperdb = true Shardsvr = true ReplSet = shard2 Port = 20002 OplogSize = 100 Pidfilepath =/usr/local/mongodb26/shard2/mongodb. pid Logpath =/usr/local/mongodb26/log/shard2.log Logappend = true Profile = 1 Slowms = 5 Fork = true

The two machines in the multipart storage, respectively, create these two files.
4, 192.168.10.209 route server
The route has two configuration files.

The code is as follows:

Copy code

# Cat/usr/local/mongodb26/conf/configsvr. conf // route configuration, configuration file Pidfilepath =/usr/local/mongodb26/configsvr/mongodb. pid Dbpath =/usr/local/mongodb26/configsvr Directoryperdb = true Configsvr = true // declare that this is the config service of a cluster, default port 27019, default directory/data/configdb Port = 20000 Logpath =/usr/local/mongodb26/log/configsvr. log Logappend = true Fork = true # Cat/usr/local/mongodb26/conf/mongos. conf // route configuration, configuration file Configdb = 192.168.10.209: 20000 // The Configuration Server for the listener. Only one or three configuration servers are allowed. Port = 30000 ChunkSize = 1 // use 100 or delete to create an environment in mb. After deletion, the default value is 64. Logpath =/usr/local/mongodb26/log/mongos. log Logappend = true Fork = true

4. Start the mongodb process

The code is as follows:

Copy code

1. Start of the 192.168.10.103 and 192.168.10.219 multipart storage nodes /Usr/local/mongodb26/bin/mongod-f/usr/local/mongodb26/conf/shard1.conf /Usr/local/mongodb26/bin/mongod-f/usr/local/mongodb26/conf/shard2.conf 2. Start on the 192.168.10.209 route node /Usr/local/mongodb26/bin/mongod-f/usr/local/mongodb26/conf/configsvr. conf /Usr/local/mongodb26/bin/mongos-f/usr/local/mongodb26/conf/mongos. conf If the following error is reported: #/Usr/local/mongodb26/bin/mongos-f/usr/local/mongodb26/conf/mongos. conf BadValue need either 1 or 3 configdbs Try '/usr/local/mongodb26/bin/mongos -- help' for more information Causes and Solutions: The configdb parameter in the mongos. conf file is incorrect. Here, the number of routes can only be 1 or 3. Configdb = 192.168.10.209: 20000 5. Check whether the startup is successful. 1. Start of the 192.168.10.103 and 192.168.10.219 multipart storage nodes [Root @ localhost conf] # netstat-tpnl | grep mongod Tcp 0 0 0.0.0.0: 20001 0.0.0.0: * LISTEN 21556/mongod Tcp 0 0 0.0.0.0: 20002 0.0.0.0: * LISTEN 21976/mongod 2. Start on the 192.168.10.209 route node [Root @ localhost mongodb26] # netstat-tpnl | grep mon Tcp 0 0 0.0.0.0: 30000 0.0.0.0: * LISTEN 3521/mongos Tcp 0 0 0.0.0.0: 20000 0.0.0.0: * LISTEN 3946/mongod

If all listening ports have been started, it indicates that they have been added successfully.
6. Configure the mongodb replica set node
1. Configure port 20001

The code is as follows:

Copy code

#/Usr/local/mongodb26/bin/mongo -- port 20001   > Use admin   > Config = {_ id: "shard1", members :[   {_ Id: 0, host: "192.168.10.103: 20001 "},   {_ Id: 1, host: "192.168.10.219: 20001 "}       ]  }; {  "_ Id": "shard1 ", "Members ":[         {  "_ Id": 0, "Host": "192.168.10.103: 20001" },         {  "_ Id": 1, "Host": "192.168.10.219: 20001"         }      ]  }    > Rs. initiate (config );   > Exit;

2. Configure port 20002

The code is as follows:

Copy code

#/Usr/local/mongodb26/bin/mongo -- port 20002   > Use admin   > Config = {_ id: "shard2", members :[   {_ Id: 0, host: "192.168.10.103: 20002 "},   {_ Id: 1, host: "192.168.10.219: 20002 "}     ]  }; {  "_ Id": "shard2 ", "Members ":[         {  "_ Id": 0, "Host": "192.168.10.103: 20002" },         {  "_ Id": 1, "Host": "192.168.10.219: 20002"         }      ]  }    > Rs. initiate (config );   > Exit;

Perform the multipart, 192.168.10.103, and 192.168.10.219 operations on any machine. The two ports are configured once.
7. mongos route section configuration

The code is as follows:

Copy code

#/Usr/local/mongodb26/bin/mongo -- port 30000   Mongos> use admin   Mongos> db. runCommand ({addshard: "shard1/192.168.10.103: 20001,192.168 .10.219: 20001", name: "shard1", maxsize: 20480 }) Mongos> db. runCommand ({addshard: "shard2/192.168.10.103: 20002,192.168 .10.219: 20002", name: "shard2", maxsize: 20480 });   Mongos> db. runCommand ({listshards: 1 }) {  "Shards ":[         {  "_ Id": "shard1 ", "Host": "shard1/192.168.10.103: 20001,192.168 .10.219: 20001" },         {  "_ Id": "shard2 ", "Host": "shard2/192.168.10.103: 20002,192.168 .10.219: 20002"         }  ], "OK": 1 }

Here, the replica set + shard function is configured. Note: Although the configuration is complete, you must declare that the database and table must be sharded.
8. Declare that the database and table must be sharded.

The code is as follows:

Copy code

Mongos> use admin // switch to the admin Library   Mongos> db. runCommand ({enablesharding: "test2"}); // declare that the test2 database allows sharding. {"OK": 1}   Mongos> db. runCommand ({shardcollection: "test2.books", key: {id: 1}); // declare the books table to be sharded {"Collectionsharded": "test2.books", "OK": 1}   Mongos> use test2 // switch to test2 Mongos> db. stats (); // view the database status Mongos> db. books. stats (); // view the table status If the following error is reported: {"OK": 0, "errmsg": "sharding not enabled for db"}, because the library does not allow sharding

9. Test
1. Test script

The code is as follows:	Copy code
Mongos> for (var I = 1; I <= 20000; I ++) db. books. save ({id: I, name: "12345678", sex: "male", age: 27, value: "test "});

2. Test results

The code is as follows:

Copy code

Mongos> db. books. stats (); {  "Sharded": true, "SystemFlags": 1, "UserFlags": 1, "Ns": "test2.books ", "Count": 20000, "NumExtents": 10, "Size": 2240000, "StorageSize": 5586944, "TotalIndexSize": 1267280, "IndexSizes ":{ "_ Id _": 678608, "Id_1": 588672 }, "AvgObjSize": 112, "Nindexes": 2, "Nchunks": 4, "Shards ":{ "Shard1 ":{ "Ns": "test2.books ", "Count": 5499, "Size": 615888, "AvgObjSize": 112, "StorageSize": 2793472, "NumExtents": 5, "Nindexes": 2, "LastExtentSize": 2097152, "PaddingFactor": 1, "SystemFlags": 1, "UserFlags": 1, "TotalIndexSize": 367920, "IndexSizes ":{ "_ Id _": 196224, "Id_1": 171696 }, "OK": 1 }, "Shard2 ":{ "Ns": "test2.books ", "Count": 14501, "Size": 1624112, "AvgObjSize": 112, "StorageSize": 2793472, "NumExtents": 5, "Nindexes": 2, "LastExtentSize": 2097152, "PaddingFactor": 1, "SystemFlags": 1, "UserFlags": 1, "TotalIndexSize": 899360, "IndexSizes ":{ "_ Id _": 482384, "Id_1": 416976 }, "OK": 1         }  }, "OK": 1 }

3. Test results and data analysis
1), data records, all inserted to shard2
2), shard2 data remains unchanged, and shard1 data begins to increase.
3), shard1 data remains unchanged, and shard2 data is reduced.
Data records are inserted at a time, which can be balanced in about 5 seconds. You can repeat this command in db. book. stats (), and you will find data flow changes.