MongoDB replica sets + sharding solution and chunks block and chip Key Analysis

The following figure shows the mongdb cluster architecture to be built. Create the first replset folder ------------------ create directory mkdir-P/data/replset_sharding/replset1/r0mkdir-P/data/replset_sharding/replset1/r1mkdir-P /data/replset_sharding/replset1/r2mkdir-P/data/replset_sharding/replset1/log can be changed to the following one (this is for the convenience of viewing logs, view it directly in the command line output)/apps/Mongo/Bi N/mongod -- dbpath/data/replset_sharding/replset1/R0 -- replset replset1 -- Port 18010 -- directoryperdb -- rest/apps/Mongo/bin/mongod -- dbpath/data/replset_sharding/replset1/ r1 -- replset replset1 -- Port 18011 -- directoryperdb -- rest/apps/Mongo/bin/mongod -- dbpath/data/replset_sharding/replset1/R2 -- replset replset1 -- Port 18012 -- directoryperdb -- rest Initialization replica set: /apps/Mongo/bin/Mongo -- Port 18010config _ Replset1 = {_ ID: "replset1", members: [{_ ID: 0, host: "127.0.0.1: 18010", priority: 4}, {_ ID: 1, Host: "127.0.0.1: 18011", Priority: 2}, {_ ID: 2, host: "127.0.0.1: 18012", arbiteronly: true}]} note that the arbiter arbitration node only votes, do not receive copied data! Rs. Initiate (config_replset1); then -------------------- create the second replset folder -------------------- create directory mkdir-P/data/replset_sharding/replset2/r0mkdir-P/data/repl Set_sharding/replset2/r1mkdir-P/data/replset_sharding/replset2/r2mkdir-P/data/replset_sharding/replset2/log use the following one (this is to facilitate reading logs, directly view the command line output) /apps/Mongo/bin/mongod -- dbpath/data/replset_sharding/replset2/R0 -- replset replset2 -- Port 28010 -- directoryperdb -- rest/apps/Mongo/bin/mongod -- dbpath/Data /replset_sharding/replset2/R1 -- replset replset2 -- Port 28011 -- directoryperdb -- rest/apps/Mongo/bin /Mongod -- dbpath/data/replset_sharding/replset2/R2 -- replset replset2 -- Port 28012 -- directoryperdb -- rest initialize the replica set: /apps/Mongo/bin/Mongo -- Port 28010config_replset2 = {_ ID: "replset2", members: [{_ ID: 0, host: "127.0.0.1: 28010", priority: 4}, {_ ID: 1, host: "127.0.0.1: 28011", Priority: 2}, {_ ID: 2, host: "127.0.0.1: 28012", arbiteronly: true}]} note that the arbiter arbitration node only votes and does not receive copied data! Rs. Initiate (config_replset2); then -------------------- create the third replset folder -------------------- create directory mkdir-P/data/replset_sharding/replset3/r0mkdir-P/data/repl Set_sharding/replset3/r1mkdir-P/data/replset_sharding/replset3/r2mkdir-P/data/replset_sharding/replset3/log use the following one (this is to facilitate reading logs, directly view the command line output) /apps/Mongo/bin/mongod -- dbpath/data/replset_sharding/replset3/R0 -- replset replset3 -- Port 38010 -- directoryperdb -- rest/apps/Mongo/bin/mongod -- dbpath/Data /replset_sharding/replset3/R1 -- replset replset3 -- Port 38011 -- directoryperdb -- rest/apps/Mongo/bin /Mongod -- dbpath/data/replset_sharding/replset3/R2 -- replset replset3 -- Port 38012 -- directoryperdb -- rest initialize the replica set: /apps/Mongo/bin/Mongo -- Port ipv10config_replset3 = {_ ID: "replset3", members: [{_ ID: 0, host: "127.0.0.1: 38010", priority: 4}, {_ ID: 1, host: "127.0.0.1: 38011", Priority: 2}, {_ ID: 2, host: "127.0.0.1: 38012", arbiteronly: true}]} note that the arbiter arbitration node only votes and does not receive copied data! Rs. Initiate (config_replset3); begin certificate );------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

// Create databases and log files

Mkdir-P/data/replset_sharding/log ------------------------------------------------------------------------ 1. Start config server, config server, 40002 mkdir-P/data/replset_sharding/config0mkdir-P/data/replset_sharding/config1mkdir-P/data/replset_sharding/config2

/Apps/Mongo/bin/mongod -- dbpath/data/replset_sharding/config0 -- configsvr -- Port 40000 -- directoryperdb -- rest/apps/Mongo/bin/mongod -- dbpath/data/replset_sharding/ config1 -- configsvr -- Port 40001 -- directoryperdb -- rest/apps/Mongo/bin/mongod -- dbpath/data/replset_sharding/config2 -- configsvr -- Port 40002 -- directoryperdb -- rest

To run the program in the background, use the following: /apps/Mongo/bin/mongod -- dbpath/data/replset_sharding/config -- configsvr -- Port 40000 -- logpath/data/replset_sharding/log/config. log -- fork -- directoryperdb -- rest partitions ------------------------------------------------------------------------------------------------------------------------------The above three config servers store the cluster shard configuration information. If the config server fails, the cluster cannot be started normally!The configuration server uses the two-step commit mechanism (instead of the asynchronous replication of the normal MongoDB replica set), because the two-step commit is consistent in real time,Asynchronous replication is the final consistency!
The two-step Submission mechanism ensures that the configurations of the three servers are consistent. If a configuration server goes down, the cluster configuration information will be read-only!The client can still read cluster data and update cluster data (as long as you do not need to modify the configuration of the server, such as modification of chunks block or modification of the chip key)PS: the admin database of the entire cluster is stored in config sever, and the three config servers not only synchronize the Database Config, but also synchronize the database admin.--------------------------------------------------------------------------------2. Start route process, route process: 50000/Apps/Mongo/bin/mongos -- Port 50000 -- configdb 127.0.0.1: 40000,127.0 .0.1: 40001,127.0 .0.1: 40002 -- chunksize 1 -- chunksize 1 (MB) specifies the minimum unit capacity of the shard, set 1 m here to make it easy to view the effect. To run it in the background, use the following:/apps/Mongo/bin/mongos -- Port 50000 -- configdb 127.0.0.1: 40000 -- chunksize 50 -- logpath =/data/replset_sharding/log/route. log -- fork --------------------------------------------------------------------------------3. Connect to mongos to configure shardingLog on to route process/apps/Mongo/bin/Mongo -- Port 50000 use admin with Mongo shell (remember to run this command and switch to the admin database)// Add a shard node. Each Shard is a replica set.DB. runcommand ({addshard: "replset1/127.0.0.1: 18010,127.0 .0.1: 18011,127.0 .0.1: 18012", allowlocal: true}) dB. runcommand ({addshard: "replset2/127.0.0.1: 28010,127.0 .0.1: 28011,127.0 .0.1: 28012", allowlocal: true}) dB. runcommand ({addshard: "replset3/127.0.0.1: 38010,127.0 .0.1: 38011,127.0 .0.1: 38012", allowlocal: true}) (allowlocal: true: only when development is complete, the parts are configured locally, this is not the case during production )-----------------------------------------------------------------------------------: The configuration information is added to the shards set of the config database of the config server:You will find that because the arbiter arbitration node does not receive data, it is directly removed by mongos in the configuration phase,DB. runcommand ({addshard: "replset1/127.0.0.1: 18010,127.0 .0.1: 18011,127.0 .0.1: 18012", allowlocal: true })

Equivalent to: DB. runcommand ({addshard: "replset1/127.0.0.1: 18010,127.0 .0.1: 18011", allowlocal: true })

Bytes -----------------------------------------------------------------------------------

// A. Configure the database mydb and enable the shard use admindb. runcommand ({enablesharding: "mydb "})In this case, the database mydb is based on the shard replset3 (primary shard)After mydb is configured to enable sharding, the configuration information is stored in the databases set of the config database of the configuration server!
// B. Set the set to be sharded: Enable the set users to shard the DB with the partition key _ id. runcommand ({shardcollection: "mydb. users ", key: {_ ID: 1}) after the users set of mydb is configured to enable sharding, the configuration information is stored in the collections set of the config database of the configuration server!

------------------------------------------------------------------------------ You can see that the Routing Server does not store configuration information (dbpath is also the reason, but the configuration on the server will be cached !) 5. sharding: insert data using the incremental partition key/apps/Mongo/bin/Mongo -- Port 50000 use mydb test to insert 0.6 million pieces of data for (VAR I = 1; I <= 600000; I ++) dB. users. insert ({age: I, name: "Jack", ADDR: "Guangzhou", Country: "China"}) Use admin to execute: DB. runcommand ({enablesharding: "mydb"}) enables the database to enable shard dB. runcommand ({shardcollection: "mydb. users ", key: {_ ID: 1}) splits the set users with _ id as the sharding key and waits for a few minutes. The data of the Set users is evenly distributed to each shard, fragment completion 1. Let's take a look at each trunk. here we need to test it. We have set 1 MB/trunk. Each trunk records the range of the chip key. In the partition, because the sharding key is objectid, objectid is like an incremental partition key. It cannot be evenly routed to each partition during insertion. In this case, the write load is uneven and concentrated on one machine!
: The balancer is moving the chunk of the shard.
When the incremental partition key is used, shards of the data are uneven after the data is inserted. Therefore, after the data is inserted, mongos uses balancer to load balance the data! For example, the balancer is executing load balancing, and the lock is used at this time. MongoDB 2.2.0 introduces the DB-Level Lock. I don't know if it is also a DB-Level Lock during sharding? This locks the database mydb graph of the shard: The balancer of mongos is executing the load balancing graph: The balancer of mongos is executing the Load Balancing

Figure: The balancer of mongos is performing Load Balancing for more than 10 minutes. At this time, the slice should be "even"... notice carefully, how can we "even ?? The chunks of each slice are even, and the chunks of each slice are 33, because the replset2 is the primary slice () of the database mydb, and because it is an incremental slice, at the beginning, all the 0.6 million records were inserted into this film, and because the chunksize we set for mongos is 1 MB, so when we configure two pieces of balanced data for replset2, so that the size of each trunk is balanced to nearly 1 MB (actually 1.1), 6, sharding, insert data/apps/Mongo/bin/Mongo -- Port 50000 use admin in random chip key mode and execute: DB. runcommand ({enablesharding: "mydb"}) enables the database to enable shard dB. runcommand ({shardcollection: "mydb. user2 ", key: {shardkey: 1}) makes the set user2 sharding key with shardkey as sharding key. The empty set user2 uses shardk After the key is sharding, a chunks (-∞, + ∞) block will be allocated to the database mydb substrate replset3 on the configuration server. The substrate is very important, for a non-sharding set, it is stored in the substrate, and for the sharding set, each infinite chunks block is also allocated to the substrate! Use mydb to insert 0.6 million data records. The shardkeyfor (VAR I = 1; I <= 600000; I ++) dB is generated here. user2.insert ({name: "a long name", age: I, ADDR: "A Long Address Guangzhou", Country: "China", shardkey: math. random ()}) because at the beginning, there were no chunks in the configuration server, and the first infinite chunks were located on the substrate, later, the chunks are split while inserting data into the database, and the chunks are mainly distributed on the shard000 substrate. As a result, the insert operation is concentrated on the substrate! Remove all the data in the Set user2! The set is the information of the remaining chunks blocks (in the configuration server) figure: after the set user2 is cleared, the number of chunks blocks of each slice is uneven. Figure: no data, the number of chunks blocks will also be adjusted until the number is average! (Note again:The essence of mongos routing load balancing is:Perform load balancing on each slice in the background until the number of chunks of each slice is equal!)Insert a 0.3 million random key document for (VAR I = 1; I <= 300000; I ++) dB into an empty set with 69 chunks (even blocks. user2.insert ({name: "a long name", age: I, ADDR: "A Long Address Guangzhou", Country: "China", shardkey: math. random ()})

Visible,Insert 0.3 million random keys into an empty set with 69 chunks blocks (even blocks). The entire insertion process is quite even! The number of blocks is only increased by 6. That is to say, almost all the 0.3 million documents "hit" existing blocks! Bytes -------------------------------------------------------------------------------------------8. Summary of Testing 6 and 7
1: If the chunks block information of the new set does not exist in the configuration server at the beginning, whether it is an incremental key or a random key,The data insertion process is not even, and may even be concentrated on a machine. Then mongos will execute Server Load balancer.
2: The mongos route performs load balancing on each slice in the background until the number of chunks blocks for each slice is equal!
3: For the sharding cluster of Server Load balancer (the number of chunks in each slice is equal), the operation on random keys is very effective, and the entire process is quite even.At this time, the incremental key operation will still cause serious load imbalance!
Partition 5: Execute sharding/apps/Mongo/bin/Mongo -- Port 50000 use mydb to insert 0.6 million data records for (VAR I = 1; I <= 600000; I ++) dB. users. insert ({age: I, name: "irelandken", ADDR: "Guangzhou", Country: "China"}) and wait a few minutes, the data in the users set is evenly distributed to each shard. The shards are completed ------------------------------------------------------------------ 6. The Shard server is removed and the data dB is recycled. runcommand ({"Removeshard": "127.0.0.1: 38010"}) Because 127.0.0.1: 38010 is the primary chip of the database test and mydb "primary": "shard0000, therefore, you need to manually move the database substrate such as/* 1 */{"_ id": "test", "partitioned": True, "primary ": "shard0000"}/* 2 */{"_ id": "mydb", "partitioned": True, "primary": "shard0000"} manually modify the substrate of the database test, change to 127.0.0.1: 38011 run: mongos> dB. runcommand ({"moveprimary": "test", "to": "127.0.0.1: 38011"}) {"primary": "shard0001: 127.0.0. 1: 38011 "," OK ": 1} manually modify the database test substrate and change it to 127.0.0.1: 38011. Execute: mongos> dB. runcommand ({"moveprimary": "mydb", "to": "127.0.0.1: 38011"}) {"primary": "shard0001: 127.0.0.1: 38011", "OK ": 1} after all the dependencies on the slices to be deleted are deleted, run the following command again: DB. runcommand ({"removeshard": "127.0.0.1: 38010"}) mongos> dB. runcommand ({"removeshard": "127.0.0.1: 38010"}) {"MSG": "removeshard completed successfully", "State": "completed", "shard": "shard" 0000 "," OK ": 1} mongos collects the data of the film, averages the data to other pieces, and then removes the film from the sharding cluster! You can see the progress in the middle of this sentence after multiple calls! This operation is completely transparent to users and does not require downtime! ------------------------------------------------------------------------------ 7. added the shard server to connect to mongos because there are shards of the database test and mydb collections in each shard, the newly added mongod cannot contain the same database (if the database to be added contains mydb and contains a certain amount of data, this data must be deleted) the newly added shard server to mongos must not contain the same database as other slices! Use admindb. runcommand ({addshard: "127.0.0.1: 38010", allowlocal: true}) after a new slice is added, mongos performs Load Balancing again to evenly distribute data to each slice! Slave from the log of each part, only the node shard server and the route process are "busy", and the config server seems to be just synchronizing the configuration of the route process, because the routing route process does not persist data, the config server stores the configuration and feels that the routing route process is a facade, and the external server seems to be a database.