Build high usable MongoDB cluster (iv): Fragment __mongodb

According to the previous section, "Building a highly available MongoDB cluster (iii)--in-depth replica set" after the establishment of two problems are not resolved: from the node each of the above data is a full copy of the database, from the node pressure will not be too large. Data pressure to the machine can not support the automatic expansion.

In the early stage of the system, the amount of data will not cause too much problem, but as the amount of data continues to increase, sooner or later there will be a machine hardware bottleneck problem. And MongoDB is the mass data structure, he can not solve the massive data how to line. No way. "Fragmentation" uses this to solve this problem.

Traditional database How to do mass data reading and writing. In fact, a word summary: divide and conquer. As you can see on the diagram above, the following Taobao Yeu Xuqiang the architecture diagram mentioned in Infoq:

The above figure has a TDDL, is a Taobao data access layer component, his main role is SQL parsing, routing processing. Resolves the currently accessed SQL judge in which business database, which table accesses the query, and returns the data results, based on the functionality of the application request. Detailed diagram:

Having said so much about the structure of the traditional database, how did NoSQL do that? MySQL to do the automatic expansion of the need to add a data access layer to expand the program, database additions, deletions, backups need to control the program. One but the database node more than one, to maintain is also very headache. But mongodb all of this through his own internal mechanism can be done. Suddenly petrified, so cow X. Or the diagram above to see MongoDB through which mechanisms to achieve routing, fragmentation:

You can see from the diagram that there are four components: MONGOs, config server, shard, replica set.

MONGOs, the portal of the database cluster request, all requests are coordinated through MONGOs, no need to add a routing selector in the application, MONGOs is a request distribution center, it is responsible for the corresponding data request request to the corresponding Shard server. In the production environment there is usually more than MONGOs as the entrance of the request, preventing one of them from hanging all MONGODB requests have no way to operate.

Config server, as the name implies, is configured to store all database meta information (routing, fragmentation) configuration. The mongos itself does not physically store fragmented servers and data routing information, but is cached in memory, and the configuration server actually stores the data. MONGOs The first time you start or turn off the reboot will load the configuration information from config server, then if the configuration server information changes will notify all the MONGOs update their status, so MONGOs can continue to accurately route. There are usually multiple config server configuration servers in a production environment, because it stores the metadata for a fragmented route, which cannot be lost. Even if you hang one of them, as long as there is inventory, the MongoDB cluster will not hang.

Shard, this is the legend of the fragment. The above mentioned a machine even if the ability to have a ceiling, like the Army war, a person again drink blood bottle also spell each other's one division. As the saying goes, heads the top of Zhuge Liang, this time the strength of the team is highlighted. In the Internet is also the case, a common machine can not do many machines to do, the following figure:

A data table for a machine Collection1 stores 1T of data, and the pressure is too high. After the 4 machines were divided, each machine was 256G, and the pressure on one machine was apportioned. Maybe someone asked a machine hard drive to increase a little more, why should be divided to four machines. Do not think of storage space, the actual running of the database also has hard disk read and write, network IO, CPU and memory bottlenecks. As long as the fragment rule is set up in the MongoDB cluster, the corresponding data operation request can be forwarded to the corresponding slicing machine automatically through the MONGOs operation database. In the production environment fragment key can be set up, this affects how the data evenly distributed to a number of pieces of machine, do not appear in one of the machines divided into 1T, other machines do not have the situation, so as not to fragment.

Replica set, the last two sections have been detailed about this dongdong, how here again to join the fun. In fact, the above 4 slices if there is no replica set is an incomplete schema, assuming that one of the fragments hang out that one-fourth of the data is lost, so in the high availability of the fragmented architecture also need for each fragment build replica set replica set to ensure fragmentation of the reliability. The production environment is usually 2 copies + 1 arbitrations.

Say so much, still come to combat how to build a highly available MongoDB cluster:

First determine the number of components, MONGOs 3, config server 3, the data is 3 Shard server 3, each shard has a copy of a quorum that is 3 * 2 = 6, a total of 15 instances need to be deployed. These examples can be deployed in stand-alone machines can also be deployed in a machine, we have limited testing resources here, only prepared 3 machines, in the same machine as long as the port is different can, look at the physical deployment diagram:

The architecture is set up and the software is installed. 1, prepare the machine, the IP is set separately: 192.168.0.136, 192.168.0.137, 192.168.0.138. 2, respectively on each machine to establish MongoDB fragment corresponding Test folder.

Xhtml

1 2 3 4 5

#存放mongodb数据文件 mkdir-p/data/mongodbtest #进入mongodb文件夹 cd/data/mongodbtest

3, download the MongoDB installation package