How to implement one-click delivery of production-level MongoDB sharding clusters using containers

Author Introduction

Wei
Haven Court co-founder and CTO, responsible for back-end research and development of the Docker-based automated service management platform. Good at Docker related technology, pass platform architecture and enterprise automation operation and maintenance system design and development. Before starting a business in Sina SAE platform as a technical manager, from 09 SAE project to the second half of 13, four years, has been responsible for the Saepass platform of various web-based service management system design and development, in the DevOps has accumulated rich experience. Docker Engine code contributor, the number of code contributions in the Docker Open source project is around 50 worldwide.

Opening

MONGO supported replication set and sharding clusters very early. After so many years of precipitation, the maturity of the MONGO cluster has been very stable, by a large number of companies used in their own production environment. How to implement a one-click delivery deployment of MONGO clusters using container technology, shielding the details of the underlying implementation, is a topic that many people care about.

This article will introduce the process-based container technology to realize the one-click deployment of the MONGO sharding cluster, which fully demonstrates the powerful power of the container.

What is MONGO sharding cluster

Mongo (mongodb.com), as a popular document database, has many interesting features, such as built-in failover, support for file storage, support for MapReduce, and the ability to run JS scripts directly on the server side. Of course, what we care about most today is its built-in sharding functionality. The design of distributed system has always been a high challenge, in recent years, with the development of the Internet, distributed practice gradually began to be popularized and emphasized.

For an in-depth understanding of the MONGO sharding principle, refer to:

Https://docs.mongodb.com/manual/sharding

Cluster Orchestration Planning

MONGO clusters involve multiple services, including:

• ConfigDB, the central configuration of the sharding cluster, is a replicaset cluster that listens on 27019 ports
• MONGOs, as the access and management portal for the sharding cluster, listens on port 27017
• MONGO slices, data shard node
• MONGO Replicaset, each data shard is a replicaset cluster that guarantees no single point and provides automatic failover capabilities
• Mongo Express, MongoDB's management dashboard

Topology diagram

This deployment, CONFIGDB3 station, MONGO data node 12 units, divided into 4 shards, each Shard 1 Master 2 from

Prepare MONGO Mirror First

MONGO has provided an image of the MONGO 3.2, which can be used as a mirror for the MONGO cluster. To reduce subsequent maintenance efforts, we try not to build our own images.

Below we follow the MONGO Sharding official deployment document, step by step to orchestrate the cluster.

https://docs.mongodb.com/manual/tutorial/deploy-shard-cluster/

The order of deployment is: Mongo-express, mongo-replicaset/mongo-slices, ConfigDB, MONGOs

Deploying MONGO ConfigDB Preparing the ConfigDB configuration file

# cat mongo-config.confsystemLog:   0operationProfiling:   3000processManagement:   falsestorage:   dbPath: /data/db   journal:      true   engine: wiredTigerreplication:   replSetName: {{.ReplSetName}}sharding:   clusterRole: configsvrnet:   27019

Preparing the Init JS script for the CONFIGDB cluster

# cat configdb-init.js$s"mongo-configsvr" }}config = {    "{{.ReplSetName}}",     members : [     $c$s.Containers }}{_id : {{$c"{{$c.Domain}}:27019"},{{end}}     ]}rs.initiate(config)rs.status()

Start Orchestration ConfigDB Service first step create template

Step two Select MONGO image, specify cmd

The third step is to set the container parameters, associate the configuration file

Fourth Step Health Check strategy

The fifth step sets the deployment policy, the default number of deployment containers

Initialize CONFIGDB to Replicaset cluster

We do this by adding a one-off service that configsvr-init will automatically exit after the initialization action is performed

The first step is to select MONGO image, specify cmd

The second step is to set the container parameters, associated JS script

The third step is to set the deployment strategy with the priority number set to one (ConfigDB priority 10)

At this point, ConfigDB's arrangement is complete. We'll then orchestrate the MONGOs service

Preparing the MONGOs configuration file

# cat mongos.confreplication:   2000$port := .CfgPort }}{{$s"mongo-configsvr"}}sharding:   $i,$c$s.Containers}}{{if$i0}},{{end}}{{$c.Domain}}:{{$port}}{{end}}

MONGO profile We usually put it under/etc/.

Choreography MONGOs Template First Step select image, specify cmd parameter

Notice that the CMD executes the mongos.

The second step is to set the container parameters, Associate MONGOs Configuration

Third Step Health Check

The fourth step is to set the deployment strategy, note the priority

The MONGOs must run after the ConfigDB cluster is initialized, and then we begin to orchestrate the MONGO data node service, which is the most complex part of the cluster

Preparing the configuration file for the Shard node

cat mongo-shard.confsystemLog:   0operationProfiling:   3000processManagement:   falsestorage:   dbPath: /data/db   journal:      true   $num1$cn1$rsn$cn$num }}replication:   replSetName: RS{{$rsn}}

The variable REPLICA_NUM represents a few from each master, with a default of 2
Here, the 123 containers are compiled as rs0,456 rs1,789 and rs2,10,11,12 as RS3

Orchestration MONGO Shard Service first step add MONGO mirror, specify cmd parameter

Second step setting parameters, associating configuration

Third Step Health Check

Step fourth set up the deployment

With the above data node, we find that we have not initialized the Replset, and have not added each replset to sharding, let's do shard initialization work

Initializing the sharding cluster

#!/bin/bashSet - e{{$s: = Service"MONGO"}}{{$n: = (parseint. Replica_num) |add1}}{{$slices:=$s. Containernum |divide$n}}{{$port:= . Port}}EchoBegin to init replicaSet... forIinch{0.. {{$slices|subtract1}}}; Dobn=$ (($i* {{$n}} +1)) en=$ ($bn+ {{. Replica_num})) rs_node= rs_node_config= ii=$bn   while[$ii-le$en]; DoRs_node=$RS _node$ii. Mongo. {{. instance.name}}.csphere.local:{{$port}}, rs_node_config=$RS _node_config"{_id:$ii, host: \"$ii. mongo.{ {. instance.name}}.csphere.local:{{$port}}\ "},"ii=$ (($ii+1)) Doners_nodes=$ (Echo $RS _node|sed' s/,$//') rs_nodes_config=$ (Echo $RS _node_config|sed' s/,$//') Cat >/tmp/rs$i. js <<eofconfig = {_id:"RS$i", Members: [$RS _nodes_config]}rs.initiate (config) rs.status () EOF# # Connect to 0.mongo/3.mongo ...begin_mongo=$bn. Mongo. {{. Instance.name}}.csphere.local MONGO$begin _mongo/tmp/rs$i. JS Sleep3j=0   while[$j -lt Ten]; DoMongo$begin _mongo--eval"Rs.status ()"| grep PRIMARY && Breakj=$ (($j+1)) sleep2   Done  if[$j -eq Ten]; Then    EchoRs$iInitianized failedExit 1  fi  EchoRs$iinitinized OKEchoBegin to add RS$iTo sharding MONGO--eval"Sh.addshard (\" RS${i}/$RS _nodes\ ")" 1. MONGOs. {{. Instance.name}}.csphere.local Done

The script above is a little bit more complicated, and the basic logic is to initialize each replicaset cluster RS0 first. RS3, and add each RS to the sharding cluster as a shard slices. Refer to the deployment topology diagram above for details.

Start Orchestration Shard-init Service

The service is one-off executed

The first step is to add the image, specifying CMD as init.sh

The second step is to set the parameters and synchronize the script init.sh

Step three set deployment priorities

In this case, the MONGO sharding cluster is basically finished. Next we add a visualization panel for the MONGO service, which makes it easy for us to use

Orchestration Mongo-express Service First step add MONGO Express mirror

Setting environment variables in the second step

Third Step Health Check

Step fourth set deployment priorities

At this point, the orchestration is all done. Let's test the results of the work just now.

Deployment test

Click Deploy to generate an application instance that we named m .

Let's take a look at the initialization output of the Shard-init:

Let's look at the output of the Panel Express:

This is the home page of MONGO Express and you can view a summary of the visits to the MONGOs

We look at the data in the Config DB table, especially with regard to the cluster configuration:

MONGOs table:

What are the shards? We saw the RS0 RS1 RS2 RS3:

We can create a DB test named Csphere:

Follow-up, you can refer to the MONGO documentation, in development, set up your sharding key and so on. For further MongoDB management, please refer to the official documentation

Summarize

Looking at the entire orchestration from the automated delivery of the MongoDB cluster, we find that the key behind the magic is: DISCOVERY all, discovering everything and making it programmable.

Q&a

Q1: Hello, how is the actual application of the present situation? What are the challenges?

A1: The actual test runs well. The challenge lies in the orchestration process, the ability to discovery the ability to meet the automation behind. In real production, you can optimize your profile, such as challenge timeout, or log loglevel

Q2: What language is the template file rendered in, and what process is found to launch the container information?

A2: Haven Court Csphere platform behind a set of powerful configuration template engine, Golang template. The platform's focal center can discover various parameters and be programmed to address automation challenges

Q3: Hello MongoDB cluster performance bottleneck, monitoring metrics What do you think is the most important thing to pay attention to when expanding?

A3:mongo Cluster In use note the choice of storage engine, before 3.2 is mmap,3.2 has been adjusted, the new default engine can better handle the problem of large data volume. MONGO level expansion itself is good, note that MONGOs agent deploys a little more, to avoid traffic flowing out from a single MONGOs

Contact Haven Court

Phone 010-62249349

email [email protected]

Official website http://csphere.cn

How to implement one-click delivery of production-level MongoDB sharding clusters using containers