MongoDB分區群集

標籤：primary   and   sys   mongod   table   實驗   ffffff   檔案儲存   通過   

MongoDB分區群集什麼是分區

高資料量和輸送量的資料庫應用會對單機的效能造成較大的壓力，大的查詢量會把單機CPU耗盡，大的資料量對單機的儲存壓力大，最終會消耗系統的記憶體而將壓力轉移到磁碟IO上

MongoDB分區的優勢

分區為應對高輸送量與大資料量提供了方法

• 使用分區減少了對需要處理的請求，因此，通過水平拓展，群集可以提高自己的儲存量和輸送量。
• 使用分區減少了每個分區儲存的資料

分區伺服器分為以下幾個伺服器組成

• 路由伺服器 （router）
• 設定管理員 （configsvr）
• 分區伺服器 （shardsvr）

我們通過路由伺服器發送請求給設定管理員，設定管理員調用下面的分區伺服器結構圖如下

MongoDB分區群集的部署====（註：分區群集在3.2和3.2以上的版本不同3.2版本在做分區伺服器的時候可以一個分區伺服器對應一台資料庫，但是在3.2以上的版本它要求每個分區伺服器必須做群集）====這邊我們用的是3.2版本的原始碼包實驗步驟：做設定管理員

---------------------安裝MongoDB 3.2----------------------------#安裝開發包yum install openssl-devel -y#解壓原始碼包tar zxvf mongodb-linux-x86_64-3.2.1.tgz -C /opt/#重新命名並移動到/usr/local檔案裡面mv mongodb-linux-x86_64-3.2.1/ /usr/local/mongodb#建立資料存放位置mkdir -p /data/mongodb/mongodb{1,2,3,4}#建立日誌存放位置mkdir /data/mongodb/logs建立以log為結尾的日誌存放touch /data/mongodb/logs/mongodb{1,2,3,4}.log#給記錄檔最大全許可權chmod -R 777 /data/mongodb/logs/*.log#修改最大並發串連ulimit -n 25000ulimit -u 25000

上面是開始安裝MongoDB，下面開始配置，設定管理員的設定檔

cd /usr/local/mongodb/bin/vim mongodb1.conf#設定管理員連接埠號碼port=37017       #資料檔案儲存位置dbpath=/data/mongodb/mongodb1#記錄檔儲存位置logpath=/data/mongodb/logs/mongodb1.loglogappend=truefork=true#同時承受的並發串連數maxConns=5000storageEngine=mmapv1#配置為設定管理員configsvr=true

某節點記憶體不足時，從其他節點分配記憶體

sysctl -w vm.zone_reclaim_mode=0echo never > /sys/kernel/mm/transparent_hugepage/enabledecho never > /sys/kernel/mm/transparent_hugepage/defrag#建立軟串連便於管理ln -s /usr/local/mongodb/bin/mongo /usr/bin/mongoln -s /usr/local/mongodb/bin/mongod /usr/bin/mongod#開啟設定管理員mongod -f /usr/local/mongodb/bin/mongodb1.conf

分區伺服器 1和分區伺服器 2同樣配置改連接埠號碼資料檔案記錄檔位置加入shardsvr=true刪除原有的設定管理員configsvr=true

#連接埠號碼改為47017port=37017        #資料檔案儲存位置dbpath=/data/mongodb/mongodb2#記錄檔儲存位置logpath=/data/mongodb/logs/mongodb.loglogappend=truefork=true#同時承受的並發串連數maxConns=5000storageEngine=mmapv1#配置為分區伺服器shardsvr=true#分別開啟分區伺服器1和2

啟動路由伺服器

cd /usr/local/mongodb/bin/#啟動mongos 路由伺服器的連接埠號碼為27017日誌存放目錄在/usr/local/mongodb/bin/下面的route.log裡面佈建服務IP：連接埠號碼 --chunkSize 1 塊大小為1./mongos --port 27017 --fork --logpath=/usr/local/mongodb/bin/route.log --configdb 192.168.235.204:37017 --chunkSize 1

直接啟動mongo路由伺服器

[[email protected] bin]# mongoMongoDB shell version: 3.2.1connecting to: testWelcome to the MongoDB shell.For interactive help, type "help".For more comprehensive documentation, see    http://docs.mongodb.org/Questions? Try the support group    http://groups.google.com/group/mongodb-userServer has startup warnings: 2018-07-18T14:52:06.622+0800 I CONTROL  [main] ** WARNING: You are running this process as the root user, which is not recommended.2018-07-18T14:52:06.622+0800 I CONTROL  [main] mongos>#查看資料庫mongos> show dbsconfig  0.031GBmongos> sh.status()   #shards下為空白，沒有分區伺服器--- Sharding Status ---   sharding version: {    "_id" : 1,    "minCompatibleVersion" : 5,    "currentVersion" : 6,    "clusterId" : ObjectId("5b4ee3961c914459f603260f")}  shards:  active mongoses:    "3.2.1" : 1  balancer:    Currently enabled:  yes    Currently running:  no    Failed balancer rounds in last 5 attempts:  0    Migration Results for the last 24 hours:         No recent migrations  databases:  #添加分區伺服器  mongos> sh.addShard("192.168.32.207:47018"){ "shardAdded" : "shard0000", "ok" : 1 }mongos> sh.addShard("192.168.32.207:47017"){ "shardAdded" : "shard0001", "ok" : 1 }mongos> sh.status()--- Sharding Status ---   sharding version: {    "_id" : 1,    "minCompatibleVersion" : 5,    "currentVersion" : 6,    "clusterId" : ObjectId("5b4ee3961c914459f603260f")}  shards:  #下面兩個是分區伺服器    {  "_id" : "shard0000",  "host" : "192.168.32.207:47018" }    {  "_id" : "shard0001",  "host" : "192.168.32.207:47017" }  active mongoses:    "3.2.1" : 1  balancer:    Currently enabled:  yes    Currently running:  no    Failed balancer rounds in last 5 attempts:  0    Migration Results for the last 24 hours:         No recent migrations  databases:

實驗分區功能

添加兩個分區伺服器後，資料庫和集合還未啟用分區使用mongoimport命令匯入sales.txt資料到kgc資料庫的sales表

mongos> db.createCollection(‘users‘){ "ok" : 1 }mongos> show collectionssystem.indexesusersmongos> exitbye[[email protected] bin]# mongoMongoDB shell version: 3.2.1connecting to: testServer has startup warnings: 2018-07-18T14:52:06.622+0800 I CONTROL  [main] ** WARNING: You are running this process as the root user, which is not recommended.2018-07-18T14:52:06.622+0800 I CONTROL  [main] mongos> show dbsconfig  0.031GBkgc     0.078GBmongos> for (var i=1;i<=10000;i++)db.users.insert({"id":1,"name":"jaap"+i}) #插入10000條資訊WriteResult({ "nInserted" : 1 })mongos> show tablessystem.indexesusersmongos> db.users.find() #查看資訊{ "_id" : ObjectId("5b4ef398b83d741f7c50491a"), "id" : 1, "name" : "zhang" }{ "_id" : ObjectId("5b4ef407b83d741f7c50491b"), "id" : 1, "name" : "jaap1" }{ "_id" : ObjectId("5b4ef407b83d741f7c50491c"), "id" : 1, "name" : "jaap2" }{ "_id" : ObjectId("5b4ef407b83d741f7c50491d"), "id" : 1, "name" : "jaap3" }{ "_id" : ObjectId("5b4ef407b83d741f7c50491e"), "id" : 1, "name" : "jaap4" }{ "_id" : ObjectId("5b4ef407b83d741f7c50491f"), "id" : 1, "name" : "jaap5" }{ "_id" : ObjectId("5b4ef407b83d741f7c504920"), "id" : 1, "name" : "jaap6" }{ "_id" : ObjectId("5b4ef407b83d741f7c504921"), "id" : 1, "name" : "jaap7" }{ "_id" : ObjectId("5b4ef407b83d741f7c504922"), "id" : 1, "name" : "jaap8" }{ "_id" : ObjectId("5b4ef407b83d741f7c504923"), "id" : 1, "name" : "jaap9" }{ "_id" : ObjectId("5b4ef407b83d741f7c504924"), "id" : 1, "name" : "jaap10" }{ "_id" : ObjectId("5b4ef407b83d741f7c504925"), "id" : 1, "name" : "jaap11" }{ "_id" : ObjectId("5b4ef407b83d741f7c504926"), "id" : 1, "name" : "jaap12" }{ "_id" : ObjectId("5b4ef407b83d741f7c504927"), "id" : 1, "name" : "jaap13" }{ "_id" : ObjectId("5b4ef407b83d741f7c504928"), "id" : 1, "name" : "jaap14" }{ "_id" : ObjectId("5b4ef407b83d741f7c504929"), "id" : 1, "name" : "jaap15" }{ "_id" : ObjectId("5b4ef407b83d741f7c50492a"), "id" : 1, "name" : "jaap16" }{ "_id" : ObjectId("5b4ef407b83d741f7c50492b"), "id" : 1, "name" : "jaap17" }{ "_id" : ObjectId("5b4ef407b83d741f7c50492c"), "id" : 1, "name" : "jaap18" }{ "_id" : ObjectId("5b4ef407b83d741f7c50492d"), "id" : 1, "name" : "jaap19" }Type "it" for moremongos> db.users.find().limit(5)  #查看指定資訊{ "_id" : ObjectId("5b4ef398b83d741f7c50491a"), "id" : 1, "name" : "zhang" }{ "_id" : ObjectId("5b4ef407b83d741f7c50491b"), "id" : 1, "name" : "jaap1" }{ "_id" : ObjectId("5b4ef407b83d741f7c50491c"), "id" : 1, "name" : "jaap2" }{ "_id" : ObjectId("5b4ef407b83d741f7c50491d"), "id" : 1, "name" : "jaap3" }{ "_id" : ObjectId("5b4ef407b83d741f7c50491e"), "id" : 1, "name" : "jaap4" }mongos> sh.status()  #查看資料庫分區資訊--- Sharding Status ---   sharding version: {    "_id" : 1,    "minCompatibleVersion" : 5,    "currentVersion" : 6,    "clusterId" : ObjectId("5b4ee3961c914459f603260f")}  shards:    {  "_id" : "shard0000",  "host" : "192.168.32.207:47018" }    {  "_id" : "shard0001",  "host" : "192.168.32.207:47017" }  active mongoses:    "3.2.1" : 1  balancer:    Currently enabled:  yes    Currently running:  no    Failed balancer rounds in last 5 attempts:  0    Migration Results for the last 24 hours:         No recent migrations  databases:    {  "_id" : "kgc",  "primary" : "shard0000",  "partitioned" : false }

啟用分區

mongos> sh.enableSharding("kgc"){ "ok" : 1 }mongos> sh.status()--- Sharding Status ---   sharding version: {    "_id" : 1,    "minCompatibleVersion" : 5,    "currentVersion" : 6,    "clusterId" : ObjectId("5b4ee3961c914459f603260f")}  shards:    {  "_id" : "shard0000",  "host" : "192.168.32.207:47018" }    {  "_id" : "shard0001",  "host" : "192.168.32.207:47017" }  active mongoses:    "3.2.1" : 1  balancer:    Currently enabled:  yes    Currently running:  no    Failed balancer rounds in last 5 attempts:  0    Migration Results for the last 24 hours:         No recent migrations  databases:    {  "_id" : "kgc",  "primary" : "shard0000",  "partitioned" : true#對users表中id建立主鍵索引mongos> db.users.createIndex({"id":1})#表分區mongos> sh.shardCollection("kgc.users",{"id":1})#查看分區情況mongos> sh.status()--- Sharding Status ---   sharding version: {    "_id" : 1,    "minCompatibleVersion" : 5,    "currentVersion" : 6,    "clusterId" : ObjectId("5b4ee3961c914459f603260f")}  shards:    {  "_id" : "shard0000",  "host" : "192.168.32.207:47018" }    {  "_id" : "shard0001",  "host" : "192.168.32.207:47017" }  active mongoses:    "3.2.1" : 1  balancer:    Currently enabled:  yes    Currently running:  no    Failed balancer rounds in last 5 attempts:  0    Migration Results for the last 24 hours:         No recent migrations  databases:    {  "_id" : "kgc",  "primary" : "shard0000",  "partitioned" : true }        kgc.users            shard key: { "id" : 1 }            unique: false            balancing: true            chunks:                shard0000   1            { "id" : { "$minKey" : 1 } } -->> { "id" : { "$maxKey" : 1 } } on : shard0000 Timestamp(1, 0)

添加標籤

#添加分區伺服器mongod -f mongodb4.confmongomongos> sh.addShard("192.168.235.205:47019")mongos> sh.status()chunks:    shard0000   4    shard0001   4    shard0002   3#刪除分區節點mongos> use adminmongos> db.runCommand({"removeshard":"192.168.235.205:47019"})

MongoDB分區群集