MongoDB database Random query performance test for 100 million records _MONGODB

MONGDB Performance pressure test, random query, data volume 100 million records

Operating system centos6.4x64 bit

From the test results, when MongoDB all the data into memory, query speed according to the size of the document, performance bottlenecks are usually in the network traffic and CPU processing performance (in this test when all the data in memory, pure query speed can be stabilized in 10w/s around, System load can be maintained under 1, because at this time the CPU has been used to the limit, when the concurrent large load value will soar, the performance of a sharp decline.

Basic configuration of pressure generation server and MongoDB server

CPU Model: Intel (R) Xeon (r) CPU e5-2630 0 @ 2.30GHz

Memory: 64G

Hard disk: 10K Speed 6 block to do Raid0,raid card cache 1G (IO performance of the disk determines MongoDB to load the non-hotspot data to the memory speed, when the hotspot data all in memory, IO performance can be ignored)

Network card speed: Access Gigabit interactive machine, speed 1000MB

Data structure, user_id for the index key value of 0-99999999, the total size of data about 37G, all loaded into memory about 32G

> Db.thing.findOne ();

{

"_id": ObjectId ("51fd6830c6db9d15676f89ef"),

"IP": "192.168.168.254",

"G_roup": "Kiwi",

"Mac": "of:fd:67:8c:2f:8f",

"Address": "Hongmei1801num",

"user_id": 10000000,

"Name": "user10000000",

"title": "System",

"Database": "MongoDB",

"Telphone": Numberlong ("15718441234"),

"Mail": "Yj@chinapnr.com",

"OS": "Win7",

"Company": "CHINAPNR"

}

Pressure generation script opens 40 processes, concurrent random queries

[root@mdb-166 app]# Cat mselectmongodb.py

Import time,pymongo,multiprocessing,random,string

Class Sqltomongo:

def m_sql (self,x,y):

Server= "Mongodb://python:oracle@192.168.4.167:27017/syslog"

Conn=pymongo. Connection (server)

Db=conn.syslog

Col=db.thing

Start=x*y

End=start+x

For I in Xrange (start,end):

D=random.randint (Start,end)

Val=col.find ({"user_id":d})

A=list (Val)

def gen_load (X,taskid):

Task=sqltomongo ()

Print "task%s start!"% TaskID

Task.m_sql (X,taskid)

if __name__ = = "__main__":

inser_number=2500000

Pro_pool = multiprocessing. Pool (processes=101)

Print time.strftime ('%y-%m-%d:%h-%m-%s ', Time.localtime (Time.time ()))

Start_time=time.time ()

Manager = multiprocessing. Manager ()

For I in Xrange (40):

Taskid=i

Pro_pool.apply_async (gen_load,args= (Inser_number,taskid))

Pro_pool.close ()

Pro_pool.join ()

elapsed = Time.time ()-start_time

Print elapsed

Time.sleep (1)

Print "sub-process (es) done."

The pressure generates the server load, and when 80 concurrent is turned on, it cannot withstand

top-17:12:59 up 2 days, 7:34, 4 users, Load average:35.87, 29.41, 18.16

tasks:487 Total, running, 423 sleeping, 0 stopped, 0 zombie

Cpu0:88.1%us, 10.6%sy, 0.0%ni, 1.3%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st

Cpu1:87.7%us, 10.9%sy, 0.0%ni, 1.3%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st

Cpu2:87.7%us, 11.0%sy, 0.0%ni, 1.3%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st

Cpu3:88.0%us, 10.3%sy, 0.0%ni, 1.7%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st

Cpu4:87.7%us, 11.0%sy, 0.0%ni, 1.3%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st

Cpu5:88.7%us, 9.9%sy, 0.0%ni, 1.3%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st

Cpu6:40.2%us, 3.3%sy, 0.0%ni, 0.3%id, 0.0%wa, 0.0%hi, 56.1%si, 0.0%st

Cpu7:89.7%us, 8.9%sy, 0.0%ni, 1.3%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st

Cpu8:88.1%us, 10.6%sy, 0.0%ni, 1.3%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st

Cpu9:89.7%us, 9.3%sy, 0.0%ni, 1.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st

Cpu10:88.7%us, 9.9%sy, 0.0%ni, 1.3%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st

Cpu11:88.4%us, 10.2%sy, 0.0%ni, 1.3%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st

mem:65909128k Total, 5409148k used, 60499980k free, 90680k buffers

swap:8388600k Total, 0k used, 8388600k free, 4392420k cached

PID USER PR NI virt RES SHR S%cpu%mem time+ COMMAND

 13342 root      20   0  178m 9056 1480 R 15.9  0.0   5:15.79 python mselectmongodb.py                                                                           

 13430 root      20   0  178m 8976 1468 R 15.9  0.0   3:05.26 python mselectmongodb.py                                                                           

13321 Root 0 178m 9000 1464 R 15.6 0.0 5:17.26 python mselectmongodb.py

........

13311 Root 0 178m 8980 1464 R 14.9 0.0 5:21.70 python mselectmongodb.py

System load,

[root@mdb-167 ~]# Top

System load for concurrent process 80

[root@mdb-167 ~]# Top

Top-17:08:33 up 4:49, 5 users, Load average:0.00, 0.00, 1.05

tasks:287 Total, 2 running, 285 sleeping, 0 stopped, 0 zombie

Cpu0:60.1%us, 31.4%sy, 0.0%ni, 8.5%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st

Cpu1:57.8%us, 33.0%sy, 0.0%ni, 9.2%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st

Cpu2:58.4%us, 31.4%sy, 0.0%ni, 10.1%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st

Cpu3:54.8%us, 33.9%sy, 0.0%ni, 11.3%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st

Cpu4:55.8%us, 31.8%sy, 0.0%ni, 12.3%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st

Cpu5:53.4%us, 32.9%sy, 0.0%ni, 13.7%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st

Cpu6:31.4%us, 12.7%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 55.9%si, 0.0%st

Cpu7:61.0%us, 31.7%sy, 0.0%ni, 7.2%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st

Cpu8:59.7%us, 32.3%sy, 0.0%ni, 8.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st

Cpu9:59.0%us, 32.4%sy, 0.0%ni, 8.6%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st

Cpu10:59.4%us, 31.4%sy, 0.0%ni, 9.2%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st

Cpu11:57.8%us, 32.1%sy, 0.0%ni, 10.1%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st

mem:65909128k Total, 34281160k used, 31627968k free, 50416k buffers

swap:8388600k Total, 0k used, 8388600k free, 33502796k cached

   PID user      pr  ni  virt  res  SHR S%cpu%mem     time+  command                                                                                             

  2924 mdb       20   0 80.7g  31g  31g S 1092.6 50.7 458:42.39 /app/mongodb/bin/mongod-f/app/mongodb/etc/mdb.conf                                           

  1967 root      20   0     0    0     0 s  0.3  0.0   0:05.40 [kondemand/6]                                                                                       

1996 Root 0 10964 720 432 S 0.3 0.0 0:05.12 irqbalance

3045 mdb 0 84500 38m 7076 S 0.3 0.1 0:01.99 mongostat-h 127.0.0.1--port 27017-u xxxxx

Mongstat Monitoring

Insert Query Update delete getmore command flushes mapped vsize res faults locked db idx Miss% QR|QW Netin Netout Conn Time

*0 92408 *0 *0 0 1|0 0 40.1g 80.7g 31.9g 0 syslog:0.0% 0 0|0 20|0 5m 28m 82 17:08:32

*0 92827 *0 *0 0 3|0 0 40.1g 80.7g 31.9g 0 syslog:0.0% 0 0|0 19|0 5m 28m 82 17:08:33

.......

    *0  93717     *0     *0        0     1|0       0  40.1g  80.7g  31.9g       0  syslog:0.0%          0        0|0    25|0     5m    28m     82   17:08:38 

    *0  92193     *0     *0        0     3|0       0  40.1g  80.7g  31.9g       0  syslog:0.1%          0        1|0    11|0     5m    28m     82   17:08:39 

    *0  92250     *0     *0        0     1|0       0  40.1g  80.7g  31.9g       0  syslog:0.0%          0        0|0    25|0     5m    28m     82   17:08:40 

*0 93790 *0 *0 0 3|0 0 40.1g 80.7g 31.9g 0 syslog:0.0% 0 1|0 16|0 5m 28m 82 17:08:41

Network traffic

[root@mdb-167 app]#./check_netflow.py-l 1-i p1p1-t 1

In 1 seconds network transmit and receive average (KB)

transmit=34101,receive=11370

transmit=34354,receive=11452

.....

transmit=33957,receive=11322

transmit=34045,receive=11353

transmit=34244,receive=11416

Disk IO

[root@mdb-167 ~]# tail-f Mselectmongodb

device:rrqm/s wrqm/s r/s w/s rkb/s wkb/s avgrq-sz avgqu-sz await SVCTM%util

SDA 0.00 7.00 0.00 4.00 0.00 48.00 24.00 0.00 0.00 0.00 0.00

device:rrqm/s wrqm/s r/s w/s rkb/s wkb/s avgrq-sz avgqu-sz await SVCTM%util

SDA 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00

.....

SDA 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00

device:rrqm/s wrqm/s r/s w/s rkb/s wkb/s avgrq-sz avgqu-sz await SVCTM%util

SDA 0.00 7.00 0.00 5.00 0.00 48.00 19.20 0.00 0.00 0.00 0.00

Second, the concurrent process reaches 120 when the system load (due to CPU processing capacity, the system has been unable to withstand)

top-16:57:57 up 3:47, 5 users, Load average:58.18, 30.82, 16.40

tasks:288 Total, 1 running, 287 sleeping, 0 stopped, 0 zombie

Cpu0:59.6%us, 40.2%sy, 0.0%ni, 0.2%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st

Cpu1:59.3%us, 40.5%sy, 0.0%ni, 0.2%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st

Cpu2:59.0%us, 40.8%sy, 0.0%ni, 0.2%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st

Cpu3:58.7%us, 41.0%sy, 0.0%ni, 0.3%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st

Cpu4:58.8%us, 40.9%sy, 0.0%ni, 0.3%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st

Cpu5:58.8%us, 40.9%sy, 0.0%ni, 0.3%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st

Cpu6:27.2%us, 13.1%sy, 0.0%ni, 0.1%id, 0.0%wa, 0.0%hi, 59.6%si, 0.0%st

Cpu7:60.5%us, 39.3%sy, 0.0%ni, 0.2%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st

Cpu8:60.2%us, 39.5%sy, 0.0%ni, 0.2%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st

Cpu9:59.8%us, 39.9%sy, 0.0%ni, 0.3%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st

Cpu10:59.5%us, 40.2%sy, 0.0%ni, 0.3%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st

Cpu11:59.4%us, 40.4%sy, 0.0%ni, 0.3%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st

mem:65909128k Total, 34278248k used, 31630880k free, 45652k buffers

swap:8388600k Total, 0k used, 8388600k free, 33501056k cached

PID USER PR NI virt RES SHR S%cpu%mem time+ COMMAND

2924 mdb 0 80.7g 31g 31g S 1193.2 50.4 165:06.51/app/mongodb/bin/mongod-f/app/mongodb/etc/mdb.conf

Mongstat Monitoring

Insert Query Update delete getmore command flushes mapped vsize res faults locked db idx Miss% QR|QW Netin Netout Conn Time

*0 103334 *0 *0 0 1|0 0 40.1g 80.7g 31.7g 0 syslog:0.1% 0 0|0 56|0 6m 31m 122 16:55:19

*0 103299 *0 *0 0 3|0 0 40.1g 80.7g 31.7g 0 syslog:0.0% 0 0|0 55|0 6m 31m 122 16:55:20

......

*0 102858 *0 *0 0 3|0 0 40.1g 80.7g 31.7g 0 syslog:0.0% 0 0|0 66|0 6m 31m 122 16:55:26

*0 103201 *0 *0 0 1|0 0 40.1g 80.7g 31.7g 0 syslog:0.0% 0 0|0 60|0 6m 31m 122 16:55:27

*0 102381 *0 *0 0 3|0 0 40.1g 80.7g 31.7g 0 syslog:0.1% 0 1|0 89|0 6m 31m 122 16:55:28

Disk IO

[root@mdb-167 ~]# tail-f Mselectmongodb

device:rrqm/s wrqm/s r/s w/s rkb/s wkb/s avgrq-sz avgqu-sz await SVCTM%util

SDA 0.00 7.00 0.00 4.00 0.00 48.00 24.00 0.00 0.00 0.00 0.00

device:rrqm/s wrqm/s r/s w/s rkb/s wkb/s avgrq-sz avgqu-sz await SVCTM%util

.....

device:rrqm/s wrqm/s r/s w/s rkb/s wkb/s avgrq-sz avgqu-sz await SVCTM%util

SDA 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00

device:rrqm/s wrqm/s r/s w/s rkb/s wkb/s avgrq-sz avgqu-sz await SVCTM%util

SDA 0.00 7.00 0.00 5.00 0.00 48.00 19.20 0.00 0.00 0.00 0.00

Network traffic

transmit=37567,receive=12525

......

transmit=37691,receive=12568

transmit=37592,receive=12534

transmit=37480,receive=12498

The following is the MongoDB query speed change process from the start of the test, and when the data is gradually loaded into memory, the query speed is gradually increased, which takes about 20 minutes to load all the data into memory.

[mdb@mdb-167 ~]$ more Mselectmongodb

Connected to:127.0.0.1:27017

Insert Query Update delete getmore command flushes mapped vsize res faults locked db idx Miss% QR|QW Netin Netout Conn Time

*0 679 *0 *0 0 1|0 0 40.1g 80.6g 494m 671 syslog:1.0% 0 0|0 0|0 40 K 213k 42 16:37:16

*0 562 *0 *0 0 3|0 0 40.1g 80.6g 500m 625 syslog:0.9% 0 36|0 2|0 33 K 176k 42 16:37:17

..........

*0 626 *0 *0 0 1|0 0 40.1g 80.6g 550m 667 syslog:1.0% 0 36|0 1|0 36 K 196k 42 16:37:24

*0 649 *0 *0 0 3|0 1 40.1g 80.6g 557m 671 syslog:0.9% 0 35|0 1|0 38 K 202k 42 16:37:25

Insert Query Update delete getmore command flushes mapped vsize res faults locked db idx Miss% QR|QW Netin Netout Conn Time

*0 706 *0 *0 0 1|0 0 40.1g 80.6g 565m 709 syslog:1.0% 0 39|0 1|0 41 K 221k 42 16:37:26

..........

    *0    992     *0     *0        0     1|0       0  40.1g  80.6g   954m    751 syslog:1.1%          0        0|0     1|0    58k   309k     42   16:38:12 

*0 942 *0 *0 0 3|0 0 40.1g 80.6g 964m 759 syslog:0.9% 0 0|0 0|0 55 K 294k 42 16:38:13

*0 1026 *0 *0 0 1|0 0 40.1g 80.6g 974m 813 syslog:0.8% 0 0|0 0|0 60 K 319k 42 16:38:14

*0 1062 *0 *0 0 3|0 0 40.1g 80.6g 985m 826 syslog:0.8% 0 0|0 0|0 62 K 331k 42 16:38:15

*0 1044 *0 *0 0 1|0 0 40.1g 80.6g 995m syslog:0.9% 0 0|0 0|0 61 K 325k 42 16:38:16

Insert Query Update delete getmore command flushes mapped vsize res faults locked db idx Miss% QR|QW Netin Netout Conn Time

*0 1005 *0 *0 0 3|0 0 40.1g 80.6g 1005m 785 syslog:0.7% 0 30|0 1|0 59 K 313k 42 16:38:17

*0 1008 *0 *0 0 1|0 0 40.1g 80.6g 1015m 737 syslog:0.9% 0 0|0 1|0 59 K 314k 42 16:38:18

*0 1137 *0 *0 0 3|0 0 40.1g 80.6g 1g 853 syslog:0.7% 0 0|0 0|0 67 K 354k 42 16:38:19

*0 993 *0 *0 0 1|0 0 40.1g 80.6g 1.01g 768 syslog:0.9% 0 0|0 0|0 58 K 310k 42 16:38:20

*0 1114 *0 *0 0 3|0 0 40.1g 80.6g 1.02g 811 syslog:1.0% 0 0|0 0|0 65 K 346k 42 16:38:21

*0 1001 *0 *0 0 1|0 0 40.1g 80.6g 1.03g 744 syslog:1.0% 0 10|0 1|0 59 K 312k 42 16:38:22

..........

*0 1892 *0 *0 0 1|0 0 40.1g 80.6g 2.03g 900 syslog:0.7% 0 1|0 1|0 111 K 587k 42 16:39:44

*0 1881 *0 *0 0 3|0 0 40.1g 80.6g 2.04g 894 syslog:0.7% 0 0|0 0|0 111 K 583k 42 16:39:45

*0 1927 *0 *0 0 1|0 0 40.1g 80.6g 2.06g 902 syslog:0.8% 0 0|0 0|0 113 K 598k 42 16:39:46

Insert Query Update delete getmore command flushes mapped vsize res faults locked db idx Miss% QR|QW Netin Netout Conn Time

*0 1950 *0 *0 0 3|0 0 40.1g 80.6g 2.07g 912 syslog:0.6% 0 37|0 1|0 115 K 605k 42 16:39:47

*0 2086 *0 *0 0 1|0 0 40.1g 80.6g 2.09g 901 syslog:0.8% 0 0|0 1|0 123 K 647k 42 16:39:48

*0 1774 *0 *0 0 3|0 0 40.1g 80.6g 2.1g 846 syslog:0.8% 0 0|0 0|0 105 K 551k 42 16:39:49

.......

*0 2589 *0 *0 0 1|0 0 40.1g 80.6g 2.56g 933 syslog:0.8% 0 0|0 1|0 15 2k 803k 42 16:40:18

*0 2289 *0 *0 0 3|0 0 40.1g 80.6g 2.58g 849 syslog:0.9% 0 0|0 0|0 13 5k 710k 42 16:40:19

*0 2523 *0 *0 0 1|0 0 40.1g 80.6g 2.59g 930 syslog:0.7% 0 0|0 0|0 14 8k 781k 42 16:40:20

*0 2636 *0 *0 0 3|0 0 40.1g 80.6g 2.61g 878 syslog:0.8% 0 0|0 1|0 15 5k 817k 42 16:40:21

*0 2509 *0 *0 0 1|0 0 40.1g 80.6g 2.63g 889 syslog:0.7% 0 0|0 1|0 14 8k 777k 42 16:40:22

*0 2637 *0 *0 0 3|0 0 40.1g 80.6g 2.65g 916 syslog:0.7% 0 0|0 0|0 15 5k 817k 42 16:40:23

*0 2575 *0 *0 0 1|0 0 40.1g 80.6g 2.67g 911 syslog:0.8% 0 0|0 0|0 15 1k 798k 42 16:40:24

*0 2516 *0 *0 0 3|0 1 40.1g 80.6g 2.68g 873 syslog:6.5% 0 16|0 0|0 14 8k 779k 42 16:40:25

*0 2498 *0 *0 0 1|0 0 40.1g 80.6g 2.7g 879 syslog:0.8% 0 0|0 0|0 14 7k 774k 42 16:40:26

Insert Query Update delete getmore command flushes mapped vsize res faults locked db idx Miss% QR|QW Netin Netout Conn Time

*0 2536 *0 *0 0 3|0 0 40.1g 80.6g 2.72g 872 syslog:0.7% 0 0|0 0|0 14 9k 786k 42 16:40:27

*0 2826 *0 *0 0 1|0 0 40.1g 80.6g 2.74g 959 syslog:0.8% 0 0|0 1|0 16 6k 876k 42 16:40:28

*0 2676 *0 *0 0 3|0 0 40.1g 80.6g 2.75g 925 syslog:0.8% 0 0|0 0|0 15 8k 829k 42 16:40:29

.......

*0 8934 *0 *0 0 1|0 0 40.1g 80.6g 4.85g 1088 syslog:0.6% 0 0|0 1|0 52 7k 2m 42 16:41:50

*0 9583 *0 *0 0 3|0 0 40.1g 80.6g 4.89g 1161 syslog:0.6% 0 0|0 2|0 56 5k 2m 42 16:41:51

*0 8982 *0 *0 0 1|0 0 40.1g 80.6g 4.93g 1107 syslog:0.6% 0 23|0 3|0 53 0k 2m 42 16:41:52

*0 9950 *0 *0 0 3|0 0 40.1g 80.6g 4.97g 1067 syslog:0.5% 0 0|0 0|0 58 7k 3m 42 16:41:53

*0 9580 *0 *0 0 1|0 1 40.1g 80.6g 5.01g 1084 syslog:0.7% 0 0|0 1|0 56 5k 2m 42 16:41:54

.......

*0 92475 *0 *0 0 3|0 0 40.1g 80.7g 31.9g 1 syslog:0.0% 0 0|0 18|0 5m 28m 82 16:53:28

Insert Query Update delete getmore command flushes mapped vsize res faults locked db idx Miss% QR|QW Netin Netout Conn Time

    *0  92832     *0     *0        0     1|0       0  40.1g  80.7g  31.9g       0  syslog:0.0%          0        0|0    19|0     5m    28m     82   16:53:29 

    *0  92077     *0     *0        0     3|0       0  40.1g  80.7g  31.9g       0  syslog:0.0%          0        0|0    12|0     5m    28m     82   16:53:30 

    *0  93296     *0     *0        0     1|0       0  40.1g  80.7g  31.9g       0  syslog:0.1%          0        0|0    18|0     5m    28m     82   16:53:31 

*0 91269 *0 *0 0 3|0 0 40.1g 80.7g 31.9g 0 syslog:0.0% 0 1|0 10|0 5m 28m 82 16:53:32

*0 93878 *0 *0 0 1|0 0 40.1g 80.7g 31.9g 0 syslog:0.0%