MongoDB --- performance optimization --- (1)

MONGODB Data Architecture


Upgrade Solution

Problem Discovery

The number of users that application servers suddenly flood into, resulting in slow server response


Check the server and find that the response speed is extremely slow, which is 30ops.


Check Process


. Slow database query found
. Check the index
. Check memory usage.
. Check TCP Connection




Comparison Test 1


. Hardware configuration:
. Client:
. Alibaba Cloud, 2 CPU, 2 GB memory, windows2008
. Server:
. A. Li cloud, router: 4 CPU, 4G memory, centos6.5 64-bit
. A. Li cloud, mongod: 2 CPU, 8 GB memory, centos6.5 64-bit
1. perform a test on the router, with QPS per second.
. 2. perform a test on ipvd, with QPS per second.
. Conclusion: The sharding architecture is used to make queries slower than the direct query of a single mongo.




Comparison Test 2


. Hardware configuration:
. Client:
. A. Li cloud, 2 CPU, 2g memory, windows2008
. A. Li cloud, 2 CPU, GB memory, windows2008
. Server:
. A. Li cloud, mongod: 2 CPU, 2g memory, windows2008
1. Perform a local 2 CPU, 2g memory, and windows2008 test. The query speed reaches QPS, and sometimes exceeds QPS.
. 2. Perform a 2-CPU, GB memory, and windows2008 test on Objective D. the query speed is QPS.
. Conclusion: tcp has a great impact on the... network.




Comparison Test 3


. Hardware configuration:
. Client:
. Physical Machine, 2 CPU, GB memory, windows2008
. Server:
. Physical Machine, Memory D: 2 CPU, 4.0g memory, centos6.5 64-bit
. Physical Machine, router: 2 CPU, 4.0g memory, centos6.5 64-bit
1. perform a test on mongod. The query speed is QPS.
2. perform a test on the router. The query speed is QPS.
. Conclusion: Alibaba Cloud has certain limitations, which limit the speed.




Comparison Test 4


. Hardware configuration:
. Client:
. Ucloud, 2 CPU, 4.0g memory, windows2008
. Server:
. Ucloud, mongod: 2 CPU, 4.0g memory, centos6.5 64-bit
. Ucloud, router: 2 CPU, 4.0g memory, centos6.5 64-bit
1. perform a test on mongod. The query speed is 2000qps.
2. perform a test on the router. The query speed is QPS.
. Conclusion: ucloud. BI. The cloud machine has good performance.




Comparison Test 5


. Hardware configuration:
. Client:
. Ucloud, 16 CPU, 6 4.0g memory, windows2008
. Server:
. Ucloud, mongod: 16 CPU, 64.0g memory, centos6.5 64-bit
. Ucloud, router: 16 CPU, 64.0g memory, centos6.5 64-bit
1. perform a test on mongod. The query speed is 13000qps.
2. perform a test on the router. The query speed is QPS.
. Conclusion: the root cause of the problem is found. The number of client cpu cores determines the mongodb query speed.




Conclusion


1. Alibaba Cloud's cloud host has some restrictions, resulting in no matter how to set the system or database configuration, it cannot break through. qps of an order of magnitude.
2. The number of client cpu cores determines the mongodb query speed.
3. tcp restrictions can be avoided through the Intranet.
4. linux is better than windows. Therefore, the mongodb architecture is better in linux than in windows.


Latest Architecture




Latest Architecture




Ultimate performance test


. The above section shows the latest architecture. perform a test:
. Open four iis clients and query the router,
The router can query up to 20000 times per second. For business comparison, you can handle 2000 people per second.




Performance conclusion


To ensure normal operation of online services, 8 CPUs and 16 GB memory are required for the server where the database is located.
To ensure the integrity of the backup, the slave database is backed up and the router is backed up. The backup is retained for 30 days.