Causes and solutions for high MongoDB connections

In the MongoDBSharding architecture, the number of connections is very high. Here, the number of connections is divided into several concepts: the number of tcp connections can be measured by netstat, which is generally the highest. If mongodmongos is on the same server, it is more obvious. Reference command: netstat-ant | awk {print $5} | awk-F: {print $1} | sort | uniq-c | sort-rnmo

In the MongoDB Sharding architecture, the number of connections is very high. The number of connections here is divided into several concepts: the number of tcp connections can be measured by netstat, which is generally the highest. If mongod/mongos is on the same server, it is more obvious. Reference command: netstat-ant | awk {print $5} | awk-F: {print $1} | sort | uniq-c | sort-rn mo

In the MongoDB Sharding architecture, the number of connections is very high. The number of connections is divided into several concepts:
The number of tcp connections netstat can be counted, which is generally the highest. If mongod/mongos is on the same server, it is more obvious.
Reference command: netstat-ant | awk '{print $5}' | awk-F: '{print $1}' | sort | uniq-c | sort-rn
The number of mongos/mongod connections was STAT/db. serverStatus ()/connPoolStats.
How high is the number of connections?
This depends on the number of application server instances connected to the mongodb cluster, and the qps (add, delete, modify, and query.
If the qps of a single application server is <100, the number of mongos connections exceeds 2000, it must be high. This is generally caused by unreasonable connection pool configurations.
Mongod/mongos maxConns = 20000 by default. versions 2.4 and earlier cannot exceed this value. version 2.6 (Development version 2.5) cancels this limit.
Related http://nosqldb.org/topic/50ca8a50ee680fee790001f2

Why is the connection count too high?

• Unreasonable connection pool Configuration
  In the case of slice, the phenomenon is that the number of tcp connections is too high (for example, 20000), and the number of mongos connections is too high (for example, more than 10000)
  For java, connectionsPerHost should not be configured too large. The official default value is changed from 10 to 100, and the default value is 5 times the multiplier (threadsAllowedToBlockForConnectionMultiplier), generally 20 ~ 50.
• Too many application server instances

In our scenario, when the number of application servers connected to mongos (such as the number of Tomcat instances) exceeds 200 or even nearly 15000, the number of tcp connections exceeds, check that the number of connections on the corresponding port of mongod is more than 8000, and the number of mongos is more than 2000. At this time, ops (query, insert, update, delete) is less than 200 per second ,. Regular mongos restart (for example, once a Monday) can alleviate this problem.

• The reason for mongodb itself is that the number of mongos connections is not high (such as 1000 +), and the number of mongod connections is relatively high (such as 8000 + ).

Solution
To sum up, there are several scenarios with high connection scores:
When there are too many application server instances, you can calculate the number of connections established by each instance, and appropriately reduce the connection pool parameter.
The number of mongos connections is high, which is a configuration problem. Change the connection pool parameters.
The number of mongos connections is not high, and the number of mongod connections is relatively high. For example, if the connection pool configuration is reasonable and still relatively high, try to enable the releaseConnectionsAfterResponse parameter (version 2.2.4 or later). This parameter is
Hide the releaseConnectionsAfterResponse Parameter

mongos> use adminswitched to db adminmongos> db.runCommand({ setParameter : 1, releaseConnectionsAfterResponse : true }){ "was" : false, "ok" : 1 }

Or

shell> mongos --setParameter "releaseConnectionsAfterResponse=true" --configdb ... 

Note:
The write operation needs to call getLastError immediately (w = 1, that is, the safe write mode), w = 2 (waiting for the slave database to write confirmation), there may be some errors.
After the upgrade, or after the mongos process is restarted, You need to reset this parameter, which takes effect only for a single mongos.
Enable the releaseConnectionsAfterResponse parameter, and the number of tcp connections is significantly reduced to a relatively stable number. In a few hours, the number of tcp connections dropped from more than 8000 to more than 4000, and the effect was good.

• ReleaseConnectionsAfterResponse parameter Principle

Generally, each mongos-> mongod connection is independently cached and cannot be reused, even if the connection is idle, it cannot be used again until the connection is closed and returned to the connection pool. After the releaseConnectionsAfterResponse parameter is enabled, mongos-> mongod connections can be reused after a read operation or secure write operation is completed (put the connections in the connection pool instead of the cache, that is, they are returned to the connection pool earlier ), the releaseConnectionsAfterResponse parameter is simply described as the mongos-> mongod connection is returned to the connection pool earlier, so that too many connections are not opened, thus reducing the number of connections.
Create a new serverParameter for mongos, "releaseConnectionsAfterResponse," which enables returning ShardConnections from the per-socket pool to the global pool after each read operation. this shocould reduce the total number of outgoing mongos connections to each shard.
The option allows better use of the connection pool, it doesn't invalidate the connections in the pool. normally, mongos-> mongod connections for insert/update/delete/query are cached inpidually for each incoming connection, and can't be re-used until the incoming connection is closed, even if they are idle and there are other active incoming connections.
What the releaseConnectionsAfterResponse option does is allow the mongos-> mongod connection to be re-used (returned to the pool) after any read op (including getLastError (), so after safe writes as well ). it shouldn't have a significant performance impact-the connection isn't destroyed, it's just returned from the incoming connection cache to the shared pool early.

Code:
Https://github.com/mongodb/mongo/commit/706459a8af0b278609d70e7122595243df6aeee8
Https://github.com/mongodb/mongo/commit/74323d671a216c8c87fcb295ed743f830d5212ee
Https://github.com/mongodb/mongo/commit/5d5fe49dfb5f452832b9d44fddbfb2a4e8b42f2a

====================
-ConnPoolTimeout settings

(This parameter is not officially available)
Effect

mongos> db.runCommand({ setParameter : 1, connPoolTimeout : 900 }){ "was" : 1800, "ok" : 1 }

Preliminary tests show that the effect is not obvious.

• ReleaseConnections

You are planning to add the command releaseConnections to run from mongod to reduce the number of replica set connections.

Reprinted from: http://nosqldb.org/topic/518510c8735345ad0a04fef8

Original article address: causes and solutions for high MongoDB connections, thanks to the original author for sharing.