The reason and solution of high MongoDB connection number

 

The number of connections in the MongoDB sharding architecture is easy to reach very high, where the number of connections is divided into several concepts:
Number of TCP connections netstat can be counted, which is generally the highest. If the Mongod/mongos is on the same server, it is more obvious.
Reference command: Netstat-ant|awk ' {print $} ' |awk-f: ' {print $} ' |sort |uniq-c|sort-rn
Mongos/mongod Number of Connections Mongostat/db.serverstatus ()/connpoolstats can be counted.
How high is the number of connections?
This depends on the number of instances connected to the MongoDB cluster application server, the QPS (increase and deletion) and other judgments.
Application Server single, if qps<100,, MONGOs connection number more than 2000, must be high. This is typically caused by an unreasonable connection pool configuration.
Mongod/mongos default maximum number of connections maxconns=20000,2.4 version and previous version maximum cannot exceed this value, version 2.6 (version 2.5) cancels this limit.
RELATED LINKS http://nosqldb.org/topic/50ca8a50ee680fee790001f2

What causes the number of connections to be too high

• The connection pool is not configured properly
  In the case of sharding, the phenomenon is that the number of TCP connections is too high (if 20000 is reached) and the MONGOs connection is too high (if more than 10000)
  Java, for example, Connectionsperhost should not be configured too large, the official default value from the original 10 changed to 100, and there is a default of 5 times times the multiplier (threadsallowedtoblockforconnectionmultiplier), General 20 ~50 can do it.

• Too many application server instances

The scenario we encountered, when connected to the MONGOs application server (such as the number of Tomcat instances) hundred, even reached nearly 200, the number of TCP connections is very high, reached more than 15000, view mongod corresponding port connections of up to 8,000, MONGOs more than 2000. At this time OPS (Query,insert,update,delete) is below 200 per second. Periodic restarts (such as once a week) MONGOs can mitigate this problem appropriately.

• The reason for MongoDB itself is that the number of MONGOs connections is not high (such as 1000+), with a higher number of mongod connections (such as 8000+).

How to Solve
To summarize, the high number of connections is divided into several scenarios:
Too many application server instances, you can count the number of connections established per instance, and properly lower the connection pool parameters.
MONGOs connection number is high, this is the configuration problem, change the connection pool parameters.
MONGOs connection number is not high, Mongod connection number is higher, such as more than 5000, if the connection pool configuration is reasonable and relatively high, try to enable the Releaseconnectionsafterresponse parameter (2.2.4 version above), this parameter is
Hide parameter Releaseconnectionsafterresponse

2. MONGOs> Use admin
4. Switched to DB admin
6. MONGOs> db. RunCommand({ setparameter : 1, releaseconnectionsafterresponse : True })
8. {" was" : false, "OK" : 1 }

Or

shell> mongos --setParameter "releaseConnectionsAfterResponse=true" --configdb ...

Considerations for this parameter:
The write operation requires immediate invocation of GetLastError (W=1, or secure write mode), and there may be errors when w=2 (waiting for write confirmation from the library).
After the upgrade, or after restarting the MONGOs process, you need to reset the parameter, which takes effect only for a single mongos.
With the Releaseconnectionsafterresponse parameter enabled, the number of TCP connections is significantly reduced to a relatively stable number. A few hours, the number of TCP connections from 8,000 to more than 4,000, the effect is good.

• Releaseconnectionsafterresponse parameter principle

Typically, each mongos->mongod connection is cached separately, and the connection cannot be reused, even if the connection is idle, until the connection closes and the connection is returned to the connection pool for reuse ; When the Releaseconnectionsafterresponse parameter is enabled, the connection between the Mongos->mongod can be reused after a read or secure write operation is completed (placing the connection in the connection pool instead of the cache, i.e. earlier regression into the connection pool) , the Releaseconnectionsafterresponse parameter is simply that the connection of the Mongos->mongod is back to the connection pool earlier, so that there is not too much connection, thus reducing the number of connections.
Create a new serverparameter for MONGOs, "Releaseconnectionsafterresponse," which enables returning shardconnections from The Per-socket pool to the global pool through each read operation. This should reduce the total number of outgoing mongos connections to each shard.
The option allows better use of the connection pool, it doesn ' t invalidate the connections in the pool. Normally, mongos->mongod connections for insert/update/delete/query is cached individually for each incoming CONNECTI On, and can ' t be re-used until the incoming connection was closed, even if they is idle and there was other active incomin G connections.
What the Releaseconnectionsafterresponse option does are allow the Mongos->mongod connection to being re-used (returned to The pool) after any read OP (including GetLastError (), so after safe writes as well). It shouldn ' t have a significant performance impact-the connection isn ' t destroyed, it's just returned from the incoming Connection cache to the shared pool early.

Code:
Https://github.com/mongodb/mongo/commit/706459a8af0b278609d70e7122595243df6aeee8
Https://github.com/mongodb/mongo/commit/74323d671a216c8c87fcb295ed743f830d5212ee
Https://github.com/mongodb/mongo/commit/5d5fe49dfb5f452832b9d44fddbfb2a4e8b42f2a

===============
-Connpooltimeout Settings

(This parameter is not official)
Effect

2.  mongos> Db. ({ Setparameter : 1, Connpooltimeout : 900 })
4. {" was" : 1800, "OK" : 1 } /c15>

Initial test, the effect is not obvious.

• Releaseconnections

Plan to add a command releaseconnections, run from Mongod, and reduce the number of replica set connections.

The reason and solution of high MongoDB connection number