Awesome MongoDB-PHP-Driver connection pool in PHP-FPM Mode

@ Zheng summary creation date: 2013/1 problem occurrence environment:

• Nginx
• PHP 5.3.10 as php-fpm extension to nginx
• Mongodb-php-driver 1.2.12
• Apsaradb for MongoDB 2.2

This problem is caused by official features of MongoDB PHP Driver 1.2.x. For the description, see PHP-202 and PHP-347. Simply put, in the PHP-FPM mode, every PHP Worker process has its own independent mongodb connection pool, which leads to excessive connections and increased memory. I. PHP service background:A Web application provides services through the PHP program running Nginx + factcgi. The maximum number of child processes of the PHP-FPM, Which is set through the max_children parameter of the php-fpm.conf (or pm = dynamic, determined by the spare_servers + start_servers parameter ). This value has been set 512. Ii. MongoDB service background:The maximum number of connections of a mongodb instance can be set through maxConns in the startup parameter:

• MaxConns: the default value depends on system restrictions (such as ulimit and file descriptor ). If this parameter is not set, mongodb does not limit the number of connections. However, you cannot set more than 20,000.

Generally, maxConns parameters are not set intentionally.

  Iii. What are the terrible connection pool features of MongoDB PHP Driver ?)Officially provided by MongoDB Mongodb-php-driverVersions earlier than 1.3.0 (1.2.0 ~ 1.2.1x) has a terrible connection pool implementation solution. When any query is executed, a connection will be requested from the connection pool, and then returned to the connection pool. Completion means that the variable holding the connection leaves its scope. In PHP-FPM mode, a PHP Web application can calculate the number of concurrent connections set up for MongoDB instance as follows:

• Number of processes: max-children = 512, so it is 512 processes;
• A MongoDB instance corresponds to a connection pool: the master station has two replica set instances: 165 and 166;
• Connections in the connection pool: mongodb-php-driver does not limit this. It can be infinitely increased until the handle is exhausted.

------ Zheng Jing: This calculation method is from mongo. connecting. pools ------ according to the official mongodb documentation, Although the infinite increase in the number of connections is theoretically possible, the actual observation found that the number of connections between a Web Server and a mongodb instance is usually stable in a value, there will be no big fluctuations. Then, assume that The number of connections a PHP Web application initiates to a mongodb-165 is: 750The MongoDB instance must maintain at least 750x10 MB by default. 7.5 GB 

5. Solution

Quickly upgrade to mongodb-php driver 1.3.2. Reference:1) 2012-12-9, Connection Handling with the MongoDB PHP driver, English document, Chinese translation; 2) test result of Li DAN: "test the AB result after the driver is upgraded to the 1.3.2 stable version, and find that the close effect is obvious, and the number of connections quickly drops to the number before the test. Although the peak value is larger than the old driver, it should be able to solve the problem of high connection continuity on the existing online ." 3) mongodb Connection Pooling (version 1.2.0-1.2.12 * only *); 4) mongodb-java-driver defines the maximum number of connections that an application can establish with a mongodb instance, (connectionsPerHost × threadsAllowedToBlockForConnectionMultiplier) connections:

• Mongo. options. connectionsPerHost: Maximum number of physical connections that can be established between each Application and MongoDB instance. The default value is 10;
• Mongo. options. threadsAllowedToBlockForConnectionMultiplier: Maximum number of threads available in the pool, which is 5 by default.

5) crazyshell, 2012, MongoDB maxConns parameters;

Several images are presented: Spymemcached related articles: 1) spymemcached useNagle problem and TCP/IP delay data sending 2) spymemcached: the feature of Auto-Reconnect when an mc node times out for more than 998 consecutive times. 3) implementation of Multiget hole: spymemcached

Recommended reading:

Three instances demonstrate Java Thread Dump Log Analysis