Causes and solutions for MongoDB query timeout exceptions

Causes and solutions for MongoDB query timeout exceptions

Aggregate a collection of more than one million records.

 
 

  
  
DBObject match=(DBObject)JSON.parse("{$match:{logType:{'$in':[5,9]}}}"); 
  
  
DBObject group=(DBObject)JSON.parse("{$group:{'_id':'$domainUrl','count':{'$sum':1}}}"); 
  
  
AggregationOutput output = logCollection.aggregate(match,group); 
 
 

Occasionally, a Read timed out exception occurs.

 
 

  
  
com.mongodb.MongoException$Network: Read operation to server /192.168.10.202:27017 failed on database adLogTable 
  
  
at com.mongodb.DBTCPConnector.innerCall(DBTCPConnector.java:253) 
  
  
at com.mongodb.DB.command(DB.java:261) 
  
  
at com.mongodb.DB.command(DB.java:243) ... 
  
  
Caused by: java.net.SocketTimeoutException: Read timed out 
  
  
at java.net.SocketInputStream.socketRead0(Native Method) 
  
  
at java.net.SocketInputStream.read(SocketInputStream.java:152) 
 
 

After multiple tests, it is found that the average time for performing an aggregation operation is 5 s. If the time exceeds 5 s, an error is reported!

Then, view the MongoDB configuration information:

 
 

  
  
socket-timeout="5000" //5s 
 
 

The default socket-timeout configuration is 0, that is, there is no limit.

There is no timeout limit, and it is not easy to find out a problem in the system. A reasonable timeout time should be provided according to the actual situation.

Multiple tests show that the maximum execution time is 6 seconds, and the timeout time is set to 10000.

 
 

  
  
socket-timeout="10000" //10s 
 
 

Note: When MongoDB is integrated with Spring, if you want to configure multiple MongDB sources, only the last <mongo: options> configuration is enabled.

The parameter configuration information should be stored in the properties file.

 
 

  
  
<mongo:mongo host="${mongodb.ip}" id="mongo202" port="${mongodb.port}"> 
  
  
<mongo:options connections-per-host="200" 
  
  
threads-allowed-to-block-for-connection-multiplier="100" 
  
  
connect-timeout="1000" 
  
  
max-wait-time="1000" 
  
  
auto-connect-retry="true" 
  
  
socket-keep-alive="true" 
  
  
socket-timeout="10000" 
  
  
slave-ok="true" 
  
  
write-number="1" 
  
  
write-timeout="0" 
  
  
write-fsync="true" /> 
  
  
</mongo:mongo> 
 
 

Get configuration parameters through Java API

 
 

  
  
DBCollection logCollection = mongoTemplate.getCollection(collName); 
  
  
MongoOptions mongoOptions = logCollection.getDB().getMongo().getMongoOptions(); 
  
  
System.out.println(mongoOptions.getSocketTimeout()); 
 
 

Last point: ConnectionTimeOut and SocketTimeOut:

A complete request consists of three phases:

• Establish a connection
• Data Transmission
• Disconnect

If the connection time for a server (database) request exceeds ConnectionTimeOut, ConnectionTimeOutException is thrown, that is, the server connection times out and no connection is established within the specified time.

If the connection to the server is successful, data transmission starts.

If the time used by the server to process data exceeds SocketTimeOut, SocketTimeOutExceptin is thrown. That is, the server response times out and the server does not return data to the client within the specified time.