談MongoDB讀取連線逾時的異常

標籤：des   style   blog   http   color   io   os   java   ar   

今天在現場的哥們發來異常，讓我解決，錯誤資訊如下：

HTTP Status 500 - Read operation to server 192.168.1.110:20001 failed on database wpdb; nested exception is com.mongodb.MongoException$Network: Read operation to server 192.168.1.110:20001 failed on database wpdb--------------------------------------------------------------------------------type Exception reportmessage Read operation to server 192.168.1.110:20001 failed on database wpdb; nested exception is com.mongodb.MongoException$Network: Read operation to server 192.168.1.110:20001 failed on database wpdbdescription The server encountered an internal error that prevented it from fulfilling this request.exception org.springframework.dao.DataAccessResourceFailureException: Read operation to server 192.168.1.110:20001 failed on database wpdb; nested exception is com.mongodb.MongoException$Network: Read operation to server 192.168.1.110:20001 failed on database wpdborg.springframework.data.mongodb.core.MongoExceptionTranslator.translateExceptionIfPossible(MongoExceptionTranslator.java:56)org.springframework.data.mongodb.core.MongoTemplate.potentiallyConvertRuntimeException(MongoTemplate.java:1913)org.springframework.data.mongodb.core.MongoTemplate.executeFindOneInternal(MongoTemplate.java:1743)org.springframework.data.mongodb.core.MongoTemplate.doFindOne(MongoTemplate.java:1557)org.springframework.data.mongodb.core.MongoTemplate.findOne(MongoTemplate.java:497)com.geoway.ime.tile.dao.impl.TileDaoImpl.getTile(TileDaoImpl.java:60)com.geoway.ime.tile.service.impl.TileServiceImpl.getTile(TileServiceImpl.java:105)sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)java.lang.reflect.Method.invoke(Method.java:597)org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317)org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)org.springframework.cache.interceptor.CacheInterceptor$1.invoke(CacheInterceptor.java:58)org.springframework.cache.interceptor.CacheAspectSupport.execute(CacheAspectSupport.java:211)org.springframework.cache.interceptor.CacheInterceptor.invoke(CacheInterceptor.java:66)org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)com.sun.proxy.$Proxy50.getTile(Unknown Source)com.geoway.ime.server.rest.TileServer.getTileKVP(TileServer.java:60)sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)java.lang.reflect.Method.invoke(Method.java:597)com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$TypeOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:185)com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:302)com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108)com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84)com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1511)com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1442)com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1391)com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1381)com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:416)com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:538)com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:716)javax.servlet.http.HttpServlet.service(HttpServlet.java:723)org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:88)org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:106)root cause com.mongodb.MongoException$Network: Read operation to server 192.168.1.110:20001 failed on database wpdbcom.mongodb.DBTCPConnector.innerCall(DBTCPConnector.java:308)com.mongodb.DBTCPConnector.call(DBTCPConnector.java:279)com.mongodb.DBCollectionImpl.find(DBCollectionImpl.java:84)com.mongodb.DBCollectionImpl.find(DBCollectionImpl.java:66)com.mongodb.DBCollection.findOne(DBCollection.java:870)com.mongodb.DBCollection.findOne(DBCollection.java:844)com.mongodb.DBCollection.findOne(DBCollection.java:790)org.springframework.data.mongodb.core.MongoTemplate$FindOneCallback.doInCollection(MongoTemplate.java:1980)org.springframework.data.mongodb.core.MongoTemplate$FindOneCallback.doInCollection(MongoTemplate.java:1965)org.springframework.data.mongodb.core.MongoTemplate.executeFindOneInternal(MongoTemplate.java:1739)org.springframework.data.mongodb.core.MongoTemplate.doFindOne(MongoTemplate.java:1557)org.springframework.data.mongodb.core.MongoTemplate.findOne(MongoTemplate.java:497)com.geoway.ime.tile.dao.impl.TileDaoImpl.getTile(TileDaoImpl.java:60)com.geoway.ime.tile.service.impl.TileServiceImpl.getTile(TileServiceImpl.java:105)sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)java.lang.reflect.Method.invoke(Method.java:597)org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317)org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)org.springframework.cache.interceptor.CacheInterceptor$1.invoke(CacheInterceptor.java:58)org.springframework.cache.interceptor.CacheAspectSupport.execute(CacheAspectSupport.java:211)org.springframework.cache.interceptor.CacheInterceptor.invoke(CacheInterceptor.java:66)org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)com.sun.proxy.$Proxy50.getTile(Unknown Source)com.geoway.ime.server.rest.TileServer.getTileKVP(TileServer.java:60)sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)java.lang.reflect.Method.invoke(Method.java:597)com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$TypeOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:185)com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:302)com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108)com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84)com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1511)com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1442)com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1391)com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1381)com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:416)com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:538)com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:716)javax.servlet.http.HttpServlet.service(HttpServlet.java:723)org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:88)org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:106)root cause java.net.SocketTimeoutException: Read timed outjava.net.SocketInputStream.socketRead0(Native Method)java.net.SocketInputStream.read(SocketInputStream.java:129)java.io.BufferedInputStream.fill(BufferedInputStream.java:218)java.io.BufferedInputStream.read1(BufferedInputStream.java:258)java.io.BufferedInputStream.read(BufferedInputStream.java:317)org.bson.io.Bits.readFully(Bits.java:48)org.bson.io.Bits.readFully(Bits.java:35)org.bson.io.Bits.readFully(Bits.java:30)com.mongodb.Response.<init>(Response.java:42)com.mongodb.DBPort$1.execute(DBPort.java:141)com.mongodb.DBPort$1.execute(DBPort.java:135)com.mongodb.DBPort.doOperation(DBPort.java:164)com.mongodb.DBPort.call(DBPort.java:135)com.mongodb.DBTCPConnector.innerCall(DBTCPConnector.java:300)com.mongodb.DBTCPConnector.call(DBTCPConnector.java:279)com.mongodb.DBCollectionImpl.find(DBCollectionImpl.java:84)com.mongodb.DBCollectionImpl.find(DBCollectionImpl.java:66)com.mongodb.DBCollection.findOne(DBCollection.java:870)com.mongodb.DBCollection.findOne(DBCollection.java:844)com.mongodb.DBCollection.findOne(DBCollection.java:790)org.springframework.data.mongodb.core.MongoTemplate$FindOneCallback.doInCollection(MongoTemplate.java:1980)org.springframework.data.mongodb.core.MongoTemplate$FindOneCallback.doInCollection(MongoTemplate.java:1965)org.springframework.data.mongodb.core.MongoTemplate.executeFindOneInternal(MongoTemplate.java:1739)org.springframework.data.mongodb.core.MongoTemplate.doFindOne(MongoTemplate.java:1557)org.springframework.data.mongodb.core.MongoTemplate.findOne(MongoTemplate.java:497)com.geoway.ime.tile.dao.impl.TileDaoImpl.getTile(TileDaoImpl.java:60)com.geoway.ime.tile.service.impl.TileServiceImpl.getTile(TileServiceImpl.java:105)sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)java.lang.reflect.Method.invoke(Method.java:597)org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317)org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)org.springframework.cache.interceptor.CacheInterceptor$1.invoke(CacheInterceptor.java:58)org.springframework.cache.interceptor.CacheAspectSupport.execute(CacheAspectSupport.java:211)org.springframework.cache.interceptor.CacheInterceptor.invoke(CacheInterceptor.java:66)org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)com.sun.proxy.$Proxy50.getTile(Unknown Source)com.geoway.ime.server.rest.TileServer.getTileKVP(TileServer.java:60)sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)java.lang.reflect.Method.invoke(Method.java:597)com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$TypeOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:185)com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:302)com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108)com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84)com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1511)com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1442)com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1391)com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1381)com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:416)com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:538)com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:716)javax.servlet.http.HttpServlet.service(HttpServlet.java:723)org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:88)org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:106)note The full stack trace of the root cause is available in the Apache Tomcat/6.0.41 logs.

　　由異常來看，是由於MongoDB連線逾時造成的。於是，往MongoDB上找原因。首先上MongoDB的設定檔：

mongodb.replica-set=192.98.12.242:30007,192.98.12.245:30007mongodb.hostname=192.98.12.245mongodb.port=30007mongodb.connections-per-host=1000mongodb.threads-allowed-to-block-for-connection-multiplier=200mongodb.connect-timeout=2000mongodb.max-wait-time=2000mongodb.auto-connect-retry=truemongodb.socket-keep-alive=truemongodb.socket-timeout=1500mongodb.slave-ok=truemongodb.write-number=1mongodb.write-timeout=1000mongodb.write-fsync=truemongodb.dbname=shardvectordbmongodb.dbuser=mongodb.dbpassword=

　　連線時間2秒。然後在定位mongodb的查詢，結果為6、7秒。原來是這個原因。然後定位，一個查詢為什麼耗時那麼久呢？

經過排查，原來是查詢欄位沒有建立索引。然後把索引建立，查詢幾十毫秒，ok問題全部解決。

談MongoDB讀取連線逾時的異常