CrawlerProgramAn exception occurred while running,
Java.net. socketexception: no buffer space available (maximum connections reached ?) : Jvm_bin
Httpclient is used in crawlers, And the thread pool is used.
After searching, I found that someone else has a similar problem. The solution is as follows: I changed the program to see if this problem will occur in the future.
From http://9iopen.group.javaeye.com/group/blog/144545
A crawler encounters the following exceptions:
Java.net. socketexception no buffer space available (maximum connections reached ?) : Jvm_bind
We know that the operating system has the maximum number of file handles that it can hold. During network connection, each socket request occupies one file handle resource. If it is not released in time, the file handle resource may be exhausted.
Pass CheckCodeIt is found that resources are not released when httpclient is used to send GET requests, and resources are easily exhausted due to multi-threaded programs.
The modified code is as follows:
Java code
-
- Httpclient client =
New
Httpclient ();
- Getmethod method =
Null
;
-
- Try
{
-
- Method =
New
Getmethod (fig. getuuri (). getescapeduri ());
-
- Statuscode = client.exe cutemethod (method );
-
- //...
-
- }
-
- Finally
{
- If
(
Null
! = Method)
-
- Method. releaseconnection ();
-
- }
The answer to Sun's technical forum is as follows:
Chances are you are forgetting to close a socket, a database
Connection, or some other connection that uses sockets internally. See
Example program below.
The alternative is that your program (or the sum of all programs
Running on your computer) really needs a lot of connections. In this
Case you'll need to find out how to increase the amount of socket
Buffer space that your operating system allocates. I 'd start
Googling for instructions. Or maybe you cocould redesign your program so
That it doesn' t use so much resources.
For more answers to this question, please refer to the following:
Http://forum.java.sun.com/thread.jspa? Threadid = 556382