Java.net. socketexception: no buffer space available (maximum connections reached ?) : Jvm_bind

A friend asked me to help solve the problem of a Java application on his server. I used the business time to rush to the site twice and finally solved the problem and kept a record, I don't want to help the people who have been as confused as me. I hope I don't forget it.

I. Problem Performance

First, describe the environment. The server is Windows Server 2003. I am not very familiar with the features of this operating system. I just use it as an ordinary personal operating system. I usually install linux on Java servers.

SQL Server 2005 and SP2 used by the database. His database was originally from SQL Server 2000 and I helped him migrate it last week.

Middleware uses resin 2.1.17. This is a servlet container that I like very much and are familiar with. Adding or not adding Apache has good web service performance.

Anti-virus software uses rising, dual network card, hardware firewall, what brand I don't care about.

So what is the problem?

This!

Com. Microsoft. sqlserver. JDBC. sqlserverexception: The TCP/IP connection to the host fails. Java.net. socketexception: no buffer space available (maximum connections reached ?) : Jvm_bind

After resin is started, it will be normal for a short period of time, but if there is no access, it will die several hours later. There is no problem with the static page, that is, the HTTP web server is not dead, and database connection is not involved.

Ii. Analysis

This is the first time I encountered this error.

I first ruled out the program problem by myself. Some people on the Internet said that the database connection was not closed, and such comrades were new to me. I can also be certain that the program is certainly no problem, this error occurs at the bottom of the database.

I read a lot of articles on the Internet. The reasons are as follows.

1. Windows server runs automatic updates. I still don't know why the system consumes so many socket connections after automatic updates are run. However, automatic updates will definitely occupy network bandwidth and CPU usage. on the server, I think automatic updates should be disabled and manually updated.

2. Trojans are in the machine. Trojan horse will open many backdoors and create a large number of connections, including local network bots and Internet. This also causes the machine to be unable to access the Internet, or the program to be unable to connect. In the search results, a large number of results are SQL Server database servers, which can easily attract Trojans and viruses. Most of them are SQL server problems, not accidental.

3. The virtual memory configuration is too low. After I found this problem for the first time, I went to the site to open the machine and reported a virtual memory error, saying that the virtual content was insufficient.

4. an improper program creates a large number of short connections. Due to the program's robustness, the socket is busy or time_wait status, or the server and client have an exception, while the other side is not notified, it is still connected and constantly performs write operations.

3. Hands-on solutions

After analysis, I wrote down the measures to be taken one by one. My head is not very easy to use and it is easy to forget things.

The measures are as follows: What to do first, what to do later, and the friends who see this article can adjust it by themselves.

1. Optimize the resin configuration and appropriately expand the number of connection pools. Although this is not the problem, it is still slightly extended. Remote monitoring is configured, directory browsing that is not performed by default is disabled, and errors such as 404 are blocked. Use httpd-install to install Resin into the service. In the past, my friends started the console directly, this can easily cause the program to stop after the console caches it to the top. The show_ SQL switch of Hibernate is disabled.

2. Disable Automatic Updates and disable the Background Intelligent Transfer Service. When you open the service, the service window is automatically closed five seconds later, and there are many strange services. At the same time, we can see that Norton has been automatically disabled, and it cannot be opened. This machine has a lot of viruses.

3. Anti-Virus. I don't know. I was shocked. My buddy is really a big male. Fuck, all kinds of viruses are in progress, and they are still providing services. This server is really poor! If the host is ill, the host will not be able to see the doctor. It is impossible for the host to serve the country without medical insurance. No matter whether your uncle is here to treat you. After analysis, the viruses include USB flash drives, runauto .. viruses, gray pigeons, legendary Trojans, various unknown backdoors and Trojans, and remote control. I downloaded a 360 file and wanted to be lazy. The installation was not possible. Deleted runauto... (my previous blog specifically described how to deal with this stuff). As a result, drive C cannot be opened and can only be opened in the resource manager. It's quite troublesome. So I found a few hiding places for the virus, analyzed the unknown service, and failed to delete it using DOS. It was still useful to smash 360 of the files. What is C:/Windows/lsass.exe, and so on? example .exe.exe is too common. The specific process is not very clear, in short, the last 360 can be installed and updated. 360 This is really good, and all Trojans are cleared. Now the service window is not automatically closed, Norton is also monitored normally, the machine does not open the port abnormally, there is no inexplicable service, well, it's almost done.

4. Optimize server settings. In fact, I do not know such optimization. Looked at, the virtual memory is in the C disk, the C disk space is already very small, find something to see clearly, move to another disk, do not understand, do not dare to give people to move. Re-open the service window. All the useless services are open. Close all the services I have identified and what server services are still running ADMIN $. Close them all, this machine is useless. What else does IIS and Apache work well? I think tomcat has been installed. Simply add resin to open it. It's really busy. Of course I don't want this to happen. It's all done.

5. check whether a program has a vulnerability. Because I configured a small tool under resin that only my own Google robots could not find (and also refused robot scanning, previously, we saw that a large number of connections were created from time to time by an Internet IP address, but they were all in the time_wait status. This is also terrible. After reading this, in addition to this web application, there is also a small program that automatically receives data every day. I don't know whether this program uses a socket persistent connection or a short connection. If you are interested in this content, You can Google it in detail. However, this program is indeed not configured to run automatically upon startup (I added it). It may be that clients on the IP address of the Internet are constantly seeking for connections! Socket. Close () is set to the time_wait status instead of releasing the connection immediately. The default time is 240 s, that is, 4 minutes. This is too long. It takes 30 seconds (minimum value. When modifying this default value, we also found that this machine has not enabled the large port service, that is, the port above 5000. This will also cause the error "the system lacks enough buffer space or the operation on the socket cannot be performed because the queue is full! The Web Service is configured to port 10 thousand or above! The procedure is as follows:

1. Start Registry Editor. HKEY_LOCAL_MACHINE/system/CurrentControlSet/services/TCPIP/parameters
2. New

Value Name: maxuserport
Value Type: DWORD
Value Data: 65534 (it can be written in decimal format. If it is in hexadecimal format, it is fffe, because 4 f is 65535)
Valid range: 5000-65534 (decimal)
Default Value: 0x1388 (decimal)

New

Value Name: tcptimedwaitdelay
Value Type: DWORD
Value Data: 0000001e (that is, 30. It is very convenient to use a Windows Calculator. This indicates the number of seconds for time_wait)

OK!

After some tossing, I finally calmed down this crying server in the sound of an ear-shaking roar. I also tame a few Trojans, and I had a good tutorial on my Resin server. By the way, there is an EMR server behind me. It sounds like a snoring, often scaring me ......

I hope that the summary I wrote over one hour will be useful to myself and help more friends. If you reprint the summary, please indicate the source.

(This blog post is from Sunday mushroom-thirty things, address http://giscool.blog.163.com/blog/static/56088547200810151151355)