Recently, another strange thing was that a very simple program called the web to read a batch of data from the database and showed that the program had no problems at the beginning, after a while, it suddenly died, and all the queries did not respond.
Exploration + torture:
The method used at the beginning is really very primitive. Because no exception is reported when the system gets stuck, it is likely that no timeout is set. Check and adjust the timeout settings for web connections and database connections, but the problem persists.
It seems that the database cannot get the link. This may be because the database connection pool is exhausted and the connection should not be released. Check the current number of connections in mysql, which is normal and does not feel like a connection problem. When the program gets stuck, other programs connected to the database are normal, and the restart of the program is normal. Therefore, it should not be the problem of connection depletion. During this period, the current thread of the program is printed out, and the number of threads is not abnormally expanded.
The log output is added before and after every statement executed by the program. It is surprising that the program stops at the place where the service calls dao. No logs are output in the first line of the dao method. Can an error still occur during program calls ?! I have not figured out this problem yet. It is estimated that it is a ghost of spring. So we simply remove the dao layer and copy the code to the service layer. The result code is stuck in the database query sentence.
Check that CPU and memory consumption is low, but the program stops for no reason. This phenomenon is the same as the deadlock. However, the lock is useless in the program, so it can be ruled out.
I thought that I was using dbcp. I found that there were a lot of people with bugs on the Internet. I was wondering if I wanted to upgrade dbcp or change it to c3p0. In desperation, the database configuration is still wrong. Instead, the data source with the problem is not replaced with c3p0, leading to the problem still not solved.
Wake up:
There are too many things done by SB, so it's silly to count the number of threads. Why don't you directly check what program stops ?! When the program gets stuck again, kill-3 [pid] to print out the memory status. The result will be displayed on the getConnection method of dbcp. Apparently, I encountered the legendary dbcp bug. During this change, I found that the last configuration file was wrong. It was a tragedy. Upgrade dbcp to the latest version. After several days of running, the problem has not been reproduced.
The kill command can only be used in linux. In fact, you can also use the built-in jdk tool, such as using jstack to view the thread running status, which can also be used in windows.
Summary:
Java is a very open program, and most of the tools provided are easy to locate in case of problems. In the middle, you need to know what tools are available to assist you. At least you need to check the built-in tools under jdk/bin; in addition, we need to have the awareness of using tools to solve the problem. If we still know that the output statements are printed everywhere when we first learned to write code, the level of solving the problem is too low, it is also difficult to solve the problem.