Remember once Java application CPU utilization too high debugging experience

1, phenomenon

Write a storm application that synchronizes the table to hbase primarily via MySQL's binlog. After running for a period of time found that the CPU usage often soared to more than 200%, and then a variety of message accumulation alarm and so on, there are various problems

2, research process

CPU usage is high, so first find the process, through the top command, monitor the utilization of the process, and then through H, see the CPU utilization of each thread, note the high CPU utilization of the thread ID, and then through the Jstack PID, get the individual line stacks, After listening to the thread ID of top gets converted to 16, the thread stack that found the problem is as follows:

The main problem was found on the HashMap put. It is strange that only one spout should be single-threaded, but later found that the client side is multithreaded from NSQ to fetch data. HashMap is not thread-safe, so in multi-threading, a dead loop can cause high CPU usage. Specific reasons can be see: http://coolshell.cn/articles/9606.html

3, processing method:

Find the problem, through code discovery conversion JSON to map after not specified Concurrenthashmap, and then put the data caused, directly by replacing into Concurrenthashmap just fine.

4, summary

HashMap is not thread-safe and must not be used in multithreaded processes. In addition, encountered in Java application CPU usage is high, should be able to use the above solutions to locate.

Remember once Java application CPU utilization too high debugging experience