Case
Runbroker.sh java_opt uses the default parameters, the broker runs the process of less generation garbage collection, frequent Laosheng generation garbage collection CMS GC, and Laosheng generation of memory is not lost. Causes stuttering, message delivery failed. Eventually, the Java heap is exhausted and the broker goes down. For advice
Memory Analysis:
After the broker is paralyzed, use JMAP to generate the dump file, analyze it using mat, and discover that the Haservice instance retained a lot of memory.
The server mode is 3m and no slave is used, but the service from the machine brush disk consumes a lot of memory and cannot be recycled. The problem is obvious.
Workaround One: Start slave
Solution Two: Refactor code Rocketmq_store.jar, remove slave correlation.
After further investigation, Haservice found that there are a large number of store.log in the corresponding log file.
Haservice receive new connection,/172.16.50.105:xxxxx
And start a lot of
Writesocketservice Service started
And is not closed, there should be writesocketservice service end, but only a very small portion of the log.
Consistent with problem Suspect 1!
Solution: 172.16.50.105 has been identified as a monitor, frequent connections cause the above problems. Stop monitoring the broker server.
Problem: Virtual machine Laosheng generation garbage collection occurs frequently