java線程阻塞問題排查方法，java線程阻塞排查

java線程阻塞問題排查方法，java線程阻塞排查

我開發的worker，每隔幾個月線上都會阻塞一次，一直都沒查出問題。今天終於了了這個心結。把解決過程總結下和大家分享。

首先用jstack命令打出這個進程的全部線程堆棧。拿到線程dump檔案之後，搜尋自己的worker名字。

 

"DefaultQuartzScheduler_Worker-10" prio=10 tid=0x00007f55cd54d800 nid=0x3e2e waiting for monitor entry [0x00007f51ab8f7000]   java.lang.Thread.State: BLOCKED (on object monitor)at com.jd.chat.worker.service.impl.NewPopAccountSyncServiceImpl.addAccounts(NewPopAccountSyncServiceImpl.java:86)- waiting to lock <0x0000000782359268> (a com.jd.chat.worker.service.impl.NewPopAccountSyncServiceImpl)at com.jd.chat.worker.service.timer.AccountIncSyncTimer.run(AccountIncSyncTimer.java:114)at com.jd.chat.worker.service.timer.AbstractTimer.start(AbstractTimer.java:44)at com.jd.chat.worker.service.timer.AbstractTimer.doJob(AbstractTimer.java:49)at com.jd.chat.worker.web.context.StartAppListener$TimerJob.execute(StartAppListener.java:188)at org.quartz.core.JobRunShell.run(JobRunShell.java:202)at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:573)- locked <0x0000000783641c68> (a java.lang.Object)

很快便找到了線程在哪一行被阻塞。但是酒瓶這麼點資訊，並不能查出問題的真正原因，這裡推薦一個工具，叫tda.bat。同事給我的，網上應該有下載。把這個dump檔案匯入到tda中。找到阻塞的線程。阻塞的線程是紅色的。

 

之所以說這個軟體好，是因為當你找到blocked的線程後，介面的下方，會打出阻塞的更詳細的線程堆棧。截取這個堆棧的部分資訊。

 

at org.mariadb.jdbc.MySQLPreparedStatement.execute(MySQLPreparedStatement.java:141)at org.apache.commons.dbcp.DelegatingPreparedStatement.execute(DelegatingPreparedStatement.java:172)at org.apache.commons.dbcp.DelegatingPreparedStatement.execute(DelegatingPreparedStatement.java:172)at com.ibatis.sqlmap.engine.execution.SqlExecutor.executeUpdate(SqlExecutor.java:80)at com.ibatis.sqlmap.engine.mapping.statement.MappedStatement.sqlExecuteUpdate(MappedStatement.java:216)at com.ibatis.sqlmap.engine.mapping.statement.MappedStatement.executeUpdate(MappedStatement.java:94)at com.ibatis.sqlmap.engine.impl.SqlMapExecutorDelegate.update(SqlMapExecutorDelegate.java:457)at com.ibatis.sqlmap.engine.impl.SqlMapSessionImpl.update(SqlMapSessionImpl.java:90)at org.springframework.orm.ibatis.SqlMapClientTemplate$9.doInSqlMapClient(SqlMapClientTemplate.java:380)at org.springframework.orm.ibatis.SqlMapClientTemplate$9.doInSqlMapClient(SqlMapClientTemplate.java:1)at org.springframework.orm.ibatis.SqlMapClientTemplate.execute(SqlMapClientTemplate.java:200)at org.springframework.orm.ibatis.SqlMapClientTemplate.update(SqlMapClientTemplate.java:378)at com.jd.im.data.dataresource.ImSqlMapClientTemplate.retriedWithoutAnyInterventionUpdate(ImSqlMapClientTemplate.java:169)at com.jd.im.data.dataresource.ImSqlMapClientTemplate.update(ImSqlMapClientTemplate.java:137)at com.jd.chat.dao.impl.WriteDaoImpl.update(WriteDaoImpl.java:21)at com.jd.chat.zone.service.impl.GroupServiceImpl.updateRoute(GroupServiceImpl.java:766)at com.jd.chat.worker.service.impl.NewPopAccountSyncServiceImpl.addAccounts(NewPopAccountSyncServiceImpl.java:267)- locked <0x0000000782359268> (a com.jd.chat.worker.service.impl.NewPopAccountSyncServiceImpl)

這個才是真正有用的堆棧！它告訴了我程式是在執行SQL的時候，SQL發生死結，於是線程被阻塞。它還提供了更有用的資訊，那就是到底是哪個SQL導致的死結。堆棧的倒數第三行指示了導致死結的SQL。

但是一定要用這個工具才能找到具體的原因嗎？答案當然是NO！

告訴大家怎麼不通過工具找到阻塞的真正原因！

剛剛通過“BLOCKED”關鍵字搜到了線程堆棧，找到它的線程名“DefaultQuartzScheduler_Worker-10”。OK，然後，把最後的10改成1，也就是“DefaultQuartzScheduler_Worker-1”，然後再拿這個關鍵字搜尋整個進程堆棧。

 

"DefaultQuartzScheduler_Worker-1" prio=10 tid=0x00007f55cd2aa000 nid=0x3e25 runnable [0x00007f51b02c0000]   java.lang.Thread.State: RUNNABLEat java.net.SocketInputStream.socketRead0(Native Method)at java.net.SocketInputStream.read(SocketInputStream.java:129)at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)at java.io.BufferedInputStream.read1(BufferedInputStream.java:258)at java.io.BufferedInputStream.read(BufferedInputStream.java:317)- locked <0x0000000791370d50> (a java.io.BufferedInputStream)at org.mariadb.jdbc.internal.common.packet.buffer.ReadUtil.readFully(ReadUtil.java:82)at org.mariadb.jdbc.internal.common.packet.buffer.ReadUtil.readFully(ReadUtil.java:92)at org.mariadb.jdbc.internal.common.packet.RawPacket.nextPacket(RawPacket.java:77)at org.mariadb.jdbc.internal.common.packet.SyncPacketFetcher.getRawPacket(SyncPacketFetcher.java:67)at org.mariadb.jdbc.internal.mysql.MySQLProtocol.getResult(MySQLProtocol.java:891)at org.mariadb.jdbc.internal.mysql.MySQLProtocol.executeQuery(MySQLProtocol.java:982)at org.mariadb.jdbc.MySQLStatement.execute(MySQLStatement.java:280)- locked <0x0000000791370678> (a org.mariadb.jdbc.internal.mysql.MySQLProtocol)at org.mariadb.jdbc.MySQLPreparedStatement.execute(MySQLPreparedStatement.java:141)

貼出這個進程堆棧的一部分。這個進程堆棧其實也就是剛剛tda軟體介面下方展示的導致線程阻塞的真正的堆棧！這個線程是runnable狀態的，可惜mysql是鎖死的。也就是說阻塞在了mysql裡。