The Hadoop process takes up 800% of the CPU

I. Phenomenon

Hadoop process CPU accounted for 800%

Second, check the problem

1, Top

Z Highlight

H Display Thread

<> page

2, Jstatack

Sudo-u Admin Jstack 97932 | Pager

2014-03-20 21:45:45

Full thread dump OpenJDK (Taobao) 64-bit Server VM (20.0-b12-internal mixed mode):

"Attach Listener" daemon prio=10 tid=0x00007f6640a15800 nid=0xf350 waiting on condition [0x0000000000000000]

Java.lang.Thread.State:RUNNABLE

"Thread-218" prio=10 tid=0x00007f663c909800 nid=0x1813b runnable [0x0000000042ffd000]

Java.lang.Thread.State:RUNNABLE

At Java.util.HashMap.getEntry (hashmap.java:347)

At Java.util.HashMap.containsKey (hashmap.java:335)

At Java.util.HashSet.contains (hashset.java:184)

At Org.apache.hadoop.hive.ql.QueryPlan.updateCountersInQueryPlan (queryplan.java:306)

At Org.apache.hadoop.hive.ql.QueryPlan.getQueryPlan (queryplan.java:421)

At Org.apache.hadoop.hive.ql.QueryPlan.toString (queryplan.java:593)

At Org.apache.hadoop.hive.ql.history.HiveHistory.logPlanProgress (hivehistory.java:510)

At Org.apache.hadoop.hive.ql.exec.Task.executeTask (task.java:168)

At Org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential (taskrunner.java:57)

At Org.apache.hadoop.hive.ql.exec.TaskRunner.run (taskrunner.java:47)

"Thread-217" prio=10 tid=0x00007f663d75b800 nid=0x1813a runnable [0x0000000043b08000]

Java.lang.Thread.State:RUNNABLE

At Java.util.HashMap.getEntry (hashmap.java:347)

At Java.util.HashMap.containsKey (hashmap.java:335)

At Java.util.HashSet.contains (hashset.java:184)

At Org.apache.hadoop.hive.ql.QueryPlan.updateCountersInQueryPlan (queryplan.java:306)

At Org.apache.hadoop.hive.ql.QueryPlan.getQueryPlan (queryplan.java:421)

At Org.apache.hadoop.hive.ql.QueryPlan.toString (queryplan.java:593)

At Org.apache.hadoop.hive.ql.history.HiveHistory.logPlanProgress (hivehistory.java:510)

At Org.apache.hadoop.hive.ql.exec.Task.executeTask (task.java:168)

At Org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential (taskrunner.java:57)

At Org.apache.hadoop.hive.ql.exec.TaskRunner.run (taskrunner.java:47)

"Thread-152" prio=10 tid=0x00007f663cc68000 nid=0x180f7 runnable [0x0000000040904000]

Java.lang.Thread.State:RUNNABLE

At Java.util.HashMap.getEntry (hashmap.java:347)

At Java.util.HashMap.containsKey (hashmap.java:335)

At Java.util.HashSet.contains (hashset.java:184)

At Org.apache.hadoop.hive.ql.QueryPlan.updateCountersInQueryPlan (queryplan.java:306)

At Org.apache.hadoop.hive.ql.QueryPlan.getQueryPlan (queryplan.java:421)

At Org.apache.hadoop.hive.ql.QueryPlan.toString (queryplan.java:593)

At Org.apache.hadoop.hive.ql.history.HiveHistory.logPlanProgress (hivehistory.java:510)

3. Process Name

PS aux | grep 97932

[www.linuxidc.com @linux ~]$ ps aux | grep 97932

56506 68038 0.0 0.0 65304 836 pts/0 s+ 21:55 0:00 grep 97932

Admin 97932 326696 0.6 767276 315552? Sl Jan14 21868789:28/home/yunti/java-current/bin/java-xmx256m-dhadoop.log.dir=/home/yunti/hadoop-current/bin/. /logs-dhadoop.log.file=hadoop.log-dhadoop.home.dir=/home/yunti/hadoop-current/bin/. -dhadoop.id.str=-dhadoop.root.logger=info,console-dhadoop.root.logger.appender=console- dhadoop.root.logger.level=info-djava.library.path=/home/yunti/hadoop-current/bin/. /lib/native/linux-amd64-64-classpath/home/yunti/hadoop-current/bin/. /conf:/home/yunti/java-current/lib/tools.jar:/home/yunti/hadoop-current/bin/..:/ home/yunti/hadoop-current/bin/. /hadoop-*-core.jar:/home/yunti/hadoop-current/bin/.

Iii. Conclusion

Hive the execution, the cycle is dead.