Hadoop进程占CPU 800%

 

 

首页服务器应用

背景:

阅读新闻

Hadoop进程占CPU 800%

[日期:2014-03-21]   来源:Linux社区  作者:fjqzju   [字体:]  

一、现象

Hadoop进程CPU占800%

二、查问题

1、TOP

Hadoop进程占CPU 800%

z      高亮

H      显示线程

<>    翻页

2、Jstatack

Hadoop进程占CPU 800%

sudo -u admin jstack 97932 | less

2014-03-20 21:45:45
Full thread dump OpenJDK (Taobao) 64-Bit Server VM (20.0-b12-internal mixed mode):


"Attach Listener" daemon prio=10 tid=0x00007f6640a15800 nid=0xf350 waiting on condition [0x0000000000000000]
  java.lang.Thread.State: RUNNABLE


"Thread-218" prio=10 tid=0x00007f663c909800 nid=0x1813b runnable [0x0000000042ffd000]
  java.lang.Thread.State: RUNNABLE
        at java.util.HashMap.getEntry(HashMap.java:347)
        at java.util.HashMap.containsKey(HashMap.java:335)
        at java.util.HashSet.contains(HashSet.java:184)
        at org.apache.hadoop.hive.ql.QueryPlan.updateCountersInQueryPlan(QueryPlan.java:306)
        at org.apache.hadoop.hive.ql.QueryPlan.getQueryPlan(QueryPlan.java:421)
        at org.apache.hadoop.hive.ql.QueryPlan.toString(QueryPlan.java:593)
        at org.apache.hadoop.hive.ql.history.HiveHistory.logPlanProgress(HiveHistory.java:510)
        at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:168)
        at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57)
        at org.apache.hadoop.hive.ql.exec.TaskRunner.run(TaskRunner.java:47)


"Thread-217" prio=10 tid=0x00007f663d75b800 nid=0x1813a runnable [0x0000000043b08000]
  java.lang.Thread.State: RUNNABLE
        at java.util.HashMap.getEntry(HashMap.java:347)
        at java.util.HashMap.containsKey(HashMap.java:335)
        at java.util.HashSet.contains(HashSet.java:184)
        at org.apache.hadoop.hive.ql.QueryPlan.updateCountersInQueryPlan(QueryPlan.java:306)
        at org.apache.hadoop.hive.ql.QueryPlan.getQueryPlan(QueryPlan.java:421)
        at org.apache.hadoop.hive.ql.QueryPlan.toString(QueryPlan.java:593)
        at org.apache.hadoop.hive.ql.history.HiveHistory.logPlanProgress(HiveHistory.java:510)
        at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:168)
        at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57)
        at org.apache.hadoop.hive.ql.exec.TaskRunner.run(TaskRunner.java:47)


"Thread-152" prio=10 tid=0x00007f663cc68000 nid=0x180f7 runnable [0x0000000040904000]
  java.lang.Thread.State: RUNNABLE
        at java.util.HashMap.getEntry(HashMap.java:347)
        at java.util.HashMap.containsKey(HashMap.java:335)
        at java.util.HashSet.contains(HashSet.java:184)
        at org.apache.hadoop.hive.ql.QueryPlan.updateCountersInQueryPlan(QueryPlan.java:306)
        at org.apache.hadoop.hive.ql.QueryPlan.getQueryPlan(QueryPlan.java:421)
        at org.apache.hadoop.hive.ql.QueryPlan.toString(QueryPlan.java:593)
        at org.apache.hadoop.hive.ql.history.HiveHistory.logPlanProgress(HiveHistory.java:510)

3、进程名
ps aux | grep 97932

[ @linux ~]$ ps aux | grep 97932 
56506    68038  0.0  0.0  65304  836 pts/0    S+  21:55  0:00 grep 97932
admin    97932 326696  0.6 767276 315552 ?    Sl  Jan14 21868789:28 /home/yunti/java-current/bin/java -Xmx256m -Dhadoop.log.dir=/home/yunti/hadoop-current/bin/../logs -Dhadoop.log.file=hadoop.log -Dhadoop.home.dir=/home/yunti/hadoop-current/bin/.. -Dhadoop.id.str= -Dhadoop.root.logger=INFO,console -Dhadoop.root.logger.appender=console -Dhadoop.root.logger.level=info -Djava.library.path=/home/yunti/hadoop-current/bin/../lib/native/Linux-amd64-64 -classpath /home/yunti/hadoop-current/bin/../conf:/home/yunti/java-current/lib/tools.jar:/home/yunti/hadoop-current/bin/..:/home/yunti/hadoop-current/bin/../hadoop-*-core.jar:/home/yunti/hadoop-current/bin/..

三、结论
hive执行的时候死循环了

相关阅读

Ubuntu 13.04上搭建Hadoop环境

Ubuntu 12.10 +Hadoop 1.2.1版本集群配置

Ubuntu上搭建Hadoop环境(单机模式+伪分布模式)

Ubuntu下Hadoop环境的配置

单机版搭建Hadoop环境图文教程详解

搭建Hadoop环境(在Winodws环境下用虚拟机虚拟两个Ubuntu系统进行搭建)

内容版权声明:除非注明,否则皆为本站原创文章。

转载注明出处:https://www.heiqu.com/bda6f4a6c27e2d972f9db595e96b8991.html