Ganglia running processes是怎么算出来的?ganglia是通过 cat /proc/loadavg获得running processes的。
可得到如下值:0.00 0.28 0.61 1/591 2993。其中,1是running process,591是total process。
为了追踪ganglia图上突然出现的14个running processes,调查了一下,ps Haxh查询出来的total processes和/proc/loadavg的total processes是一致的,状态为R的即是running process。于是又写了个脚本,在启动namenode的期间,每隔0.5秒打印出/proc/loadavg 和 ps Haxh的数据,查看何时出现14个running processes的。
最后发现,不管有没有这个patch,都有可能出现running processes从1上升至6,甚至14的情况。这些processes是namenode的子线程,在某些情况下状态为Rl,R是运行态,l是多线程,在/proc/loadavg中被计算成了running processes。这些子线程运行的时间很短,ganglia是每分钟获得一次数据,很有可能没有采集到。
因此,我之前测试时ganglia图示上的差别,只是巧合,导致我认为加上patch之后有问题。我通过几次实验,看到没有patch时,也会出现ganglia图中running processes上升的情况。
PROCESS STATE CODES
Here are the different values that the s, stat and state output specifiers (header "STAT" or "S") will display to describe the state of a process.
D Uninterruptible sleep (usually IO)
R Running or runnable (on run queue)
S Interruptible sleep (waiting for an event to complete)
T Stopped, either by a job control signal or because it is being traced.
W paging (not valid since the 2.6.xx kernel)
X dead (should never be seen)
Z Defunct ("zombie") process, terminated but not reaped by its parent.
For BSD formats and when the stat keyword is used, additional characters may be displayed:
< high-priority (not nice to other users)
N low-priority (nice to other users)
L has pages locked into memory (for real-time and custom IO)
s is a session leader
l is multi-threaded (using CLONE_THREAD, like NPTL pthreads do)
+ is in the foreground process group
更多Hadoop相关信息见Hadoop 专题页面 ?tid=13