Linux CentOS下Hadoop伪分布模式安装笔记(2)

2.2.1 进入 /usr/local/hadoop/hadoop-0.20.2/bin目录下,格式化namenode

[root@master bin]# hadoop namenode -format
10/07/19 10:46:41 INFO namenode.NameNode: STARTUP_MSG:
/************************************************************
STARTUP_MSG: Starting NameNode
STARTUP_MSG:   host = master/202.173.253.36
STARTUP_MSG:   args = [-format]
STARTUP_MSG:   version = 0.20.2
STARTUP_MSG:   build = https://svn.apache.org/repos/asf/hadoop/common/branches/branch-0.20 -r 911707; compiled by 'chrisdo' on Fri Feb 19 08:07:34 UTC 2010
************************************************************/
Re-format filesystem in /usr/local/hadoop/hdfs/name ? (Y or N) Y
10/07/19 10:46:43 INFO namenode.FSNamesystem: fsOwner=root,root,bin,daemon,sys,adm,disk,wheel
10/07/19 10:46:43 INFO namenode.FSNamesystem: supergroup=supergroup
10/07/19 10:46:43 INFO namenode.FSNamesystem: isPermissionEnabled=true
10/07/19 10:46:43 INFO common.Storage: Image file of size 94 saved in 0 seconds.
10/07/19 10:46:43 INFO common.Storage: Storage directory /usr/local/hadoop/hdfs/name has been successfully formatted.
10/07/19 10:46:43 INFO namenode.NameNode: SHUTDOWN_MSG:
/************************************************************
SHUTDOWN_MSG: Shutting down NameNode at master/202.173.253.36
************************************************************/ 

 2.2.2 启动hadoop所有进程

在/usr/local/hadoop/hadoop-0.20.2/bin 目录下,执行start-all.sh命令

启动完成后,可用jps命令查看hadoop进程是否启动完全。正常情况下应该有如下进程:

10910 NameNode
11431 Jps
11176 SecondaryNameNode
11053 DataNode
11254 JobTracker
11378 TaskTracker  

       我在搭建过程中,在此环节出现的问题最多,经常出现启动进程不完整的情况,要不是datanode无法正常启动,就是namenode或是TaskTracker启动异常。解决的方式如下:

1.在Linux下关闭防火墙:使用service iptables stop命令;

2.再次对namenode进行格式化:在/usr/local/hadoop/hadoop-0.20.2/bin 目录下执行hadoop namenode -format命令

3.对服务器进行重启

4.查看datanode或是namenode对应的日志文件,日志文件保存在/usr/local/hadoop/hadoop-0.20.2/logs目录下。仔细查看日志报错的原因,(上次日志报错的信息忘记了)解决方法是进入/usr/local/hadoop/hdfs/name 和usr/local/hadoop/hdfs/data目录下,将目录下的文件全部删除。

5.再次在/bin目录下用start-all.sh命令启动所有进程,通过以上的几个方法应该能解决进程启动不完全的问题了。

 

2.2.3  查看集群状态

在 bin目录下执行:hadoop dfsadmin  -report

[root@master bin]# hadoop dfsadmin -report
Configured Capacity: 304427253760 (283.52 GB)
Present Capacity: 282767941632 (263.35 GB)
DFS Remaining: 282767904768 (263.35 GB)
DFS Used: 36864 (36 KB)
DFS Used%: 0%
Under replicated blocks: 0
Blocks with corrupt replicas: 0
Missing blocks: 0

-------------------------------------------------
Datanodes available: 1 (1 total, 0 dead)

Name: 202.173.253.36:50010
Decommission Status : Normal
Configured Capacity: 304427253760 (283.52 GB)
DFS Used: 36864 (36 KB)
Non DFS Used: 21659312128 (20.17 GB)
DFS Remaining: 282767904768(263.35 GB)
DFS Used%: 0%
DFS Remaining%: 92.89%
Last contact: Mon Jul 19 11:07:22 CST 2010

 

2.3 在WEB页面下查看Hadoop工作情况

打开IE浏览器输入部署Hadoop服务器的IP:

 

:50070;

Linux <a href=CentOS下Hadoop伪分布模式安装笔记" src="https://www.linuxidc.com/upload/2011_04/110402060369001.jpg" width=600 border=0>

 

:50030。

Linux <a href=CentOS下Hadoop伪分布模式安装笔记" src="/uploads/allimg/200607/0J15S2c_0.jpg" border=0>

内容版权声明:除非注明,否则皆为本站原创文章。

转载注明出处:https://www.heiqu.com/wyzyss.html