Hadoop2.6版本集群环境搭建(2)

<configuration>
 <property>
  <name>dfs.namenode.secondary.http-address</name>
  <value>S1PA11:9001</value>
 </property>

  <property>
   <name>dfs.namenode.name.dir</name>
   <value>file:/home/spark/opt/hadoop-2.6.0/dfs/name</value>
 </property>

 <property>
  <name>dfs.datanode.data.dir</name>
  <value>file:/home/spark/opt/hadoop-2.6.0/dfs/data</value>
  </property>

 <property>
  <name>dfs.replication</name>
  <value>3</value>
 </property>

 <property>
  <name>dfs.webhdfs.enabled</name>
  <value>true</value>
 </property>

</configuration>

4.6、配置  mapred-site.xml 文件-->>增加mapreduce配置（使用yarn框架、jobhistory使用地址以及web地址）

<configuration>
  <property>
   <name>mapreduce.framework.name</name>
   <value>yarn</value>
 </property>
 <property>
  <name>mapreduce.jobhistory.address</name>
  <value>S1PA11:10020</value>
 </property>
 <property>
  <name>mapreduce.jobhistory.webapp.address</name>
  <value>S1PA11:19888</value>
 </property>
</configuration>

4.7、配置   yarn-site.xml  文件-->>增加yarn功能

<configuration>
  <property>
   <name>yarn.nodemanager.aux-services</name>
   <value>mapreduce_shuffle</value>
  </property>
  <property>
   <name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>
   <value>org.apache.hadoop.mapred.ShuffleHandler</value>
  </property>
  <property>
   <name>yarn.resourcemanager.address</name>
   <value>S1PA11:8032</value>
  </property>
  <property>
   <name>yarn.resourcemanager.scheduler.address</name>
   <value>S1PA11:8030</value>
  </property>
  <property>
   <name>yarn.resourcemanager.resource-tracker.address</name>
   <value>S1PA11:8035</value>
  </property>
  <property>
   <name>yarn.resourcemanager.admin.address</name>
   <value>S1PA11:8033</value>
  </property>
  <property>
   <name>yarn.resourcemanager.webapp.address</name>
   <value>S1PA11:8088</value>
  </property>

</configuration>

5、将配置好的hadoop文件copy到另一台slave机器上

[spark@S1PA11 opt]$ scp -r hadoop-2.6.0/ spark@10.126.34.43:~/opt/

四、验证

1、格式化namenode:

[spark@S1PA11 opt]$ cd hadoop-2.6.0/
[spark@S1PA11 hadoop-2.6.0]$ ls
bin  dfs  etc  include  input  lib  libexec  LICENSE.txt  logs  NOTICE.txt  README.txt  sbin  share  tmp
[spark@S1PA11 hadoop-2.6.0]$ ./bin/hdfs namenode -format

[spark@S1PA222 .ssh]$ cd ~/opt/hadoop-2.6.0
[spark@S1PA222 hadoop-2.6.0]$ ./bin/hdfs  namenode -format

2、启动hdfs:

[spark@S1PA11 hadoop-2.6.0]$ ./sbin/start-dfs.sh 
15/01/05 16:41:04 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Starting namenodes on [S1PA11]
S1PA11: starting namenode, logging to /home/spark/opt/hadoop-2.6.0/logs/hadoop-spark-namenode-S1PA11.out
S1PA222: starting datanode, logging to /home/spark/opt/hadoop-2.6.0/logs/hadoop-spark-datanode-S1PA222.out
Starting secondary namenodes [S1PA11]
S1PA11: starting secondarynamenode, logging to /home/spark/opt/hadoop-2.6.0/logs/hadoop-spark-secondarynamenode-S1PA11.out
15/01/05 16:41:21 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
[spark@S1PA11 hadoop-2.6.0]$ jps
22230 Master
30889 Jps
22478 Worker
30498 NameNode
30733 SecondaryNameNode
19781 ResourceManager

3、停止hdfs: