Hadoop 1.0.0的分布式配置(一个namenode 两个个data

我用了两台虚拟机:

10.232.35.43 namenode datanode

10.232.35.44 datanode

条件:

1.配置完SSH和RSA(网上攻略很多咯)namenode 和datanode之间通信 正常

2.JAVA配置,所有机器上安装JDK-7u2(最新版了的~)

3.在1.0.0中,首先将conf 目录下的Hadoop-env.sh中的JAVA_HOME路径指向你java的安装目录。

4.其次,在conf下有三个配置文件需要更新。分别是core-site.xml,hdfs-site.xml,mapred-site.xml,其实我觉得分开不分开都是一样的。大家不要按照网上的分别去配这三个文件,只要把所有的配置全放在一个文件,其他的全部拷贝!!

我的配置如下:

<configuration>
<property>
<name>fs.default.name</name><value>hdfs://localhost:9000</value><description>HDFS URI file system</description>
</property>
<property>
<name>hadoop.tmp.dir</name><value>/home/fangpei.pt/hadoop/hadooptmp</value><description>hadoop temp filefolder</description>
</property>
<property>
<name>dfs.name.dir</name><value>/home/fangpei.pt/hadoop/hdfs/name</value><description>HDFS name matadata</description>
</property>
<property>
<name>dfs.data.dir</name><value>/home/fangpei.pt/hadoop/hdfs/data</value><description>the physical data block position of datanode</description>
</property>
<property>
<name>dfs.replication</name><value>1</value><description>replication</description>
</property>
<property>
<name>mapred.job.tracker</name><value>hdfs://localhost:9001</value><description>jobtracker</description>
</property>
<property>
<name>mapred.local.dir</name><value>/home/fangpei.pt/hadoop/mapred/local</value><description>mapred local dir</description>
</property>
<property>
<name>mapred.sysytem.dir</name><value>/home/fangpei.pt/hadoop/mapred/system</value><description>hdfs dir</description>
</property>
</configuration>


将此配置复制到三个文件中。

5.slave和master都这样配置即可。

6.然后根据需要,配置conf下的slave 和master文件,我的是master下是10.232.35.43 slave下是10.232.35.43 10.232.35.44    (最好不要用localhost,那样不好复制...),将配置好的这两个文件再次全部复制,拷贝到其他机器上。

7.启动你配置为master的机器。./start-all.sh,这就不用说了吧。启动成功,输出:

starting namenode, logging to /home/fangpei.pt/hadoop-1.0.0/libexec/../logs/hadoop-fangpei.pt-namenode-v035043.sqa.cm4.out
10.232.35.44: starting datanode, logging to /home/fangpei.pt/hadoop-1.0.0/libexec/../logs/hadoop-fangpei.pt-datanode-v035044.sqa.cm4.out
localhost: starting datanode, logging to /home/fangpei.pt/hadoop-1.0.0/libexec/../logs/hadoop-fangpei.pt-datanode-v035043.sqa.cm4.out
localhost: starting secondarynamenode, logging to /home/fangpei.pt/hadoop-1.0.0/libexec/../logs/hadoop-fangpei.pt-secondarynamenode-v035043.sqa.cm4.out
starting jobtracker, logging to /home/fangpei.pt/hadoop-1.0.0/libexec/../logs/hadoop-fangpei.pt-jobtracker-v035043.sqa.cm4.out
10.232.35.44: starting tasktracker, logging to /home/fangpei.pt/hadoop-1.0.0/libexec/../logs/hadoop-fangpei.pt-tasktracker-v035044.sqa.cm4.out
localhost: starting tasktracker, logging to /home/fangpei.pt/hadoop-1.0.0/libexec/../logs/hadoop-fangpei.pt-tasktracker-v035043.sqa.cm4.out


8.如有疑问或想法,欢迎交流~~

注意事项:

1.所有上面配置中提到的目录,要至少自己创建到二级目录。

内容版权声明:除非注明,否则皆为本站原创文章。

转载注明出处:http://www.heiqu.com/706bdb42ea68ed042b2e4bf7d701d745.html