Ubuntu 14.04下Hadoop集群安装(3)

①在master和slave节点上更改"core-site.xml"文件,master和slave节点应该使用相同"fs.defaultFS"值,而且必须指向master节点;在“configuration”中间添加如下配置:

<property>
  <name>hadoop.tmp.dir</name>
  <value>/home/hduser/tmp</value>
  <description>Temporary Directory.</description>
</property>
 
<property>
  <name>fs.defaultFS</name>
  <value>hdfs://master:54310</value>
  <description>Use HDFS as file storage engine</description>
</property>

最终core-site.xml配置文件如下图所示:

Ubuntu 14.04下Hadoop集群安装

如果tmp目录不存在,需要手动创建一个:

$ mkdir /home/hduser/tmp
$ chown -R hduser:hdgroup /home/hduser/tmp //非hduser用户创建虚赋权

②只在master节点上更改"mapred-site.xml"文件,由于没有这个文件,需要需要复制那个template文件生成一个:

$ cd /home/hduser/hadoop/
$ cp -av etc/hadoop/mapred-site.xml.template etc/hadoop/mapred-site.xml

编辑xml配置文件,在“configuration”中间添加如下配置:

<property>
 <name>mapreduce.jobtracker.address</name>
 <value>master:54311</value>
 <description>The host and port that the MapReduce job tracker runs
  at. If “local”, then jobs are run in-process as a single map
  and reduce task.
</description>
</property>
<property>
 <name>mapreduce.framework.name</name>
 <value>yarn</value>
 <description>The framework for running mapreduce jobs</description>
</property>

③在master和slave节点上更改"hdfs-site.xml"文件,在“configuration”中间添加如下配置:

<property>
 <name>dfs.replication</name>
 <value>2</value>
 <description>Default block replication.
  The actual number of replications can be specified when the file is created.
  The default is used if replication is not specified in create time.
 </description>
</property>
<property>
 <name>dfs.namenode.name.dir</name>
 <value>/data/hduser/hdfs/namenode</value>
 <description>Determines where on the local filesystem the DFS name node should store the name table(fsimage). If this is a comma-delimited list of directories then the name table is replicated in all of the directories, for redundancy.
 </description>
</property>
<property>
 <name>dfs.datanode.data.dir</name>
 <value>/data/hduser/hdfs/datanode</value>
 <description>Determines where on the local filesystem an DFS data node should store its blocks. If this is a comma-delimited list of directories, then data will be stored in all named directories, typically on different devices. Directories that do not exist are ignored.
 </description>

内容版权声明:除非注明,否则皆为本站原创文章。

转载注明出处:https://www.heiqu.com/66394ea0c449127a5f04c2f44caac939.html