<property>
<name>dfs.ha.automatic-failover.enabled.cluster1</name>
<value>true</value>
</property>
【指定cluster1是否启动自动故障恢复，即当NameNode出故障时，是否自动切换到另一台NameNode】
<property>
<name>dfs.client.failover.proxy.provider.cluster1</name>
<value>org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider</value>
</property>
【指定cluster1出故障时，哪个实现类负责执行故障切换】
<property>
<name>dfs.journalnode.edits.dir</name>
<value>/data0/hadoop2/hdfs/journal</value>
</property>
【指定JournalNode集群在对NameNode的目录进行共享时，自己存储数据的磁盘路径】
<property>
<name>dfs.ha.fencing.methods</name>
<value>sshfence</value>
</property>
【一旦需要NameNode切换，使用ssh方式进行操作】
<property>
<name>dfs.ha.fencing.ssh.private-key-files</name>
<value>/root/.ssh/id_rsa</value>
</property>
【如果使用ssh进行故障切换，使用ssh通信时用的密钥存储的位置】
</configuration>
1.2.4. slaves
hadoop1
hadoop2
hadoop2
1.3. 把以上配置的内容复制到hadoop1、hadoop2、hadoop3节点上
1.4. 修改hadoop1、hadoop2、hadoop3上的配置文件内容
1.4.1. 修改hadoop2上的core-site.xml内容
fs.defaultFS的值改为hdfs://cluster2
1.4.1. 修改hadoop2上的hdfs-site.xml内容
把cluster1中关于journalnode的配置项删除，增加如下内容
<property>
<name>dfs.namenode.shared.edits.dir</name>
<value>qjournal://hadoop0:8485;hadoop1:8485;hadoop2:8485/cluster2</value>
</property>
1.4.3. 开始启动
1.4.3.1. 启动journalnode
在hadoop0、hadoop1、hadoop2上执行
sbin/hadoop-daemon.sh startjournalnode
1.4.3.1. 格式化ZooKeeper
在hadoop0、hadoop2上执行
bin/hdfs zkfc -formatZK
zkCli.sh-->ls->/Hadoop-ha/cluster1
1.4.3.3. 对hadoop0节点进行格式化和启动
bin/hdfs namenode -format
sbin/hadoop-daemon.sh start namenode
1.4.3.4. 对hadoop1节点进行格式化和启动
bin/hdfs namenode -bootstrapStandby
sbin/hadoop-daemon.sh start namenode
1.4.3.5. 在hadoop0、hadoop1上启动zkfc
1 sbin/hadoop-daemon.sh start zkfc
我们的hadoop0、hadoop1有一个节点就会变为active状态。
1.4.3.6. 对于cluster2执行类似操作
1.4.4. 启动datanode
在hadoop0上执行命令
sbin/hadoop-daemons.sh start datanode
1.5. 配置Yarn
1.5.1. 修改文件mapred-site.xml
<property>
<name>mapreduce.framework.name</name>
<value>yarn</value>
</property>
<property>
<property>
<name>mapreduce.jobhistory.address</name>
<value>hadoop0:10020</value>
</property>
<property>
<name>mapreduce.jobhistory.webapp.address</name>
<value>hadoop0:19888</value>
</property>
<property>
<name>mapreduce.task.io.sort.factor</name>
<value>20</value>
<description>The number of streams to merge at once while sorting
files. This determines the numberof open file handles.</description>
</property>
<property>
<name>mapreduce.reduce.shuffle.parallelcopies</name>
<value>40</value>
<description>The default number of parallel transfers run byreduce
during the copy(shuffle) phase.
</description>
</property>
<property>
<name>mapreduce.job.reduce.slowstart.completedmaps</name>
<value>0.80</value>
<description>Fraction of the number of maps in the job whichshould be
complete before reduces are scheduled for the job.
</description>
</property>
<property>
<name>mapreduce.task.io.sort.mb</name>
<value>300</value>
<description>The total amount of buffer memory to use whilesorting
files, in megabytes. By default,gives each merge stream 1MB, which
should minimize seeks.</description>
</property>
<property>
<name>mapreduce.map.output.compress</name>
<value>true</value>
<description>Should the outputs of the maps be compressed beforebeing
sent across the network. UsesSequenceFile compression.
</description>
</property>
<property>
<name>mapreduce.client.submit.file.replication</name>
<value>5</value>
<description>默认10，The replication level for submitted job files. This
should be around thesquare root of the number of nodes.
</description>
</property>