Hadoop动态添加删除节点datanode及恢复(2)

[hadoop@hadoop-master hadoop]$ hdfs dfsadmin -report
Configured Capacity: 58663657472 (54.63 GB)
Present Capacity: 15487176704 (14.42 GB)
DFS Remaining: 15486873600 (14.42 GB)
DFS Used: 303104 (296 KB)
DFS Used%: 0.00%
Under replicated blocks: 5
Blocks with corrupt replicas: 0
Missing blocks: 0
Missing blocks (with replication factor 1): 0

-------------------------------------------------
Live datanodes (2):

Name: 192.168.48.131:50010 (slave2)
Hostname: 183.221.250.11
Decommission Status : Normal
Configured Capacity: 38588669952 (35.94 GB)
DFS Used: 8192 (8 KB)
Non DFS Used: 36887191552 (34.35 GB)
DFS Remaining: 1701470208 (1.58 GB)
DFS Used%: 0.00%
DFS Remaining%: 4.41%
Configured Cache Capacity: 0 (0 B)
Cache Used: 0 (0 B)
Cache Remaining: 0 (0 B)
Cache Used%: 100.00%
Cache Remaining%: 0.00%
Xceivers: 1
Last contact: Thu Mar 01 19:36:33 PST 2018

Name: 192.168.48.132:50010 (slave1)
Hostname: slave1
Decommission Status : Normal
Configured Capacity: 20074987520 (18.70 GB)
DFS Used: 294912 (288 KB)
Non DFS Used: 6289289216 (5.86 GB)
DFS Remaining: 13785403392 (12.84 GB)
DFS Used%: 0.00%
DFS Remaining%: 68.67%
Configured Cache Capacity: 0 (0 B)
Cache Used: 0 (0 B)
Cache Remaining: 0 (0 B)
Cache Used%: 100.00%
Cache Remaining%: 0.00%
Xceivers: 1
Last contact: Thu Mar 01 19:36:35 PST 2018

[hadoop@hadoop-master hadoop]$

7动态删除datanode

7.1配置NameNode的hdfs-site.xml，

适当减小dfs.replication副本数，增加dfs.hosts.exclude配置

[hadoop@hadoop-master hadoop]$ pwd
/usr/hadoop/hadoop-2.7.5/etc/hadoop
[hadoop@hadoop-master hadoop]$ cat hdfs-site.xml
<configuration>
<property>
      <name>dfs.replication</name>
      <value>3</value>
</property>
  <property>
      <name>dfs.name.dir</name>
      <value>/usr/local/hadoop-2.7.5/tmp/dfs/name</value>
</property>
    <property>
      <name>dfs.data.dir</name>
      <value>/usr/local/hadoop-2.7.5/tmp/dfs/data</value>
    </property>
<property>
    <name>dfs.hosts.exclude</name>
    <value>/usr/hadoop/hadoop-2.7.5/etc/hadoop/excludes</value>
  </property>

</configuration>

7.2在namenode对应路径（/etc/hadoop/）下新建excludes文件，

并写入待删除DataNode的ip或域名

[hadoop@hadoop-master hadoop]$ pwd
/usr/hadoop/hadoop-2.7.5/etc/hadoop
[hadoop@hadoop-master hadoop]$ vi excludes
####slave2
192.168.48.131[hadoop@hadoop-master hadoop]$

7.3在NameNode上刷新所有DataNode

hdfs dfsadmin -refreshNodes
sbin/start-balancer.sh

7.4在namenode查看当前集群情况,

确认信节点已经正常删除,结果中已无slave2

[hadoop@hadoop-master hadoop]$ hdfs dfsadmin -report

或者可以在web检测界面（ip:50070）上可以观测到DataNode逐渐变为Dead。

:50070/

在datanode项,Admin state已经由“In Service“变为”Decommissioned“，则表示删除成功

7.5停止已删除的节点相关进程

[hadoop@slave2 hadoop-2.7.5]$ jps
9530 Jps
3897 DataNode
6772 NodeManager
[hadoop@slave2 hadoop-2.7.5]$ sbin/hadoop-daemon.sh stop datanode
stopping datanode
[hadoop@slave2 hadoop-2.7.5]$ sbin/yarn-daemon.sh stop nodemanager
stopping nodemanager
[hadoop@slave2 hadoop-2.7.5]$ jps
9657 Jps
[hadoop@slave2 hadoop-2.7.5]$

8恢复已删除节点

执行7.2 中删除相关信息，然后4，5，6即可。

Hadoop2.3-HA高可用集群环境搭建  https://www.linuxidc.com/Linux/2017-03/142155.htm
Hadoop项目之基于CentOS7的Cloudera 5.10.1（CDH）的安装部署  https://www.linuxidc.com/Linux/2017-04/143095.htm
Hadoop2.7.2集群搭建详解（高可用）  https://www.linuxidc.com/Linux/2017-03/142052.htm
使用Ambari来部署Hadoop集群（搭建内网HDP源）  https://www.linuxidc.com/Linux/2017-03/142136.htm
Ubuntu 14.04下Hadoop集群安装  https://www.linuxidc.com/Linux/2017-02/140783.htm
CentOS 6.7安装Hadoop 2.7.2  https://www.linuxidc.com/Linux/2017-08/146232.htm
Ubuntu 16.04上构建分布式Hadoop-2.7.3集群  https://www.linuxidc.com/Linux/2017-07/145503.htm
CentOS 7 下 Hadoop 2.6.4 分布式集群环境搭建  https://www.linuxidc.com/Linux/2017-06/144932.htm
Hadoop2.7.3+Spark2.1.0完全分布式集群搭建过程  https://www.linuxidc.com/Linux/2017-06/144926.htm