Hadoop2.5.0伪分布式环境搭建

本章主要介绍下在Linux系统下的Hadoop2.5.0伪分布式环境搭建步骤。首先要搭建Hadoop伪分布式环境,需要完成一些前置依赖工作,包括创建用户、安装JDK、关闭防火墙等。

一、创建hadoop用户

使用root账户创建hadoop用户,为了在实验环境下便于操作,赋予hadoop用户sudo权限。具体操作代码如下:

useradd hadoop # 添加hadoop用户
passwd hadoop # 设置密码
visudo
hadoop ALL=(root)NOPASSWD:ALL

二、Hadoop伪分布式环境搭建

1、关闭Linux中的防火墙和selinux

禁用selinux,代码如下:

sudo vi /etc/sysconfig/selinux # 打开selinux配置文件
SELINUX=disabled # 修改SELINUX属性值为disabled

关闭防火墙,代码如下:

sudo service iptables status # 查看防火墙状态
sudo service iptables stop # 关闭防火墙
sudo chkconfig iptables off # 关闭防火墙开机启动设置

2、安装jdk

首先,查看系统中是否有安装自带的jdk,如果存在,则先卸载,代码如下:

rpm -qa | grep java # 查看是否有安装jdk
sudo rpm -e --nodeps java-1.6.0-openjdk-1.6.0.0-1.50.1.11.5.el6_3.x86_64 tzdata-java-2012j-1.el6.noarch java-1.7.0-openjdk-1.7.0.9-2.3.4.1.el6_3.x86_64 # 卸载自带jdk

接着,安装jdk,步骤如下:

step1.解压安装包:

tar -zxf jdk-7u67-linux-x64.tar.gz -C /usr/local/

step2.配置环境变量及检查是否安装成功:

sudo vi /etc/profile # 打开profile文件
##JAVA_HOME
export JAVA_HOME=/usr/local/jdk1.7.0_67
export PATH=$PATH:$JAVA_HOME/bin

# 生效文件
source /etc/profile # 使用root用户操作

# 查看是否配置成功
java -version

3、安装hadoop

step1:解压hadoop安装包

tar -zxvf /opt/software/hadoop-2.5.0.tar.gz -C /opt/software/

建议:将/opt/software/hadoop-2.5.0/share下的doc目录删除。

step2:修改etc/hadoop目录下hadoop-env.sh、mapred-env.sh、yarn-env.sh三个配置文件中的JAVA_HOME

export JAVA_HOME=/usr/local/jdk1.7.0_67

step3:修改core-site.xml

<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet type="text/xsl" href="https://www.linuxidc.com/configuration.xsl"?>
<!--
  Licensed under the Apache License, Version 2.0 (the "License");
  you may not use this file except in compliance with the License.
  You may obtain a copy of the License at

Unless required by applicable law or agreed to in writing, software
  distributed under the License is distributed on an "AS IS" BASIS,
  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  See the License for the specific language governing permissions and
  limitations under the License. See accompanying LICENSE file.
-->

<!-- Put site-specific property overrides in this file. -->

<configuration>
    <property>
        <name>name</name>
        <value>my-study-cluster</value>
    </property>
    <property>
        <name>fs.defaultFS</name>
        <value>hdfs://bigdata01:8020</value>
    </property>
        <!-- 指定Hadoop系统生成文件的临时目录地址 -->
    <property>
        <name>hadoop.tmp.dir</name>
        <value>/opt/software/hadoop-2.5.0/data/tmp</value>
    </property>
    <property>
        <name>fs.trash.interval</name>
        <value>1440</value>
    </property>
    <property>
        <name>hadoop.http.staticuser.user</name>
        <value>hadoop</value>
    </property>
        <property>
                <name>hadoop.proxyuser.hadoop.hosts</name>
                <value>bigdata01</value>
        </property>
        <property>
                <name>hadoop.proxyuser.hadoop.groups</name>
                <value>*</value>
        </property>
</configuration>

step4:修改hdfs-site.xml

<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet type="text/xsl" href="https://www.linuxidc.com/configuration.xsl"?>
<!--
  Licensed under the Apache License, Version 2.0 (the "License");
  you may not use this file except in compliance with the License.
  You may obtain a copy of the License at

内容版权声明:除非注明,否则皆为本站原创文章。

转载注明出处:https://www.heiqu.com/36e8ab6c6a2434eaa2ba60283c8d3225.html