Hadoop集群部署安装

学习Hadoop,必须要亲自部署一下,找找感觉。我使用的版本是0.20.2,下面就详细说一下部署过程。

一、规划

使用VMWare Workstation创建3台虚拟机。定义如下表:

   

Hostname

 

IP

 

OS

 

Master

 

n1

 

192.168.10.101

 

Cent OS 5

 

Slave1

 

d1

 

192.168.10.102

 

Cent OS 5

 

Slave2

 

d2

 

192.168.10.103

 

Cent OS 5

 

二、前期准备

1、在所有主机上,创建相同的用户(组)

[root@n1 ~]# groupadd -g 500 hadoop

[root@n1 ~]# useradd -u 500 -g hadoop -G hadoop -d /home/hadoop hadoop

[root@n1 ~]# passwd hadoop

Changing password for user hadoop.

New UNIX password:

BAD PASSWORD: it is based on a dictionary word

Retype new UNIX password:

passwd: all authentication tokens updated successfully.

编辑/etc/hosts文件,形如:

# Do not remove the following line, or various programs

# that require network functionality will fail.

127.0.0.1      localhost.localdomain localhost

::1            localhost6.localdomain6 localhost6

192.168.10.101  n1      n1

192.168.10.102  d1      d1

192.168.10.103  d2      d2

2、安装JDK

使用WinSCP将JDK1.6安装包传送到服务器主机上。

[root@n1 ~]# chown -R hadoop:hadoop /software

[root@n1 ~]# su - hadoop

[hadoop@n1 ~]$ ll /software/

total 70144

-rw-r--r-- 1 hadoop hadoop 71748681 Jul 15 22:33 jdk-6u33-linux-i586.bin

[hadoop@n1 ~]$ chmod -R 755 /software/

[hadoop@n1 ~]$ ll /software/

total 70144

-rwxr-xr-x 1 hadoop hadoop 71748681 Jul 15 22:33 jdk-6u33-linux-i586.bin

然后在另外两个节点执行下面命令,使用scp复制jdk安装包。

[root@d2 ~]# mkdir /software

[root@d2 ~]# scp root@192.168.10.101:/software/jdk-6u33-linux-i586.bin /software/.

The authenticity of host '192.168.10.101 (192.168.10.101)' can't be established.

RSA key fingerprint is 48:ef:fc:67:3f:43:88:0e:fa:e6:a6:ec:75:b0:c4:7f.

Are you sure you want to continue connecting (yes/no)? yes

Warning: Permanently added '192.168.10.101' (RSA) to the list of known hosts.

root@192.168.10.101's password:

jdk-6u33-linux-i586.bin                      100%  68MB  5.3MB/s  00:13

现在开始进行jdk安装。

[root@n1 ~]# su - hadoop

[hadoop@n1 ~]$ cd /software/

[hadoop@n1 software]$ ll

total 70144

-rwxr-xr-x 1 hadoop hadoop 71748681 Oct  9 02:20 jdk-6u33-linux-i586.bin

[hadoop@n1 software]$ ./jdk-6u33-linux-i586.bin

Unpacking...

Checksumming...

。。。。。。。

Java(TM) SE Development Kit 6 successfully installed.

Product Registration is FREE and includes many benefits:

* Notification of new versions, patches, and updates

* Special offers on Oracle products, services and training

* Access to early releases and documentation

Product and system data will be collected. If your configuration

supports a browser, the JDK Product Registration form. will

be presented. If you do not register, none of this information

will be saved. You may also register your JDK later by

opening the register.html file (located in the JDK installation

directory) in a browser.

For more information on what data Registration collects and

how it is managed and used, see:

Press Enter to continue.....

Done.

配置JDK的环境变量。在.bash_profile文件中加入如下代码。

JAVA_HOME=/software/jdk1.6.0_33

CLASSPATH=.:$JAVA_HOME/lib

PATH=$JAVA_HOME/bin:$PATH

export JAVA_HOME CLASSPATH PATH

别忘了执行

[hadoop@n1 ~]$ . .bash_profile

内容版权声明:除非注明,否则皆为本站原创文章。

转载注明出处:http://www.heiqu.com/19efc44e9877bf4f58b3ed9981200b67.html