学习Hadoop,必须要亲自部署一下,找找感觉。我使用的版本是0.20.2,下面就详细说一下部署过程。
一、规划
使用VMWare Workstation创建3台虚拟机。定义如下表:
Hostname
IP
OS
Master
n1
192.168.10.101
Cent OS 5
Slave1
d1
192.168.10.102
Cent OS 5
Slave2
d2
192.168.10.103
Cent OS 5
二、前期准备
1、在所有主机上,创建相同的用户(组)
[root@n1 ~]# groupadd -g 500 hadoop
[root@n1 ~]# useradd -u 500 -g hadoop -G hadoop -d /home/hadoop hadoop
[root@n1 ~]# passwd hadoop
Changing password for user hadoop.
New UNIX password:
BAD PASSWORD: it is based on a dictionary word
Retype new UNIX password:
passwd: all authentication tokens updated successfully.
编辑/etc/hosts文件,形如:
# Do not remove the following line, or various programs
# that require network functionality will fail.
127.0.0.1 localhost.localdomain localhost
::1 localhost6.localdomain6 localhost6
192.168.10.101 n1 n1
192.168.10.102 d1 d1
192.168.10.103 d2 d2
2、安装JDK
使用WinSCP将JDK1.6安装包传送到服务器主机上。
[root@n1 ~]# chown -R hadoop:hadoop /software
[root@n1 ~]# su - hadoop
[hadoop@n1 ~]$ ll /software/
total 70144
-rw-r--r-- 1 hadoop hadoop 71748681 Jul 15 22:33 jdk-6u33-linux-i586.bin
[hadoop@n1 ~]$ chmod -R 755 /software/
[hadoop@n1 ~]$ ll /software/
total 70144
-rwxr-xr-x 1 hadoop hadoop 71748681 Jul 15 22:33 jdk-6u33-linux-i586.bin
然后在另外两个节点执行下面命令,使用scp复制jdk安装包。
[root@d2 ~]# mkdir /software
[root@d2 ~]# scp root@192.168.10.101:/software/jdk-6u33-linux-i586.bin /software/.
The authenticity of host '192.168.10.101 (192.168.10.101)' can't be established.
RSA key fingerprint is 48:ef:fc:67:3f:43:88:0e:fa:e6:a6:ec:75:b0:c4:7f.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '192.168.10.101' (RSA) to the list of known hosts.
root@192.168.10.101's password:
jdk-6u33-linux-i586.bin 100% 68MB 5.3MB/s 00:13
现在开始进行jdk安装。
[root@n1 ~]# su - hadoop
[hadoop@n1 ~]$ cd /software/
[hadoop@n1 software]$ ll
total 70144
-rwxr-xr-x 1 hadoop hadoop 71748681 Oct 9 02:20 jdk-6u33-linux-i586.bin
[hadoop@n1 software]$ ./jdk-6u33-linux-i586.bin
Unpacking...
Checksumming...
。。。。。。。
Java(TM) SE Development Kit 6 successfully installed.
Product Registration is FREE and includes many benefits:
* Notification of new versions, patches, and updates
* Special offers on Oracle products, services and training
* Access to early releases and documentation
Product and system data will be collected. If your configuration
supports a browser, the JDK Product Registration form. will
be presented. If you do not register, none of this information
will be saved. You may also register your JDK later by
opening the register.html file (located in the JDK installation
directory) in a browser.
For more information on what data Registration collects and
how it is managed and used, see:
Press Enter to continue.....
Done.
配置JDK的环境变量。在.bash_profile文件中加入如下代码。
JAVA_HOME=/software/jdk1.6.0_33
CLASSPATH=.:$JAVA_HOME/lib
PATH=$JAVA_HOME/bin:$PATH
export JAVA_HOME CLASSPATH PATH
别忘了执行
[hadoop@n1 ~]$ . .bash_profile