在Ubuntu11.10下构建Hadoop实验环境笔记

1,安装虚拟机virtual box,添加虚拟机hdp1.vdi(网络1:nat或bridge, 网络2:internal),安装Ubuntu 11.10,添加用户Hadoop作为运行hadoop之用户,并在安装好后设置root用户密码。

2,在虚拟机中安装“增强功能”,重启后右上角系统菜单“System Settings..."->"Language Support"  中 "Install / Remove Language..."安装简体中文支持, "Keyboard input method system"选ibus. (注:系统panel上的输入法图标经常会消失,可以通过命令: killall ibus-daemon; ibus-daemon -d 重启ibus显示)

更多Hadoop相关信息见Hadoop 专题页面 ?tid=13

3,更改apt源:

$ sudo cp /etc/sources.list /etc/sources.list.bak

$ sudo gedit /etc/sources.list

内容如下:
deb oneiric main restricted universe multiverse
deb oneiric-updates main restricted universe multiverse
deb oneiric-security main restricted universe multiverse
deb oneiric-backports main restricted universe multiverse
deb oneiric main restricted universe multiversedeb

deb oneiric main universe restricted multiverse
deb-src oneiric main universe restricted multiverse
deb oneiric-security universe main multiverse restricted
deb-src oneiric-security universe main multiverse restricted
deb oneiric-updates universe main multiverse restricted
deb oneiric-proposed universe main multiverse restricted
deb-src oneiric-proposed universe main multiverse restricted
deb oneiric-backports universe main multiverse restricted
deb-src oneiric-backports universe main multiverse restricted
deb-src oneiric-updates universe main multiverse restricted

#deb oneiric main universe restricted multiverse
#deb-src oneiric main universe restricted multiverse
#deb oneiric-security universe main multiverse restricted
#deb-src oneiric-security universe main multiverse restricted
#deb oneiric-updates universe main multiverse restricted
#deb-src oneiric-updates universe main multiverse restricted

4,$ sudo apt-get update

5,$ sudo apt-get install vim-gtk, axel, mutliget   (注:下载工具)

6,$ sudo apt-get install chkconfig sysv-rc-conf  (注:管理系统服务用)

7,$ sudo apt-get install openjdk-7-jdk rsync openssh-server openssh-client (注:hadoop需用的工具)

这一步可顺便配置好java的环境变量:

$ sudo vi /etc/profile.d/bash_completion.sh, 在文件末尾添加内容如下:

export JAVA_HOME=/usr/lib/jvm/java-7-openjdk-i386/
export PATH=$PATH:$JAVA_HOME/bin
export CLASSPATH=.:$JAVA_HOME/lib

8,下载hadoop: $ axel

9, 安装hadoop:$ sudo dpkg -i hadoop_1.0.1-1_i386.deb

10,查看hadoop安装了哪些文件:$ sudo dpkg -L hadoop

11,更改系统网卡2的网络配置为;静态ip, 地址-> 192.168.0.1 (注:以后其它系统相应改为192.168.0.2, 192.168.0.3 ......)

12,更改系统配置:

$ sudo vi /etc/hostname 内容为:hdp1 (注:以后其它系统相应改为hdp2, hdp3 ......)

$ sudo vi /etc/hosts 增加内容:

192.168.0.1     hdp1
192.168.0.2     hdp2
192.168.0.3     hdp3
192.168.0.4     hdp4

注意:应确保hostname和hosts文件设置的主机名一致,否则reduce过程无法完成。

13,产生SSH密钥,实现无密码安全联机:

$ ssh-keygen -t rsa -P ""

$ cd ~/.ssh

$ cp id_ras.pub authorized_keys

$ ssh hdp1 ( 注:本步测试上面各步骤是否正确,正确方能成功)

$ exit

注:以下步骤配置hadoop

14,选定hdp1为Master并作为NameNode和JobTracker

$ sudo vi /etc/hadoop/masters 内容为:hdp1

15,将所有服务器(包括主服务器hdp1)全部添加为slaves次服务器,当作DataNode和TaskTracker

$ sudo vi /etc/hadoop/slaves 内容为:

hdp1

hdp2

hdp3

内容版权声明:除非注明,否则皆为本站原创文章。

转载注明出处:http://www.heiqu.com/929c1c07a2f5c55a9d16f4689caa4d9f.html