Hadoop2.7.1 集群搭建记录

      我们需要三台虚拟机,可以先装一台虚拟机,下载好hadoop,配置好JDK,设置好环境变量后拷贝虚拟机

安装第一台虚拟机

安装的步骤不说了,说一下注意点

注意共享粘贴板

Hadoop2.7.1 集群搭建记录

多添加一块网卡,选择"HostOnly",这样我们应该就有两个网卡一个NAT,一个HostOnly.

Hadoop2.7.1 集群搭建记录

Hadoop2.7.1 集群搭建记录

安装的时候会有一步下载语言包,这步直接Skip.

 安装需要用到的软件和一些设置,其实这块才是最麻烦的

查看一下你当前的组groups

Hadoop2.7.1 集群搭建记录

  sudo chown -R hadoop /opt

  sudo chgrp -R hadoop /opt  

安装openssh-server,linuxmint默认应该是没有装过的.  sudo apt-get install openssh-server

关闭防火墙   sudo ufw disable

查看防火墙状态 sudo ufw status inactive

安装vim,sudo apt-get install vim

修改hostname(三台机器的hostname最好不一样,比如我是master-hadoop,slave1-hadoop,slave2-hadoop,为了好区分)

安装JDK

把下载的jdk上传到/opt/lib中(用MX直接可以拖放进去)

解压jdk tar -zxvf jdk-8u92-linux-x64.tar.gz

mv jdk1.8.0_92 jdk8  重命名一下文件夹名称

看一下现在的目录结构,注意下own和grp 都是hadoop(也可以不是hadoop,但是最好和hadoop相关的文件目录都属于一个组,防止权限不足等情况)

 

hadoop@hadoop-pc / $ cd /opt/ hadoop@hadoop-pc /opt $ ll total 16 drwxr-xr-x 4 hadoop hadoop 4096 Jul 2 00:33 ./ drwxr-xr-x 23 root root 4096 Jul 1 23:23 ../ drwxr-xr-x 3 hadoop hadoop 4096 Nov 29 2015 firefox/ drwxr-xr-x 3 hadoop hadoop 4096 Jul 2 01:04 lib/ hadoop@hadoop-pc /opt $ cd lib/ hadoop@hadoop-pc /opt/lib $ ll total 177156 drwxr-xr-x 3 hadoop hadoop 4096 Jul 2 01:04 ./ drwxr-xr-x 4 hadoop hadoop 4096 Jul 2 00:33 ../ drwxr-xr-x 8 hadoop hadoop 4096 Apr 1 12:20 jdk8/ -rw-rw-r-- 1 hadoop hadoop 181389058 Jul 2 01:00 jdk-8u92-linux-x64.tar.gz hadoop@hadoop-pc /opt/lib $ mkdir package hadoop@hadoop-pc /opt/lib $ mv jdk-8u92-linux-x64.tar.gz package/ hadoop@hadoop-pc /opt/lib $ ll total 16 drwxr-xr-x 4 hadoop hadoop 4096 Jul 2 01:08 ./ drwxr-xr-x 4 hadoop hadoop 4096 Jul 2 00:33 ../ drwxr-xr-x 8 hadoop hadoop 4096 Apr 1 12:20 jdk8/ drwxrwxr-x 2 hadoop hadoop 4096 Jul 2 01:08 package/ hadoop@hadoop-pc /opt/lib $ cd jdk8/ hadoop@hadoop-pc /opt/lib/jdk8 $ ll total 25916 drwxr-xr-x 8 hadoop hadoop 4096 Apr 1 12:20 ./ drwxr-xr-x 4 hadoop hadoop 4096 Jul 2 01:08 ../ drwxr-xr-x 2 hadoop hadoop 4096 Apr 1 12:17 bin/ -r--r--r-- 1 hadoop hadoop 3244 Apr 1 12:17 COPYRIGHT drwxr-xr-x 4 hadoop hadoop 4096 Apr 1 12:17 db/ drwxr-xr-x 3 hadoop hadoop 4096 Apr 1 12:17 include/ -rwxr-xr-x 1 hadoop hadoop 5090294 Apr 1 11:33 Javafx-src.zip* drwxr-xr-x 5 hadoop hadoop 4096 Apr 1 12:17 jre/ drwxr-xr-x 5 hadoop hadoop 4096 Apr 1 12:17 lib/ -r--r--r-- 1 hadoop hadoop 40 Apr 1 12:17 LICENSE drwxr-xr-x 4 hadoop hadoop 4096 Apr 1 12:17 man/ -r--r--r-- 1 hadoop hadoop 159 Apr 1 12:17 README.html -rw-r--r-- 1 hadoop hadoop 525 Apr 1 12:17 release -rw-r--r-- 1 hadoop hadoop 21104834 Apr 1 12:17 src.zip -rwxr-xr-x 1 hadoop hadoop 110114 Apr 1 11:33 THIRDPARTYLICENSEREADME-JAVAFX.txt* -r--r--r-- 1 hadoop hadoop 177094 Apr 1 12:17 THIRDPARTYLICENSEREADME.txt hadoop@hadoop-pc /opt/lib/jdk8 $ 

设置JAVA_HOME和环境变量

# /etc/profile: system-wide .profile file for the Bourne shell (sh(1)) # and Bourne compatible shells (bash(1), ksh(1), ash(1), ...). if [ "$PS1" ]; then if [ "$BASH" ] && [ "$BASH" != "/bin/sh" ]; then # The file bash.bashrc already sets the default PS1. # PS1='\h:\w\$ ' if [ -f /etc/bash.bashrc ]; then . /etc/bash.bashrc fi else if [ "`id -u`" -eq 0 ]; then PS1='# ' else PS1='$ ' fi fi fi # The default umask is now handled by pam_umask. # See pam_umask(8) and /etc/login.defs. if [ -d /etc/profile.d ]; then for i in /etc/profile.d/*.sh; do if [ -r $i ]; then . $i fi done unset i fi #ADD HERE JAVA_HOME=/opt/lib/jdk8 CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar PATH=$JAVA_HOME/bin:$PATH export JAVA_HOME export CLASSPATH export PATH 

检查JAVA版本和环境变量

内容版权声明:除非注明,否则皆为本站原创文章。

转载注明出处:https://www.heiqu.com/a2b2c121b5ef6fd2cad8095933d60bae.html