Ubuntu 14.04 LTE下安装Hadoop 1.2.1(集群分布式模式)(6)

$ ls -l
total 239424
drwxr-xr-x 2 hadoop hadoop      4096 Dec 27 04:19 Desktop
drwxr-xr-x 2 hadoop hadoop      4096 Dec 27 04:19 Documents
drwxr-xr-x 2 hadoop hadoop      4096 Dec 27 04:19 Downloads
-rw-r--r-- 1 hadoop hadoop      8980 Dec 27 03:27 examples.desktop
-rw-rw-r-- 1 hadoop hadoop  63851630 Dec 30 05:25 hadoop-1.2.1.tar.gz
-rw-rw-r-- 1 hadoop hadoop 181260798 Dec 27 06:49 jdk-8u65-linux-x64.tar.gz
drwxr-xr-x 2 hadoop hadoop      4096 Dec 27 04:19 Music
drwxr-xr-x 2 hadoop hadoop      4096 Dec 27 04:19 Pictures
drwxr-xr-x 2 hadoop hadoop      4096 Dec 27 04:19 Public
drwxr-xr-x 2 hadoop hadoop      4096 Dec 27 04:19 Templates
drwxr-xr-x 2 hadoop hadoop      4096 Dec 27 04:19 Videos
$ exit
Connection to slave2 closed.
hadoop@master:~/.ssh$

格式化hdfs的namenode

hadoop@master:/opt/hadoop-1.2.1/bin$ ./hadoop namenode -format16/01/09 23:33:18 INFO namenode.NameNode: STARTUP_MSG:
/************************************************************
STARTUP_MSG: Starting NameNode
STARTUP_MSG:  host = master/127.0.1.1
STARTUP_MSG:  args = [-format]
STARTUP_MSG:  version = 1.2.1
STARTUP_MSG:  build = https://svn.apache.org/repos/asf/hadoop/common/branches/branch-1.2 -r 1503152; compiled by 'mattf' on Mon Jul 22 15:23:09 PDT 2013
STARTUP_MSG:  java = 1.8.0_65
************************************************************/
Re-format filesystem in /hadoop/hdfs/name ? (Y or N) Y
16/01/09 23:34:09 INFO util.GSet: Computing capacity for map BlocksMap
16/01/09 23:34:09 INFO util.GSet: VM type      = 64-bit
16/01/09 23:34:09 INFO util.GSet: 2.0% max memory = 1013645312
16/01/09 23:34:09 INFO util.GSet: capacity      = 2^21 = 2097152 entries
16/01/09 23:34:09 INFO util.GSet: recommended=2097152, actual=2097152
16/01/09 23:34:10 INFO namenode.FSNamesystem: fsOwner=hadoop
16/01/09 23:34:10 INFO namenode.FSNamesystem: supergroup=supergroup
16/01/09 23:34:10 INFO namenode.FSNamesystem: isPermissionEnabled=true
16/01/09 23:34:10 INFO namenode.FSNamesystem: dfs.block.invalidate.limit=100
16/01/09 23:34:10 INFO namenode.FSNamesystem: isAccessTokenEnabled=false accessKeyUpdateInterval=0 min(s), accessTokenLifetime=0 min(s)
16/01/09 23:34:10 INFO namenode.FSEditLog: dfs.namenode.edits.toleration.length = 0
16/01/09 23:34:10 INFO namenode.NameNode: Caching file names occuring more than 10 times
16/01/09 23:34:11 INFO common.Storage: Image file /hadoop/hdfs/name/current/fsimage of size 112 bytes saved in 0 seconds.
16/01/09 23:34:11 INFO namenode.FSEditLog: closing edit log: position=4, editlog=/hadoop/hdfs/name/current/edits
16/01/09 23:34:11 INFO namenode.FSEditLog: close success: truncate to 4, editlog=/hadoop/hdfs/name/current/edits
16/01/09 23:34:11 INFO common.Storage: Storage directory /hadoop/hdfs/name has been successfully formatted.
16/01/09 23:34:11 INFO namenode.NameNode: SHUTDOWN_MSG:
/************************************************************
SHUTDOWN_MSG: Shutting down NameNode at master/127.0.1.1
************************************************************/
hadoop@master:/opt/hadoop-1.2.1/bin$

在master上启动hadoop的所有服务

hadoop@master:/opt/hadoop-1.2.1/bin$ ./start-all.sh
starting namenode, logging to /opt/hadoop-1.2.1/libexec/../logs/hadoop-hadoop-namenode-master.out
slave2: starting datanode, logging to /opt/hadoop-1.2.1/libexec/../logs/hadoop-hadoop-datanode-slave2.out
slave1: starting datanode, logging to /opt/hadoop-1.2.1/libexec/../logs/hadoop-hadoop-datanode-slave1.out
master: starting datanode, logging to /opt/hadoop-1.2.1/libexec/../logs/hadoop-hadoop-datanode-master.out
master: starting secondarynamenode, logging to /opt/hadoop-1.2.1/libexec/../logs/hadoop-hadoop-secondarynamenode-master.out
starting jobtracker, logging to /opt/hadoop-1.2.1/libexec/../logs/hadoop-hadoop-jobtracker-master.out
slave2: starting tasktracker, logging to /opt/hadoop-1.2.1/libexec/../logs/hadoop-hadoop-tasktracker-slave2.out
slave1: starting tasktracker, logging to /opt/hadoop-1.2.1/libexec/../logs/hadoop-hadoop-tasktracker-slave1.out
master: starting tasktracker, logging to /opt/hadoop-1.2.1/libexec/../logs/hadoop-hadoop-tasktracker-master.out
hadoop@master:/opt/hadoop-1.2.1/bin$ jps
5088 JobTracker
5010 SecondaryNameNode
4871 DataNode
4732 NameNode
5277 Jps
5230 TaskTracker
hadoop@master:/opt/hadoop-1.2.1/bin$

观察slave1

hadoop@slave1:~/.ssh$ jps
3669 TaskTracker
3718 Jps
3560 DataNode
hadoop@slave1:~/.ssh$

观察slave2

hadoop@slave2:~$ jps
3216 Jps
2982 TaskTracker
3095 DataNode

过程中需要注意的问题:

你既然是用hadoop用户去启动hadoop服务的,那这个用户至少得对java、hadoop程序有访问执行权限吧,这点要注意。

其次,这个用户对hadoop的hdfs的文件和mapreduce文件都需要有写入权限吧

问题一:

内容版权声明:除非注明,否则皆为本站原创文章。

转载注明出处:https://www.heiqu.com/e386ef54f6ced7b3c6d250b30537cfd3.html