<configuration>
<property>
<name>mapred.job.tracker</name>
<value>localhost:54311</value>
<description>The host and port that the MapReduce job tracker runs
at. If "local", then jobs are run in-process as a single map
and reduce task.
</description>
</property>
</configuration>
hdfs-site.xml如下:
<configuration>
<property>
<name>dfs.replication</name>
<value>1</value>
<description>Default block replication.
The actual number of replications can be specified when the file is created.
The default is used if replication is not specified in create time.
</description>
</property>
</configuration>
vim hadoop-env.sh
export JAVA_HOME=/usr/lib/jdk1.6.0_33
ok,配置完毕
格式化HDFS:
/opt/hadoop/bin/hadoop namenode -format
输出
root@Master:/opt/hadoop# /opt/hadoop/bin/hadoop namenode -format
12/07/13 14:27:29 INFO namenode.NameNode: STARTUP_MSG:
/************************************************************
STARTUP_MSG: Starting NameNode
STARTUP_MSG: host = Master.Hadoop/127.0.1.1
STARTUP_MSG: args = [-format]
STARTUP_MSG: version = 0.20.2
STARTUP_MSG: build = https://svn.apache.org/repos/asf/hadoop/common/branches/branch-0.20 -r 911707; compiled by 'chrisdo' on Fri Feb 19 08:07:34 UTC 2010
************************************************************/
Re-format filesystem in /opt/hadoop-datastore/dfs/name ? (Y or N) y
Format aborted in /opt/hadoop-datastore/dfs/name
12/07/13 14:27:35 INFO namenode.NameNode: SHUTDOWN_MSG:
/************************************************************
SHUTDOWN_MSG: Shutting down NameNode at Master.Hadoop/127.0.1.1
************************************************************/
启动HDFS和MapReduce
改为hadoop用户
/opt/hadoop/bin/start-all.sh
输出
starting namenode, logging to /opt/hadoop/bin/../logs/hadoop-root-namenode-Master.Hadoop.out
The authenticity of host 'localhost (127.0.0.1)' can't be established.
ECDSA key fingerprint is 3e:55:d8:be:47:46:21:95:29:9b:9e:c5:fb:02:f4:d2.
Are you sure you want to continue connecting (yes/no)? yes
localhost: Warning: Permanently added 'localhost' (ECDSA) to the list of known hosts.
root@localhost's password:
localhost: starting datanode, logging to /opt/hadoop/bin/../logs/hadoop-root-datanode-Master.Hadoop.out
root@localhost's password:
localhost: starting secondarynamenode, logging to /opt/hadoop/bin/../logs/hadoop-root-secondarynamenode-Master.Hadoop.out
starting jobtracker, logging to /opt/hadoop/bin/../logs/hadoop-root-jobtracker-Master.Hadoop.out
root@localhost's password:
localhost: starting tasktracker, logging to /opt/hadoop/bin/../logs/hadoop-root-tasktracker-Master.Hadoop.out
9、停止服务的脚本是:
/opt/hadoop/bin/stop-all.sh
10.启动成功后,用jps查看下。
5.运行wordcount.java
在Hadoop所在目录里有几个jar文件,其中hadoop-examples-0.20.203.0.jar就是我们需要的,它里面含有wordcount,咱们使用命令建立测试的文件
(1)先在本地磁盘建立两个输入文件file01 和file02:
$ echo “Hello World Bye World” > file01
$ echo “Hello Hadoop Goodbye Hadoop” > file02
./hadoop fs -ls /
(2)在hdfs 中建立一个input 目录:./hadoop fs -mkdir input
删除./hadoop dfs -rmr input
(3)将file01 和file02 拷贝到hdfs 中:
./hadoop fs -copyFromLocal /home/linuxidc/file0* input
./hadoop fs -ls /user/root/input
Found 2 items
-rw-r--r-- 1 root supergroup 22 2012-07-13 15:07 /user/root/input/file01
-rw-r--r-- 1 root supergroup 28 2012-07-13 15:07 /user/root/input/file02
root@Master:/opt/hadoop/bin# ./hadoop fs -cat /user/root/input/file01/
Hello World Bye World
(4)执行wordcount:
$ Hadoop jar hadoop-0.20.1-examples.jar wordcount input output
$ bin/hadoop jar hadoop-0.20.1-examples.jar wordcount input output
Exceptioninthread"m
ain" java.io.IOException: Error openingjobjar: hadoop-0.20.2-examples.jar
at org.apache.hadoop.util.RunJar.main(RunJar.java:90)
Caused by: java.util.zip.ZipException: error in opening zip file
at java.util.zip.ZipFile.open(Native Method)
at java.util.zip.ZipFile.<init>(ZipFile.java:114)
at java.util.jar.JarFile.<init>(JarFile.java:135)
at java.util.jar.JarFile.<init>(JarFile.java:72)
at org.apache.hadoop.util.RunJar.main(RunJar.java:88)
解决办法:
注意路径:
./hadoop jar /opt/hadoop/hadoop-0.20.2-examples.jar wordcount input output
输出