10.MapReduce测试:
hduser@master:/usr/local/Hadoop$ mkdir /tmp/test
hduser@master:/usr/local/hadoop$ cd /tmp/test
hduser@master:/tmp/test$ rz
rz waiting to receive.
开始 zmodem 传输。 按 Ctrl+C 取消。
100% 615 KB 615 KB/s 00:00:01 0 Errors
100% 502 KB 502 KB/s 00:00:01 0 Errors
100% 813 KB 813 KB/s 00:00:01 0 Errors
hduser@master:/tmp/test$ /usr/local/hadoop/bin/start-all.sh
starting namenode, logging to /usr/local/hadoop/libexec/../logs/hadoop-hduser-namenode-master.out
slave2: starting datanode, logging to /usr/local/hadoop/libexec/../logs/hadoop-hduser-datanode-slave2.out
slave1: starting datanode, logging to /usr/local/hadoop/libexec/../logs/hadoop-hduser-datanode-slave1.out
master: starting datanode, logging to /usr/local/hadoop/libexec/../logs/hadoop-hduser-datanode-master.out
localhost: starting secondarynamenode, logging to /usr/local/hadoop/libexec/../logs/hadoop-hduser-secondarynamenode-master.out
starting jobtracker, logging to /usr/local/hadoop/libexec/../logs/hadoop-hduser-jobtracker-master.out
slave1: starting tasktracker, logging to /usr/local/hadoop/libexec/../logs/hadoop-hduser-tasktracker-slave1.out
slave2: starting tasktracker, logging to /usr/local/hadoop/libexec/../logs/hadoop-hduser-tasktracker-slave2.out
master: starting tasktracker, logging to /usr/local/hadoop/libexec/../logs/hadoop-hduser-tasktracker-master.out
hduser@master:~$ /usr/local/hadoop/bin/hadoop dfs -copyFromLocal /tmp/test /user/hduser/test
hduser@master:~$ /usr/local/hadoop/bin/hadoop dfs -ls /user/hduser
Found 1 items
drwxr-xr-x - hduser supergroup 0 2013-04-09 08:06 /user/hduser/test
hduser@master:~$ /usr/local/hadoop/bin/hadoop dfs -ls /user/hduser/test
Found 3 items
-rw-r--r-- 2 hduser supergroup 630010 2013-04-09 08:06 /user/hduser/test/1.epub
-rw-r--r-- 2 hduser supergroup 514824 2013-04-09 08:06 /user/hduser/test/2.epub
-rw-r--r-- 2 hduser supergroup 832882 2013-04-09 08:06 /user/hduser/test/3.epub
hduser@master:~$ cd /usr/local/hadoop/
hduser@master:/usr/local/hadoop$ bin/hadoop jar hadoop-examples-1.0.4.jar wordcount /user/hduser/test /user/hduser/test-output
13/04/09 08:18:45 INFO input.FileInputFormat: Total input paths to process : 3
13/04/09 08:18:45 INFO util.NativeCodeLoader: Loaded the native-hadoop library
13/04/09 08:18:45 WARN snappy.LoadSnappy: Snappy native library not loaded
13/04/09 08:18:45 INFO mapred.JobClient: Running job: job_201304090758_0001
13/04/09 08:18:46 INFO mapred.JobClient: map 0% reduce 0%
13/04/09 08:19:10 INFO mapred.JobClient: map 66% reduce 0%
13/04/09 08:19:19 INFO mapred.JobClient: map 100% reduce 0%
13/04/09 08:19:31 INFO mapred.JobClient: map 100% reduce 100%
13/04/09 08:19:36 INFO mapred.JobClient: Job complete: job_201304090758_0001
13/04/09 08:19:37 INFO mapred.JobClient: Counters: 29
13/04/09 08:19:37 INFO mapred.JobClient: Job Counters
13/04/09 08:19:37 INFO mapred.JobClient: Launched reduce tasks=1
13/04/09 08:19:37 INFO mapred.JobClient: SLOTS_MILLIS_MAPS=42658
13/04/09 08:19:37 INFO mapred.JobClient: Total time spent by all reduces waiting after reserving slots (ms)=0
13/04/09 08:19:37 INFO mapred.JobClient: Total time spent by all maps waiting after reserving slots (ms)=0
13/04/09 08:19:37 INFO mapred.JobClient: Launched map tasks=3
13/04/09 08:19:37 INFO mapred.JobClient: Data-local map tasks=3
13/04/09 08:19:37 INFO mapred.JobClient: SLOTS_MILLIS_REDUCES=20867
13/04/09 08:19:37 INFO mapred.JobClient: File Output Format Counters
13/04/09 08:19:37 INFO mapred.JobClient: Bytes Written=3216032
13/04/09 08:19:37 INFO mapred.JobClient: FileSystemCounters
13/04/09 08:19:37 INFO mapred.JobClient: FILE_BYTES_READ=3421949
13/04/09 08:19:37 INFO mapred.JobClient: HDFS_BYTES_READ=1978040
13/04/09 08:19:37 INFO mapred.JobClient: FILE_BYTES_WRITTEN=6930267
13/04/09 08:19:37 INFO mapred.JobClient: HDFS_BYTES_WRITTEN=3216032
13/04/09 08:19:37 INFO mapred.JobClient: File Input Format Counters
13/04/09 08:19:37 INFO mapred.JobClient: Bytes Read=1977716
13/04/09 08:19:37 INFO mapred.JobClient: Map-Reduce Framework
13/04/09 08:19:37 INFO mapred.JobClient: Map output materialized bytes=3421961
13/04/09 08:19:37 INFO mapred.JobClient: Map input records=14841
13/04/09 08:19:37 INFO mapred.JobClient: Reduce shuffle bytes=2449921
13/04/09 08:19:37 INFO mapred.JobClient: Spilled Records=64440
13/04/09 08:19:37 INFO mapred.JobClient: Map output bytes=3685555
13/04/09 08:19:37 INFO mapred.JobClient: CPU time spent (ms)=10830
13/04/09 08:19:37 INFO mapred.JobClient: Total committed heap usage (bytes)=496644096
13/04/09 08:19:37 INFO mapred.JobClient: Combine input records=36177
13/04/09 08:19:37 INFO mapred.JobClient: SPLIT_RAW_BYTES=324
13/04/09 08:19:37 INFO mapred.JobClient: Reduce input records=32220
13/04/09 08:19:37 INFO mapred.JobClient: Reduce input groups=31501
13/04/09 08:19:37 INFO mapred.JobClient: Combine output records=32220
13/04/09 08:19:37 INFO mapred.JobClient: Physical memory (bytes) snapshot=614944768
13/04/09 08:19:37 INFO mapred.JobClient: Reduce output records=31501
13/04/09 08:19:37 INFO mapred.JobClient: Virtual memory (bytes) snapshot=3845349376
13/04/09 08:19:37 INFO mapred.JobClient: Map output records=36177
hduser@master:/usr/local/hadoop$ mkdir /tmp/test-output
hduser@master:/usr/local/hadoop$ bin/hadoop dfs -getmerge /user/hduser/test-output /tmp/test-output
13/04/09 08:22:48 INFO util.NativeCodeLoader: Loaded the native-hadoop library
$head /tmp/test-output/test-output
查看内容~
接下来是HBase的安装: