Hadoop集群安装Hbase实验环境搭建(4)

10.MapReduce测试:
 
hduser@master:/usr/local/Hadoop$ mkdir /tmp/test
 hduser@master:/usr/local/hadoop$ cd /tmp/test
 hduser@master:/tmp/test$ rz
 rz waiting to receive.
 开始 zmodem 传输。  按 Ctrl+C 取消。
  100%    615 KB  615 KB/s 00:00:01      0 Errors
  100%    502 KB  502 KB/s 00:00:01      0 Errors
  100%    813 KB  813 KB/s 00:00:01      0 Errors
 
hduser@master:/tmp/test$ /usr/local/hadoop/bin/start-all.sh
 starting namenode, logging to /usr/local/hadoop/libexec/../logs/hadoop-hduser-namenode-master.out
 slave2: starting datanode, logging to /usr/local/hadoop/libexec/../logs/hadoop-hduser-datanode-slave2.out
 slave1: starting datanode, logging to /usr/local/hadoop/libexec/../logs/hadoop-hduser-datanode-slave1.out
 master: starting datanode, logging to /usr/local/hadoop/libexec/../logs/hadoop-hduser-datanode-master.out
 localhost: starting secondarynamenode, logging to /usr/local/hadoop/libexec/../logs/hadoop-hduser-secondarynamenode-master.out
 starting jobtracker, logging to /usr/local/hadoop/libexec/../logs/hadoop-hduser-jobtracker-master.out
 slave1: starting tasktracker, logging to /usr/local/hadoop/libexec/../logs/hadoop-hduser-tasktracker-slave1.out
 slave2: starting tasktracker, logging to /usr/local/hadoop/libexec/../logs/hadoop-hduser-tasktracker-slave2.out
 master: starting tasktracker, logging to /usr/local/hadoop/libexec/../logs/hadoop-hduser-tasktracker-master.out
 


hduser@master:~$ /usr/local/hadoop/bin/hadoop dfs -copyFromLocal /tmp/test /user/hduser/test
 hduser@master:~$ /usr/local/hadoop/bin/hadoop dfs -ls /user/hduser
 Found 1 items
 drwxr-xr-x  - hduser supergroup          0 2013-04-09 08:06 /user/hduser/test
 hduser@master:~$ /usr/local/hadoop/bin/hadoop dfs -ls /user/hduser/test
 Found 3 items
 -rw-r--r--  2 hduser supergroup    630010 2013-04-09 08:06 /user/hduser/test/1.epub
 -rw-r--r--  2 hduser supergroup    514824 2013-04-09 08:06 /user/hduser/test/2.epub
 -rw-r--r--  2 hduser supergroup    832882 2013-04-09 08:06 /user/hduser/test/3.epub
 

hduser@master:~$ cd /usr/local/hadoop/
 hduser@master:/usr/local/hadoop$ bin/hadoop jar hadoop-examples-1.0.4.jar wordcount /user/hduser/test /user/hduser/test-output
 13/04/09 08:18:45 INFO input.FileInputFormat: Total input paths to process : 3
 13/04/09 08:18:45 INFO util.NativeCodeLoader: Loaded the native-hadoop library
 13/04/09 08:18:45 WARN snappy.LoadSnappy: Snappy native library not loaded
 13/04/09 08:18:45 INFO mapred.JobClient: Running job: job_201304090758_0001
 13/04/09 08:18:46 INFO mapred.JobClient:  map 0% reduce 0%
 13/04/09 08:19:10 INFO mapred.JobClient:  map 66% reduce 0%
 13/04/09 08:19:19 INFO mapred.JobClient:  map 100% reduce 0%
 13/04/09 08:19:31 INFO mapred.JobClient:  map 100% reduce 100%
 13/04/09 08:19:36 INFO mapred.JobClient: Job complete: job_201304090758_0001
 13/04/09 08:19:37 INFO mapred.JobClient: Counters: 29
 13/04/09 08:19:37 INFO mapred.JobClient:  Job Counters
 13/04/09 08:19:37 INFO mapred.JobClient:    Launched reduce tasks=1
 13/04/09 08:19:37 INFO mapred.JobClient:    SLOTS_MILLIS_MAPS=42658
 13/04/09 08:19:37 INFO mapred.JobClient:    Total time spent by all reduces waiting after reserving slots (ms)=0
 13/04/09 08:19:37 INFO mapred.JobClient:    Total time spent by all maps waiting after reserving slots (ms)=0
 13/04/09 08:19:37 INFO mapred.JobClient:    Launched map tasks=3
 13/04/09 08:19:37 INFO mapred.JobClient:    Data-local map tasks=3
 13/04/09 08:19:37 INFO mapred.JobClient:    SLOTS_MILLIS_REDUCES=20867
 13/04/09 08:19:37 INFO mapred.JobClient:  File Output Format Counters
 13/04/09 08:19:37 INFO mapred.JobClient:    Bytes Written=3216032
 13/04/09 08:19:37 INFO mapred.JobClient:  FileSystemCounters
 13/04/09 08:19:37 INFO mapred.JobClient:    FILE_BYTES_READ=3421949
 13/04/09 08:19:37 INFO mapred.JobClient:    HDFS_BYTES_READ=1978040
 13/04/09 08:19:37 INFO mapred.JobClient:    FILE_BYTES_WRITTEN=6930267
 13/04/09 08:19:37 INFO mapred.JobClient:    HDFS_BYTES_WRITTEN=3216032
 13/04/09 08:19:37 INFO mapred.JobClient:  File Input Format Counters
 13/04/09 08:19:37 INFO mapred.JobClient:    Bytes Read=1977716
 13/04/09 08:19:37 INFO mapred.JobClient:  Map-Reduce Framework
 13/04/09 08:19:37 INFO mapred.JobClient:    Map output materialized bytes=3421961
 13/04/09 08:19:37 INFO mapred.JobClient:    Map input records=14841
 13/04/09 08:19:37 INFO mapred.JobClient:    Reduce shuffle bytes=2449921
 13/04/09 08:19:37 INFO mapred.JobClient:    Spilled Records=64440
 13/04/09 08:19:37 INFO mapred.JobClient:    Map output bytes=3685555
 13/04/09 08:19:37 INFO mapred.JobClient:    CPU time spent (ms)=10830
 13/04/09 08:19:37 INFO mapred.JobClient:    Total committed heap usage (bytes)=496644096
 13/04/09 08:19:37 INFO mapred.JobClient:    Combine input records=36177
 13/04/09 08:19:37 INFO mapred.JobClient:    SPLIT_RAW_BYTES=324
 13/04/09 08:19:37 INFO mapred.JobClient:    Reduce input records=32220
 13/04/09 08:19:37 INFO mapred.JobClient:    Reduce input groups=31501
 13/04/09 08:19:37 INFO mapred.JobClient:    Combine output records=32220
 13/04/09 08:19:37 INFO mapred.JobClient:    Physical memory (bytes) snapshot=614944768
 13/04/09 08:19:37 INFO mapred.JobClient:    Reduce output records=31501
 13/04/09 08:19:37 INFO mapred.JobClient:    Virtual memory (bytes) snapshot=3845349376
 13/04/09 08:19:37 INFO mapred.JobClient:    Map output records=36177

hduser@master:/usr/local/hadoop$ mkdir /tmp/test-output
 hduser@master:/usr/local/hadoop$ bin/hadoop dfs -getmerge /user/hduser/test-output /tmp/test-output
 13/04/09 08:22:48 INFO util.NativeCodeLoader: Loaded the native-hadoop library
 

$head /tmp/test-output/test-output
 
查看内容~
 
接下来是HBase的安装:

内容版权声明:除非注明,否则皆为本站原创文章。

转载注明出处:http://www.heiqu.com/4ce53d8ee7a2c55a2709567d6625af3d.html