关于MR的工作原理不做过多叙述，本文将对MapReduce的实例WordCount(单词计数程序)做实践，从而理解MapReduce的工作机制。
WordCount:
1.应用场景，在大量文件中存储了单词，单词之间用空格分隔
2.类似场景：搜索引擎中，统计最流行的N个搜索词，统计搜索词频率，帮助优化搜索词提示。
3.采用MapReduce执行过程如图
3.1MapReduce将作业的整个运行过程分为两个阶段
3.1.1Map阶段和Reduce阶段
Map阶段由一定数量的Map Task组成
输入数据格式解析：InputFormat
输入数据处理：Mapper
数据分组：Partitioner
3.1.2Reduce阶段由一定数量的Reduce Task组成
数据远程拷贝
数据按照key排序
数据处理：Reducer
数据输出格式：OutputFormat
4.介绍代码结构
4.1 pom.xml
<?xml version="1.0" encoding="UTF-8"?> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 "> <modelVersion>4.0.0</modelVersion> <groupId>hadoop</groupId> <artifactId>hadoop.mapreduce</artifactId> <version>1.0-SNAPSHOT</version> <repositories> <repository> <id>aliyun</id> <url></url> </repository> </repositories> <dependencies> <dependency> <groupId>org.apache.hadoop</groupId> <artifactId>hadoop-yarn-client</artifactId> <version>2.7.3</version> </dependency> <dependency> <groupId>org.apache.hadoop</groupId> <artifactId>hadoop-common</artifactId> <version>2.7.3</version> </dependency> <dependency> <groupId>org.apache.hadoop</groupId> <artifactId>hadoop-mapreduce-client-jobclient</artifactId> <version>2.7.3</version> </dependency> </dependencies> <build> <plugins> <plugin> <artifactId>maven-assembly-plugin</artifactId> <version>2.3</version> <configuration> <classifier>dist</classifier> <appendAssemblyId>true</appendAssemblyId> <descriptorRefs> <descriptor>jar-with-dependencies</descriptor> </descriptorRefs> </configuration> <executions> <execution> <id>make-assembly</id> <phase>package</phase> <goals> <goal>single</goal> </goals> </execution> </executions> </plugin> </plugins> </build> </project>