64位Ubuntu下重新编译Hadoop2.2.0步骤

Hadoop官方网站中只提供了32位的Hadoop-2.2.0.tar.gz,如果要在64位Ubuntu下部署Hadoop-2.2.0,就需要重新编译源码包,生成64位的部署包。

建议以下操作使用root账户,避免出现权限不足的问题。

Linux系统:64位Ubuntu 14.04.1

安装JDK

请参考文章《Ubuntu 14.04.1中安装JDK》

安装Maven

请参考文章《Ubuntu 14.04.1中安装Maven》

下载Hadoop源码

wget http://mirror.bit.edu.cn/apache/hadoop/common/hadoop-2.2.0/hadoop-2.2.0-src.tar.gz

解压

tar -xzf hadoop-2.2.0-src.tar.gz

编译源代码

cd hadoop-2.2.0-src mvn package -Pdist,native -DskipTests -Dtar

第1次编译:失败(hadoop pom.xml的bug)

错误信息:

[ERROR] Failed to execute goal on project hadoop-auth: Could not resolve dependencies for project org.apache.hadoop:hadoop-auth:jar:2.2.0: Could not transfer artifact org.mortbay.jetty:jetty:jar:6.1.26 from/to central (https://repo.maven.apache.org/maven2): GET request of: org/mortbay/jetty/jetty/6.1.26/jetty-6.1.26.jar from central failed: SSL peer shut down incorrectly -> [Help 1] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException [ERROR] [ERROR] After correcting the problems, you can resume the build with the command [ERROR] mvn -rf :hadoop-auth

解决办法:
这是hadoop的一个bug,在pom.xml中添加下面patch即可,详见https://issues.apache.org/jira/browse/HADOOP-10110 。

编辑`hadoop-common-project/hadoop-auth/pom.xml`文件:

vi hadoop-common-project/hadoop-auth/pom.xml

在<dependencys></dependencys>节点中插入:

<dependency> <groupId>org.mortbay.jetty</groupId> <artifactId>jetty-util</artifactId> <scope>test</scope> </dependency>

第2次编译:失败(未安装protoc)

错误信息:

[ERROR] Failed to execute goal org.apache.hadoop:hadoop-maven-plugins:2.2.0:protoc (compile-protoc) on project hadoop-common: org.apache.maven.plugin.MojoExecutionException: 'protoc --version' did not return a version -&gt; [Help 1] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException [ERROR] [ERROR] After correcting the problems, you can resume the build with the command [ERROR] mvn -rf :hadoop-common

解决办法:

根据错误信息可以知道是因为没有安装protoc。

wget https://protobuf.googlecode.com/files/protobuf-2.5.0.tar.gz tar -xzf protobuf-2.5.0.tar.gz cd protobuf-2.5.0 ./configure make make check make install

其中,在执行./configure命令是会报如下错误:

checking whether to enable maintainer-specific portions of Makefiles... yes checking build system type... x86_64-unknown-linux-gnu checking host system type... x86_64-unknown-linux-gnu checking target system type... x86_64-unknown-linux-gnu checking for a BSD-compatible install... /usr/bin/install -c checking whether build environment is sane... yes checking for a thread-safe mkdir -p... /bin/mkdir -p checking for gawk... gawk checking whether make sets $(MAKE)... no checking for gcc... no checking for cc... no checking for cl.exe... no configure: error: in `/home/hadoop/protobuf-2.5.0': configure: error: no acceptable C compiler found in $PATH See `config.log' for more details

提示我们找不到C编译器,因此我们还需要安装C编译器。

ubuntu提供了集成gcc等编译器的基本编译工具`build-essential`,安装起来也比较方便,只需要一行命令:

apt-get install build-essential

安装过程中可能会提示包找不到,建议先更新下软件源:

apt-get update

安装之后验证protobuf的时候可能会报错以下错误:

内容版权声明:除非注明,否则皆为本站原创文章。

转载注明出处:https://www.heiqu.com/99b8d6d6949aa7d51b6cd535c04a42e1.html