cdh版本的hue安装配置部署以及集成hadoop hbase hive mysql等权威指南

hue下载地址:https://github.com/cloudera/hue

hue学习文档地址

我目前使用的是hue-3.7.0-cdh5.3.6

hue(HUE=Hadoop User Experience)

Hue是一个开源的Apache Hadoop UI系统,由Cloudera Desktop演化而来,最后Cloudera公司将其贡献给Apache基金会的Hadoop社区,它是基于Python Web框架Django实现的。

通过使用Hue我们可以在浏览器端的Web控制台上与Hadoop集群进行交互来分析处理数据,例如操作HDFS上的数据,运行MapReduce Job,执行Hive的SQL语句,浏览HBase数据库等等。

hue特点:

  能够支持各个版本的hadoop
  hue默认数据库:sql lite
  文件浏览器:对数据进行增删改查
  hue下载src包进行一次编译,二次编译,在这用的是已经一次编译

hue部署:

1、下载依赖包:yum源安装

sudo yum -y install ant asciidoc cyrus-sasl-devel cyrus-sasl-gssapi gcc gcc-c++ krb5-devel libtidy libxml2-devel libxslt-devel mvn mysql mysql-devel openldap-devel python-devel sqlite-devel openssl-devel  

sudo yum install ant asciidoc cyrus-sasl-devel cyrus-sasl-gssapi cyrus-sasl-plain gcc gcc-c++ krb5-devel libffi-devel libxml2-devel libxslt-devel make mysql mysql-devel openldap-devel python-devel sqlite-devel gmp-devel

2、解压hue tar包

  tar -zxvf hue-3.7.0-cdh5.3.6.tar.gz -C /指定的目录

3、二次编译

  进入hue目录:执行make apps  会出现个build目录

错误:(centos 7会有)
error: static declaration of ‘X509_REVOKED_dup’ follows non-static declaration
static X509_REVOKED * X509_REVOKED_dup(X509_REVOKED *orig) {
^
In file included from /usr/include/openssl/ssl.h:156:0,
from OpenSSL/crypto/x509.h:17,
from OpenSSL/crypto/crypto.h:30,
from OpenSSL/crypto/crl.c:3:
/usr/include/openssl/x509.h:751:15: note: previous declaration of ‘X509_REVOKED_dup’ was here
X509_REVOKED *X509_REVOKED_dup(X509_REVOKED *rev);
^
error: command 'gcc' failed with exit status 1

给下面两个删掉:/usr/include/openssl/x509.h -》751、752行
X509_REVOKED *X509_REVOKED_dup(X509_REVOKED *rev);
X509_REQ *X509_REQ_dup(X509_REQ *req);
##必须删掉,注释不行

4、进入到hue-3.7.0-cdh5.3.6/desktop/conf

配置hue.ini文件:

secret_key=jFE93j;2[290-eiw.KEiwN2s3['d;/.q[eIW^y#e=+Iei*@Mn<qW5o
http_host=hadoop01.xningge.com
http_port=8888
time_zone=Asia/Shanghai

cdh版本的hue安装配置部署以及集成hadoop hbase hive mysql等权威指南

5、启动hue

  两种方式

1-->cd build/env/bin---》./supervisor
2-->build/env/bin/supervisor

 6、浏览器访问hue

  主机名+8888
  创建用户名和密码

cdh版本的hue安装配置部署以及集成hadoop hbase hive mysql等权威指南

hue和hadoop的组件配置

1、hdfs的配置

  在hdfs-site.xml中配置

<property>
  <name>dfs.webhdfs.enabled</name>
  <value>true</value>
</property>

 

  在core-site.xml中配置

  

<property>
  <name>hadoop.proxyuser.hue.hosts</name>
  <value>*</value>
</property>
<property>
  <name>hadoop.proxyuser.hue.groups</name>
  <value>*</value>
</property>

2、重新启动hdfs进程

  sbin/start-yarn.sh

3、hue配置

  [[hdfs_clusters]]
  # HA support by using HttpFs

[[[default]]]
# Enter the filesystem uri
fs_defaultfs=hdfs://hadoop01.xningge.com:8020

# NameNode logical name.
## logical_name=

# Use WebHdfs/HttpFs as the communication mechanism.
# Domain should be the NameNode or HttpFs host.
# Default port is 14000 for HttpFs.
webhdfs_url=http://hadoop01.xningge.com:50070/webhdfs/v1

# This is the home of your Hadoop HDFS installation
hadoop_hdfs_home=http://www.likecs.com/opt/modules/cdh/hadoop-2.5.0-cdh5.3.6

# Use this as the HDFS Hadoop launcher script
hadoop_bin=http://www.likecs.com/opt/modules/cdh/hadoop-2.5.0-cdh5.3.6/bin

# Change this if your HDFS cluster is Kerberos-secured
## security_enabled=false

# Default umask for file and directory creation, specified in an octal value.
## umask=022

# Directory of the Hadoop configuration
hadoop_conf_dir=http://www.likecs.com/opt/modules/cdh/hadoop-2.5.0-cdh5.3.6/etc/hadoop

[[yarn_clusters]]

[[[default]]]
# Enter the host on which you are running the ResourceManager
resourcemanager_host=hadoop01.xningge.com

# The port where the ResourceManager IPC listens on
resourcemanager_port=8032

# Whether to submit jobs to this cluster
submit_to=True

# Resource Manager logical name (required for HA)
## logical_name=

# Change this if your YARN cluster is Kerberos-secured
## security_enabled=false

# URL of the ResourceManager API
resourcemanager_api_url=http://hadoop01.xningge.com:8088

# URL of the ProxyServer API
proxy_api_url=http://hadoop01.xningge.com:8088

# URL of the HistoryServer API
history_server_api_url=http://hadoop01.xningge.com:19888

eg:此配置都是伪分布式模式

4、启动hue服务

  build/env/bin/supervisor

hue与hive配置

1、hive配置

  在hive-site.xml配置

内容版权声明:除非注明,否则皆为本站原创文章。

转载注明出处:https://www.heiqu.com/zzffzj.html