RedHat下搭建Hive环境
1、首先要确认Hadoop和hive的版本,参考如下:
Hive-0.10.0对应Hadoop 0.20.x, 0.23.x.y, 1.x.y, 2.x.y
Hive-0.9.0 对应Hadoop 0.20.x and 1.x
Hive-0.8.1对应Hadoop0.20.1 and 0.20.2
2、下载必备软件
hadoop-1.1.1.tar.gz
hive-0.10.0.tar.gz
JDK1.6
mysql-connector-java-5.1.18-bin.jar
3、安装并设置环境变量
Ø 新建系统用户uhive
Ø 安装JDK
Ø 将hadoop和hive解压到uhive用户的根路径下
Ø 配置环境变量
在用户uhive的根路径下的.bash_profile文件中添加如下信息:
exportJAVA_HOME=/home/weblogic/jdk160_14_R27.6.5-32/jre
export HADOOP_HOME=/home/uhive/hadoop
export HIVE_HOME=/home/uhive/hive
export HIVE_CONF_DIR=$HIVE_HOME/conf
export HIVE_LIB=$HIVE_HOME/lib
exportCLASSPATH=$HIVE_HOME/lib:$JAVA_HOME/lib:$JAVA_HOME/jre/lib:$HADOOP_HOME
exportPATH=$HIVE_HOME/bin:$HADOOP_HOME/bin:$JAVA_HOME/bin:/sbin/:/bin:$PATH
其中jdk的路径以个人安装路径为准;
将mysql-connector-java-5.1.18-bin.jar文件拷贝到$HIVE_HOME/lib路径下;
Ø 建立分布式文件系统
/home/uhive/hadoop/bin/hadoop fs -ls /home/uhive/fsdir
Ø 建立mysql用户
mysql> create user 'hive' identified by'hive';
mysql> grant all privileges on *.* to 'hive'@'%' with grant option;
mysql> grant all privileges on *.* to 'hive'@'localhost' with grant option;
mysql> update mysql.user setpassword=password('hive') where User="hive" andHost="localhost";
mysql> flush privileges;
mysql> create database hive;
Ø 修改配置文件
进入$HIVE_HOME/conf目录下,执行如下命令:
cp hive-default.xml.template hive-default.xml
cp hive-exec-log4j.properties.template hive-exec-log4j.properties
cp hive-log4j.properties.template hive-log4j.properties
建立路径home/uhive/fsdir
建立文件hive-site.xml,内容如下:
<?xml version="1.0"?>
<?xml-stylesheettype="text/xsl" href="https://www.linuxidc.com/configuration.xsl"?>
<configuration>
<property>
<name>hive.metastore.local</name>
<value>true</value>
</property>
<property>
<name>javax.jdo.option.ConnectionURL</name>
<value>jdbc:mysql://localhost:3306/hive?createDatabaseIfNotExist=true</value>
</property>
<property>
<name>javax.jdo.option.ConnectionDriverName</name>
<value>com.mysql.jdbc.Driver</value>
</property>
<property>
<name>javax.jdo.option.ConnectionUserName</name>
<value>hive</value>
</property>
<property>
<name>javax.jdo.option.ConnectionPassword</name>
<value>hive</value>
</property>
<property>
<name>datanucleus.fixedDatastore</name>
<value>false</value>
</property>
<property>
<name>hive.metastore.warehouse.dir</name>
<value>/home/uhive/fsdir/warehouse</value>
<description>location of default database for thewarehouse</description>
</property>
</configuration>
Ø 使用验证
A、 使用hive进入命令行,如果出现如下信息:
WARNING:org.apache.hadoop.metrics.jvm.EventCounter is deprecated. Please useorg.apache.hadoop.log.metrics.EventCounter in all the log4j.properties files
请在hive-log4j.properties中修改将log4j.appender.EventCounter的值修改为org.apache.hadoop.log.metrics.EventCounter
B、 创建数据表create table hwz2(id int, name string) rowformat delimited FIELDS TERMINATED BY ',';
C、 加载数据load data local inpath'/home/uhive/testdir/hwz2.txt' overwrite into table hwz2;