HBase之单机模式与伪分布式模式安装(5)

硬件环境:CentOS 6.5 服务器4台(一台为Master节点,三台为Slave节点)
软件环境:Java 1.7.0_45、Eclipse Juno Service Release 2、Hadoop-1.2.1、hbase-0.94.20。

1、Hbase API介绍 1)几个相关类与HBase数据模型之间的对应关系

HbaseAdmin用于数据库的创建与删除,HbaseConfiguration用于数据库的配置,Htable用数据库表的相关操作,HtableDescriptor用于数据库表列族的相关操作,Put用于数据库表记录的添加,Get用户数据库表记录的获取,Scanner用于数据库表全表查询

hbase-api

2)HBaseConfiguration

类名:org.apache.hadoop.hbase.HBaseConfiguration
作用:对HBase进行配置
常用方法:void set(String name, String value),通过属性名来设置值。
用法示例:

Configuration conf = HBaseConfiguration.create(); conf.set("hbase.zookeeper.quorum", "Master"); 3)HBaseAdmin

类名:org.apache.hadoop.hbase.client.HBaseAdmin
作用:提供了一个接口来管理HBase数据库的表信息。它提供的方法包括:创建表,删除表,列出表项,使表有效或无效,以及添加或删除表列族成员等。

hbase-HBaseAdmin

4)HTableDescriptor

类名:org.apache.hadoop.hbase.HTableDescriptor
作用:包含了表的名字极其对应表的列族。
常用方法:void addFamily(HcolumnDescriptor family) 添加一个列族。其详细用法如下所示,向tb_user表中添加了一个content列族。

HTableDescriptor tableDescriptor = new HTableDescriptor("tb_user"); HColumnDescriptor col = new HColumnDescriptor("content:"); tableDescriptor.addFamily(col); 5)HColumnDescriptor

类名:org.apache.hadoop.hbase.HColumnDescriptor
作用:维护着关于列族的信息,例如版本号,压缩设置等。它通常在创建表或者为表添加列族的时候使用。列族被创建后不能直接修改,只能通过删除然后重新创建的方式。列族被删除的时候,列族里面的数据也会同时被删除。

6)HTable

类名:org.apache.hadoop.hbase.client.HTable
作用:可以用来和HBase表直接通信。此方法对于更新操作来说是非线程安全的。
用法示例:

HTable table = null; ResultScanner rs = null; try { Scan scan = new Scan(); table = new HTable(conf, tableName); rs = table.getScanner(scan); } catch (IOException e) { e.printStackTrace(); }

hbase-HTable

7) Put

类名:org.apache.hadoop.hbase.client.Put
作用:用来对单个行执行添加操作
用法示例:

HTable table = null; try { table = new HTable(conf, tabelName); Put putRow1 = new Put(rowKey.getBytes()); putRow1.add(family.getBytes(), qualifier.getBytes(),value.getBytes()); table.put(putRow1); table.close(); } catch (IOException e) { e.printStackTrace(); }

hbase-Put

8)Get

类名:org.apache.hadoop.hbase.client.Get
作用:用来获取单个行的相关信息
用法示例:

Get query = new Get(rowKey.getBytes()); query.addColumn(Bytes.toBytes(family), Bytes.toBytes(qualifier)); if(table.exists(query)){ Result result = table.get(query); System.out.format("ROW\t%s\n",new String(result.getRow())); for(KeyValue kv : result.raw()){ System.out.format("COLUMN\t %S:%s\t%s\n",new String(kv.getFamily()),new String(kv.getQualifier()),new String(kv.getValue())); } }

hbase-Get

2、HBase API实战 1)创建表

create(String tableName,String… families)实现了创建名为tablename的表,同时添加若干families列族,列族个数不定,详细源码请参考:HBaseAPI/src/com/zonesion/hbase/CreateTable.java。

public static void create(String tableName,String... families) { Configuration conf = HBaseConfiguration.create(); conf.set("hbase.zookeeper.quorum", "Master"); HBaseAdmin admin = null; try { admin = new HBaseAdmin(conf); HTableDescriptor tableDescriptor = new HTableDescriptor( tableName.getBytes()); if (admin.tableExists(tableName)) {// 表存在 System.out.println(tableName + "已经存在!"); } else { for(String family :families){ tableDescriptor.addFamily(new HColumnDescriptor(family)); } admin.createTable(tableDescriptor); } } catch (MasterNotRunningException e) { e.printStackTrace(); } } 2)添加记录

put(String tabelName,String rowKey,String family,String qualifier,String value)实现了向tablename表主键为rowkey、列族为family、列为qualifier添加值为value的记录,详细源码请参考:HBaseAPI/src/com/zonesion/hbase/PutRow.java。

public static void put(String tabelName,String rowKey,String family,String qualifier,String value) { Configuration conf = HBaseConfiguration.create(); conf.set("hbase.zookeeper.quorum", "Master"); HTable table = null; try { table = new HTable(conf, tabelName); Put putRow1 = new Put(rowKey.getBytes()); putRow1.add(family.getBytes(), qualifier.getBytes(),value.getBytes()); table.put(putRow1); table.close(); } catch (IOException e) { e.printStackTrace(); } } 3)获取记录

getRow(String tableName,String rowKey) 实现了获取tablename表主键为rowkey的所有记录,详细源码请参考:HBaseAPI/src/com/zonesion/hbase/GetRow.java。

public static void getRow(String tableName,String rowKey) { Configuration conf = HBaseConfiguration.create(); conf.set("hbase.zookeeper.quorum", "Master"); HTable table = null; try { table = new HTable(conf, tableName); Get query = new Get(rowKey.getBytes()); if(table.exists(query)){ Result result = table.get(query); System.out.format("ROW\t%s\n",new String(result.getRow())); for(KeyValue kv : result.raw()){ System.out.format("COLUMN\t %S:%s\t%s\n",new String(kv.getFamily()),new String(kv.getQualifier()),new String(kv.getValue())); } } table.close(); } catch (IOException e) { e.printStackTrace(); } } 4)遍历表

list(String tableName)实现了遍历表tableName,并打印所有遍历的记录,详细源码请参考:HBaseAPI/src/com/zonesion/hbase/GetScanner.java。

public static void list(String tableName) { Configuration conf = HBaseConfiguration.create(); conf.set("hbase.zookeeper.quorum", "Master"); HTable table = null; ResultScanner rs = null; try { Scan scan = new Scan(); table = new HTable(conf, tableName); rs = table.getScanner(scan); for(Result row : rs){ System.out.format("ROW\t%s\n",new String(row.getRow())); for(Map.Entry<byte[], byte[]> entry : row.getFamilyMap("info".getBytes()).entrySet()){ String column = new String(entry.getKey()); String value = new String(entry.getValue()); System.out.format("COLUMN\t info:%s\t%s\n",column,value); } for(Map.Entry<byte[], byte[]> entry : row.getFamilyMap("address".getBytes()).entrySet()){ String column = new String(entry.getKey()); String value = new String(entry.getValue()); System.out.format("COLUMN\t address:%s\t%s\n",column,value); } } } catch (IOException e) { e.printStackTrace(); } }

更多详细的用法请参考:HBaseAPI/src/com/zonesion/hbase目录,其中CreateTable.java用于创建表,DeleteTable.java用于删除表,FilterQuery.java用于过滤查询,GetColumn.java用于获取指定列记录,GetFamily.java用于获取指定列族的所有记录,GetRow.java用于获取指定rowkey的所有记录,GetScanner.java用于遍历表,PutRow.java用于添加记录,HBaseAPI.java整合了上述所有资源以命令的形式进行调用,在实验过程中我们使用的HBaseAPI.java命令调用方式,用户也可单独执行以上所有资源。

3、部署运行 1) 启动Hadoop集群和Hbase服务 [hadoop@K-Master ~]$ start-dfs.sh #启动hadoop HDFS文件管理系统 [hadoop@K-Master ~]$ start-mapred.sh #启动hadoop MapReduce分布式计算服务 [hadoop@K-Master ~]$ start-hbase.sh #启动Hbase [hadoop@K-Master ~]$ jps #查看进程 22003 HMaster 10611 SecondaryNameNode 22226 Jps 21938 HQuorumPeer 10709 JobTracker 22154 HRegionServer 20277 Main 10432 NameNode

特别注意:用户可先通过jps命令查看Hadoop集群和Hbase服务是否启动,如果Hadoop集群和Hbase服务已经启动,则不需要执行此操作。

2)部署源码 #设置工作环境 [hadoop@K-Master ~]$ mkdir -p /usr/hadoop/workspace/Hbase #部署源码 将HBaseAPI 文件夹拷贝到/usr/hadoop/workspace/Hbase/ 路径下;

… 你可以直接 下载 HBaseAPI

------------------------------------------分割线------------------------------------------

免费下载地址在

用户名与密码都是

具体下载目录在 /2015年资料/3月/8日/HBase入门基础教程/

下载方法见

------------------------------------------分割线------------------------------------------

3)修改配置文件

a)查看hbase核心配置文件hbase-site.xml的hbase.zookeeper.quorum属性
使用如下命令查看hadoop核心配置文件hbase-site.xml的hbase.zookeeper.quorum属性值,当前K-Master服务器的hbase.zookeeper.quorum属性值为K-Master;

[hadoop@K-Master ~]$ cat /usr/hbase/conf/hbase-site.xml <?xml version="1.0"?> <?xml-stylesheet type="text/xsl" href="https://www.linuxidc.com/configuration.xsl"?> <!-- Put site-specific property overrides in this file. --> <configuration> <!--zookeeper运行的机器,要为奇数个,使得投票更公平--> <property> <name>hbase.zookeeper.quorum</name> <value>K-Master</value> </property> </configuration>

b)修改项目HBaseAPI/src/config.properties属性文件
将项目HBaseAPI/src/config.properties属性文件的hbase.zookeeper.quorum属性值修改为上一步查询到的属性值,保持config.properties文件的hbase.zookeeper.quorum属性值与hbase-site.xml文件的hbase.zookeeper.quorum属性值一致;

#切换工作目录 [hadoop@K-Master ~]$ cd /usr/hadoop/workspace/Hbase/HBaseAPI/ #修改属性值 [hadoop@K-Master HBaseAPI]$ vim src/config.properties hbase.zookeeper.quorum=K-Master #拷贝src/config.properties到bin文件夹 [hadoop@K-Master HBaseAPI]$ cp src/config.properties bin/ 4)编译文件 #执行编译 [hadoop@K-Master HBaseAPI]$ javac -classpath /usr/hadoop/hadoop-core-1.2.1.jar:/usr/hadoop/lib/commons-cli-1.2.jar:lib/zookeeper-3.4.5.jar:lib/hbase-0.94.20.jar -d bin/ src/com/zonesion/hbase/*.java #查看编译是否成功 [hadoop@K-Master HBaseAPI]$ ls bin/com/zonesion/hbase/ -la total 52 drwxrwxr-x 2 hadoop hadoop 4096 Dec 30 16:18 . drwxrwxr-x 3 hadoop hadoop 4096 Dec 30 16:18 .. -rw-rw-r-- 1 hadoop hadoop 3283 Dec 30 16:18 CreateTable.class -rw-rw-r-- 1 hadoop hadoop 2702 Dec 30 16:18 DeleteTable.class -rw-rw-r-- 1 hadoop hadoop 3781 Dec 30 16:18 FilterQuery.class -rw-rw-r-- 1 hadoop hadoop 2870 Dec 30 16:18 GetColumn.class -rw-rw-r-- 1 hadoop hadoop 2789 Dec 30 16:18 GetFamily.class -rw-rw-r-- 1 hadoop hadoop 2511 Dec 30 16:18 GetRow.class -rw-rw-r-- 1 hadoop hadoop 3519 Dec 30 16:18 GetScanner.class -rw-rw-r-- 1 hadoop hadoop 3085 Dec 30 16:18 HBaseAPI.class -rw-rw-r-- 1 hadoop hadoop 4480 Dec 30 16:18 PropertiesHelper.class -rw-rw-r-- 1 hadoop hadoop 2148 Dec 30 16:18 PutRow.class 5)打包Jar文件 #拷贝lib文件夹到bin文件夹 [hadoop@K-Master HBaseAPI]$ cp –r lib/ bin/ #打包Jar文件 [hadoop@K-Master HBaseAPI]$ jar -cvf HBaseAPI.jar -C bin/ . added manifest adding: lib/(in = 0) (out= 0)(stored 0%) adding: lib/zookeeper-3.4.5.jar(in = 779974) (out= 721150)(deflated 7%) adding: lib/protobuf-java-2.4.0a.jar(in = 449818) (out= 420864)(deflated 6%) adding: lib/hbase-0.94.20.jar(in = 5475284) (out= 5038635)(deflated 7%) adding: com/(in = 0) (out= 0)(stored 0%) adding: com/zonesion/(in = 0) (out= 0)(stored 0%) adding: com/zonesion/hbase/(in = 0) (out= 0)(stored 0%) adding: com/zonesion/hbase/GetScanner.class(in = 2681) (out= 1422)(deflated 46%) adding: com/zonesion/hbase/GetColumn.class(in = 2229) (out= 1127)(deflated 49%) adding: com/zonesion/hbase/DeleteTable.class(in = 2058) (out= 1162)(deflated 43%) adding: com/zonesion/hbase/CreateTable.class(in = 2534) (out= 1395)(deflated 44%) adding: com/zonesion/hbase/HBaseAPI.class(in = 2947) (out= 1319)(deflated 55%) adding: com/zonesion/hbase/FilterQuery.class(in = 3114) (out= 1540)(deflated 50%) adding: com/zonesion/hbase/GetRow.class(in = 1914) (out= 1017)(deflated 46%) adding: com/zonesion/hbase/PutRow.class(in = 1600) (out= 888)(deflated 44%) adding: com/zonesion/hbase/GetFamily.class(in = 2170) (out= 1106)(deflated 49%) 6)运行实例

a)Help命令

[hadoop@K-Master HBaseAPI]$ hadoop jar HBaseAPI.jar com.zonesion.hbase.HBaseAPI HBaseAPI action ... create <tableName> [family...] delete <tableName> put <tableName> <rowKey> <family> <column> <value> scan <tableName> get <tableName> <rowKey> get <tableName> <rowKey> <family> get <tableName> <rowKey> <family> <column>

b)create命令

#查看create帮助命令 [hadoop@K-Master HBaseAPI]$ hadoop jar HBaseAPI.jar com.zonesion.hbase.HBaseAPI create create <tableName> [family...] #创建tb_admin表,其中该表包含info、address两个列族 [hadoop@K-Master HBaseAPI]$ hadoop jar HBaseAPI.jar com.zonesion.hbase.HBaseAPI create tb_admin info address

c)put命令

#查看put帮助命令 [hadoop@K-Master HBaseAPI]$ hadoop jar HBaseAPI.jar com.zonesion.hbase.HBaseAPI put put <tableName> <rowKey> <family> <column> <value> #使用put命令向表为tb_admin、<rowkey>是'andieguo'添加一系列记录 [hadoop@K-Master HBaseAPI]$ hadoop jar HBaseAPI.jar com.zonesion.hbase.HBaseAPI put tb_admin andieguo info age 25 [hadoop@K-Master HBaseAPI]$ hadoop jar HBaseAPI.jar com.zonesion.hbase.HBaseAPI put tb_admin andieguo info birthday 1990-09-08 [hadoop@K-Master HBaseAPI]$ hadoop jar HBaseAPI.jar com.zonesion.hbase.HBaseAPI put tb_admin andieguo info company zonesion [hadoop@K-Master HBaseAPI]$ hadoop jar HBaseAPI.jar com.zonesion.hbase.HBaseAPI put tb_admin andieguo address country China [hadoop@K-Master HBaseAPI]$ hadoop jar HBaseAPI.jar com.zonesion.hbase.HBaseAPI put tb_admin andieguo address province hubei [hadoop@K-Master HBaseAPI]$ hadoop jar HBaseAPI.jar com.zonesion.hbase.HBaseAPI put tb_admin andieguo address city wuhan

d)scan命令

#查看scan帮助命令 [hadoop@K-Master HBaseAPI]$ hadoop jar HBaseAPI.jar com.zonesion.hbase.HBaseAPI scan scan <tableName> #使用scan命令查看tb_admin表里的所有记录 [hadoop@K-Master HBaseAPI]$ hadoop jar HBaseAPI.jar com.zonesion.hbase.HBaseAPI scan tb_admin ROW andieguo COLUMN info:age 25 COLUMN info:birthday 1990-09-08 COLUMN info:company zonesion COLUMN address:city wuhan COLUMN address:country China COLUMN address:province hubei

e)get命令

#查看get帮助命令 [hadoop@K-Master HBaseAPI]$ hadoop jar HBaseAPI.jar com.zonesion.hbase.HBaseAPI get get <tableName> <rowKey> get <tableName> <rowKey> <family> get <tableName> <rowKey> <family> <column> #使用get命令查看表tb_admin中rowkey为andieguo的所有记录 [hadoop@K-Master HBaseAPI]$ hadoop jar HBaseAPI.jar com.zonesion.hbase.HBaseAPI get tb_admin andieguo ROW andieguo COLUMN ADDRESS:city wuhan COLUMN ADDRESS:country China COLUMN ADDRESS:province hubei COLUMN INFO:age 25 COLUMN INFO:birthday 1990-09-08 COLUMN INFO:company zonesion #使用get命令查看表tb_admin中rowkey为andieguo、列族为info的所有记录 [hadoop@K-Master HBaseAPI]$ hadoop jar HBaseAPI.jar com.zonesion.hbase.HBaseAPI get tb_admin andieguo info ROW andieguo COLUMN INFO:age 25 COLUMN INFO:birthday 1990-09-08 COLUMN INFO:company zonesion #使用get命令查看表tb_admin中rowkey为andieguo、info为age的所有记录 [hadoop@K-Master HBaseAPI]$ hadoop jar HBaseAPI.jar com.zonesion.hbase.HBaseAPI get tb_admin andieguo info age ROW andieguo COLUMN INFO:age 25

f)delete命令

#查看delete帮助命令 [hadoop@K-Master HBaseAPI]$ hadoop jar HBaseAPI.jar com.zonesion.hbase.HBaseAPI delete delete <tableName> #使用delete命令删除tb_admin表 [hadoop@K-Master HBaseAPI]$ hadoop jar HBaseAPI.jar com.zonesion.hbase.HBaseAPI delete tb_admin 14/12/19 19:46:37 INFO client.HBaseAdmin: Disabled tb_admin 14/12/19 19:46:38 INFO client.HBaseAdmin: Deleted tb_admin 关闭HBaseAdmin

内容版权声明:除非注明,否则皆为本站原创文章。

转载注明出处:https://www.heiqu.com/6690979460adcf00c692f31564e2097a.html