Hive entry 3-Integration of Hive and HBase

Opening Remarks: Hive and HBase integration function is to use their own external API interface communication, mutual communication is mainly rely on hive_hbase-handler.jar tool class (HiveStorageHandlers: there is still some interest in hive_hbase-handler.jar this stuff, free to grind.

Opening Remarks: Hive and HBase integration function is to use their own external API interface communication, mutual communication is mainly rely on the hive_hbase-handler.jar tool class (Hive Storage Handlers: there is still some interest in hive_hbase-handler.jar this stuff, free to grind.

Opening remarks:
Hive and HBase integration function implementation is to use the two itself external API interface communication, mutual communication is mainly rely on the hive_hbase-handler.jar tool class (Hive Storage Handlers:

Saliva:
There is still some interest in hive_hbase-handler.jar this stuff, free to grind.

I. two considerations:
1. the required software includes Hadoop, Hive, Hbase, and Zookeeper. Integration of Hive and HBase requires the Hive version, so do not download it. earlier than 0.6.0 old version, Hive.0.6.0 version only support with HBase docking, so in the Hive lib directory can see more hive_hbase-handler.jar jar package, he is Hive extended storage Handler, we recommend that you use version 0.20.6 for HBase. This time I didn't start the HDFS cluster environment. All the testing environments are on one machine.

2. When running Hive, the following error may occur, indicating that your JVM is not allocated enough space,Error MessageAs follows:
Invalid maximum heap size:-Xmx4096m
The specified size exceeds the maximum representable size.
Cocould not create the Java virtual machine.

Solution:
/Work/hive/bin/ext # vim util/execHiveCmd. sh file contains row 33rd
Modify,
Hadoops _heapsize = 4096
Is
Hadoops _heapsize = 256

Add export $ HIVE_HOME =/work/hive to/etc/profile /.

Ii. Start the runtime environment
1. Start Hive
Hive-auxpath/work/hive/lib/hive_hbase-handler.jar,/work/hive/lib/hbase-0.20.3.jar,/work/hive/lib/zookeeper-3.2.2.jar-hiveconf hbase. master = 127.0.0.1: 60000
Load the tools required by Hive and point to the master server address of HBase. My HBase master server and Hive run on the same machine, so I point to the local machine.

2. Start HBase
/Work/hbase/bin/hbase master start

3. Start Zookeeper
/Work/zookeeper/bin/zkServer. sh start

Iii. Execution
Create a table in Hive and associate it with each other
Create table hbase_table_1 (key int, value string) stored by 'org. apache. hadoop. hive. hbase. hbasestoragehandler' with serdeproperties ("hbase. columns. mapping "=": key, cf1: val ") TBLPROPERTIES (" hbase. table. name "=" xyz ");

Run a table creation statement in Hive and import the data
Create a table
Create table pokes (foo INT, bar STRING );
Data Import
Load data local inpath'/work/hive/examples/files/kv1.txt 'Overwrite into table pokes;

Insert a data entry into the table associated with Hive and HBase.
Insert overwrite table hbase_table_1 SELECT * FROM pokes WHERE foo = 98;
After running successfully ,:

Data is inserted using the MapReduce policy algorithm and written to HBase at the same time ,:

Run the scan 'xyz' and describe "xyz" commands in HBase shell to view the table structure and run the following results:

Xyz is a table created in Hbase through Hive. The ing attribute is specified in the Hive table creation statement."Hbase. columns. mapping" = ": key, cf1: val"And create a table name in HBase"Hbase. table. name" = "xyz"

Insert a record in hbase when running the put command
Put 'xyz', '123', 'cf1: val', 'www .javabloger.com'

Run the query statement on hive to check whether the data inserted in hbase has been synchronized,
Select * from hbase_table_1 WHERE key = 10001;
:

Final Effect
The above integration process and operation steps have been completed. NowAdd records to HBase in Hive. You also add records to Hive in HBase.Indicates that Hive and HBase are successfully integrated. Can we write massive amounts of data in HBase and query the message in Hive? HBase does not support complex queries, but HBase can obtain one or more rows of data based on keys, scan data intervals, and filter operations. Complex queries can be completed by Hive, one as the storage entry (HBase) and the other as the query entry (Hive ). As shown in.


Haha, I laughed. The above is just my point of view.

In this case, I will continue to update later. Thank you for your reading.

Related Articles:
Apache Hive entry 2
Apache Hive Entry 1

HBase entry 4
HBase entry 3
HBase entry 2
HBase Basics

-End-

Original article address: Hive entry 3-Integration of Hive and HBase. Thank you for sharing it with me.