How to convert a relational data table to an hbase data table

How to convert a relational data table to an hbase data table

First, we need to increase the number of data tables in the relational database from vertical extension to horizontal extension of the HBase data table"

I. Hbase Storage Structure

A) HBase stores data in the form of tables (HTable)

B) HTable includes many rows. Each row is uniquely marked by RowKey, and rows are arranged in the lexicographically ordered order of RowKey. The table is divided into multiple HRegion rows.

C) each row contains one RowKey and multiple Column families. Data is physically cut according to Column Family, that is, data of different Column families is placed in different stores, and data of one Column Family is placed in one Strore.

D) HRegion consists of multiple stores. A Store is physically stored in one MemStrore (in memory) and multiple storefiles (hfiles ).

Ii. Design Principles:

(1) rowkey

A) rowkey is the key in hbase's key-value storage. Generally, the field to be queried is used as the rowkey.,Query Result As value,In HBase, rowkeys are arranged alphabetically.
 

(2) The design of Column Family should follow:

A) data of different Column families is physically separated, and the columns to be obtained by a single request are not distributed in different Column families;

B) The number of CF files should be as small as possible because too many columnfamily files will affect each other.

(3) column

For applications whose columns need to be extended, the column can be designed in a common way. For applications with relatively fixed columns, it is best to encapsulate a row of records into one column, this saves storage space. Protocolbuffer is recommended for encapsulation.

Iii. Example:

(1) Oracle relational data tables have 10 billion IOV metric data

X represents the longitude, y represents the latitude, creat_at: Creation Time, type: 1 represents the alarm point, 2 represents the normal point, org_id: Organization id

Required: a. query the trajectory of a specified vehicle on a specified day. B. Download all alarm points of a specified unit on a specified day,

A table structure of hbase is provided.

Rowkey	ColumnFamily
Car_id + create_at + org_id + type	Location 1 (x, y)	Location 2 (x, y)