Xia mengzhu talks about the differences between hive vs. hbase

For users who are new to big data, it is difficult to distinguish hive from hbase. This article will try to analyze its own definitions, features, restrictions, and application scenarios for reference.

What is hive?

Apache hiveIs built on the top layer of hadoop (distributed system infrastructure ).Data WarehouseNote that this is not a database. Hive can be seen as a user programming interface, which does not store and compute data. It relies on HDFS (hadoop Distributed File System) and mapreduce (a programming model, ing and simplification; used for parallel big data operations ). Its operations on HDFS are similar to SQL-named hql. It provides a wide range of SQL query methods to analyze data stored in HDFS; after hql is compiled and converted into a mapreduce job, It queries and analyzes the required content through its own SQL; as a result, even if you are not familiar with mapreduce, you can use the SQL language to query, summarize, and analyze data. Mapreduce developers can use their own Mapper and reducer as plug-ins to support hive for more complex data analysis.

What is hbase?

Apache hbaseIt is a nosql (not only SQL) database system running on the top layer of HDFS. Unlike hive, hbase provides the instant read/write function and is a column-oriented database. Hbase stores data in the form of a table. The table consists of rows and columns, and the columns are divided into several row families ). For example, a message column cluster contains the sender, receiver, sending date, message title, and message content. Each key value in hbase is defined as a cell. The key consists of Row-key, column cluster, column, and timestamp. In hbase, each row represents a key-value ing combination identified by the row key. Hbase's goal is to increase computing and storage capabilities by increasing cheap commercial servers.

Features

The JDBC-compliant hive not only allows users with SQL knowledge to execute mapreduce jobs indirectly, but also integrates the current SQL-based operation tools. However, because the default data reading function is full table traversal, the time consumption is inevitably relatively large. Even so, the amount of data read through different hive partition methods can also be limited. The hive partition allows you to filter and query data stored in an independent file. The returned data is filtered data. For example, to access a log file of a date, the premise is that the file name of this type contains the date information.

Hbase stores data as key-value pairs. It includes four main data operations:

 

1. Add or update data rows
2. Scan to obtain cells in a certain range
3. Returns the corresponding cells for a specific data row.
4. Deletes a row or column from a data table.

 

The column information can be used to obtain values before data changes (you can delete the column information history to release the storage space through the hbase compression policy ).

Restrictions

Hive does not support regular SQL update statements, such as data insertion, update, and deletion. Because its operations on data target the entire data table. At the same time, this feature allows data to be queried in minutes or even hours. In addition, the mapreduce conversion process must follow the predefined conversion rules.

Hbase has a set of SQL-like operating languages for data query. In addition, Zookeeper is required to run hbase. Zookeeper is a reliable coordination system for large-scale distributed systems. It provides the following functions: configuration maintenance, Name Service, distributed synchronization, and group service.

Application Example

Hive is suitable for querying large volumes of data such as network logs and static data. For example, user consumption behavior records and Website access footprints. However, it is not applicable to online real-time online queries.

Hbase can be used for online real-time query of big data. For example, fackbook performs online real-time analysis on messages transmitted between users.

Summary

Hive and hbase are based on different hadoop technologies. Hive is an SQL programming interface that can execute mapreduce jobs. hbase is a non-relational database structure. Combined with the characteristics of the two, the combined use of the two may have a complementary effect. For example, hive is used to process static offline data, hbase is used for online real-time query, and then the result sets between the two are integrated and merged, so that the data is complete and always young, it provides good support for further business analysis.