2. Hive Introduction

One: Hive basic concept                                    

1. Hive is a Hadoop-based data warehousing tool that maps a structured data file into a database table and provides full SQL query functionality that can be translated into a mapreduce task to run.

The advantage is that the learning cost is low, the simple mapreduce statistics can be quickly realized through the class SQL statements, and it is very suitable for the statistical analysis of data Warehouse without developing specialized mapreduce applications.

2. Hive is the SQL parsing engine that translates SQL statements into m/r jobs and executes them in Hadoop.

3. The Hive table is actually an HDFs directory/file, separating the folders by the table name. If it is a partitioned table, the partition value is a subfolder that can be used directly in the M/R job.

Hive's awesome place is:

1. Based on MapReduce, supports SQL syntax

2. There is no format requirement for data uploaded to the Data Warehouse

II: The system architecture of Hive

1. There are three main user interfaces: Cli,jdbc/odbc and WebUI

1) CLI, Shell command line

2) JDBC/ODBC is the Java of Hive, similar to the way you use traditional database JDBC

3) WebGui is a browser access Hive

2. Hive stores metadata in the database (Metastore)

Currently only MySQL, Derby is supported.

The metadata in Hive includes the name of the table, the columns and partitions of the table and their properties, the properties of the table (whether external tables, etc.), the directory where the table's data resides, and so on

3. Parsing, compiling and running of HQL

The interpreter, compiler, optimizer completes the generation of HQL query statements from lexical analysis, parsing, compiling, optimization, and query Planning (plan). The generated query plan is stored in HDFs, and then a mapreduce call is executed

4. Data storage location for Hive

In HDFs, most of the queries are done by MapReduce

(Query with *, exception: SELECT * FROM table does not generate Mapredcue task)

2. Hive Introduction