Introduction to the Hive for Hadoop notes (architecture of Hive)

<title>Getting Started with Hive (ii)</title> Getting Started with Hive (ii) metadata for hive architecture 0Hive

• Hive stores metadata in the database (Metastore), supports databases such as MySQL, Derby, and Oracle, and Hive defaults to the Derby database
• The metadata in hive includes the name of the table, the columns and partitions of the table and their properties, the properties of the table (whether external tables, etc.), the directory where the table's data resides, and so on
  

0HQL Execution Process

• The interpreter, compiler, optimizer completes the generation of HQL query statements from lexical analysis, parsing, compiling, optimization, and Query Planning (PLAN) . The generated query plan is stored in HDFs, and then a mapreduce call is executed

Detailed process:

1. Hql--select
2. Parser-Lexical analysis
3. Compiler--Generate HQL Execution plan ( javac java–>. Class)
4. Optimizer-Generate the best execution plan
5. Perform

We have an employee information sheet that goes to Oracle:

SQL> ——查询10号部门的员工信息
forselectfromwhere deptno=10:

已解释。

SQL> ——查看该select的执行计划
selectfrom table(dbms_xplan.display):

Perform a full table scan, of course the cost of a full table scan is relatively high
The department number is indexed below

indexon emp(deptno):

索引已创建。

forselectfromwhere deptno=10:

已解释。

selectfrom table(dbms_xplan.display):

It's an index-based scan that's faster for full-table scanning.

It's almost like Oracle for Hive.

So:

0hadoop

• Use HDFS for storage and compute with MapReduce

0 Meta Data storage (Metastore)

• Typically stored in a relational database such as MySQL, Derby

Introduction to the Hive for Hadoop notes (architecture of Hive)