Hive and Impala load and store data function exposure

Hive and Impala are data query tools built on top of Hadoop, so how do they load and store data in real-world applications?

Hive and Impala store and load tables, like all relational databases, have their own data management structure, from its server to database to tables and views.

In other databases, tables are stored in their own specific file format, such as Oracle has its own storage format, and for hive, a table is an HDFs directory containing one or more files, which is the content under the table, the default storage path:/user/hive/ Warehouse/<table_name> support for multiple storage formats.

The above is the storage of data, then each table, each structure has its own column or type definition of information, how to save it? They are stored in metastore, and all the data is stored on HDFS, so we want to get the table structure information, we need to know the meaning and structure of each table in hive metadata. In hive, there are simple commands to approximate the structure of the table: describe formatted tableName; Hive Metastore The table structure is as follows:

650) this.width=650; "Src=" Http://s2.51cto.com/wyfs02/M01/8C/22/wKioL1hjXsOgsyH2AABjuxp9PmY517.png-wh_500x0-wm_3 -wmp_4-s_3348256513.png "title=" 11.png "alt=" Wkiol1hjxsogsyh2aabjuxp9pmy517.png-wh_50 "/>

because Hive and Impala use the same data , table in HDFs, metadata in Metastore, so the above storage and structure introduction also applies to Impala.

Sample Data loading and storage :

650) this.width=650; "Src=" Http://s4.51cto.com/wyfs02/M02/8C/26/wKiom1hjXtPAopMDAAFbDIZ58h4447.png-wh_500x0-wm_3 -wmp_4-s_2626090314.png "title=" 22.png "alt=" Wkiom1hjxtpaopmdaafbdiz58h4447.png-wh_50 "/>

Here we have to differentiate between the two concepts: data and meta data. Data refers to the information you store and process, such as billing records, sensor readings, and service logs. Metadata is used to describe the shape of the data, such as field names and order.

650) this.width=650; "Src=" Http://s1.51cto.com/wyfs02/M02/8C/22/wKioL1hjXuOyLoUbAAAtsSRx2mA404.png-wh_500x0-wm_3 -wmp_4-s_445803288.png "title=" 33.png "alt=" Wkiol1hjxuoyloubaaatssrx2ma404.png-wh_50 "/>

The introduction of hive and Impala, is the sharing of their own experience, hoping to help you understand and recognize the functions of hive and Impala, in the ordinary time we do technology can also pay more attention to some big data information. For example, the service number "Big Data cn", "Big Data Times Learning Center", for the improvement of our own knowledge structure is a great help and promotion.

This article is from the "11872756" blog, please be sure to keep this source http://11882756.blog.51cto.com/11872756/1886883

Hive and Impala load and store data function exposure