Learn about hive and Impala must-see classic parsing

Hive and Impala as a data query tool, how do they query the data? What tools do we use to interact with Impala and hive?

We first make clear Hive and the Impala the interface for the corresponding query is provided separately:

(1) command Line Shell :

1. Impala : Impala Shell

2. Hive : Beeline (Early Hive the command-line version is Hive Shell , not used now)

(2) Hue Web UI :

1.Hue the inside provides Hive Query Editor

2.the Impala Query Editor is available in Hue

3. The Meta Data Manager is available in Hue and can be accessed directly from the metadata.

( 3 ) provides a JDBC and the ODBC Support

Here is a detailed description:

First, Impala

(1) Use Impala Shell

Impala Shellis similar toMySQLInteractive tool that can be started directly at the terminalImpala Shell, butImpala Shellwhere to start is very fastidious. Impalaitself is a distributed architecture, it'sImpaladis in eachslavenode in the. SoImpalaThe following is the case of direct execution.slavenode that is running in the.

650) this.width=650; "Src=" Http://s1.51cto.com/wyfs02/M01/8C/2E/wKioL1hkv4_TrMN6AACnWw5_3po611.png-wh_500x0-wm_3 -wmp_4-s_2206939752.png "title=" 11.png "alt=" Wkiol1hkv4_trmn6aacnww5_3po611.png-wh_50 "/>

if not in slave node, you can specify its Server , as in the following way, by - I. the parameters, specifying 21000 Port execution.

650) this.width=650; "Src=" Http://s3.51cto.com/wyfs02/M01/8C/32/wKiom1hkv5vjt8qQAABdJUftXlU853.png-wh_500x0-wm_3 -wmp_4-s_2625108586.png "title=" 22.png "alt=" Wkiom1hkv5vjt8qqaabdjuftxlu853.png-wh_50 "/>

Impala Shell and all the SQL , enter the semicolon as the end of the statement, using the quit Command Exit Shell . Use Impala-shell--help to see a complete list of options if you need to see Help. Because any tool has a lot of parameters, be sure to get used to it and learn to use it to find some of the parameters that you want, and then solve some complex data analysis and processing.

in the Impala Shell examples of executing queries inside:

650) this.width=650; "Src=" Http://s3.51cto.com/wyfs02/M02/8C/32/wKiom1hkv6vzYi92AAFZNJdkEuA294.png-wh_500x0-wm_3 -wmp_4-s_2279070837.png "title=" 33.png "alt=" Wkiom1hkv6vzyi92aafznjdkeua294.png-wh_50 "/>

(2) Impala interacting with the operating system

in the Impala inside, we sometimes need to perform some Linux commands, such as query date, remote connection, access to directory data, etc., we do not need to exit Impala Shell , direct operation is possible, for example:

650) this.width=650; "Src=" Http://s4.51cto.com/wyfs02/M01/8C/2E/wKioL1hkv7qirGzNAAA6YMVJVjo018.png-wh_500x0-wm_3 -wmp_4-s_2629519679.png "title=" 44.png "alt=" Wkiol1hkv7qirgznaaa6ymvjvjo018.png-wh_50 "/>

but Impala not directly supported HDFS command, but you can use the Shell Run HDFs DFS command to create a directory, as follows:

650) this.width=650; "Src=" Http://s2.51cto.com/wyfs02/M02/8C/32/wKiom1hkv8zTg2mmAAA5kcPLOFc819.png-wh_500x0-wm_3 -wmp_4-s_2337094073.png "title=" 55.png "alt=" Wkiom1hkv8ztg2mmaaa5kcplofc819.png-wh_50 "/>

(3) run from the command line Impala query, can not login Impala Shell can be executed, such as:

1. Use - F option to execute the file containing the query

650) this.width=650; "Src=" Http://s3.51cto.com/wyfs02/M00/8C/2E/wKioL1hkv96SazHrAAAHSFkBnZk507.png-wh_500x0-wm_3 -wmp_4-s_1169598003.png "title=" 66.png "alt=" Wkiol1hkv96sazhraaahsfkbnzk507.png-wh_50 "/>

2. Use - Q option to run the query directly on the command line

650) this.width=650; "Src=" Http://s3.51cto.com/wyfs02/M01/8C/32/wKiom1hkv_LQVkqjAAAHue8HSSY024.png-wh_500x0-wm_3 -wmp_4-s_3103692904.png "title=" 77.png "alt=" Wkiom1hkv_lqvkqjaaahue8hssy024.png-wh_50 "/>

3. Use - o to output the results to a file

650) this.width=650; "Src=" Http://s2.51cto.com/wyfs02/M01/8C/2E/wKioL1hkv_7QsO_zAAAN471mgLA206.png-wh_500x0-wm_3 -wmp_4-s_2141334432.png "title=" 88.png "alt=" Wkiol1hkv_7qso_zaaan471mgla206.png-wh_50 "/>

Second, Hive

( 1 ) Start Beeline

Hive Shellis aHiveearlier versions, now usingBeeline Shell, withImpala Shellsimilar, but it is based onJDBCand theODBC, if you need to useBeelinewords that need to be startedHive2, by providingHive2Server-SpecifiedURLto startBeeline, you also need to specify the user name and password as needed, as follows:

650) this.width=650; "Src=" Http://s2.51cto.com/wyfs02/M02/8C/2E/wKioL1hkwByDIvjHAAB2HOsV2q4667.png-wh_500x0-wm_3 -wmp_4-s_629505803.png "title=" 99.png "alt=" Wkiol1hkwbydivjhaab2hosv2q4667.png-wh_50 "/>

( 2 ) in Beeline Execute Query

and the SQL end With a semicolon, execute the query and Impala Shell similar, but the result format is slightly different, such as:

650) this.width=650; "Src=" Http://s3.51cto.com/wyfs02/M02/8C/32/wKiom1hkwDKQMuPEAAEV6z9Pa74192.png-wh_500x0-wm_3 -wmp_4-s_2543090050.png "title=" 10.png "alt=" Wkiom1hkwdkqmupeaaev6z9pa74192.png-wh_50 "/>

( 3 ) using Beeline

There are some differences with other tools, and executing queries is normal. SQL input, but if it is a managed command, such as making a connection, interrupting, exiting, executing Beeline commands need to be brought up! " ", no terminator is required. Common Command Description:

1 , !connect URL – Connect the different Hive2 Server

2 , !exit – Exit Shell

3 , !help – Show all Commands list

4 , !verbose – Show details for query append

Example :

650) this.width=650; "Src=" Http://s4.51cto.com/wyfs02/M00/8C/2E/wKioL1hkwFigOM55AAAIJCmNVHY999.png-wh_500x0-wm_3 -wmp_4-s_2901877024.png "title=" 101.png "alt=" Wkiol1hkwfigom55aaaijcmnvhy999.png-wh_50 "/>

( 4 ) is executed from the command line Hive Enquiry

1 , using - F option to execute the containing HiveQL Code of the file

650) this.width=650; "Src=" Http://s2.51cto.com/wyfs02/M01/8C/32/wKiom1hkwGmTEXsLAAAG6o2us28331.png-wh_500x0-wm_3 -wmp_4-s_1669666377.png "title=" 102.png "alt=" Wkiom1hkwgmtexslaaag6o2us28331.png-wh_50 "/>

2 , using - e option to run directly on the command line HiveQL

650) this.width=650; "Src=" Http://s3.51cto.com/wyfs02/M01/8C/2E/wKioL1hkwI_jSLprAAAH30ck2zQ110.png-wh_500x0-wm_3 -wmp_4-s_2968116554.png "title=" 103.png "alt=" Wkiol1hkwi_jslpraaah30ck2zq110.png-wh_50 "/>

3 , using --silent to prevent the message output from being notified, or to - e or - F option is used together with the

650) this.width=650; "Src=" Http://s3.51cto.com/wyfs02/M02/8C/2E/wKioL1hkwJ6B40q5AAAF-LFE4eY065.png-wh_500x0-wm_3 -wmp_4-s_1267766158.png "title=" 104.png "alt=" Wkiol1hkwj6b40q5aaaf-lfe4ey065.png-wh_50 "/>

Third, Hue

( 1 hue access Span style= "FONT-SIZE:19PX;" >hive and impala they each have editors hue query editors we can find the hive and impala

as follows :

650) this.width=650; "Src=" Http://s4.51cto.com/wyfs02/M00/8C/2E/wKioL1hkwLKxAQdcAAB7nvNthQs179.png-wh_500x0-wm_3 -wmp_4-s_3827362448.png "title=" 105.png "alt=" Wkiol1hkwlkxaqdcaab7nvnthqs179.png-wh_50 "/>

and the meta-database we can Data Browsers inside you can see some of its tools, such as:

650) this.width=650; "Src=" Http://s5.51cto.com/wyfs02/M01/8C/32/wKiom1hkwMPjVt6MAABKKT-1Qxg030.png-wh_500x0-wm_3 -wmp_4-s_2515345216.png "title=" 106.png "alt=" Wkiom1hkwmpjvt6maabkkt-1qxg030.png-wh_50 "/>

( 2 ) Hue Query Editor, querying Impala and the Hive Query Editor is almost the same, the specific operation of the schematic:

650) this.width=650; "Src=" Http://s3.51cto.com/wyfs02/M00/8C/32/wKiom1hkwNCCYSZRAAHWtFCRJaA357.png-wh_500x0-wm_3 -wmp_4-s_2081585811.png "title=" 107.png "alt=" Wkiom1hkwnccyszraahwtfcrjaa357.png-wh_50 "/>

with the Impala and the Hive interaction, these are the tools we can use and use, but in the actual operation we also need to strengthen their actual combat ability to better grasp and understanding. Big data as the current is still improving the development of technology, need every want to engage and have been engaged in its personnel continue to learn and accumulate, more need to communicate and share, and common progress. "Big Data cn", "Big Data Era Learning Center" is my usual self-learning process to find the service number, the introduction of big data on the knowledge of big data and the development of some industry knowledge is very good, ordinary people can see.

This article is from the "11872756" blog, please be sure to keep this source http://11882756.blog.51cto.com/11872756/1887331

Learn about hive and Impala must-see classic parsing