Hadoop external data file path Query

Source: Internet
Author: User

In Hive, appearance is a very important component, which facilitates data sharing.
Because normal tables copy data files to their own directories, you can only save multiple copies of data to share data.
But the appearance solves this problem well.

Create external table sunwg_test09 (ID int, name string)
Row format delimited
Fields terminated by '\ t'
Location '/sunwg/test08 ′;

In fact, the External table can not only specify the HDFS directory, but also the local directory.
For example:
Create external table test10 (ID int, name string)
Row format delimited
Fields terminated by '\ t'
Location 'file: // home/hjl/sunwg /';


There is no directory in the hive. External Store. Warehouse. Dir of the External table. If we create an external table, how can we find the data file? For all the table structures, we have selected the storage method for hive configuration. The most common method is to store them in MySQL.

The configuration snippets in hive-site.xml are as follows:

<Property> <Name> hive. MetaStore. Local </Name> <value> true </value> </property>
<Property> <Name> javax. JDO. Option. connectionurl </Name> <value> JDBC: mysql: // centos1: 3306/hive? Createdatabaseifnotexist = true </value> </property>
<Property> <Name> javax. JDO. Option. connectiondrivername </Name> <value> com. MySQL. JDBC. Driver </value> </property>
<Property> <Name> javax. JDO. Option. connectionusername </Name> <value> hadoop </value> </property>
<Property> <Name> javax. JDO. Option. connectionpassword </Name> <value> hadoop </value> </property>

This section is the storage address, user name, and password of the hive metadata we saved. We can use the mysql client to log on.

Mysql> select * From Tsung where tbl_name = 'sunwg _ test09 ′;
Error 2006 (hy000): MySQL server has gone away
No connection. Trying to reconnect...
Connection ID: 16
Current Database: hjl

+ --- + ----- + --- + ------ + --- + ---- + --- + ------ + ------- +
| Tbl_id | create_time | db_id | last_access_time | Owner | retention | sd_id | tbl_name | tbl_type | view_expanded_text | view_original_text |
+ --- + ----- + --- + ------ + --- + ---- + --- + ------ + ------- +
| 15 | 1299519817 | 1 | 0 | hjl | 0 | 15 | sunwg_test09 | external_table | null |
+ --- + ----- + --- + ------ + --- + ---- + --- + ------ + ------- +
1 row in SET (0.01 Sec)

The table type is external_table.

Mysql> select * from SDS where sd_id = 15;
+ --- + -------------- + ----- + ----------- + ----- + -------------------- + ---- +
| Sd_id | input_format | is_compressed | location | num_buckets | output_format | serde_id |
+ --- + -------------- + ----- + ----------- + ----- + -------------------- + ---- +
| 15 | Org. apache. hadoop. mapred. textinputformat | HDFS: // hadoop00: 9000/hjl/test08 |-1 | Org. apache. hadoop. hive. QL. io. hiveignorekeytextoutputformat | 15 |
+ --- + -------------- + ----- + ----------- + ----- + -------------------- + ---- +
1 row in SET (0.00 Sec)

The data file path of sunwg_test09 in Table SDS is HDFS: // hadoop00: 9000/hjl/test08.

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.