Hadoop's--hive Notes

Reprint Please specify source: http://blog.csdn.net/l1028386804/article/details/46496123

1.hive1.1 is the role of the Data Warehouse in the Hadoop ecosystem. He is able to manage data in Hadoop and can query data in Hadoop.
In essence, hive is a SQL parsing engine. Hive can run SQL queries into a job in MapReduce.
Hive has a mapping tool that transforms SQL into a job in mapreduce, converting tables and fields in SQL to files in HDFs (folders) and columns in files.
This set of mapping tools is called Metastore, usually stored in Derby, MySQL.
The default location for 1.2 hive in HDFs is/user/hive/warehouse, which is determined by the properties Hive.metastore.warehouse.dir in the configuration file Hive-conf.xml.
2.hive installation (1) Unzip, rename, set environment variables (2) Modify the configuration file

Under Directory $hive_home/conf/, perform the command MV hive-default.xml.template hive-site.xml Rename

Under Directory $hive_home/conf/, perform the command MV hive-env.sh.template hive-env.sh Rename
(3) Modify the configuration file for Hadoop hadoop-env.sh

The contents of the amendment are as follows:

Export hadoop_classpath=.: $CLASSPATH: $HADOOP _classpath: $HADOOP _home/bin
(4) Under directory $hive_home/bin, modify the file hive-config.sh

Add the following content:

Export JAVA_HOME=/USR/LOCAL/JDK
Export Hive_home=/usr/local/hive
Export Hadoop_home=/usr/local/hadoop
3. Install MySQL (1) to remove the MySQL-related library information that is already installed on Linux.

RPM-E xxxxxxx--nodeps

Execute command rpm-qa |grep mysql check whether to remove clean
(2) Execute command rpm-i mysql-server-******** install MySQL server (3) Start MySQL server, execute command mysqld_safe & (4) Execute command rpm-i MYSQL-CL ient-******** Install the MySQL client (5) Execute the command mysql_secure_installation set the root user password 4. Use MySQL as Hive's Metastore (1) to place the MySQL JDBC driver into the Lib directory of Hive (2) to modify the Hive-site.xml file

The contents of the amendment are as follows:

<property>        <name>javax.jdo.option.ConnectionURL</name>        <value>jdbc:mysql:// hadoop0:3306/hive?createdatabaseifnotexist=true</value>    </property>    <property>        <name>javax.jdo.option.ConnectionDriverName</name>        <value>com.mysql.jdbc.driver</ value>    </property>    <property>        <name>javax.jdo.option.connectionusername</ name>        <value>root</value>    </property>    <property>        <name> javax.jdo.option.connectionpassword</name>        <value>admin</value>    </property>

5. Internal Table CREATE table T1 (id int);
LOAD DATA LOCAL inpath '/root/id ' into TABLE T1;
CREATE TABLE T2 (id int, name string) ROW FORMAT delimited fields TERMINATED by ' \ t ';
6. partition Table CREATE table t3 (id int) partitioned by (day int);
LOAD DATA LOCAL inpath '/root/id ' into TABLE T1 PARTITION (day=22);
7. bucket table CREATE table t4 (ID int) clustered by (ID) into 4 buckets;
Set hive.enforce.bucketing = true;
Insert INTO table T4 Select id from T3;
8. External table Create external table t5 (id int) location '/external ';

Hadoop's--hive Notes