Hive-based hive Environment Construction

Hive default metadata information is stored in Derby. Derby's built-in relational database and single session (only single client connection is supported, and an error will be reported when two clients are connected in the past );

Hive supports storing metadata in relational databases, such as MySQL and Oracle;

In this case, the hive metadata is stored in MySQL. Therefore, you must first install the MySQL database and use centos6.4.

 

Install MySQL

Install using Yum:

Yum install MySQL # Install MYSQL client Yum install mysql-server # Install MySQL Server

Determine whether MySQL has been installed:

chkconfig --list|grep mysql

Start the MySQL service:

Service mysqld start
Or
/Etc/init. d/mysqld start

Check whether the MySQL service is enabled:

/etc/init.d/mysqld status

Set MySQL startup:

chkconfig mysqld on 

Check whether MySQL boot is configured successfully:

chkconfig --list|grep mysql

Display 2 3 4 5 as on indicates that the startup configuration is successful

Create a root administrator account for MySQL:

mysqladmin -uroot password root

Log on to MySQL

mysql -uroot -proot

 

Install hive

Install dependency

Dependencies required for hive installation:

1) JDK 6 or above;

2) hadoop installation is complete;

 

Installation version

Hive-0.12.0-cdh5.0.0.tar.gz

: Http://archive.cloudera.com/cdh5/cdh/5/

Generally, hive is installed on namenode.

 

Installation Steps

1) decompress

tar -zxvf hive-0.12.0-cdh5.0.0.tar.gz -C /app

2) modify the hive configuration file

CD/APP/hive-0.12.0-cdh5.0.0/confcp hive-env.sh.template hive-env.shvi hive-env.sh
Hadoop_home =/APP/hadoop-2.3.0-cdh5.0.0 # multiple hadoop clusters need to manually point to the hadoop cluster you need export hive_conf_dir =/APP/hive-0.12.0-cdh5.0.0/conf # multiple hive clusters need to manually point to the conf to install hive directory

Hive-site.xml configured as follows

If the hive-site.xml file does not exist, copy one from the hive-default.xml.template

CP hive-default.xml.template hive-site.xml

<?xml version="1.0"?><?xml-stylesheet type="text/xsl" href="configuration.xsl"?><configuration>    <property>      <name>javax.jdo.option.ConnectionURL</name>      <value>jdbc:mysql://localhost:3306/hive?createDatabaseIfNotExist=true</value>    </property>

<property>      <name>javax.jdo.option.ConnectionDriverName</name>      <value>com.mysql.jdbc.Driver</value>    </property>    <property>      <name>javax.jdo.option.ConnectionUserName</name>      <value>root</value>    </property>    <property>      <name>javax.jdo.option.ConnectionPassword</name>      <value>root</value>    </property></configuration>

Note: The Hive database does not need to be created manually. After hive is started properly, the hive database is automatically created. After hive is started successfully, a series of tables are automatically created in the hive database of MySQL;

The default data storage directory of hive is/user/hive/warehouse. You can also manually specify

<property>　　<name>hive.metastore.warehouse.dir</name>　　<value>/user/hive/warehouse</value></property>

3) copy the MySQL driver package to hive/lib

cp /home/hadoop/software/mysql-connector-java-5.1.17-bin.jar /app/hive-0.12.0-cdh5.0.0/lib/

4) Assign permissions to users

Run the following command in MySQL:

grant all on hive.* to [email protected]‘%‘ identified by ‘root‘ WITH GRANT OPTION ;

This means that any object in the hive database can be remotely accessed by the root user of any host using the password as root.

5) Configure Environment Variables

Configure hive_home and add $ hive_home/bin to path for convenient operations

6) Start

Run the following command in Linux: hive directly uses hive because $ hive_home/bin has been added to the bin.

Supplement:

A) Enable hive by specifying the log level:

hive --hiveconf hive.root.logger=DEBUG,console

B) Start hive remote access (in thriftserver mode). For example, the Java code calls hive

hive --service hiveserver &

C) browser access

Add in hive-site.xml

<property>  <name>hive.hwi.war.file</name>  <value>lib/hive-hwi-0.12.0-cdh5.0.0.jar</value></property>

Start hive UI

hive --service hwi

Browser access: http: // hostname: 9999/Hwi

 

Common hive commands

First, we will introduce several frequently-used hive commands. Later, we will introduce the usage of various commands in detail.

View All databases: Show Databases

Switch to hive Database: Use hive

Show tables

Create Table Test (ID int, name string)

View the table structure. Only field information is included: DESC table_name

View the table structure details. Not only field information, but also storage path information: DESC extended table_name

View function: DESC function function_name