3. hive-0.12 Installation Guide

Hive Installation Deployment

(Installation will have version issue hadoop1.0 version above please install hive-0.90 test

hadoop2.0 above Please install hive-0.12.0 or the latest version of the test)

Hive-0.9.0:http://pan.baidu.com/s/1rj6f8

hive-0.12.0:http://mirrors.hust.edu.cn/apache/hive/hive-0.12.0/

hadoop1.2.1 pseudo-Distributed installation: http://www.cnblogs.com/Jiangzl/p/4198544.html

1. Copy the hive-0.12.0.tar.gz to/HOME/HADOOP2. Unzip the hive-0. 12.0.tar.gz and renaming

#cd/usr/local

#tar-ZXVF hive-0. 12.0.tar.gz

#mv hive-0. 12.0 Hive

3. Modifying environment variables

Modify the/etc/profile file.

#vi/etc/profile

Increase

Export HIVE_home=/hadoop/hadoop/hive

Modify

Exportpath= $JAVA _home/bin: $PATH: $HADOOP _home/bin: $HIVE _home/bin

Save exit

#source/etc/profile

4. Configure the Hive System File 4.1: Modify the template file under the Conf directory

L CD $HIVE _home/conf

L MV Hive-env.sh.template hive-env.sh

L MV Hive-default.xml.template Hive-site.xml

4.2. Modify the hive-config.sh of $hive_home/bin and add the following three lines

Export JAVA_HOME=/USR/LOCAL/JDK

Export Hive_home=/home/hadoop/hive

Export HADOOP_HOME=/HOME/HADOOP/HADOOP2

4.3. Start hive

When starting hive, remember to start Hadoop first (because hive is manipulating data in HDFs)

JPS command to view the currently started Java program

#hive

4.4. Error-Please modify Hive-site.xml: (vi edit:/auth)

[FatalError] hive-site.xml:2002:16:the element type "value" must beterminated by the matching End-tag "</value>".

2002 <value>auth</auth>

(In 2002 Line 16th character there: <value>auth</value>)

Hive>show tables;

At this time also reported error:

Failed:execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask.java.lang.RuntimeException:Unable To Instantiateorg.apache.hadoop.hive.metastore.HiveMetaStoreClient

Finally, the value of hive-site.xml inside hive.metastore.schema.verification is changed to false , there is no error.

Hive>create table Test (Idint,name string);

hive>quit;

5. Verify that the creation of the table is successful

Method One: Observation: #hadoop fs-ls/user/hive

Parameter: Hive.metastore.warehouse.dir

Method Two:

http://cloud4:50070 If the configuration is successful, view/user/hive exists

6.Hive of Metastore

L Metastore is the central place where hive metadata is stored. Metastore uses the built-in Derby database as the storage engine by default

L Derby Engine Disadvantage: Only one session can be opened at a time

L use MySQL as an external storage engine to simultaneously access multiple users

So we usually recommend using MySQL, but we need to configure

6.1. Configure MySQL metastore6.1.1. Upload mysql-connector-java-5.1.10.jar to $hive_home/lib6.1.2. Log in to MySQL to create a database HIVE

L #mysql-uroot-padmin

L Mysql>create database hive;

L Mysql>grant all on hive.* to [email protected] '% ' identified by ' admin ';

l Mysql>flush Privileges;

L mysql>set globalbinlog_format= ' MIXED '; (If the error is not to control)

6.1.3. Change MySQL database character type to Latin1 (alertdatabase)

Method 1: Manual command: (here hive is the database name)

ALTER DATABASE hive Character Set Latin1

6.1.4. Modify $hive_home/conf/hive-site.xml

1<property>2<name>javax.jdo.option.ConnectionURL</name>3HADOOP0 is the same machine as hive, or gateway IP cloud4: native; 192.168.56. 1 Gateway IP4 If you use the hive machine, which is the native, you need to install MySQL on Linux, see the following supplemental mSQL installation process5 but it seems that if it is CLOUD4 will show tables error, change to localhost is good!!! 6<value>jdbc:mysql://hadoop0:3306/hive?createDatabaseIfNotExist=true</value>7</property>8<property>9<name>javax.jdo.option.ConnectionDriverName</name>Ten<value>com.mysql.jdbc.Driver</value> One</property> A<property> -<name>javax.jdo.option.ConnectionUserName</name> -<value>root</value> the</property> -<property> -<name>javax.jdo.option.ConnectionPassword</name> -<value>admin</value> +</property>

7.Hive run mode is the task's execution Environment 1 start hive Command-line mode:

1: Direct input #/hive/bin/hive of the execution program,

2: or input #hive--service CLI

L divided into two types, local and cluster

We can use Mapred.job.tracker to indicate

How to set it up:

Hive > SET mapred.job.tracker=local

2.hive Verifying the Startup method

L 1, the Hive Web interface (port number 9999) Start mode

#hive--service Hwi &

Used to access hive through the browser

http://hadoop0:9999/hwi/

L 2, Hive remote Service (port number 10000) Start mode

#hive--service Hiveserver &

8.Hive and Traditional database

Data type of 9.Hive

L Basic Data types

Tinyint/smallint/int/bigint

Float/double

Boolean

String

• Complex data types

Array/map/struct

• No Date/datetime

10.Hive of data storage: Features

L Hive data storage is based on Hadoop HDFS

L Hive does not have a dedicated data storage format

L storage structure mainly includes: database, file, table, view

Hive can load this file directly (textfile) by default, and also supports sequence file

When creating a table, specify the column delimiter and row delimiter for hive data, and hive can parse the data

Data model for 11.Hive-database

L database similar to traditional databases

L default Database "default"

After using the #hive command, do not use the Hive>use < database name, the system default database.

Hive> Use default can be used explicitly;

Create a new library

Hive > CREATE Database test_dw;

11.1. Modify the Warehouse directory:/hive/conf/hive-site.xml

Warehouse is the directory of the data warehouse specified in Hive-site.xml by ${hive.metastore.warehouse.dir}

We can change the value to:/hive

Each Table has a corresponding directory store data in Hive. For example, a table test whose path in HDFs is:/warehouse/test.

All Table data (not including externaltable) is stored in this directory.

When you delete a table, both metadata and data are deleted

L Common operation

L Create a data file T1.dat

L Create a table

L Hive>create Table T1 (key string);

L Load Data

L hive>load data local inpath '/root/inner_table.dat ' into table T1;

L View Data

L SELECT * from t1

L SELECT COUNT (*) from T1

L Delete Tables drop table T1

3. hive-0.12 Installation Guide