Hive deployment and installation (note)

1. Download Hive: wget http://mirrors.cnnic.cn/apache/hive/hive-0.12.0/hive-0.12.0.tar.gz
2. Unzip the hive Installation File tar-zvxf hive-0.12.0.tar.gz
3. Configure the hive environment variables to initialize the hive working directory on HDFS (therefore, before deploying hive, ensure that hadoop is fully deployed and the related environment is set, hadoop version 2.2.0 is consistent with hive version)
VI/etc/profile add environment variable value
Export hive_home =/home/likehua/hive-0.12.0
Export Path = $ hive_home/bin: $ path
Source/etc/profile make the modified environment variable take effect immediately

Initialize hadoop Environment Variables
./Hadoop FS-mkdir/tmp
./Hadoop FS-mkdir/usr/hive/warehouse
./Hadoop FS-chmod g + w/tmp
./Hadoop FS-chmod g + w/usr/hive/warehouse

Configure the hive-related configuration file:/home/likehua/hive-0.12.0/Conf
Change hive-default.xml.template to hive-site.xml
Hive-log4j.properties.template to hive-log4j.properties
Hive-exec-log4j.properties.template to hive-exec-log4j.properties

Run CD $ hive_home/bin./hive.
By default, the system will go to the hive console and execute: show tables. If no error occurs, it indicates that the default version of hive is successfully installed (in fact, an error is reported, which is an XML error, I didn't think of the hive release version, and XML would have errors! The start tag and end tag of XML must be
Consistent, according to the error message, change the auth tag to value, OK ).
I gave it the default version, because the hive MetaStore is stored in a database named Derby by default.
A database is an embedded database. If there are two or more operations at the same time, an error is reported. The following describes how to configure the metasotre of hive
The information is stored in a relational database. This document uses oracle as an example. The specific steps are as follows:
1) download the oracle JDBC driver ojdbc6.jar and put it in the $ hive_home/lib folder. If there is no Lib, create it.
2) modify the configuration file hive-site.xml under $ hive_home/CONF. Major changes: connection string, driver, user name, password, and so on.
Specific parameters include:
Connection string
<Property>
<Name> javax. JDO. Option. connectionurl </Name>
<Value> JDBC: oracle: thin: @ 192.98.12.60: 1521/orcl </value>
<Description> [JDBC: thin: db11gx64.geoway.net/db11g1_jdbc connect string for a JDBC MetaStore </description>
</Property>
Database JDBC driver
<Property>
<Name> javax. JDO. Option. connectiondrivername </Name>
<Value> oracle. JDBC. Driver. oracledriver </value>
<Description> driver class name for a JDBC MetaStore </description>
</Property>
User Name
<Property>
<Name> javax. JDO. Option. connectionusername </Name>
<Value> portal </value>
<Description> username to use against MetaStore database </description>
</Property>
Password
<Property>
<Name> javax. JDO. Option. connectionpassword </Name>
<Value> portal </value>
<Description> password to use against MetaStore database </description>
</Property>
3) after the preceding changes are made, run the./hive command under the $ hive_home/bin directory to enter the console and run show talbes. If no error is reported
The table name is successfully deployed. If the table name fails, you need to locate the error information in the log file (deploy the program in Linux, if there is no log information, it is very tragic. Remember the steps above to modify the configuration file
In the hive-log4j.properties, the log storage path is recorded, default path: hive. log. dir =/tmp/$ {user. hive. in the log file, $ {USR. name} indicates
The user name for logon. If the logon user is root, the log access path is/tmp/root/hive. log .)
Errors in log files mainly include: a. The database driver does not exist. B. The database connection parameters are incorrect. C. Table or view does not exist.
The third case is many, because when you modify the default MetaStore storage mode, a lot of metadata in the new database is not initialized. We need to manually initialize it in the target database. Related
The initialization script is stored in the $ hive_home/scripts/MetaStore/upgrade directory. By default, the initialization or update scripts for databases such as Derby, MySQL, Oracle, and ipvs are stored in this directory.
Because I use the Oracle database, I will go to the $ hive_home/scripts/MetaStore/upgrade/Oracle directory. after entering the directory, you will find many SQL scripts, which are a mess and I don't know how to update them.
Or what should I do during initialization? I will teach you how to execute all the methods, and then start hive back and forth to view the logs to solve the problem. In the end, the problem will be solved. However, in fact, find
010-hive-3072.oracle. SQL, 011-hive-3649.oracle. SQL, 012-hive-1362.oracle. SQL, 013-hive-3255.oracle. SQL, 014-hive-3764.oracle. SQL are executed in sequence.
, Since my hive version is hive-0.12.0, then I execute the hive-schema-0.12.0.oracle. SQL script, and then go to hive shell to execute, show tables; if no error is reported, it means everything is OK,
If there are still errors, search for the solution on the Internet based on the log error information.

4) Another method is to initialize the MetaStore initialization of hive on a relational database, but I have never initialized it successfully, that is, to execute commands After configuring hive. /hive -- service MetaStore,
This issue will be studied later.