Introduction and installation of Chao Wu teacher course--hive

1.Hive
1.1 Roles that belong to the Data warehouse in the Hadoop ecosystem. He is able to manage data in Hadoop and can query data in Hadoop.
In essence,hive is a SQL parsing engine . Hive can run SQL queries into a job in MapReduce .
Hive has a mapping tool that transforms SQL into a job in mapreduce, converting tables and fields in SQL to files in HDFs (folders) and columns in files.
This set of mapping tools is called Metastore, typically stored in Derby, MYSQL.

The default location for 1.2 hive in HDFs is /user/hive/warehouse, which is determined by the properties in the profile Hive-conf.xml Hive.metastore.warehouse.dir decision-making.

2.hive installation
  (1) Unzip, rename, Set environment variable
  (2) under directory $hive_home/conf/, execute command MV Hive-default.xml.template   hive-site.xml rename
      under directory $hive_home/conf/, execute command MV Hive-env.sh.template         hive-env.sh Rename

1 CD  $HIVE _home2mv  hive-env.  SH. Template  hive-env.  SH3mv  hive-default.xml.template  hive-site.xml

(3) Modify the Hadoop configuration file hadoop-env.sh, and modify the contents as follows:
Export hadoop_classpath=.: $CLASSPATH: $HADOOP _classpath: $HADOOP _home/bin

Export hadoop_classpath=.: $CLASSPATH: $HADOOP _classpath: $HADOOP _home/bin

(4) Under directory $hive_home/bin, modify the file hive-config.shto add the following:
Export JAVA_HOME=/USR/LOCAL/JDK
Export Hive_home=/usr/local/hive
Export Hadoop_home=/usr/local/hadoop

1 Export java_home=/usr/local/jdk2 export hive_home=/usr/local/HIVE3 Export Hadoop_home=/usr/local/hadoop

3. Install MySQL
(1) Execute command rpm-qa | grep mysql check if MySQL software is installed. Delete if already installed
Delete the MySQL-related library information that is already installed on Linux. rpm-e xxxxxxx--nodeps
Execute Command rpm-qa | grep mysql check whether to remove clean
(2) install mysql server : execute command rpm-i mysql-server-********
(3) start mysql server: Execute command mysqld_safe &
(4) install mysql client : Execute command rpm-i mysql-client-********
(5) set root user password : Execute command mysql_secure_installation
Note: MySQL user name: Root
mysql Password: 465213a
start MySQL:mysql-uroot-p465213a (note is connected, not separate)

4. Using MySQL as the hive Metastore
(1) put mysql jdbc driver software into Hive's lib directory
(2) Modify the Hive-site.xml file and modify the contents as follows:
<property>
<name>Javax.jdo.option.ConnectionURL</name>
<value>jdbc:mysql://hadoop:3306/hive?createDatabaseIfNotExist=true</value>
</property>
<property>
<name>javax.jdo.option.ConnectionDriverName</name>
<value>com.mysql.jdbc.Driver</value>
</property>
<property>
<name>javax.jdo.option.ConnectionUserName</name>
<value>root</value>
</property>
<property>
<name>javax.jdo.option.ConnectionPassword</name>
<value>465213a</value>
</property>

Modify $hive_home/conf/hive-Site.xml<property> <name>javax.jdo.option.ConnectionURL</name> <value>jdbc:mysql://  hadoop:3306/hive?createdatabaseifnotexist=true</value></property><property> <name>javax.jdo.option.ConnectionDriverName</name> <value>Com.mysql.jdbc.Driver</value></property><property> <name>javax.jdo.option.ConnectionUserName</name> <value>Root</value></property><property> <name>javax.jdo.option.ConnectionPassword</name> <value>465213a</value></property>

Set up MySQL remote connection : execute statement under MySQL : Grant all onhive.* to "root" @ "%" identitied by "465213a"; /c4>

This allows you to connect to the Hive table under the virtual machine using the MySQL software under Windows: User name: root, Password: 465213a

Implement remote connections.

configuring Metastore for MySQL1. Upload mysql-connector-java-5.1.Ten. Jar to $hive_home/Lib2. Log in to MySQL, create a database hive#mysql -uroot- padminMySQL>CREATE DATABASE Hive;mysql>GRANT all on hive.* to [email protected]'%' identified by '465213a  ' ; MySQL> flush privileges; MySQL>Set global binlog_format='MIXED';3. Change the MySQL database character type to Latin1

5.Hive of Metastore
Metastore is the central place where hive metadata resides. Metastore uses the built-in Derby database as the storage engine by default
The disadvantage of the Derby engine: only one session can be opened at a time
Use MySQL as an external storage engine to simultaneously access multiple users

6.Hive operating mode

The run mode of hive is the execution environment of the task
Divided into local and cluster two kinds of
We can use Mapred.job.tracker to indicate
How to set it up:
Hive > SET mapred.job.tracker=local;

How to start the 7,hive

(1), hive command line mode, direct input #/hive/bin/hive execution program, or input #hive--service CLI
(2), the Hive Web interface (port number 9999) Start Mode #Hive--service hwi &
used to access hive through the browser # http://hadoop:9999/hwi/
(3), Hive remote Service (port number 10000) Start Mode #hive--service Hiveserver &

Introduction and installation of Chao Wu teacher course--hive