Real usage of hive + MySQL installation and hive. MetaStore. Local attribute

This article summarizes the installation of hive + MySQL. Later, we will put some information about hive:

1. Installation objectives:

Hive with remote MySQL

2. Summary:

1). MySQL Configuration:

Assume that hive connects to MYSQL as a hive user and logs on to MYSQL as a root user:

Drop user hive @ '% ';

Grant all on DB. * to hive @ '%' identified by 'Password'; (allows you to remotely connect to MySQL)

Grant all on DB. * to hive @ 'localhost' identified by 'Password'; (allows you to locally connect to MySQL)
Flush privileges;

2) install Hive:

A) matching of the installation package and environment:

I installed Apache hadoop-1.0.x + hive0.8.

B) install hive

Decompress the installation package and go ~ /. Add the hive_home environment variable to the bash_profile and modify the path;

Download your favorite version of The mysql-connector-java-version-bin.jar from the MySQL website, put it under $ hive_home/LIB;

Rename the hive-site.xml.example as a hive-site.xml under $ hive_home/conf;

Set the hive. MetaStore. Local attribute to true.

Other attributes to be modified are:

<property>  <name>javax.jdo.option.ConnectionURL</name>  <value>jdbc:mysql://192.168.1.234:3306/hivedb?createDatabaseIfNotExist=true</value>  <description>JDBC connect string for a JDBC metastore</description></property><property>  <name>javax.jdo.option.ConnectionDriverName</name>  <value>com.mysql.jdbc.Driver</value>  <description>Driver class name for a JDBC metastore</description></property><property>  <name>javax.jdo.option.ConnectionUserName</name>  <value>hive</value>  <description>username to use against metastore database</description></property><property>  <name>javax.jdo.option.ConnectionPassword</name>  <value>hive</value>  <description>password to use against metastore database</description></property>

<Property>
<Name> hive. Cluster. Delegation. Token. Store. zookeeper. connectstring </Name>
<Value> 192.168.7.14: 2181 </value>
<Description> the zookeeper token store connect string. </description>
</Property>

3. Description of metadata remote storage and hive. MetaStore. Local attributes:

The description of the hive. MetaStore. Local attribute on the official website is: local or remote MetaStore (removed as of hive 0.10: IfHive. MetaStore. UrisIs emptyLocalMode is assumed,RemoteOtherwise); that is, local or remote metadata (delete this attribute in hive0.10; If hive. metaStore. if the Uris attribute is null, the local mode is used by default; otherwise, the remote mode is used ).

Hive stores metadata in RDBMS. There are three modes to connect to the database:

1) ingle user mode: This mode connects to an in-memory database Derby, which is generally used for unit test.

2) Multi User Mode: connects to a database over the network, which is the most frequently used mode.

3) Remote Server Mode: used for non-Java clients to access metadatabase. A metastoreserver is started on the server. The client uses the thrift protocol to access metadatabase through metastoreserver.

If you configure remote storage of metadata, you must first start the hive metadata Storage Server Service: hive -- service MetaStore on the remote end; configure the same as described above. The default port number is 9083. No grep exists in all files in the conf directory, but netstat-anp | grep 9083 indicates that the port is listening. After the service is stopped, port is released, so the port number of this version is written to the program.

Client Configuration:

Hive. MetaStore. Uris	Thrift: //	Host and port for the thrift MetaStore Server
Hive. MetaStore. Local	False	This is local store
Hive. MetaStore. Warehouse. dir	<Base HDFS path>	Default location for hive tables.

After the client side is configured, execute the hive command to the server side to retrieve meta data, and the server side data is stored in MySQL, the metadata storage and hive service are separated, detailed official documentation: https://cwiki.apache.org/Hive/adminmanual-metastoreadmin.html#AdminManualMetastoreAdmin-RemoteMetastore

At this point, I finally understand the role of the column hive. MetaStore. Local attribute!