Hive Metastore three ways to configure

Hive's meta data supports the following three storage methods, two of which belong to local storage and one for remote storage. Remote Storage is more suitable for production environments. The Hive official Wiki describes these three ways in detail, linked to: Hive Metastore.

First, local derby

This is the simplest way to store it, and you only need to do the following configuration in Hive-site.xml

<?xml version="1.0"? ><?xml-stylesheet type="text/xsl"href="configuration.xsl"?> <configuration> <property> <name>javax.jdo.option.ConnectionURL</name> <value> Jdbc:derby:;d atabasename=metastore_db;create=true</value></property> <property> <name>javax.jdo.option.ConnectionDriverName</name> <value>org.apache.derby.jdbc.EmbeddedDriver</value></property> <property> <name> Hive.metastore.local</name> <value>true</value></property> <property> <name>hive.metastore.warehouse.dir</name> <value >/user/hive/warehouse</value></property>
</configuration>

Note: When using the Derby storage mode, running hive generates a derby file and a metastore_db directory in the current directory. The disadvantage of this type of storage is that only one hive client can use the database in the same directory, otherwise it will prompt the following error

 hive> show tables; Failed:error  in  metadata:javax.jdo.JDOFatalDataStoreException:Failed To start database  " metastore_db   ", see the next exception for   details. NestedThrowables:java.sql.SQLException:Failed to start database   " metastore_db   ", see the next exception for   details. Failed:execution Error,  return  code 1  from  org.apache.hadoop.hive.ql.exec.DDLTask  

second, local MySQL

This type of storage requires a MySQL server to be run locally and configured as follows (two ways to use MySQL, the MySQL jar package needs to be copied to the $hive_home/lib directory).

<?xml version="1.0"? ><?xml-stylesheet type="text/xsl"href="configuration.xsl"?> <configuration><property> <name>hive.metastore.warehouse.dir</name> <value>/ User/hive_remote/warehouse</value></property> <property> <name>hive.metastore.local</ Name> <value>true</value></property> <property> <name>javax.jdo.option.ConnectionURL</name> < Value>jdbc:mysql://localhost/hive_remote?createdatabaseifnotexist=true</value></property> <property> <name>javax.jdo.option.ConnectionDriverName</name> <value> Com.mysql.jdbc.driver</value></property> <property> <name> Javax.jdo.option.connectionusername</name> <value>hive</value></property> <property > <name>javax.jdo.option.ConnectionPassword</name> <value>password</value></ Property></configuration>

third, remote MySQL

This type of storage requires running a MySQL server on the remote server, and the meta service needs to be started on the hive server.

Here with MySQL test server, IP bit 192.168.1.214, new Hive_remote database, character set bit latine1

<?xml version="1.0"? ><?xml-stylesheet type="text/xsl"href="configuration.xsl"?> <configuration> <property> <name>hive.metastore.warehouse.dir</name> <value>/ User/hive/warehouse</value></property> <property> <name>javax.jdo.option.connectionurl </name> <value>jdbc:mysql://192.168.1.214:3306/hive_remote?createdatabaseifnotexist=true</value></property> <property> <name>javax.jdo.option.ConnectionDriverName</name> <value> Com.mysql.jdbc.driver</value></property> <property> <name> Javax.jdo.option.connectionusername</name> <value>hive</value></property> <property > <name>javax.jdo.option.ConnectionPassword</name> <value>password</value></ property> <property> <name>hive.metastore.local</name> <value>false</value></property> <property> <name>hive.metastore.uris</name> <value>thrift ://192.168.1.188:9083</value></property> </configuration>

Note: The server and client of hive are placed on the same servers here. The server and client can be disassembled to split the Hive-site.xml configuration file into the following two parts

1), server-side configuration file

<?xml version="1.0"? ><?xml-stylesheet type="text/xsl"href="configuration.xsl"?> <configuration> <property> <name>hive.metastore.warehouse.dir</name> <value>/ User/hive/warehouse</value></property> <property> <name>javax.jdo.option.connectionurl </name> <value>jdbc:mysql://192.168.1.214:3306/hive_remote?createdatabaseifnotexist=true</value></property> <property> <name>javax.jdo.option.ConnectionDriverName</name> <value> Com.mysql.jdbc.driver</value></property> <property> <name> Javax.jdo.option.connectionusername</name> <value>root</value></property> <property > <name>javax.jdo.option.ConnectionPassword</name> <value>test1234</value></ Property></configuration>

2), client configuration file

<?xml version= " 1.0   "? ><?xml-stylesheet type="  text/xsl   " href="  configuration.xsl   "?> < configuration> <property> <name>hive.metastore.warehouse.dir</name> <value>/user/hive/ warehouse</value></property> <property> <name>hive.metastore.local</name> <value >false  </value></property> <property> <name> hive.metastore.uris</name> <value>thrift:// 192.168.1.188:9083</VALUE>  </property> </configuration> 

Start the Hive service-side program

Clients can use the Hive command directly

[Email protected]:~$ hive Hive history file=/tmp/root/hive_job_log_root_201301301416_955801255.txthive >0.736  secondshive>

Hive Metastore three ways to configure