Configure remote MetaStore in hive

How to configure remote MetaStore in hive:

1) Configure hive to use local MySQL to store MetaStore (server a 111.121.21.23) (Remote MySQL storage can also be used)

2) After the configuration is complete, start the service bin/hive -- service MetaStore (default listening port: 9083) on server)

3) configure the hive client, modify the hive-site.xml :( server B-requires a hadoop environment)

<Property>
<Name> hive. MetaStore. Local </Name>
<Value> false </value>
<Description> controls whether to connect to remove MetaStore server or open a new MetaStore server in hive client JVM </description>
</Property>

<Property>
<Name> hive. MetaStore. Uris </Name>
<Value> thrift: // 111.121.21.23: 9083. </value>
<Description> </description>
</Property>

4) Run: Bin/hive and run the hql test.

5) after the hive client is successfully connected, the hive server outputs the following logs:

Starting hive MetaStore Server
11/10/31 18:07:27 info MetaStore. hivemetastore: 0: Opening raw store with implemenation class: org. Apache. hadoop. hive. MetaStore. ObjectStore
11/10/31 18:07:27 info MetaStore. ObjectStore: ObjectStore, initialize called
11/10/31 18:07:27 error datanucleus. Plugin: bundle "org. Eclipse. jdt. Core" requires "org. Eclipse. Core. Resources" but it cannot be resolved.
11/10/31 18:07:27 error datanucleus. Plugin: bundle "org. Eclipse. jdt. Core" requires "org. Eclipse. Core. RunTime" but it cannot be resolved.
11/10/31 18:07:27 error datanucleus. Plugin: bundle "org. Eclipse. jdt. Core" requires "org. Eclipse. Text" but it cannot be resolved.
11/10/31 18:07:27 info datanucleus. Persistence: Property datanucleus. cache. level2 unknown-will be ignored
11/10/31 18:07:27 info datanucleus. Persistence: Property javax. JDO. Option. nontransactionalread unknown-will be ignored
11/10/31 18:07:27 info datanucleus. persistence: ========================== persistence configuration ======================
11/10/31 18:07:27 info datanucleus. Persistence: datanucleus persistence factory-vendor: "datanucleus" version: "2.0.3"
11/10/31 18:07:27 info datanucleus. Persistence: datanucleus persistence factory initialised for datastore url = "JDBC: mysql: // localhost: 3306/hive? Createdatabaseifnotexist = true "driver =" com. MySQL. JDBC. Driver "username =" hive"
11/10/31 18:07:27 info datanucleus. persistence: ========================================================== ==============================
11/10/31 18:07:28 info datastore. Schema: Creating table 'delete1320055648261'
11/10/31 18:07:28 info datastore. Schema: schema name cocould not be determined for this datastore
11/10/31 18:07:28 info datastore. Schema: Dropping table 'delete1320055648261'
11/10/31 18:07:28 info datastore. Schema: initialising catalog "hive", schema "" using "NONE" auto-start Option
11/10/31 18:07:28 info datastore. Schema: catalog "hive", schema "" initialised-managing 0 classes
11/10/31 18:07:28 info MetaStore. ObjectStore: Setting MetaStore object pin classes with hive. MetaStore. cache. pinobjtypes = "table, storagedescriptor, serdeinfo, partition, database, type, fieldschema, order"
11/10/31 18:07:28 info datanucleus. Metadata: Registering listener for metadata initialisation
11/10/31 18:07:28 info MetaStore. ObjectStore: initialized ObjectStore
11/10/31 18:07:28 warn datanucleus. Metadata: Metadata parser encountered an error in file "jar: file:/opt/modules/hive-0.7.1/lib/hive-metastore-0.7.1.jar! /Package. JDO "at line 11, column 6: cvc-elt.1: cannot find the declaration of element 'jdo '.-Please
Check your specification of DTD and the validity of the metadata XML that you have specified.
11/10/31 18:07:28 warn datanucleus. Metadata: Metadata parser encountered an error in file "jar: file:/opt/modules/hive-0.7.1/lib/hive-metastore-0.7.1.jar! /Package. JDO "at line 312, column 13: The content of element type" class "must match" (extension *, implements *, datastore-identity ?, Primary-key ?, Inheritance ?, Version ?, Join *, foreign-key *, Index *, unique *, column *, field *, property *, query *, fetch-Group *, extension *)".
-Please check your specification of DTD and the validity of the metadata XML that you have specified.
11/10/31 18:07:28 warn datanucleus. Metadata: Metadata parser encountered an error in file "jar: file:/opt/modules/hive-0.7.1/lib/hive-metastore-0.7.1.jar! /Package. JDO "at line 359, column 13: The content of element type" class "must match" (extension *, implements *, datastore-identity ?, Primary-key ?, Inheritance ?, Version ?, Join *, foreign-key *, Index *, unique *, column *, field *, property *, query *, fetch-Group *, extension *)".
-Please check your specification of DTD and the validity of the metadata XML that you have specified.
11/10/31 18:07:28 warn datanucleus. Metadata: Metadata parser encountered an error in file "jar: file:/opt/modules/hive-0.7.1/lib/hive-metastore-0.7.1.jar! /Package. JDO "at line 381, column 13: The content of element type" class "must match" (extension *, implements *, datastore-identity ?, Primary-key ?, Inheritance ?, Version ?, Join *, foreign-key *, Index *, unique *, column *, field *, property *, query *, fetch-Group *, extension *)".
-Please check your specification of DTD and the validity of the metadata XML that you have specified.
11/10/31 18:07:28 warn datanucleus. Metadata: Metadata parser encountered an error in file "jar: file:/opt/modules/hive-0.7.1/lib/hive-metastore-0.7.1.jar! /Package. JDO "at line 416, column 13: The content of element type" class "must match" (extension *, implements *, datastore-identity ?, Primary-key ?, Inheritance ?, Version ?, Join *, foreign-key *, Index *, unique *, column *, field *, property *, query *, fetch-Group *, extension *)".
-Please check your specification of DTD and the validity of the metadata XML that you have specified.
11/10/31 18:07:28 warn datanucleus. Metadata: Metadata parser encountered an error in file "jar: file:/opt/modules/hive-0.7.1/lib/hive-metastore-0.7.1.jar! /Package. JDO "at line 453, column 13: The content of element type" class "must match" (extension *, implements *, datastore-identity ?, Primary-key ?, Inheritance ?, Version ?, Join *, foreign-key *, Index *, unique *, column *, field *, property *, query *, fetch-Group *, extension *)".
-Please check your specification of DTD and the validity of the metadata XML that you have specified.
11/10/31 18:07:28 warn datanucleus. Metadata: Metadata parser encountered an error in file "jar: file:/opt/modules/hive-0.7.1/lib/hive-metastore-0.7.1.jar! /Package. JDO "at line 494, column 13: The content of element type" class "must match" (extension *, implements *, datastore-identity ?, Primary-key ?, Inheritance ?, Version ?, Join *, foreign-key *, Index *, unique *, column *, field *, property *, query *, fetch-Group *, extension *)".
-Please check your specification of DTD and the validity of the metadata XML that you have specified.
11/10/31 18:07:28 warn datanucleus. Metadata: Metadata parser encountered an error in file "jar: file:/opt/modules/hive-0.7.1/lib/hive-metastore-0.7.1.jar! /Package. JDO "at line 535, column 13: The content of element type" class "must match" (extension *, implements *, datastore-identity ?, Primary-key ?, Inheritance ?, Version ?, Join *, foreign-key *, Index *, unique *, column *, field *, property *, query *, fetch-Group *, extension *)".
-Please check your specification of DTD and the validity of the metadata XML that you have specified.
11/10/31 18:07:28 warn datanucleus. Metadata: Metadata parser encountered an error in file "jar: file:/opt/modules/hive-0.7.1/lib/hive-metastore-0.7.1.jar! /Package. JDO "at line 576, column 13: The content of element type" class "must match" (extension *, implements *, datastore-identity ?, Primary-key ?, Inheritance ?, Version ?, Join *, foreign-key *, Index *, unique *, column *, field *, property *, query *, fetch-Group *, extension *)".
-Please check your specification of DTD and the validity of the metadata XML that you have specified.
11/10/31 18:07:28 warn datanucleus. Metadata: Metadata parser encountered an error in file "jar: file:/opt/modules/hive-0.7.1/lib/hive-metastore-0.7.1.jar! /Package. JDO "at line 621, column 13: The content of element type" class "must match" (extension *, implements *, datastore-identity ?, Primary-key ?, Inheritance ?, Version ?, Join *, foreign-key *, Index *, unique *, column *, field *, property *, query *, fetch-Group *, extension *)".
-Please check your specification of DTD and the validity of the metadata XML that you have specified.
11/10/31 18:07:28 warn datanucleus. Metadata: Metadata parser encountered an error in file "jar: file:/opt/modules/hive-0.7.1/lib/hive-metastore-0.7.1.jar! /Package. JDO "at line 666, column 13: The content of element type" class "must match" (extension *, implements *, datastore-identity ?, Primary-key ?, Inheritance ?, Version ?, Join *, foreign-key *, Index *, unique *, column *, field *, property *, query *, fetch-Group *, extension *)".
-Please check your specification of DTD and the validity of the metadata XML that you have specified.
11/10/31 18:07:29 info datanucleus. Persistence: Managing persistence of Class: org. Apache. hadoop. hive. MetaStore. model. mdatabase [Table: 'dbs ', inheritancestrategy: New-table]
11/10/31 18:07:29 info datanucleus. Persistence: Managing persistence of field: org. Apache. hadoop. hive. MetaStore. model. mdatabase. Parameters [Table: 'database _ Params ']
11/10/31 18:07:29 info datastore. Schema: validating 2 Index (es) for table 'dbs'
11/10/31 18:07:29 info datastore. Schema: validating 0 foreign key (s) for table 'dbs'
11/10/31 18:07:29 info datastore. Schema: validating 2 unique key (s) for table 'dbs'
11/10/31 18:07:29 info datastore. Schema: validating 2 Index (es) for table 'database _ Params'
11/10/31 18:07:29 info datastore. Schema: validating 1 foreign key (s) for table 'database _ Params'
11/10/31 18:07:29 info datastore. Schema: validating 1 unique key (s) for table 'database _ Params'
11/10/31 18:07:29 info datanucleus. Metadata: Listener found initialisation for persistable class org. Apache. hadoop. hive. MetaStore. model. mdatabase
11/10/31 18:07:29 info MetaStore. hivemetastore: started the new metaserver on port [9083]...
11/10/31 18:07:29 info MetaStore. hivemetastore: Options. minworkerthreads = 200
11/10/31 18:07:29 info MetaStore. hivemetastore: Options. maxworkerthreads = 100000
11/10/31 18:07:29 info MetaStore. hivemetastore: TCP keepalive = true
11/10/31 18:09:10 info MetaStore. hivemetastore: 1: get_all_databases
11/10/31 18:09:10 info hivemetastore. Audit: ugi = rsync IP =/111.121.23.21 cmd = get_all_databases
11/10/31 18:09:10 info MetaStore. hivemetastore: 1: Opening raw store with implemenation class: org. Apache. hadoop. hive. MetaStore. ObjectStore
11/10/31 18:09:10 info MetaStore. ObjectStore: ObjectStore, initialize called
11/10/31 18:09:10 info MetaStore. ObjectStore: initialized ObjectStore
11/10/31 18:09:14 info MetaStore. hivemetastore: 1: get_database: VV
11/10/31 18:09:14 info hivemetastore. Audit: ugi = rsync IP =/111.121.23.21 cmd = get_database: VV
11/10/31 18:09:14 info MetaStore. hivemetastore: 1: get_database: VV
11/10/31 18:09:14 info hivemetastore. Audit: ugi = rsync IP =/111.121.23.21 cmd = get_database: VV
11/10/31 18:09:24 info MetaStore. hivemetastore: 1: get_table: DB = vv tbl = vv_20111031
11/10/31 18:09:24 info hivemetastore. Audit: ugi = rsync IP =/111.121.23.21 cmd = get_table: DB = vv tbl = vv_20111031
11/10/31 18:09:24 info datanucleus. datastore: the class "org. apache. hadoop. hive. metaStore. model. mfieldschema "is tagged as" embedded-only "so does not have its own datastore table.
11/10/31 18:09:24 info datanucleus. Persistence: Managing persistence of Class: org. Apache. hadoop. hive. MetaStore. model. mserdeinfo [Table: 'serde', inheritancestrategy: New-table]
11/10/31 18:09:24 info datanucleus. datastore: the class "org. apache. hadoop. hive. metaStore. model. morder "is tagged as" embedded-only "so does not have its own datastore table.
11/10/31 18:09:24 info datanucleus. Persistence: Managing persistence of Class: org. Apache. hadoop. hive. MetaStore. model. mstoragedescriptor [Table: 'sds ', inheritancestrategy: New-table]
11/10/31 18:09:24 info datanucleus. Persistence: Managing persistence of Class: org. Apache. hadoop. hive. MetaStore. model. mtable [Table: 'tsung ', inheritancestrategy: New-table]
11/10/31 18:09:24 info datanucleus. Persistence: Managing persistence of field: org. Apache. hadoop. hive. MetaStore. model. mserdeinfo. Parameters [Table: 'serde _ Params ']
11/10/31 18:09:24 info datanucleus. Persistence: Managing persistence of field: org. Apache. hadoop. hive. MetaStore. model. mtable. Parameters [Table: 'table _ Params ']
11/10/31 18:09:24 info datanucleus. Persistence: Managing persistence of field: org. Apache. hadoop. hive. MetaStore. model. mtable. partitionkeys [Table: 'partition _ keys ']
11/10/31 18:09:24 info datanucleus. Persistence: Managing persistence of field: org. Apache. hadoop. hive. MetaStore. model. mstoragedescriptor. bucketcols [Table: 'bucket _ cols']
11/10/31 18:09:24 info datanucleus. Persistence: Managing persistence of field: org. Apache. hadoop. hive. MetaStore. model. mstoragedescriptor. Cols [Table: 'columns ']
11/10/31 18:09:24 info datanucleus. Persistence: Managing persistence of field: org. Apache. hadoop. hive. MetaStore. model. mstoragedescriptor. Parameters [Table: 'sd _ Params ']
11/10/31 18:09:24 info datanucleus. Persistence: Managing persistence of field: org. Apache. hadoop. hive. MetaStore. model. mstoragedescriptor. sortcols [Table: 'sort _ cols']
11/10/31 18:09:24 info datastore. Schema: validating 1 index (es) for table 'serde'
11/10/31 18:09:24 info datastore. Schema: validating 0 foreign key (s) for table 'serde'
11/10/31 18:09:24 info datastore. Schema: validating 1 unique key (s) for table 'serde'
11/10/31 18:09:24 info datastore. Schema: validating 4 index (es) for table 'tsalesman'
11/10/31 18:09:24 info datastore. Schema: validating 2 foreign key (s) for table 'tount'
11/10/31 18:09:24 info datastore. Schema: validating 2 unique key (s) for table 'tount'
11/10/31 18:09:24 info datastore. Schema: validating 2 Index (es) for table 'sds'
11/10/31 18:09:24 info datastore. Schema: validating 1 foreign key (s) for table 'ss'
11/10/31 18:09:24 info datastore. Schema: validating 1 unique key (s) for table 'ss'
11/10/31 18:09:24 info datastore. Schema: validating 2 Index (es) for table 'columns'
11/10/31 18:09:24 info datastore. Schema: validating 1 foreign key (s) for table 'columns'
11/10/31 18:09:24 info datastore. Schema: validating 1 unique key (s) for table 'columns'
11/10/31 18:09:24 info datastore. Schema: validating 2 Index (es) for table 'serde _ Params'
11/10/31 18:09:24 info datastore. Schema: validating 1 foreign key (s) for table 'serde _ Params'
11/10/31 18:09:24 info datastore. Schema: validating 1 unique key (s) for table 'serde _ Params'
11/10/31 18:09:24 info datastore. Schema: validating 2 Index (es) for table 'sd _ Params'
11/10/31 18:09:24 info datastore. Schema: validating 1 foreign key (s) for table 'sd _ Params'
11/10/31 18:09:24 info datastore. Schema: validating 1 unique key (s) for table 'sd _ Params'
11/10/31 18:09:24 info datastore. Schema: validating 2 Index (es) for table 'bucket _ cols'
11/10/31 18:09:24 info datastore. Schema: validating 1 foreign key (s) for table 'bucket _ cols'
11/10/31 18:09:24 info datastore. Schema: validating 1 unique key (s) for table 'bucket _ cols'
11/10/31 18:09:24 info datastore. Schema: validating 2 Index (es) for table 'partition _ keys'
11/10/31 18:09:24 info datastore. Schema: validating 1 foreign key (s) for table 'partition _ keys'
11/10/31 18:09:24 info datastore. Schema: validating 1 unique key (s) for table 'partition _ keys'
11/10/31 18:09:24 info datastore. Schema: validating 2 Index (es) for table 'table _ Params'
11/10/31 18:09:24 info datastore. Schema: validating 1 foreign key (s) for table 'table _ Params'
11/10/31 18:09:24 info datastore. Schema: validating 1 unique key (s) for table 'table _ Params'
11/10/31 18:09:24 info datastore. Schema: validating 2 Index (es) for table 'sort _ cols'
11/10/31 18:09:24 info datastore. Schema: validating 1 foreign key (s) for table 'sort _ cols'
11/10/31 18:09:24 info datastore. Schema: validating 1 unique key (s) for table 'sort _ cols'
11/10/31 18:09:24 info datanucleus. Metadata: Listener found initialisation for persistable class org. Apache. hadoop. hive. MetaStore. model. mserdeinfo
11/10/31 18:09:24 info datanucleus. Metadata: Listener found initialisation for persistable class org. Apache. hadoop. hive. MetaStore. model. mstoragedescriptor
11/10/31 18:09:24 info datanucleus. Metadata: Listener found initialisation for persistable class org. Apache. hadoop. hive. MetaStore. model. mtable
11/10/31 18:09:24 info datanucleus. Metadata: Listener found initialisation for persistable class org. Apache. hadoop. hive. MetaStore. model. mfieldschema
11/10/31 18:10:10 info MetaStore. hivemetastore: 2: get_database: VV
11/10/31 18:10:10 info hivemetastore. Audit: ugi = rsync IP =/111.121.23.21 cmd = get_database: VV
11/10/31 18:10:10 info MetaStore. hivemetastore: 2: Opening raw store with implemenation class: org. Apache. hadoop. hive. MetaStore. ObjectStore
11/10/31 18:10:10 info MetaStore. ObjectStore: ObjectStore, initialize called
11/10/31 18:10:10 info MetaStore. ObjectStore: initialized ObjectStore
11/10/31 18:10:10 info MetaStore. hivemetastore: 2: get_database: VV
11/10/31 18:10:10 info hivemetastore. Audit: ugi = rsync IP =/111.121.23.21 cmd = get_database: VV
11/10/31 18:10:16 info MetaStore. hivemetastore: 2: get_table: DB = vv tbl = vv_20111031
11/10/31 18:10:16 info hivemetastore. Audit: ugi = rsync IP =/111.121.23.21 cmd = get_table: DB = vv tbl = vv_20111031