(Pit note) hadoop3.0 (v) Getting started with hive and data types

Brief mention:

Hive is a storage structure tool that can query the files in Hadoop in a SQL-like way, or let developers familiar with Mapper/reduce do custom actions, which, for one, is simply a parsing engine, Parsing the HIVEQL statement into a job task allows Hadoop to perform operations;

HDFs directory/file, separate folders by table name. If it is a partitioned table, the partition value is a subfolder that can be used directly in the M/R job

Special NOTE: Queries that contain *, such as SELECT * from TBL, do not generate mapredcue tasks

Data type

L Basic Data types

Tinyint/smallint/int/bigint

Float/double

Boolean

String

ll complex data types

Array/map/struct

Note: There is no date/datetime type

Metastore: (Centralized storage of hive metadata) (usually not the default, but with MySQL, so there is not much to mention) Hive Metastore:

①metastore uses the built-in Derby database as the storage engine by default

Disadvantages of the ②derby engine: only one session can be opened at a time
Mysql Metastore:

Use MySQL as an external storage engine for simultaneous access by multiple users.


External engine Installation Preparation: premise: On the machine with HDFS deployment to practice MySQL installation can refer to: https://www.cnblogs.com/bookwed/p/5896619.html

Add MySQL to boot

# Chkconfig Mysqld on

Allow remote logins to MySQL

Log in to MySQL data first, and then execute the following two lines of statements:

GRANT all privileges on * * to ' root ' @ '% ' identified by ' itcast ' with GRANT OPTION;

Flush privileges;

installation preparation for the Metastore:
Hive was created in the USR directory, entered, executed

wget http://mirrors.shuosc.org/apache/hive/hive-2.3.2/apache-hive-2.3.2-bin.tar.gz

TAR-ZXVF apache-hive-2.3.2-bin.tar.gz

Then modify the name to become hive

Modify the/etc/profile file to add hive to the environment variable

Vim/etc/profile

Export hive_home=/usr/hive/hive
export path= $PATH: $JAVA _home/bin: $HADOOP _home/bin: $HADOOP _home/sbin: $HIVE _ Home/bin

Source a bit, let the configuration take effect

Source/etc/profile

The specific configuration method of http://blog.csdn.net/zxz306418932/article/details/78726118//hive2.3.2

Uploading database Drivers Mysql-connector-java-5.1.45.jar to/usr/hive/hive/lib

Modify $hive_home/conf/hive-site.xml

Add the configuration to the beginning of the Hive-site.xml file (the label <value> content is defined by itself, that is, the Linux user name)

1.

<property>
     <name>system:java.io.tmpdir</name>
     <value>/home/apache-hive/tmpdir </value>
  </property>
  <property>
     <name>system:user.name</name>
     <value>root</value>
   </property>

2. Once again, enter the Conf file in the Hive directory, configure the Hive-site.xml file, and use the search function in the VI editor (enter/in ESC mode) to find the Javax.jdo.option.ConnectionURL, respectively. Javax.jdo.option.ConnectionDriverName, Javax.jdo.option.ConnectionUserName, Javax.jdo.option.ConnectionPassword These four configurations, of which four items are filled in <value> respectively: <value>jdbc:mysql://192.168.88.129:3306/hive?characterencoding=utf8&amp ; usessl=false</value>, <value>com.mysql.jdbc.driver</value>, <value>root</value>, <value>root</value>. (Note: 192.168.88.129 here is the address of the server)
3.

The./schematool-dbtype Mysql-initschema is used in the bin file of the hive directory to initialize the meta-database.

After initialization is complete, use the./hive command to start Hive, which is available when the hive> is present.

Exceptions that may be encountered during configuration:

Issue One: Connection refuse restart Hadoop, including formatting

Question two: Boot no problem, when in the time no matter what to do HIVEQL statements are reported as follows error

Start Debug mode

Hive-hiveconf Hive.root.logger=debug,console

Test Configuration

Start Hive

# hive

Create a database

hive> CREATE DATABASE test_db;

Show all databases

show databases;

Working with Database test_db

hive> use test_db;

Create student Tables

Hive> CREATE TABLE Student (ID int,name string);

See if the following folder is stored in HDFs

Http://192.168.88.129:50070/explorer.html#/user/hive/warehouse/test_db.db/student