Brief mention:
Hive is a storage structure tool that can query the files in Hadoop in a SQL-like way, or let developers familiar with Mapper/reduce do custom actions, which, for one, is simply a parsing engine, Parsing the HIVEQL statement into a job task allows Hadoop to perform operations;
HDFs directory/file, separate folders by table name. If it is a partitioned table, the partition value is a subfolder that can be used directly in the M/R job
Special NOTE: Queries that contain *, such as SELECT * from TBL, do not generate mapredcue tasks
Data type
L Basic Data types
Tinyint/smallint/int/bigint
Float/double
Boolean
String
ll complex data types
Array/map/struct
Note: There is no date/datetime type
Metastore: (Centralized storage of hive metadata) (usually not the default, but with MySQL, so there is not much to mention) Hive Metastore:
①metastore uses the built-in Derby database as the storage engine by default
Disadvantages of the ②derby engine: only one session can be opened at a time
Mysql Metastore:
Use MySQL as an external storage engine for simultaneous access by multiple users.
External engine Installation Preparation: premise: On the machine with HDFS deployment to practice MySQL installation can refer to: https://www.cnblogs.com/bookwed/p/5896619.html
Add MySQL to boot
# Chkconfig Mysqld on
Allow remote logins to MySQL
Log in to MySQL data first, and then execute the following two lines of statements:
GRANT all privileges on * * to ' root ' @ '% ' identified by ' itcast ' with GRANT OPTION;
Flush privileges;
installation preparation for the Metastore:
Hive was created in the USR directory, entered, executed
wget http://mirrors.shuosc.org/apache/hive/hive-2.3.2/apache-hive-2.3.2-bin.tar.gz
TAR-ZXVF apache-hive-2.3.2-bin.tar.gz
Then modify the name to become hive
Modify the/etc/profile file to add hive to the environment variable
Vim/etc/profile
Export hive_home=/usr/hive/hive
export path= $PATH: $JAVA _home/bin: $HADOOP _home/bin: $HADOOP _home/sbin: $HIVE _ Home/bin
Source a bit, let the configuration take effect
Source/etc/profile
The specific configuration method of http://blog.csdn.net/zxz306418932/article/details/78726118//hive2.3.2
Uploading database Drivers Mysql-connector-java-5.1.45.jar to/usr/hive/hive/lib
Modify $hive_home/conf/hive-site.xml
Add the configuration to the beginning of the Hive-site.xml file (the label <value> content is defined by itself, that is, the Linux user name)
1.
<property>
<name>system:java.io.tmpdir</name>
<value>/home/apache-hive/tmpdir </value>
</property>
<property>
<name>system:user.name</name>
<value>root</value>
</property>
2. Once again, enter the Conf file in the Hive directory, configure the Hive-site.xml file, and use the search function in the VI editor (enter/in ESC mode) to find the Javax.jdo.option.ConnectionURL, respectively. Javax.jdo.option.ConnectionDriverName, Javax.jdo.option.ConnectionUserName, Javax.jdo.option.ConnectionPassword These four configurations, of which four items are filled in <value> respectively: <value>jdbc:mysql://192.168.88.129:3306/hive?characterencoding=utf8& ; usessl=false</value>, <value>com.mysql.jdbc.driver</value>, <value>root</value>, <value>root</value>. (Note: 192.168.88.129 here is the address of the server)
3.
The./schematool-dbtype Mysql-initschema is used in the bin file of the hive directory to initialize the meta-database.
After initialization is complete, use the./hive command to start Hive, which is available when the hive> is present.
Exceptions that may be encountered during configuration:
Issue One: Connection refuse restart Hadoop, including formatting
Question two: Boot no problem, when in the time no matter what to do HIVEQL statements are reported as follows error
Start Debug mode
Hive-hiveconf Hive.root.logger=debug,console
Test Configuration
Start Hive
# hive
Create a database
hive> CREATE DATABASE test_db;
Show all databases
show databases;
Working with Database test_db
hive> use test_db;
Create student Tables
Hive> CREATE TABLE Student (ID int,name string);
See if the following folder is stored in HDFs
Http://192.168.88.129:50070/explorer.html#/user/hive/warehouse/test_db.db/student