Hadoop series hive (data warehouse) installation and configuration

Hadoop series hive (data warehouse) installation and configuration
1. Install in namenode
CD/root/soft
Tar zxvf apache-hive-0.13.1-bin.tar.gz
Mv apache-hive-0.13.1-bin/usr/local/hadoop/hive
2. Configure environment variables (each node needs to be added)
Open/etc/profile
# Add the following content:
Export hive_home =/usr/local/hadoop/hive
Export Path = $ hive_home/bin: $ path
# Environment variables take effect
Source/etc/profile
3. Install the MySQL database environment
Please refer to http://azhuang.blog.51cto.com/9176790/1551549
After the database is successfully installed, you must create the account permission and create a hive database. The operations are as follows:
Grant all privileges on hive. * To [email protected] '2017. 168.3.% 'identified by '20140901'; grant all privileges on *. * To [email protected] '2017. 168.3.% 'identified by '20140901 ';
Create Database hive Character Set Latin1; # UTF-8 encoding hive will report an error, so you need to modify the encoding to Latin1
4. Configure hive
CD/usr/local/hadoop/hive/CONF/
CP hive-default.xml.template hive-site.xml
# Vim hive-site.xml (modify the configuration between <configuration>)

 <configuration>    <property>        <name>javax.jdo.option.ConnectionURL</name>        <value>jdbc:mysql://192.168.3.10:3306/hive?characterEncoding=latin1</value>    </property>    <property>        <name>javax.jdo.option.ConnectionDriverName</name>        <value>com.mysql.jdbc.Driver</value>    </property>    <property>        <name>javax.jdo.option.ConnectionUserName</name>        <value>root</value>    </property>    <property>        <name>javax.jdo.option.ConnectionPassword</name>        <value>123</value>    </property> </configuration>

# The above four items are:
Database Connection, database driver name, user name, password.
5. Copy the JDBC driver package of MySQL to the lib directory of hive.
CP/root/soft/mysql-connector-java-commercial-5.1.30-bin.jar/usr/local/hadoop/hive/lib/
6. Copy hive to all datanode nodes
SCP-r/usr/local/hadoop/hive [email protected]:/usr/local/hadoop/
SCP-r/usr/local/hadoop/hive [email protected]:/usr/local/hadoop/
7. Simple Test
// View the current data table
Hive> show tables;
OK
Time taken: 1.069 seconds
// Import data from the local file system to the hive table
Create student.txt test text by using the "tab" key.
[[Email protected] Soft] # Cat/root/soft/student.txt
1 AA 10 121221
2 BB 20 0990
3 cc 30 120120
# Create a student test table
Hive> Create Table student (ID int, name string, age int, tel string) Row format delimited fields terminated by '\ t' stored as textfile;
OK
Time taken: 0.043 seconds
# View the current data table and structure again
Hive> show tables;
OK
Student
Hive> DESC student;
OK
Id int
Name string
Age int
Tel string
Time taken: 0.103 seconds, fetched: 4 row (s)
# Import/root/soft/student.txt local records to the student table
Hive> load data local inpath '/root/soft/student.txt' into Table student;
Copying data from file:/root/soft/student.txt
Copying file:/root/soft/student.txt
Loading data to table default. Student
Table default. Student stats: [numfiles = 1, numrows = 0, totalsize = 43, rawdatasize = 0]
OK
Time taken: 0.376 seconds
# Check the student table. If there is a record, the local data is successfully inserted.
Hive> select * from student;
OK
1 AA 10 121221
2 BB 20 0990
3 cc 30 120120
Time taken: 0.066 seconds, fetched: 3 row (s)
// Import data to the hive table on HDFS
# Upload a local file to HDFS
[[Email protected] Soft] # hdfs dfs-Put/root/soft/student.txt/hive
[[Email protected] Soft] # hdfs dfs-CAT/hive/student.txt
1 AA 10 121221
2 BB 20 0990
3 cc 30 120120
# Importing data from HDFS to hive
Hive> load data inpath'/hive/student.txt 'into Table student;
Loading data to table default. Student
Table default. Student stats: [numfiles = 2, numrows = 0, totalsize = 86, rawdatasize = 0]
OK
Time taken: 1.389 seconds
Hive> select * from student;
OK
1 AA 10 121221
2 BB 20 0990
3 cc 30 120120
1 AA 10 121221
2 BB 20 0990
3 cc 30 120120
Time taken: 0.049 seconds, fetched: 6 row (s)

This article from the "Chengdu @ A like" blog, please be sure to keep this source http://azhuang.blog.51cto.com/9176790/1553272

Hadoop series hive (data warehouse) installation and configuration