"OD Big Data Combat" hive environment construction

First, build the Hadoop environment

"OD Big Data Combat" Hadoop pseudo-distributed environment construction

Second, Hive Environment construction

1. Prepare the installation files

：

http://archive.cloudera.com/cdh5/cdh/5/

Hive-0.13.1-cdh5.3.6.tar.gz

2. Unzip

tar -zxvf hive-0.13. 1-cdh5. 3.6. tar. Gz-c/opt/modules/cdh/

3. Modify the Configuration

cd/opt/modules/cdh/hive-0.13. 1-cdh5. 3.6/confMV hive-env. sh. Template hive-env. SH mv hive-default.xml.template hive-site.xmlmv hive-exec-log4j.properties.template hive-exec-mv hive-log4j.properties.template hive-log4j.properties

1) Modify hive-env.sh

#增加一行export java_home=/opt/modules/jdk1. 7 . 0_67hadoop_home=/opt/modules/cdh/hadoop-2.5. 0-cdh5. 3.6 export Hive_conf_dir=/opt/modules/cdh/hive-0.13. 1-cdh5. 3.6/conf

2) Modify Hive-log4j.properties

Hive.log. dir=/opt/modules/cdh/hive-0.13. 1-cdh5. 3.6/logs

3) Modify Hive-exec-log4j.properties

Hive.log. dir=/opt/modules/cdh/hive-0.13. 1-cdh5. 3.6/logs

4) Modify Hive-site.xml

< Property>  <name>Hive.lazysimple.extended_boolean_literal</name>  <value>False</value>  <Description>Lazysiimpleserde uses this properties to determine if it treats ' t ', ' t ', ' f ', ' f ', ' 1 ', and ' 0 ' as extened, le    Gal Boolean literal, in addition to ' TRUE ' and ' FALSE '.  The default is false, which means only ' TRUE ' and ' false ' is treated as legal boolean literal. </Description>  </ Property><property>  <name>Hive.mapjoin.optimized.hashtable</name>  <value>True</value>  <Description>Whether Hive should use memory-optimized hash table for Mapjoin. Only works on Tez, because Memory-optimized Hashtable cannot is serialized.</Description></ Property>

4. Verifying the Hive Environment results

bin/-ls /;

Third, the MySQL environment construction 1. The goal is to install MySQL 5.1.172. Download Yum source on official website

http://dev.mysql.com/downloads/repo/yum/

http://repo.mysql.com//mysql57-community-release-el6-8.noarch.rpm

3. Install the Yum source to the/etc/yum.repos.d/directory

sudo rpm-uvh mysql57-community-release-el6-8.noarch.rpmcd/etc/yum.repos.d/

4. Modify the Yum source configuration

Modified files: Mysql-community.repo and Mysql-community-resource.repo

5.6 Enable = 1

5.7 Enable = 0

5. Install MySQL

sudo yum-y install Mysql-community-server

6. mysql Security settings

sudo mysql_secure_installation

Grant all privileges on * * to ' root ' @ '% ' identified by ' Beifeng ' with GRANT option

7. Verifying MySQL Installation results

Go to command line: mysql-uroot-p

Four, local MySQL as Metastore mode

V. Remote MySQL as Metastore mode

"OD Big Data Combat" hive environment construction