《OD大資料實戰》hive環境搭建

標籤：

一、搭建hadoop環境

《OD大資料實戰》hadoop偽分布式環境搭建

二、hive環境搭建

1. 準備安裝檔案

：

http://archive.cloudera.com/cdh5/cdh/5/

hive-0.13.1-cdh5.3.6.tar.gz

2. 解壓

tar -zxvf hive-0.13.1-cdh5.3.6.tar.gz -C /opt/modules/cdh/

3. 修改配置

cd /opt/modules/cdh/hive-0.13.1-cdh5.3.6/confmv hive-env.sh.template hive-env.shmv hive-default.xml.template hive-site.xmlmv hive-exec-log4j.properties.template hive-exec-log4j.properties mv hive-log4j.properties.template hive-log4j.properties

1）修改hive-env.sh

#增加一行export JAVA_HOME=/opt/modules/jdk1.7.0_67HADOOP_HOME=/opt/modules/cdh/hadoop-2.5.0-cdh5.3.6export HIVE_CONF_DIR=/opt/modules/cdh/hive-0.13.1-cdh5.3.6/conf

2）修改hive-log4j.properties

hive.log.dir=/opt/modules/cdh/hive-0.13.1-cdh5.3.6/logs

3）修改hive-exec-log4j.properties 

hive.log.dir=/opt/modules/cdh/hive-0.13.1-cdh5.3.6/logs

4）修改hive-site.xml

<property>  <name>hive.lazysimple.extended_boolean_literal</name>  <value>false</value>  <description>    LazySiimpleSerde uses this properties to determine if it treats ‘T‘, ‘t‘, ‘F‘, ‘f‘,    ‘1‘, and ‘0‘ as extened, legal boolean literal, in addition to ‘TRUE‘ and ‘FALSE‘.    The default is false, which means only ‘TRUE‘ and ‘FALSE‘ are treated as legal    boolean literal.  </description>  </property><property>  <name>hive.mapjoin.optimized.hashtable</name>  <value>true</value>  <description>Whether Hive should use memory-optimized hash table for MapJoin. Only works on Tez, because memory-optimized hashtable cannot be serialized.</description></property>

4. 驗證hive環境結果

bin/hivedfs -ls /;

 

三、mysql環境搭建1. 目標是安裝mysql 5.1.172. 在官網下載yum源

http://dev.mysql.com/downloads/repo/yum/

http://repo.mysql.com//mysql57-community-release-el6-8.noarch.rpm

3. 安裝yum源到/etc/yum.repos.d/目錄

sudo rpm -Uvh mysql57-community-release-el6-8.noarch.rpmcd /etc/yum.repos.d/

4. 修改yum源配置

修改檔案：mysql-community.repo 和mysql-community-resource.repo

5.6 enable = 1

5.7 enable = 0

5. 安裝mysql

sudo yum -y install mysql-community-server

6. mysql安全性設定

sudo mysql_secure_installation

grant all privileges on *.* to ‘root‘@‘%‘ identified by ‘beifeng‘ with grant option

7. 驗證mysql安裝結果

進入命令列： mysql -uroot -p

 

四、本地mysql作為metastore模式

 

五、遠程mysql作為metastore模式

 

《OD大資料實戰》hive環境搭建