Hive installation based on Hadoop cluster

Hadoop version number: hadoop-0.23.5


Hive Version number: hive-0.8.1


Derby Version number: db-derby-10.9.1.0


mysql version number: mysql-5.1.47 (installed on Linux redhat)


first is the installation of hive embedded mode,


The default database for hive embedded installation is Derby,


embedded mode installation cannot be used for actual work,


that this mode of operation environment can not support bidirectional synchronization or cluster work.


can be used to test and deploy the hive installation correctly,


until hive is properly running in embedded mode,


can be based on the configuration file to do a simple deployment of the MySQL installation configuration.


can first create a good directory:


mkdir-p/usr/hive&http://www.aliyun.com/zixun/aggregation/37954.html ">nbsp; Used to store the corresponding files after decompression of the hive compression pack


mkdir-p/usr/derby is used to store the appropriate files for Derby decompression


　　--------------------------------------------------------


hive The compressed packets from the host to the
on the virtual machine

/usr directory,


then unzip the compressed package, which is:


Tar-xzf Hive-0.8.1-bin.tar


will generate a
in the/usr directory after decompression

Hive-0.8.1-bin folder,


Switch to this folder


Cd/usr/hive-0.8.1-bin


ls


will see the appropriate folders such as the Conf Lib bin,


CP *-r/usr/hive copies all files to/usr/hive


CD/USR Switch directories to the/usr path


RM-F Hive-0.8.1-bin This step is optional, the original extracted files deleted


　　-------------------------------------------------------------------


The Derby compression package from the host copy to the/usr directory on the virtual machine,


Unzip the compressed file:


Tar-xzf Db-derby-10.9.1.0-bin.tar


After decompression build folder: Db-derby-10.9.1.0-bin


all content under this folder into the/usr/derby folder


Cd/db-derby-10.9.1.0-bin


LS executes this command and then sees some folders such as bin Lib test


CP *-r/usr/derby


cd/usr


Rm-f-R db-derby-10.9.1.0 forcibly deletes the folder recursively.


　　------------------------------------------------------------


This way derby is installed under/usr/derby


and hive are installed under/usr/hive,

The setup of the
installation path is important because it is related to the overwrite of the configuration file.


　　-------------------------------------------------------------


Vi/etc/profile


at the end add:


#set for Hive


Export Hive_home=/usr/hive


Export hive_conf_dir=/usr/hive/conf


//Indicates the hive configuration file in the system


Export Path=$path:$hive_home/bin


//This can be achieved, as long as the input hive,hive service will automatically respond, without having to enter the absolute path hive.


export hive_lib= $HIVE _home/lib


because Hadoop is already installed, the export of the Hadoop path is no longer explained.


of course for the profile file to take effect immediately:


Source/etc/profile


this can take effect.


　　-------------------------------------


Next is the provisioning of the hive configuration file


should first switch to the directory:


cd/usr/hive/conf/


ls


will see: hive-env.sh.template this file,


(. template files are template files that allow users to customize modifications and optimizations by referencing their formats)


its CP and named: hive-env.sh command as follows:


CP hive-env.sh.template hive-env.sh


VI hive-env.sh


will remove the ' # ' in front of the export hadoop_heapsize=1024


, of course, can optimize this default 1024 according to your environment


will remove the ' # ' hadoop_home in front of the export


and let it point to the directory where you have the Hadoop installed (that is, a directory that switches to folders such as Conf,lib,bin with Hadoop in this directory),


(Mine:hadoop_home=/home/hadoop/hadoop)


in fact, in the installation of hive need to specify the principle of hadoop_home basically with

The principle that
need to specify Java_home when installing Hadoop is similar.


Hadoop needs Java support, and hive needs Hadoop for support.


will export hive_conf_dir=/usr/hive/conf and remove the ' # ' number


will export Hive_aux_jars_path=/usr/hive/lib


ESC (Key)


: Wq


source/hive-env.sh (effective file)


----------------------------------------------------------Finish hive-env.sh


before modifying, create a directory to match the
in the configuration file

The path corresponds, otherwise it will error when running hive.


mkdir-p/usr/hive/warehouse


mkdir-p/usr/hive/tmp


mkdir-p/usr/hive/log


CP hive-default.xml.template Hive-site.xml


VI hive-site.xml


This file has a lot of configuration items, long space, so be patient to see.


of course, you can also use search matching strings to find:


type '/hive.metastore.warehouse.dir ' (enter)

The
is locked to the desired string.


three of them need to be modified:


<property>


<name>hive.metastore.warehouse.dir</name>


<value>/usr/hive/warehouse</value>


</property>


This is the Set data directory


　　------------------------------------------------------------------------


<property>


<name>hive.exec.scratdir</name>


<value>/usr/hive/tmp</value>


</property>


This is the set temp file directory


　　------------------------------------------------------------


//This in the author's file can not add their own


<property>


<name>hive.querylog.location</name>


<value>/usr/hive/log</value>


</property>


This is a directory for storing hive related logs


the rest without modification.


-------------------------------------------Finish Hive-site.xml


CP hive-log4j.properties.template Hive-log4j.proprties


VI hive-log4j.properties


hive.log.dir=


this is where the corresponding log documents are stored when the hive is run


(Mine:hive.log.dir=/usr/hive/log/${user.name})


Hive.log.file=hive.log


This is hive the name of the log file


The default can be, as long as you can recognize the log is good,


only a more important need to modify, otherwise it will be an error.


Log4j.appender.eventcounter=org.apache.hadoop.log.metrics.eventcounter


If not modified, it will appear:


WARNING:org.apache.hadoop.metrics.EventCounter is deprecated.


please use Org.apache.hadoop.log.metrics.EventCounter in the log4j.properties files.


(just follow the warning prompts).


-------------------------------------------------------Finish All


then just enter the command: Hive


can show:


hive>


enter command show tables; (Do not forget to add a semicolon, otherwise it will not be performed)


Enter command exit (don't forget to add semicolon)


Exit Hive