安裝Hive(獨立模式 使用mysql串連)

標籤：

安裝Hive(獨立模式 使用mysql串連)1.預設安裝了java+hadoop2.下載對應hadoop版本的安裝包3.解壓安裝包tar zxvf apache-hive-1.2.1-bin.tar.gz4.安裝mysqlyum -y install mysql-server mysql mysqldev  //需要以root身份運行   另外可能需要配置yum源mysql常用命令:service mysqld start/stopchkconfig mysqld on //加入開機啟動 以系統root使用者操作5.授權mysql(以系統hadoop身份 資料庫root身份進行操作)mysqladmin -u root password "root" //修改root密碼為rootmysql -uroot -p密碼  (初始密碼為空白)create user ‘hive‘ identified by ‘hive‘; //建立用於串連的hive使用者 密碼為hivegrant all privileges on *.* to ‘hive‘@‘%‘ identified by "hive" with grant option;flush privileges; //重新整理許可權grant all privileges on *.* to ‘hive‘@‘localhost‘ identified by "hive" with grant option;flush privileges; //重新整理許可權grant all privileges on *.* to ‘hive‘@‘hadoop.master‘ identified by "hive" with grant option;flush privileges; //重新整理許可權set global binlog_format=‘MIXED‘; //設定格式  必須執行。不然報錯exit;service mysqld restart   //重啟服務6.測試連接mysql -hhadoop.master -uhive -phive  //能進去則表示設定成功create database hive; //建立串連資料庫hivealter database hive character set latin1;7.配置環境變數(/etc/profile)#hiveexport HIVE_HOME=/opt/hive-1.2.1export HIVE_AUX_JARS_PATH=/opt/hive-1.2.1/libexport HIVE_CONF_DIR=/opt/hive-1.2.1/confexport PATH=$PATH:$HIVE_HOME/binexport CLASSPATH=$CLASSPATH:$HIVE_HOME/lib儲存退出source /etc/profile8.修改設定檔1.根據模版複製設定檔cp hive-default.xml.template hive-site.xmlcp hive-env.sh.template hive-env.shcp hive-log4j.properties.template hive-log4j.properties2.修改設定檔####hive-site.xml####//添加項目   --0.11 版本以後可不用添加此項<property><name>hive.metastore.local</name><value>false</value></property>//修改項目<property><name>javax.jdo.option.ConnectionURL</name><value>jdbc:mysql://hadoop.master:3306/hive</value><description>JDBC connect string for a JDBC metastore</description>    </property><property><name>javax.jdo.option.ConnectionDriverName</name><value>com.mysql.jdbc.Driver</value><description>Driver class name for a JDBC metastore</description>    </property><property><name>javax.jdo.option.ConnectionUserName</name><value>hive</value><description>Username to use against metastore database</description>    </property><property><name>javax.jdo.option.ConnectionPassword</name><value>hive</value><description>password to use against metastore database</description></property><property><name>hive.exec.local.scratchdir</name><value>/opt/hive-1.2.1/tmp</value>        //需要建立此目錄<description>Local scratch space for Hive jobs</description></property><property><name>hive.downloaded.resources.dir</name><value>/opt/hive-1.2.1/tmp</value><description>Temporary local directory for added resources in the remote file system.</description></property><property><name>hive.hwi.war.file</name><value>/opt/hive-1.2.1/lib/hive-hwi-1.2.1.jar</value><description>This sets the path to the HWI war file, relative to ${HIVE_HOME}. </description>    </property>####hive-env.sh####HADOOP_HOME=/opt/hadoop-2.5.2####hive-log4j.properties####hive.log.threshold=ALLhive.root.logger=INFO,DRFAhive.log.dir=/opt/hive-1.2.1/logs  //需建立相應目錄hive.log.file=hive.log9.其他配置項1.hdfs上建立相應檔案夾並修改許可權hadoop fs -mkdir -p /tmp/hivehadoop fs -chmod 777 /tmp/hivehadoop fs -mkdir -p /user/hivehadoop fs -chmod 777 /user/hive2.修改hadoop hadoop-env.sh設定檔export HADOOP_CLASSPATH=$HADOOP_CLASSPATH:$CLASSPATH3.將mysql jdbc jar包拷貝到lib目錄下cp mysql-connector-java-5.1.21.jar /opt/hive-1.2.1/lib4.將lib目錄下的jline-2.12.jar 拷貝到/opt/hadoop-2.5.2/share/hadoop/yarn/lib下,並將相應包重屬名cp /opt/hive-1.2.1/lib/jline-2.12.jar /opt/hadoop-2.5.2/share/hadoop/yarn/libmv /opt/hadoop-2.5.2/share/hadoop/yarn/lib/jline-0.9.94.jar /opt/hadoop-2.5.2/share/hadoop/yarn/lib/jline-0.9.94.jar.bak9.使用驗證    hive --service metastore & //開機首次執行hive -e"show databases;" //運行完不報錯即安裝成功也可以使用hive命令列進去執行10.常用命令1.顯示show tables;show databases;2.定義  //添加外部分區表  推薦以後就是用這種表create external table access_info(ip string,access_date string,url string)partitioned by(logdate string) row format delimited fields terminated by ‘\t‘desc access_info;3.添加資料alter table access_info add partition(logdate=‘2016-01-15‘) location ‘/access‘; --負載檔案hdfs實際路徑  access為檔案夾名稱load data local inpath ‘/home/hadoop/huangzhijian/access.txt‘ into table access_info_local_file;  --載入本地檔案3.查詢select * from access_info;4.刪除drop table access_info;  //外部表格不會刪除原本資料  內部表就會刪除原資料//注意不能update5.其他    1.hive –f test.sql  ####test.sql####  select * from t1;  select count(*) from t1;    2.hive -e ‘hql語句‘3.hive -S -e ‘select * from t1‘(用法與第一種方式的靜音一樣，不會顯示mapreduce的操作過程)4.hive -e ‘select * from t1‘  > test.txt  （將結果輸出到本地檔案中）

　　

安裝Hive(獨立模式 使用mysql串連)