Full web most detailed Apache Kylin1.5 installation (single node) and test Case---> Now it appears that Kylin needs to be installed on the Hadoop master node __kylin

Source: Internet
Author: User
Tags zookeeper

Please refer to the original author, Xie, http://m.blog.itpub.net/30089851/viewspace-2121221/



1. Version
hadoop2.7.2+hbase1.1.5+hive2.0.0
kylin-1.5.1kylin1.5 (apache-kylin-1.5.1-hbase1.1.3-bin.tar.gz)

2.Hadoop Environment compiled to support snappy decompression Library
Recompile HADOOP-2.7.2-SRC native to support snappy decompression compression library

3. Environmental preparedness
hadoop-2.7.2+zookeeper-3.4.6 fully distributed Environment (HDFS, YARN HA)
HBase1.1.5 ha distributed construction of Hadoop2.7 Combat v1.0
Configure Hadoop2.7.2 and Hbase1.1.5 support Snappy decompression Compression library

Hadoop2.7 actual Combat v1.0 of Hive-2.0.0+mysql Local mode installation

4. Configure Hadoop2.7.2 and Hbase1.1.5 support Snappy decompression Compression library

5. Host Planning
IP Host Installation Software Process
172.16.101.55 sht-sgmhadoopnn-01 apache-kylin-1.5.1-hbase1.1.3-bin.tar.gz Runjar

6. Add hive_dependency and Kylin_home environment variables
[Root@sht-sgmhadoopnn-01 ~]# Vi/etc/profile
Export Java_home=/usr/java/jdk1.7.0_67-cloudera
Export classpath=.: $JAVA _home/jre/lib/rt.jar: $JAVA _home/lib/dt.jar: $JAVA _home/lib/tools.jar

Export Hadoop_home=/hadoop/hadoop
Export Hbase_home=/hadoop/hbase
Export Zookeeper_home=/hadoop/zookeeper

Export Hive_home=/hadoop/hive
Export hive_dependency=/hadoop/hive/conf:/hadoop/hive/lib/*:/hadoop/hive/hcatalog/share/hcatalog/ Hive-hcatalog-core-2.0.0.jar

Export Kylin_home=/hadoop/kylin

Export path=.: $HADOOP _home/bin: $JAVA _home/bin: $ZOOKEEPER _home/bin: $HBASE _home/bin: $HIVE _home/bin: $KYLIN _home/ Bin: $PATH

"/etc/profile" 99L, 2592C written
[Root@sht-sgmhadoopnn-01 ~]# Source/etc/profile


7. Sync/etc/profile to cluster other nodes, effective


 8. Download installation configuration Kylin
# Execute the following command on SHT-SGMHADOOPNN-01
[root@sht-sgmhadoopnn-01 ~]# cd/hadoop
[ root@sht-sgmhadoopnn-01 hadoop]# wget HTTPS://DIST.APACHE.ORG/REPOS/DIST/RELEASE/KYLIN/APACHE-KYLIN-1.5.2.1/ apache-kylin-1.5.2.1-hbase1.x-bin.tar.gz
[root@sht-sgmhadoopnn-01 hadoop]# TAR-ZXVF APACHE-KYLIN-1.5.1-HBASE1.1.3-BIN.TAR.GZ&NBSP
[root@sht-sgmhadoopnn-01 hadoop]# ln-s/hadoop/ Apache-kylin-1.5.2.1-bin/hadoop/kylin

[root@sht-sgmhadoopnn-01 hadoop]# vi/hadoop/kylin/bin/kylin.sh
Export Kylin_home=/hadoop/kylin # changed to absolute path
Export hbase_classpath_prefix=${tomcat_root}/bin/bootstrap.jar:${ tomcat_root}/bin/tomcat-juli.jar:${tomcat_root}/lib/*: $hive _dependency: $HBASE _classpath_prefix  #在路径中添加 $ Hive_dependency


9. Modify Kylin.properties
[root@sht-sgmhadoopnn-01 conf]# VI $KYLIN _home/conf/kylin.properties
##### #修改
# List of Web servers in use, this is enables one Web server instance to sync up with the other servers.
kylin.rest.servers=sht-sgmhadoopnn-01:7070
##### #新增
kylin.job.jar= $KYLIN _home/lib/kylin-job-1.5.2.1.jar
kylin.coprocessor.local.jar= $KYLIN _home/lib/kylin-coprocessor-1.5.2.1.jar


10. Start Kylin
[root@sht-sgmhadoopnn-01 kylin]# $KYLIN _home/bin/kylin.sh start
.........
.........
Kylin_debug_settings is not set, would not enable remote debuging
Kylin_ld_library_settings is not set, usually it's okay unless you want to specify your the own path
A New Kylin instance is started by root, stop it using "kylin.sh stop"
Please visit Http://:7070/kylin

You can check the log at/hadoop/kylin/logs/kylin.log

# # #http://172.16.101.55:7070/kylin account: ADMIN Password: kylin


Now it looks like Kylin need to be installed on the Hadoop Master node __kylin ">

11. Check whether the Kylin has started successfully
[root@sht-sgmhadoopnn-01 kylin]# NETSTAT-NLP |grep 7070
TCP 0 0 0.0.0.0:7070 0.0.0.0:* LISTEN 30939/java
[Root@sht-sgmhadoopnn-01 kylin]#


12. Import the official website test case
[root@sht-sgmhadoopnn-01 kylin]# $KYLIN _home/bin/kylin.sh Stop
[Root@sht-sgmhadoopnn-01 kylin]# $KYLIN _home/bin/sample.sh
[root@sht-sgmhadoopnn-01 kylin]# $KYLIN _home/bin/kylin.sh start
1>. Select the ' kylin_sales_cube ' sample cube, click ' Actions '-> ' Build ', and select a due date, selected in this trial ' 2016-01-10 '
(Specific hours, minutes, seconds, because the final hive SQL statement resembles WHERE (kylin_sales). Part_dt >= ' 2012-01-01 ' and kylin_sales. Part_dt < ' 2016-01-10 ');)

2>: In the ' Monitor ' tab, check the progress bar by refreshing the page until 100%
Now it looks like Kylin need to be installed on the Hadoop Master node __kylin ">

3> Execute the following SQL query in the ' Insight ' tab:
Select Part_dt, SUM (price) as total_selled, COUNT (distinct seller_id) as sellers to Kylin_sales GROUP by Part_dt Order B Y Part_dt;
# # #耗时2.87s
4> Executes the same SQL query in hive to validate Kylin query results (open mapreduce job calculation)
# # #耗时65.205s
Query execution and results as shown in the figure
Now it looks like Kylin need to be installed on the Hadoop Master node __kylin ">

After the build was successful, a 3+n table was established in hive, as shown in the figure (3 official Case hive tables, n builds hive tables)
Now it looks like Kylin need to be installed on the Hadoop Master node __kylin ">

After the build was successful, a 1+n table was established in HBase, as shown in the figure (1 metadata tables, n builds hbase tables)

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.