Hive detailed and practical (hive Environment deployment +zeus+sqoop sqoop+ User Behavior analysis case)
Course Study Address: http://www.xuetuwuyou.com/course/187
The course out of self-study, worry-free network: http://www.xuetuwuyou.com
Course Description:
This course introduces basic hive architecture and environment deployment, and leads you to understand the advantages of data Warehouse hive and the specific use of hive. The use of DDL and DML in HIVEQL and the common performance optimization schemes are explained through the analysis of the actual needs of the enterprise.
Hive is a Hadoop-based data warehousing tool that maps structured data files into a single database table and provides simple SQL query functionality that translates SQL statements into MapReduce tasks. The advantage is that the learning cost is low, the simple mapreduce statistics can be quickly realized through the class SQL statements, and it is very suitable for the statistical analysis of data Warehouse without developing specialized mapreduce applications.
Course Catalogue:
1th: Hive Basic architecture and environment deployment
Comparison of 1.MapReduce analysis and SQL analysis
Introduction of 2.Hive and its development
3.Hive installation and start-up
Basic architecture of 4.Hive explained
5. Install MySQL as a metabase store
6. Configure hive to use MySQL as the metabase store
Use of basic commands in 7.Hive
Common property configurations in 8.Hive
Interactive commands that are commonly used in 9.Hive
The management and use of database in 10Hive
Management and use of table in 11.Hive
Use of external tables in 12.Hive
2nd: Hive Common DML, UDF and connection methods
Introduction to the partition table in 13.Hive
Creation and use of partitioned tables in 14.Hive
6 ways of data import in 15.Hive and its application scenarios
4 ways of data export in 16.Hive and import and export of tables
Basic syntax of HQL in 17.Hive (i)
Basic syntax for HQL in 18.Hive (ii)
Use of order BY, sort by, distribute by and cluster by in 19.Hive
Analysis function and window function in 20.Hive
Introduction to UDFs in 21.Hive
Use custom UDF for date format conversion in 22.Hive
HiveServer2 Introduction and three ways to connect
Introduction to 24.Hive metadata, fetch task, and strict mode
The 3rd chapter: Sqoop Sqoop and user behavior analysis case
Introduction to the 25.CDH version framework
Environment deployment of the CDH version framework
Introduction of 27.Sqoop and its realization principle
28.Sqoop Installation and connectivity testing
29.Sqoop importing MySQL data into HDFs (one)
30.Sqoop importing MySQL data into HDFs (ii)
Incremental import in 31.Sqoop with Sqoop job
32.Sqoop importing MySQL data into hive tables
How to export 33.Sqoop and how to use it in scripts
34. Case study-implementation of dynamic partitioning
35. Case study-partition loading creation of source table
36. Case study-indicator analysis using SQOOP export
4th: Case analysis and optimization of hive complex user behavior
37. Automatically bulk load data to hive
38.Hive script implementation of bulk load data (i)
39.Hive script implementation of bulk load data (ii)
Use of case when, cast, and Unix_timestamp in 40.HIve
41. Complex log analysis-requirements analysis
42. Complex log analysis-requirement field explanation and filtering
43. Complex Log analysis-field extraction and creation of temporary tables
44. Complex log analysis-analysis and implementation of indicator results
Introduction and comparison of storage formats for data files in 45.Hive
46. Introduction to common compression formats and mapreduce compression
47.Hadoop Compilation Configuration Snappy compression
48.Hadoop and Hive configurations support snappy Compression
Common tuning in 49.Hive
Data skew and solutions in 50.Hive-three ways to join
Data skew and solutions in 51.Hive-group by
Use regular load data in 52.Hive
Pre-processing using Python scripts in hive
The 5th Chapter: Zeus Task Resource Scheduling tool
54. Resource task Scheduling Framework Introduction
55. A common task scheduling framework in the enterprise
Introduction of 56.Zeus and basic realization principle
57.Zeus installation Deployment-Basic Environment configuration
58.Zeus installation Deployment-Modification of configuration files
59.Zeus installation Deployment-Compile package
60.Zeus scheduling using the platform
61.Zeus Platform for task scheduling application (I.)
62.Zeus Platform for task scheduling application (II.)
63.Zeus Platform for task scheduling application (III.)
Hive Video _hive Detailed and practical (hive Environment deployment +zeus+sqoop sqoop+ User Behavior analysis case)
