Hadoop real-combat development (HDFS real-combat image, MapReduce, hbase real-combat microblogging, hive application)

What is Hadoop and why do you learn Hadoop?

Hadoop is a distributed system infrastructure developed by the Apache Foundation. Users can develop distributed programs without knowing the underlying details of the distribution. Leverage the power of the cluster for high-speed operations and storage. Hadoop implements a distributed filesystem (Hadoop Distributed File System), referred to as HDFs. HDFs is characterized by high fault tolerance and is designed to be deployed on inexpensive (low-cost) hardware. And it provides high transfer rates (HI throughput) to access application data for applications with very large datasets (large data set). HDFs relaxes (relax) POSIX requirements (requirements) so that data in the form of a stream can be accessed (streaming access) in the file system.
Hadoop is a software framework that enables distributed processing of large amounts of data. But Hadoop is handled in a way that is reliable, efficient, and scalable. Hadoop is reliable because it assumes that compute elements and storage will fail, so it maintains multiple copies of working data, ensuring that it can redistribute processing against failed nodes. Hadoop is efficient because it works in parallel and speeds up processing by parallel processing. Hadoop is also scalable and can handle petabytes of data. In addition, Hadoop relies on community server, so it costs less and can be used by anyone.
Hadoop comes with a framework written in the Java language, so it's ideal to run on a Linux production platform. This course is explained by using a Linux platform for simulation, based on real-world scenarios.

Highlight one: Comprehensive technical point, perfect system

This course takes into account the improvement of knowledge system of Hadoop course, draws out the most applied, deepest and most practical technologies in practical development, and through this course, you will reach the new high point of technology and enter the world of cloud computing. In the technical aspect you will master the basic Hadoop cluster, Hadoop hdfs principle, Hadoop hdfs Basic command, namenode working mechanism, HDFS basic configuration management; MapReduce principle; hbase system architecture; HBase Table Structure HBase How to use mapreduce;mapreduce advanced programming, split implementation details, Hive Primer, hive combined with Mapreduce;hadoop cluster installation and many other knowledge points.

Highlights II: basic + Actual combat = Application, both learning and practice

Each stage of the course has a practical application project, so that students can quickly grasp the application of knowledge points, such as in the first stage, the course combined with the HDFS application, explained the image server design, and how to use the Java API to operate on HDFs, in the second phase; The course combines hbase to realize the various functions of the microblog project so that learners can ingenious. In the third stage: HBase and MapReduce combined with the realization of a single query and statistical system, in the fourth phase, hive combat, through the actual combat data statistics system, so that students in the shortest time to master hive advanced applications.

Highlight three: Lecturer-rich experience in cloud platform operation of Telecom Group

Lecturer Robby has a wealth of experience in the telecommunications group, is currently responsible for all aspects of the cloud platform, and has many years of in-house training experience. The lecture content is completely close to the enterprise demand, not on paper.

For more technical highlights, refer to the course Outline : (This outline is named in Chapter 1 to prevent more than 1 hours of chapter content)

1th Chapter:
> Hadoop Background
> HDFs Design Goals
> HDFs not suitable for the scene
> HDFs Architecture Detailed analysis
> The Fundamentals of MapReduce

2nd Chapter
Introduction to the > Hadoop version
> Installing a standalone version of Hadoop
> Installing Hadoop clusters

3rd Chapter
> HDFs command-line basic operations
The working mechanism of > Namenode
> HDFS Basic Configuration Management

4th Chapter
> HDFS Application Combat: Image Server (1)-System design
> Application Environment Build PHP + bootstrap + Java
> Writing files to HDFs using the Hadoop Java API implementation

5th Chapter
> HDFS Application Combat: Image Server (2)
> read files in HDFs using the Hadoop Java API implementation
> Get a list of HDFs directories using the Hadoop Java API implementation
> Delete files in HDFs using the Hadoop Java API implementation

6th Chapter
> The Fundamentals of MapReduce
> MapReduce's Running Process
> Building a MapReduce Java development environment
> Implementing WordCount using the MapReduce Java interface

7th Chapter
> WordCount Operation Process Analysis
> Combiner of MapReduce
> Using MapReduce to achieve data deduplication
> Using MapReduce for Data sorting
> Using MapReduce to achieve average data score calculation

8th Chapter
> HBase Detailed Introduction
System Architecture for > HBase
> HBASE table structure, RowKey, column family, and timestamp
> Master,region in hbase and region Server

9th Chapter
> Use HBase for Weibo applications (1)
> user registration, Login and logout design
> Build Environment Struts2 + JSP + bootstrap + jquery + HBase Java API
> hbase and user-related table structure design
> Implementation of User Registration

10th Chapter
> Use HBase for Weibo applications (2)
> User logon and logoff using session
> The design of "attention" function
> table structure Design of "attention" function
> The implementation of the "focus" function

11th Chapter
> Use HBase for Weibo applications (3)
> The design of "hair Weibo" function
> The table structure design of "tweet" function
> The implementation of "tweet" function
> show the operation of the entire application

12th Chapter
> HBase and MapReduce Introduction
> How HBase uses MapReduce

13th Chapter

> HBase Application Combat: Single Query and statistics (1)
Overall design for > applications
> Development Environment Construction
> Table Structure Design

14th Chapter
> HBase Application Combat: Single Query and Statistics (2)
Design and implementation of > Single storage
Design and implementation of > Single query

15th Chapter
> HBase Application Combat: Single Query and Statistics (3)
> Statistical function Design
> Statistical Function Realization

16th Chapter
> In-depth mapreduce (1)
An explanation of > Split's implementation
> Implementation of Custom Inputs
> Example Explanation

17th Chapter
> In-depth mapreduce (2)
Partition of > reduce
> Example Explanation

18th Chapter
> Getting Started with hive
> Installing Hive
> Using hive to deposit structured data into HDFs
> Basic use of Hive

19th Chapter
> Using MySQL as Hive's metabase
> Hive combined with MapReduce

20th Chapter
> Hive Application Combat: Data Statistics (1)
> Application design, table structure design

21st Chapter
> Hive Application Combat: Data Statistics (2)
> Data entry and the realization of statistics

Hadoop real-combat development (HDFS real-combat image, MapReduce, hbase real-combat microblogging, hive application)