Kylin working principle, architecture

Core idea: expected calculation.

Pre-calculates the metrics that may be used for multidimensional Analysis, saves the calculated results to a cube, and exists in hbase for direct access when queried

High-complexity aggregation operations, multi-table connections ... The operation is converted into a query for the precomputed results. Determines that Kylin has a very good fast query, high concurrency capability

Theoretical basis: space Change time

Cuboid:kylin any combination of dimensions into a Cuboid

Cube:kylin combines all the dimensions into a cube that contains all the Cubeid

To better use the Hadoop Big Data Environment, Kylin reads the source data from the hive normally used to do the Data Warehouse, uses MapReduce as the engine for Cube building, and saves the calculation results in hbase, exposing the restful api/jebc/ An ODBC query interface.

Kylin supports standard ANSI SQL, so you can seamlessly connect with common analysis tools (Tableau, Excel)

RESTful API:

An API that conforms to the rest architecture design.

The restful architecture is one of the most popular Internet software architectures available. It is well-structured, standards-compliant, easy to understand, easy to expand, and is being used by more and more websites

REST, the abbreviation for representational state transfer

If a schema conforms to the rest principle, it is called a restful architecture

What is a restful architecture:

(1) Each URI represents a resource;

(2) between the client and the server, the transmission of such resources of some kind of performance layer;

(3) The client through four HTTP verbs, the server-side resources to operate, to achieve "performance layer State transformation."

Jdbc

(Java database Connectivity,java connection) is a Java API for executing SQL statements that provides unified access to a variety of relational databases, consisting of a set of classes and interfaces written in the Java language.

JDBC provides a benchmark to build more advanced tools and interfaces that enable database developers to write database applications

Odbc

Open database Connectivity,odbc is an integral part of the database in the Microsoft Open Service Architecture (Wosa,windows Open Services Architecture), which establishes a set of specifications and provides a set of standard APIs (application programming interfaces) for database access.

These APIs use SQL to accomplish most of their tasks. ODBC itself also provides support for the SQL language, which allows users to send SQL statements directly to ODBC.

Open Database Interconnect (ODBC) is the standard that Microsoft proposes for database access interfaces.

Open Database Interconnect defines a specification for accessing database APIs that are independent of different vendors ' DBMS and specific programming languages (but Microsoft's ODBC documentation is described in C, and many of the actual ODBC drivers are written in C language. ）

The ODBC specification was later adopted by X/open and ISO/IEC as part of the SQL standard, with reference to relevant standard documents such as ISO/IEC 9075-3:1995 (E) call-level Interface (SQL/CLI).

ANSI SQL

The National Standardization Organization (ANSI) is an organization that approves a variety of industry standards.

The standard language used by SQL as a relational database was originally approved in 1986 based on the IBM implementation.

In 1987, "International Organization for Standardization (ISO)" used ANSI SQL as an international standard.

System Architecture:

　　

Kylin is a MOLAP (multidimensional on-line Analytical Processing) system that pre-computes data in hive and leverages Hadoop's MapReduce distributed computing framework to achieve

The table that Kylin gets is the star model structure. At the moment of modeling, only one fact table, Do Zhangwi table, is supported.

If the business requirements are complex, consider further processing in hive. (such as creating a large wide table, or using view instead)

A place for storing cube data for OLAP analysis in Hbase:kylin, enabling interactive querying of cubes

　　

　　

　　

Cube Build

　　

　　

　　

　　

Kylin working principle, architecture