Relationship between data warehouse, OLAP and Data Mining

To illustrate their relationship, we have to talk about business intelligence. From a technical point of view, the process of business intelligence is based on the data warehouse in the enterprise by the online analysis and processing tools, data mining tools, and the professional knowledge of decision planners, obtain useful information and knowledge from data to help enterprises get profits.

A data warehouse is a collection of data that better supports decision-making analysis and processing by enterprises or organizations. It has four features: subject-oriented, integrated, relatively stable, and constantly changing over time, separate the data warehouse from the traditional transaction-oriented database. Key Technologies of Data Warehouse include data extraction, cleaning, conversion, loading and maintenance technologies.
OLAP is a complex analysis technology based on massive data. It supports management and decision-making personnel at all levels to quickly and flexibly perform complex query and multi-dimensional analysis and processing of data in a data warehouse from different perspectives, the query and analysis results can be presented to decision makers in an intuitive and easy-to-understand manner.
The Logical Data Model Used by OLAP is a multidimensional data model.
Common OLAP multidimensional analysis operations include roll-up, drill-down, slicing, chunking, and rotation. Multi-dimensional data models are implemented in three ways: ROLAP structure, molap structure, and holap structure. ROLAP is an OLAP implementation based on relational databases, molap is an OLAP Implementation Based on Multi-Dimensional Data organizations, and holap is an OLAP implementation based on hybrid data organizations.
Data Mining is a process of extracting useful information and knowledge hidden in mass data that people do not know beforehand. Data Mining involves multiple data sources, including data warehouses, databases, and other data sources. All data needs to be selected again. The specific selection method is related to the task. Mining results must be evaluated before they can finally become useful information. According to different evaluation results, data may need to be fed back to different stages for analysis and calculation. Common Data mining methods include association analysis, classification and prediction, clustering, outlier detection, trend and Evolution Analysis.
It can be said that online analytical processing and data mining are value-added technologies on the data warehouse.
In terms of theoretical research, researchers of OLAP technology mainly come from the database field and focus on cube compression and computing, selection and maintenance of materialized views, indexing of multi-dimensional data, and multi-dimensional query and processing, this allows you to analyze the request response time in seconds on massive data volumes. Researchers of data mining technology come from the artificial intelligence, statistics, and database fields. Their research focuses on various types of data mining. Algorithm And evaluation methods, research Scalable Data mining methods, constraints-based mining methods, mining of complex data types, and so on.
The relationship between data warehouse, OLAP and data mining is online analytical processing and data mining. Although it is a value-added technology for data warehouse to obtain two different targets, however, if these two technologies can be integrated to a certain extent, they will make analysis operations intelligent and mine operations targeted, thus comprehensively enhancing the practical value of business intelligence technology. That is, on the one hand, the online analysis technology can provide the expected mining objects and objectives for data mining to avoid the blindness of data mining. On the other hand, data mining technology can make online analysis and processing intelligent, reduce the complexity of manual operations by analysts, and reduce the burden on analysts. For example, when an analyst finds out the data of an outlier during manual analysis, the analyst can use the data mining technology to find the cause and find out the malicious violation or new demand. Another example is to use data mining technology to predict the operations and data that an analyst may be interested in during data analysis, and pre-calculate or prefetch data in advance, this improves the response time of the analysis operation. Therefore, the integration and complementarity of Online Analytical Processing and data mining technologies based on data warehouses will be the future trend of business intelligence technology.