Data Warehousing Concepts (ii)-ODS

ODS (English: Operational Data Store) is the concept of a data architecture or database design that arises from the need to integrate data from multiple systems and to use it for one or more systems.

first, the concept

ODS (Operational data store), an ODS with part of the characteristics of a data warehouse and part of the characteristics of an OLTP system, is "integrated, current, or near-current, constantly changing" data that generally does not preserve the trajectory of the data, is an optional part of the Data Warehouse architecture.

ODS data is subject-oriented, integrated, variable, and the data is current or nearly 4 basic features of the current one. It is also possible to see that ODS is a data storage technology between DB and DW, and that the data organization in the ODS is subject-oriented and integrated as the Data Warehouse (DW) compared to the original application-oriented distributed db, so the data entering the ODS is integrated into the same data as the Data Warehouse. In addition, ODS only stores current or near-current data, and, if necessary, the data in the ODS can be increased, deleted and updated operations, although the data in the DW is also subject-oriented and integrated, but these data are generally not modified, so the difference between the ODS and DW mainly reflects the variability of data, the current nature, stability , the degree of summarization.

Ii. Comparison of ODS with DW

Ods	Dw
Is the same as the source system table structure, the table structure with the source system changes	Topic-oriented table model with relatively stable model
Generally keep only the latest data or a shorter period of data	Historical data, recording tracks
The pursuit of time window short, to meet the effectiveness of the report data	Longer Batch Time window
Update, Insert, merge operation	The main insert operation

III. implementation of ODS and DW applications

Implementation	features	Risk td>
Data Warehouse (DW)	generally includes a data preparation layer, first source data collection and cleaning, a stable and mature model;	big investment, high risk
Operational Data Storage (ODS)	is mainly the rapid acquisition of source data, the general will also adopt some of the technology of DW, can partially retain the historical data of fewer days, not to meet the enterprise's mid-long-term decision-making requirements; The disadvantage is that there is no stable data layer;	small investment, but more technology involved
ods+dw	General ODS is used for reporting data sources while providing data for DW; DW for decision support, providing historical data;	generally requires step Risk reduction

Implementation Plan	Implementation results	Advantage	Disadvantage
Ods	Enterprises are able to grasp the current comprehensive data on the ODS to control the current operational situation of the enterprise	Can meet the real-time monitoring and real-time business needs of enterprises	Cannot solve the enterprise's COSCO decision-making requirement
Dw	Enterprises can analyze historical data in DW for medium and long-term planning	Can solve the enterprise's Cosco decision-making needs	Inability to meet real-time enterprise monitoring and real-time business needs

Iv. Common Terminology

ETL (Extract Transform Load)	such as IBM Datastage, Informatic PowerCenter
DM (data Mart) market	A data mart can also be called a "small Data warehouse." If the Data warehouse is built on an enterprise-level data model. Then the data mart is a subset of the enterprise Data Warehouse, which is primarily for departmental business and is geared toward a specific topic. Data mart can solve the bottleneck of accessing data warehouse to some extent
Cognos	Report Development Presentation Tools
Cube	Data cube, Cognos file-based multidimensional data organization for multidimensional Analysis
Master data	Key business data, such as customer information, cards, accounts
Metadata meta data	data, including data source metadata, ETL rule metadata, OD metadata, Report metadata, interface file metadata, business rule metadata, etc.

v. Advantages and disadvantages of ODS

Key Benefits:

1, provide t+1 isomorphism table to MIS system, reduce the data access pressure to the source trading system;
2, shielding a large number of heterogeneous databases, such as db2/400, SQL Server, the physical centralized storage to the Oracle database, reducing the difficulty of MIS deployment;
3, can be unified for cleaning and simple processing, such as unified trim;
4, to provide a partial integration of the main data layer for users to access, can reduce the impact of changes in the source system;
5, the common data indicators for multiple application systems can be unified processing, the provision of public processing layer table;
6, provide the source System batch processing end sign, convenient MIS daily development;
7, the MIS System batch processing operations can provide unified scheduling;
8, to provide some of the table's historical data preservation, convenient MIS use;

Disadvantages and deficiencies:

1, the ability to improve the data quality of the source system is limited, for example: if the source system does not have the last Modified Date field, the ODS is more difficult to provide incremental data to the subsequent application system;
2, increase the probability of data errors, if the ETL tool can also directly access multiple source system data and complete data processing, in the accuracy of the data to ensure higher;
3, because can not master all the source system data, such as a wide variety of registers, a period of time also need to promote the analysis and collection of the homogeneous table;
4, there is no stable, subject-oriented data model;
5. Historical data cannot be stored in large quantities;
6, Batch processing time window can not provide data access;