General Component Transformation Data integration model

General Component Transformation Data integration model

The most common conversions are those that make the data conform to the enterprise data model. Transformations that require concrete aggregation and calculation are shifted to the subject domain, or to where they should go, and the data is transformed in the subject area.

There is usually very little in enterprise aggregation and computing, and most conversions are for specific subject areas. Figure 3.14 depicts an example of a generic component transformation Data integration subject area model.

Figure 3.14 Common component--transformation Data integration Model sample.

Note that the aggregation of the demand-settling layer has been removed from the generic component model and has been transferred to the subject area with the idea of "shifting functionality to where it is needed".

Physical subject domain Loading Data integration model

Subject domain Load data integration model based on subject area (target grouping) relies on logically splitting the target table into a group and simplifying services for the original system processing (indirect layer).

Load the data integration model subject area to perform the following functions:

Loading data

Refreshing snapshot loading

Performing change data capture

It is in the subject domain that the data integration model is loaded, the primary key and the foreign key are created, the referential integrity is validated, and the change data capture is processed.

In addition to simplifying grouped data by subject area for understanding and maintainability, grouped data by subject area logically limits the amount of data that is hosted per processing because it is important to host data as little as possible, although these processes have minimized performance impact. Figure 3.15 shows an example of a physical data integration subject domain model.

Figure 3.15 The physical subject data domain Load Data integration Model sample.

Logical data integration Model vs. physical Data integration Model

One of the things that happens in these jobs is the question: "Is there a need for a set of logical data integration models and a second set of physical data integration models?"

The answer to the data integration model is the same as the answer to the data model: "It depends." It depends on the maturity of the data management organization that will create, manage, and master the model in terms of their metadata management, and also on other data management artifacts (such as the logical data model and the physical data model).