Oracle Data Integrator 12c----Conformance CDC (consistent CDC)

The concept of change sets is introduced in conformance CDC. A change set can include several tables that have relationships with each other, such as a primary foreign key reference relationship. The CDC can guarantee data consistency when capturing and releasing changes in a change set. This exercise describes how to use a CDC that guarantees consistency.

1 Environment Preparation

SOURCE table: Odi_src. Dept,odi_src. Emp

Target table: Odi_tag. DEPT, Odi_tag. Emp

Configuration topology See the creation and configuration of the ODI studio topology (Oracle)

Model Reverse engineering See: Oracle Data Integrator 12c-model and data storage (DataStore)

2 Importing and configuring JKM

First import JKM JKM Oracle consistent in your project

JKM is a knowledge module that generates and captures logs for a table and needs to be configured on the model or table.

Edit the journal record for model ODI_SRC, select simple journaling mode, and select JKM Oracle consistent:

3 Enable CDC Journaling

The consistency CDC and the simple CDC have different implementations and functions, but they are roughly the same in use. This still includes these steps: Add to CDC, subscribe to CDC, start journaling.

First right-click Model odi_src→ changed data capture → Add to CDC

View

Right-click Model odi_src→ changed data capture → subscribers → subscriptions

Type in the pop-up window to subscribe to the user, such as consistent, and then add

Click OK to complete the subscription. This will start a session. Please check the session completion in Operator.

Next start Diary: Right-click Model odi_src→ changed data capture → start journal

This operation also executes a session that will generate some CDC-specific database objects after the session has finished executing. The generated content is similar to the simple CDC.

4 testing to see if changes can be captured

Inserts a row of data into the DEPT.

INSERT  into VALUES ('assitant' 'BOSTON');

View Journal Data

Continue inserting data

Insert  into

Values ('6000'Miller'clerk' ' 7782 ' ' 1300 ' NULL '  - ');

View Journal Data

Then query the journal table:

SELECT *  from " odi_src_work". " Jv$ddept "; SELECT *  from "Odi_src_work". " Jv$demp ";

Can query to the data

However, if you execute the following SQL query

SELECT *  from " odi_src_work". " Jv$dept "; SELECT *  from "Odi_src_work". " Jv$emp ";

You will find that the query is not content. This is because consistency CDC provides a mechanism for expanding by window to lock down and access the journaling data per user. We need this mechanism to ensure that there is no violation of consistency when capturing and distributing change data.

Designer provides an interface to extend Windows and lock subscribers.

First choose the extension window from the right-click Pop-up menu of the model ODI_SRC:

Then select Lock Subscribers from the right-click Pop-up menu in model ODI_SRC:

Select and add the subscriber you want to lock in the pop-up window, and then determine the completion lock:

Both of these steps will commit a session. View the session execution in Operator. And then query the view

We'll see that the change data is visible in the view.

SELECT *  from "Odi_src_work". " Jv$dept ";

SELECT *  from "Odi_src_work". " Jv$emp ";

5 Create interface to synchronize change data to target

Create interfaces Consistent_cdc_dept and Consistent_cdc_emp in Project Odi_exercise, with the Consistent_cdc_dept table as an example

In the diagram, add the EMP from ODI_SRC to the source, add the EMP in the ODI_TRG to the target datastore, and map it automatically.

Then select the source data store, select "Data for journaling only",

Specify the log filter, the content of the filter is for subscribers, the following example indicates that the current interface will consume consistent subscriber data:

Other and general mappings are the same

Then configure LKM (using LKM SQL to Oracle), IKM (using Oracle Increamental Update), and CKM.

Then execute two interfaces. We see that the change data is synchronized to the Dept and EMP Tables in Odi_tag.

But we'll find that the diary data is still there. This means that for CDC with consistency settings, the interface does not automatically clear the journal after the data has been synchronized.

We need to perform two other actions to clear the journal data that has been consumed: Unlock and clear the journal. The Designer also provides an interface for performing both operations.

6 Create a package to complete a change set of data synchronization

Consistency CDC creates a change set for a model with the same name as the model. The Odi_src change concentration contains 2 tables: DEPT, EMP.

Then create a package cdc_consistent_pkg. Drag the model odi_src in the diagram. Specify the type as the journaling model, select the extension window and lock subscribers, and add subscriber consistent:

Then drag the consistent_cdc_dept and consistent_cdc_emp in. In the end, drag the model odi_src in, specify the type as the journaling model, select Clear Journal and cancel subscriber lock, and add subscriber consistent:

7 Loop execution

Add Odiwaitforlogdata after the first journal record model ODI_SRC above

The name of the CDC set can be queried by the following SQL query

SELECT *  from "Odi_src_work". " Snp_cdc_set ";

Add Odisleep after the 2nd journal record model ODI_SRC above

Set the first step for the first model ODI_SRC

Oracle Data Integrator 12c----Conformance CDC (consistent CDC)