One of the Oracle ODI series (ODI knowledge module)

One of the Oracle ODI series (ODI knowledge module)

About ODI

The ODI (Oracle Data Integrator), formerly known as sunopsis Active integration Platform, was acquired by Oracle at the end of 2006 and renamed Oracle Data Integrator. It is primarily targeted for use in ETL and data integration scenarios. ODI has some notable features compared to Oracle's original ETL tool OWB, such as an ELT architecture like OWB, but more heterogeneous data sources than OWB, and so on, this series of articles will introduce some of the basic concepts of ODI.

What is the Knowledge module for ODI

Personally, the best feature of Oracle's data integrator is the concept of a knowledge module (Knowledge module). ETL development always start from the manual programming, and then gradually appear special ETL tools, in fact, we can encounter in the project some scenarios (such as loading files to the database, from the MySQL database crawl data into the Oracle database, Extracting data from DB2 and putting it in an Oracle database, etc. aside from the business part of the data extraction and transformation effort, it can always be summed up

File->oracle

Mysql->oracle

Db2->oracle

Oracle->db2

。。。

If the specific items are not discussed, these are some of the repeated use of the scene,ODI proposed the concept of knowledge module, The detailed implementation of these scenarios as a single knowledge module and the use of the Jython scripting language in conjunction with the database SQL statements recorded as a step-by-year procedure faithfully recorded, thus forming the ODI in the knowledge Module , It basically contains all the scenarios involved in all the common applications. More conveniently, the user can either directly use the ODI knowledge module to complete the data acquisition, or directly on the knowledge module to do a variety of customization, such as a business scenario may not require a specific step in the knowledge module, then you can directly remove the step to provide better performance. Of course, users can also fully develop these knowledge modules themselves.

Classification of the ODI Knowledge Module (Knowledge module)

The knowledge module of ODI is divided into several major categories (RKM,CKM,LKM,IKM,SKM), the most important of which is LKM (load KM) and IKM (integration km)

RKM: RKM completes the function of the data model by reverse engineering the structure of the source and target systems.

CKM:CKM Complete the data quality check.

LKM:LKM completes loading from the source database data into the staging table.

IKM:IKM completes the loading of data from the staging table into the target table.

SKM:SKM completes the functionality of the ODI and Web services interfaces.

SKM (Service KM): Web Services Knowledge module, which can be used to generate Web service calls

ODI believes that the flow of a data contains the process from the source table to the staging table, and then from the staging table to the target table, while the temporary table can be placed on the source or placed on the target database and can be selected by itself. (Of course, it is entirely possible to change the concept by yourself and even use a single km to do all the work).

For example, (the staging table is placed on the target database and then the transformation of the data is completed on the target database):

How to customize the Knowledge module

Here we look at the concept of knowledge modules by analyzing a specific knowledge module. A project needs to be used in the project to import the Knowledge module: Double-click the corresponding knowledge module, you will see the specific steps in the Knowledge module: As shown above a Dblink technology using Oracle to complete the data from an Oracle database into the staging area of the lkm detailed steps, Double-clicking a step displays the specific command that this step completes: below the window is the specific command to complete this step, different knowledge modules, different steps, may not show the same, some like SQL statements, some are like the Java language (Jython script), Then there will be some functions similar to Snpref.getinfo (), the specific meaning of these macro variables and functions can be found in the ODI reference document, this step ultimately generated is a SQL command to create a dblink. Each of the steps of the knowledge module can be either deleted or added, so the ODI extension can be implemented very easily. By modifying the form of a new knowledge module on an original knowledge module is the easiest way to customize, and the other way is to fully write knowledge module through the language of Jython, it is a more flexible and complex knowledge module customization way

One of the Oracle ODI series (ODI knowledge module)