IBM infosphere Data Replication product Family replication server and change data Cap

Similarities and differences between IBM infosphere data replication product Family replication server and change data capture

One, Introduction

In today's rapidly changing business age, business decisions must be made and acted upon in the first place to stay ahead of the competition. If business data is not guaranteed to be synchronized, then production and profits are bound to suffer, but it is not easy to guarantee the credibility of the data in the face of the proliferation of information and the characteristics of distributed storage. IBM's Infosphere Data Replication product family provides a series of data synchronization solutions for this problem, which are real-time capture of data source changes based on the database log, and are transmitted to the target side in real time. This product family includes two products Infosphere RS (Replication Server) and Infosphere CDC (change Data Capture).

Second, product introduction and its structure comparison

1,replication Server (RS)

Replication Server products include both SQL replication and Q-replication technologies. Where SQL Replication (formerly known as Data Propagator) released its first version in 1994, Q replicated the first release in 2004 and has now released V10FP1. Compared to SQL replication, Q-replicated data transmission technology with the help of IBM queue mechanism, so as to improve the transmission efficiency and reliability of data, can be replicated thousands of tables per second, hundreds of thousands of rows of data, at thousands of miles from the lower end to the terminal delay of less than 1 seconds (from the source DB2 commit to the target side DB2 Co MMIT). This paper will mainly introduce the comparison between Q replication technology and CDC.

Figure 1 shows the main components of Q replication and their replication principles. The main components include:

a). Qcapture program, run on the source data side, the main function is to read the change data from the log of the source database, generate MQ messages and put it into the WebSphere MQ queue, in general, each MQ message represents a database transaction, the format of the message follows a certain standard.

b). QApply program, in parallel to get MQ messages from the queue, the message contained in the database operations through conflict detection, conflict processing, such as a series of analysis and processing to generate the corresponding database operations, the source database changes reflected in the data to the target table. The target table can be a variety of databases, in addition to DB2, can also use the federated database to replicate data changes to Oracle, Sybase, Informix, SQL Server, Teradata these databases.

c). Websphere MQ provides data transfer capabilities between Qcapture and QApply, leveraging the robustness and high performance of MQ itself.

Figure 1. The main components and working principle of Q replication

2,change Data Capture (CDC)

Change Data Capture (CDC), formerly known as Datamirror,2007, was officially renamed the CDC after the acquisition of IBM, and released the first version after the acquisition of V6.3, now the latest version for V10.2. The CDC mainly uses log-based change data capture technology to realize real-time data replication for key business systems without affecting the performance of business systems. The CDC's biggest advantage is its support for heterogeneous databases and integration with IBM products, providing customers with excellent solutions in data migration, integration, synchronization, Dynamic Data warehousing and other applications.

Figure 2 shows the CDC's key components and their replication principles, and its main components include:

a). Access Server (AS): User Configuration and monitoring of the CDC engine security management tools, support for graphical interfaces (V6.5 after integration in MC) and command line

b). Management Console (MC): CDC's graphical interface tool for managing and configuring replication information, and for real-time monitoring of replication status to obtain CDC operational information

c). Source-side engine: Read the log file of the source-side database capture change data, after the row and column filter, character encoding conversion by TCP/IP sent to the target side

D. Target-side engine: receive the change data sent by the source, after numerical conversion, character encoding conversion, conflict detection, the change data applied to the target database

e). Most CDC engines can either capture change data as source-side engines and receive change data as target-side engines and apply them to specified databases; Typically, the CDC engine is called the CDC instance, and if from the AS/MC perspective, a CDC engine is also called a CDC data store

F. Metadata: Stores configuration information for the CDC instance, including database connection information, scheduled information, and table mapping information, while recording the current replication status

Figure 2. CDC key components and replication principles

3, compare

RS and CDC support replication between most of the database systems in the current market application, and the following are the data sources, targets, message queues, and operating system comparisons supported by the two:

See more highlights of this column: http://www.bianceng.cnhttp://www.bianceng.cn/Programming/extra/