Foolish Oracle Database Synchronization tool

Recently, some research and analysis on the Oracle database real-time synchronization tool has been done, focusing on LinkedIn's databus and Ali's Yugong two middleware, in which databus need to add additional columns and triggers on each table to be synchronized, the scheme is more , this article will emphatically analyze Ali's Yugong realization plan and give the Analysis research report.

1.yugong real-time synchronization principle

Using the materialized view log function of Oracle, similar to an internal trigger, the database of the original table needs to be authorized with the creation and deletion of materialized view log, and the detailed description of materialized view log can refer to the following article:

http://blog.csdn.net/tianlesoftware/article/details/7720580

2. Performance Testing

A. 1 of the original table in test environment (Test_source) new to 3 million data

B. For data that does not affect the development environment, modify the Yugong code to change the target table to Test_source_bak

C. Start the Yugong service and start synchronizing data

D. After 1.5 hours, all incremental data is synchronized

3. Data delay Test

A. Manually modified the original table data to delete the operation, in 10 seconds to achieve data synchronization

B. Add a new column to the target library, the type is the time type, the default value is the current time

C. Use a script to generate a batch of data in the original library repeatedly (50 batches)

D. Latency of all data is within 30 seconds

4. Data integrity Testing

A. Manual random sampling of data, comparing the values of each column of the same record, the data is consistent

B. Extracting a specific collection of records using JDBC, using the MD5 function, MD5 the results consistently

5. Other Test Cases

A. The original table adds a field, but the target table does not increase and the data is synchronized correctly

B. The target table first increases the field, the original table does not increment the field, the data normal synchronization

C. Both the original table and the target table add fields and the service needs to be restarted before data can be synchronized

D. No primary key test, Yugong does not support no primary key synchronization

E. Primary key is multi-column test, can be synchronized normally

F.yugong Service continuously running for 72 hours, in the middle of a synchronization interrupt, unexplained

G. Materialized view logs when rollback in a transaction, in-step rollback

H. The target table deletes a record, the original table updates the record, and the record is resynchronized to the target table

6.yugong Analysis

A. Concise code, divided into extractor (extract), translator (translation), applier (update to target library) three parts

B. Only one service, no other third-party dependencies

C. Simple principle, can change the code according to our intention

Foolish Oracle Database Synchronization tool