Database Remote Data Synchronization solution based on email push-pull Technology

This article will introduce a cheapest remote Database Synchronization solution based on the push/pull technology. The so-called push-pull technology specifies the active transmission and active receipt of data. The data to be sent is automatically transmitted to the destination, and the target object is automatically received and processed. No agreement is required between the two. In this solution, we will adopt the mature Internet mail system to implement the push-pull technology, so that the entire system has good stability and reliability. This solution is suitable for the Database Synchronization Update Requirements in various environments.

There are many Database Replication technologies currently, and various large database systems are also integrated with the database replication technology, but these Integrated Data Replication technologies have certain limitations. First, the heterogeneous database environment cannot be effectively supported. For example, Oracle's data replication technology cannot synchronize data to ms SQL Server. Second, the replication technology does not consider network connection issues. If the database to be synchronized is remote, you must ensure network connectivity. Otherwise, data synchronization is impossible. Given the preceding restrictions, it is very difficult to synchronize data between heterogeneous databases in a wide area. In terms of software technology, access between heterogeneous databases needs to be solved; in terms of network structure, WAN connection needs to be solved, which requires a lot of manpower and financial resources.

Email Technology

Email Technology in the Internet is the most widely used technology in the Internet. It features simple use, stable performance, and low cost. Using Mail technology to synchronize and copy data should be a good choice.

As shown in, the source database generates emails for data synchronization, sends emails to the target database over the Internet, and then restoresProgramCopy the synchronized data in the email to the database. After the target database is processed, the execution and other conditions will be returned by mail for the sender to process.

No special network line orVPNNetwork, only need to work with the localISPConnection is enough to save a lot of communication costs.

Push-pull Technology

The implementation of the push-pull technique mainly relies on a service program on the server. The Service Program is an interface for all databases and is responsible for parsing all task requests and updating database data.
Is the process diagram of a service program:

All transaction requests are completed by the task distributor. The task distributor generates emails Based on the task requirements and sends them to the target end. Meanwhile, the task distributor actively receives data synchronization emails sent to other database servers, after decomposition, data is written to the specified database to update the database. All operations such as email generation and receiving are automatically completed by the task distributor without human intervention.

Database Data is synchronized by means of mail, which is suitable for scenarios with low real-time requirements, because processing of emails may cause processing delay due to various reasons. It may take a long time from data update requests to data update completion. Therefore, in our solution, the direct remote data capturing function is also added. As long as the network structure permits, the database can be updated directly, and the update method can be push, it can also be a pull method to solve the data synchronization requirements with high real-time requirements.

Is a complete logical structure of remote data updates:

As shown in, for different two remote systems, you can perform multiple remote data update operations. In the gray area shown in the figure, data synchronization is completed by mail, the intermediate part indicates that the new data is updated using direct data capture.

Features of Remote Data Synchronization Based on email push-pull technology:

1. simple operation and stable performance
2. low cost, no additional network investment required
3. Supports data updates for Heterogeneous Databases
4. Powerful functions. All tasks are written in descriptive languages, which is suitable for various applications.
5. Support multiple processing methods, such as push, pull, and direct crawling.
6. Data Synchronization is fully automated without human intervention
7. High Data Security. Each transmission link in the network is encrypted and compressed.
8. The system is developed using advanced MIDAS technology and features fast performance.