How to Design the synchronization function of software that can work offline

Many software can run without network access. Data is synchronized when there is a network. In this way, you can seamlessly connect to multiple clients. Typical representatives: youdao cloud notes and financial account book. Currently, I use php to develop a software server. The software provides a web version and customers... many software can run without network access. Data is synchronized when there is a network. In this way, you can seamlessly connect to multiple clients.

Typical representatives:Youdao cloud notes,Mining a financial account book

Currently, I use php to develop a software server. The software provides a web version and a client. The main functions of the client are also available offline.

The current solution is:
Write a php extension in C language for generatingUnique int64 Value.
A field modify_num is added to the data tables to be synchronized to store the unique int64 value. modify_num is changed when the data records are added, deleted, and modified.

The client periodically performs the synchronization operation. Each synchronization includes two atomic operations: first, the client gives the server the maximum modify_num, and the server returns the data greater than the modify_num to the client.updateAfter the client processes the data, it sends the data that the client needs to add, delete, and modify to the server.commit.

Supplement:
1. Why does modify_num use a unique value instead of a timestamp:

There is a reason why we didn't directly adopt the timestamp: when the operation interval is too short, the timestamp generated will be the same value. If this is not the case, the server returns multiple identical data records (modify_num) to the client during update. The maximum modify_num value of the client is modified for each processing entry, if the client crashes because of an external cause, the maximum modify_num value is sent to the server at the next startup. Therefore, the server returns data greater than this value, this means that no data is obtained because of the crash. The unique value is used to avoid this situation.

2. About conflict handling:

After all, we are dealing with common users, not programmers. Like version management tools, we don't have the ability to merge and conflict. In conflict, we have our own simple mechanism. The basic principle is to give priority to modify_num and to the final operation.

Do you have any better solutions or ideas?

Reply content:

Many software can run without network access. Data is synchronized when there is a network. In this way, you can seamlessly connect to multiple clients.

Typical representatives:Youdao cloud notes,Mining a financial account book

Currently, I use php to develop a software server. The software provides a web version and a client. The main functions of the client are also available offline.

The current solution is:
Write a php extension in C language for generatingUnique int64 Value.
A field modify_num is added to the data tables to be synchronized to store the unique int64 value. modify_num is changed when the data records are added, deleted, and modified.

The client periodically performs the synchronization operation. Each synchronization includes two atomic operations: first, the client gives the server the maximum modify_num, and the server returns the data greater than the modify_num to the client.updateAfter the client processes the data, it sends the data that the client needs to add, delete, and modify to the server.commit.

Supplement:
1. Why does modify_num use a unique value instead of a timestamp:

There is a reason why we didn't directly adopt the timestamp: when the operation interval is too short, the timestamp generated will be the same value. If this is not the case, the server returns multiple identical data records (modify_num) to the client during update. The maximum modify_num value of the client is modified for each processing entry, if the client crashes because of an external cause, the maximum modify_num value is sent to the server at the next startup. Therefore, the server returns data greater than this value, this means that no data is obtained because of the crash. The unique value is used to avoid this situation.

2. About conflict handling:

After all, we are dealing with common users, not programmers. Like version management tools, we don't have the ability to merge and conflict. In conflict, we have our own simple mechanism. The basic principle is to give priority to modify_num and to the final operation.

Do you have any better solutions or ideas?

modify_numSimply changelast_modifyThe content is the last time the timestamp is modified. In this way, the timestamp after the last synchronization is added, and the server sends back all the changes after the last synchronization.

A num is maintained on both sides. How can this problem be solved?
If the initial version is 0, edit it on the webpage and offline client,
During synchronization, both the server and client versions are 1;
This is the solution to the conflict.
When the value of 1 is updated, the value of the server is also 1, but the content is changed;

You can consider using redo logs similar to databases;
Logs (including precise timestamps) are recorded before server and client data modification );
After the client is connected to the Internet, the logs generated during the offline period of the client are sent to the server,
Insert logs on the server by time point (merge logs on the client server) and redo logs;
Then, synchronize the data to the client;

You should refer to the internal implementation of git software. When there is a conflict, it can be automatically merged or not automatically merged.

Merge has a high threshold for common users, which is enough for users to select a version. In fact, the simplest method is to allow users to start synchronization without knowing how to do it ,.,