Disaster Tolerance overview Design for distributed databases in a multi-layer architecture

Disaster recovery is completed: the Data Disaster Tolerance overview Design of Distributed Databases in a multi-layer structure is completed. The detailed design section will be carried out after the preliminary design demonstration is correct.

I. concept:

Mappings between objects and tables:
The school has a student table, the city has a student table, and the province has a student table. Their table structure should be completely consistent, but the school's student table only contains the student information of the school, the student table in the city contains information about all the students in the city,
The student table in the province contains the student information of the province. Use city_id and school_id to record the student data of the city and school respectively.
In this design, when the school reports data, you only need to save the student information to this table by timestamp.

Timestamp:
Considering that a total of 540 schools and 11 cities and 1 province in Jiangxi have registered students, using the database timestamp technology may result in inconsistency, NTP synchronization is basically unrealistic,
The preliminary thought is the maxid + 1 solution. Due to the concurrency problem caused by maxid + 1, we will try our best to solve it.
Select isnull (max (ID), 0) + 1 from table. The following is a convenient description. The max_id + 1 method is also described as the timestamp method. Note that max (ID) + 1 refers to the unit of the school,
That is to say, for example, the school 1234 has a record number, 6, 7, 8, 9, and under the School 2345, there is also a record number, 6, 7, 8, 9, not the whole table is unique.

Delete:
There is no Delete concept in the design. Only the delete mark can be set and B _del = 1 can be set to modify the timestamp of this record.

 
Copy:
The basic student information and student transaction information table must be completely cached at school, city, and province levels. The three cached tables are identical and are copies of each other.

Implementation Method:
Because web programs are stateless and long-running large transactions have limited capabilities, Flex + ASP. NET + WebService is used for implementation.


Policy

1. Complete Cache Policy
It is cache all once.

2. timestamp synchronization Policy (incremental synchronization Policy)
When each record is synchronized to another host, the timestamp of this record as a copy is recorded. When the next two hosts exchange data, they report their own maximum timestamp to determine whether a transmits data to B or
B transmits data to A. In this case, we need to avoid both A to B and B to a during table design. This is designed as two tables.

Behavior log:
Complete records of user operations through business writing (or trigger. At present, I have not thought of the specific practical application values of this function. I will put it here first, and I will discuss the issues later.

The following describes how to report a school student table after it is entered:

1. Before the web program enters the system, check the timestamp t_base_student, which is the max ID of the biggest change. Upload the timestamp to the municipal WebService, webService returns the timestamp of the school at the municipal end.
2. The program is responsible for comparing whether the school is newer than the city () or the city is newer than the school.
If the school is newer than the city (if no data is reported), the first 100 pieces of data that are later than the timestamp are reported to the city, and the city is saved to the database. Call this function recursively, and report 100 more records until the data is reported completely and the upstream and downstream timestamps are consistent.
If the city is newer than the school (the school data is lost and the city needs to provide the first 100 pieces of data, the school will save the data to the database and call it recursively until the upstream and downstream data are completely consistent ).

In this case, data may be exchanged with the Municipal server for a long time, and JavaScript may not be suitable. You can consider using Flex + ASP. net WebService, such as progress bars and other friendly prompts.
Reference: http://www.oschina.net/code/snippet_153403_10076

3. After confirming that the table is consistent, the school collects student information based on the registration database and admission database, and then manually records 10 students.
4. After the instructor checks the information, the Report button is displayed. We mark the student data in the table as 1.
5. The program will report the data marked as 1 and reported to the Municipal end as null. After the report is complete, change null to the report time.
6. At this point, the up and down data are completely consistent, and the copy is successfully saved.
7. The city side modifies the student data, for example, helping to modify the name (of course, this is not the actual business, but just to illustrate the problem ), the timestamp of this record will be modified to the maxid + 1 recorded in this table of the school.
8. Before the school enters the student status system again, check whether there is any data to be updated and directly download the data as needed to ensure the consistency with the municipal data copy.

Summary:
The timestamp-Based Incremental COPY method can solve data inconsistency, Disaster Tolerance, and other problems, but it may also cause data copy cache failure due to various reasons, at that time, our alternative method will take effect, that is, using the full Cache Policy for a new cache.
Because the data volume may be large, you can consider using the Flex + WebService Program for the complete cache.