Discussion on database expansion scheme

causes

Each project is developed from a small project, from the initial database to the back of the thousands of tens of thousands of databases, this development process, we have to involve a technical problem: when the amount of data is too large, how to expand?

Case

Xiao Ming is now responsible for a site, the user database has 2, the site user data through the ID modulo, there are two user databases, the data is now increasing, two databases are not enough, and now need to increase the database to expand, how should Xiao Ming?

Programme

1. Downtime expansion
2. Smooth expansion

Downtime Expansion

Let's start by understanding the outage scaling scheme, a scenario that many people will use initially (several databases), with the following steps:

1. Xiao Ming first hanging announcement, tell everyone tomorrow's 02:00-06:00, the site will be down to upgrade;
2. Time has come, Xiao Ming stopped all the external services;
3. Xiaoming added 2 databases, and then wrote a program to migrate the original 2 library data to the existing 4 libraries (three-page);
4. Data migration complete, modify database service configuration;
5. Restart the service and re-open the external service.

Rollback Scenario:

Data migration failed, or the test failed after the migration, modify the service configuration back to the original two libraries and upgrade again.

Advantages:

• Simple

Disadvantages:

1. Not high available
2. Open upgrade to upgrade complete, short time, project group pressure, error -prone
3. Upgrade time is basically the middle of the night when the minimum flow, the project team tired, prone to error
4. After running for a period of time, found that the problem, difficult to rollback, can only return to the expansion before the loss of some data

applicable:

1. Small websites;
2. Most games;
3. Services that are not high-availability requirements.

Smooth Expansion

Now let's talk about the focus of this article: the best solution to smooth expansion is to expand the database is a multiple of the original database , such as: 2 databases to expand to 4 databases, is twice times the original. Steps:

(1) 2 additional databases

(2) Configure the dual-master for Data Synchronization (test first, back-line, restart service time is seconds)

(3) After the data synchronization is complete, configure the primary master double write ( because synchronization has a delay, if there are data write/update every moment, it is not accurate to ensure that the data is synchronized )

(4) After the data synchronization is completed (long time), Delete the dual master synchronization, modify the database configuration, and restart (seconds);

(5) This time has been expanded to complete, but at this time the data is not reduced, the new database with the old database as much data, at this time also need to write a program, empty the database of redundant data , such as:

1. User1 removal of the UID% 4 = 2 data;
2. User3 removal of the uid% 4 = 0 data;
3. User2 removal of the UID% 4 = 3 data;
4. User4 removal of the UID% 4 = 1 data.

Now, we have completed the smooth expansion of the database.

Advantages

1. During the expansion, the service proceeds as usual, ensuring high availability ;
2. The time is long, the project group pressure is not so big, the error rate is low ;
3. Expansion period, encountered any problems, can be debugged at any time, not afraid of affecting the online services ;
4. Half the amount of data is missing for each database.

Disadvantages

1. The program is complex, need to configure the dual-master, main master double write, detection data synchronization and other additional technologies;
2. But when the database is thousands of times, the expansion is complicated (very little, unless you put a lot of business data in the same database).

applicable:

1. Large web site;
2. Services that are highly available and require high availability.

Summary

This paper mainly explains the two schemes of database expansion, and explains the advantages and disadvantages of these two schemes, and the applicable scenarios:

• outage Expansion: simple, not high availability, error-prone, can not be rolled back after expansion, can only back the file, will lose part of the data.
• Smooth expansion: complex, high availability, error debugging easy, easy rollback, do not cause data loss,

Conclusion

1. Each scheme has its own scene, suitable for its own, is the best;
2. Small compilation experience is limited, I hope this article is helpful to everyone;
3. Make a little progress every day.

Reference Documents

58 Shenjian Teacher Architect's path: Database second-level smooth scaling architecture scheme

Discussion on database expansion scheme