The steps of large-scale system reconfiguration are simple to comb

is currently involved in the restructuring of a core large system of the company. This paper combs some steps and experiences of large-scale system reconfiguration. Overview

With the continuous development of the company's business, the increasing number of users, the performance of the system will be more and more high, and the original hastily made out of the project, its unreasonable place will be constantly exposed. If you have contact with very lucrative Internet products, you will know a small bug, the company may lose millions of or even a few billion. When the number of users of the product reaches a certain amount of time, the requirements for all aspects of the system are higher, such as QPS, CPU, disaster tolerance, demotion, current limit, scalability, maintainability and so on. In addition to dealing with a large number of concurrent requests, the system must quickly support a variety of business requirements, and the system must be heavily refactored.

Note:

Some of the steps and methods below are actually listed according to my own project.

Business Grooming

To understand the original business process, business process optimization is required if there are unreasonable business processes. This is typically handled by the company's business architect.

database Refactoring

Pre-project, due to the progress, and not enough time to design the table, resulting in a variety of redundant tables, large tables, a large number of redundant fields, poor scalability of the table. Therefore, when reconstructing the system, you can start from the table, through the current business of carding, re-organize the table.
1. The table with too many fields can be extracted into a new table according to the business attributes of some fields;
2. The table field is no longer used, delete it;
3. Can merge fields, as far as possible to merge, for example, to indicate that a product is a tourist commodity, there is no need to add a field like Is_travel, you can directly add an enumeration value in the commodity type Product_type;
4. According to the current business, some table fields are dropped to other tables, output from another dimension;
5. If there are too many extended attributes for a table, you can create a table store in addition.

Wait a minute....

Database refactoring, typically handled by a dedicated data architect. The data architect must work closely with the business architect.

Data Migration

Because of the refactoring of the database, the data of the old database must be migrated completely. Full-volume Migration: You need to do a run-only full-scale migration program, the old database in one-time migration; incremental migration: Before the new system went live, the old system was still working, and the new data had to be migrated through an incremental migration program to the database. This incremental program must run until the old system goes offline and no new data is generated.

DB data Self-test program

In order to verify that the migration program is working properly, you must also write a self-test program that is constantly compared to the data in the old and new database to see if there is data that is missing or is not equal in value.

Business Interface Design

Redesign the external business interface for newly designed tables and newly combed businesses. Of course, because of the redesign of the interface, the method of the entry, the parameters, etc. may be different. In this way, other systems will encounter some resistance when they are plugged in.

Business Interface Self-test program

You must pass a business interface self-test program that consistently matches the output of the old and new business interfaces. This is a very critical program that can help check for new data and new interface issues.

Synchronizing new requirements

The new system must also consider whether it needs to be done, as the old system is constantly having new needs to be addressed. Of course not all of the old system needs to be updated to the new system, because the new system to do business grooming, some so-called new features, in fact, has been supported.

Read Unified

When the data migration has been done correctly, through the self-test program to find the output of the new interface is already the same as the old interface output, this time, if the external system, such as a system only need to read the interface, then a system can fully use the new interface. Gradually let the only need to read the interface system on-line.

Write Unity

In order to facilitate the system access, a gateway system can be developed to allow the external system to be transparently connected to the network gateways first. The gateway system receives the write request, writes the data to the old DB first, succeeds, then writes the data to the new DB, achieves the data double writes . The advantage of this is that after the system is online, if there is a problem with the new write interface or read interface, you can switch to the old interface immediately.

The difficulty of writing unity is relatively high and needs to be very careful.

development of the joint tune

After the new interface publishes the SDK, other systems can invoke the new interface through the SDK, making a simple interface connection between developers and developers. During this time, if you encounter a business problem, you must contact the architect and data architect in a timely manner. Where appropriate, the business and the table may be adjusted.

As mentioned above, because the business interface changes relatively large, the other system access, will encounter a lot of resistance.

Testers involved

In addition to the interface function test, a complete performance test and stability test must be done. At the same time must be set up to test the environment, and other systems of testing personnel to be linked, other systems to access the new interface.

At this stage, it is best to find a reliable tester, who knows how to test technical skills and understand the business.

Access Traffic

You can first cut a few of the traffic to the new interface, try water. If you have any questions, change them in time. As long as there is traffic access, you must use a variety of monitoring system real-time monitoring, the problem of immediate alarm. In addition, developers must frequently review the log system to identify problems early. Once the new interface is very stable, all traffic can be cut into the new interface.

Observation System

–
After the new interface accesses all traffic, in addition to monitoring the system monitoring interface, developers must often look at the log system to see if the system is working properly. It's best to set a task for the developer to take turns observing the system.