Are you hungry? Offsite Dual Live Database _ database

Click to have a surprise

I share today is hungry Mody in the database and a lot of live database this piece of actual combat experience, for your reference.

Mainly share the following five points:

1, a lot of life in the difficulties

2. Multi-Live architecture

3, Database transformation

4. DBA Challenge

5, earnings and prospects one, the difficulty of living

Let's take a look at the first difficulty in living a lot: to consider how long live in the same city or more live in different places, cross-regional network delay is difficult to break the point at this stage, because the hungry is not to face the off-site, so we need to be based on the premise of delay to consider the scheme.

There is a 30 millisecond delay between Beijing and Shanghai, which can cause problems. We'll talk about it next.

The above diagram is different from the city and live in other places, the complexity and scalability of the impact on the architecture will be very different.

Let's pick a few points to talk about:

1, if just do the same city, like 30 milliseconds delay does not need to consider, because the city's delay is usually only a few milliseconds, with the same room is not very good.

2, if it is an off-site 30 millisecond delay will need to focus on, because if the application is repeatedly invoked, the magnification time is not just 30 milliseconds, may be 300 milliseconds, 500 milliseconds, for many applications is unacceptable.

In terms of scalability, if you're doing a lot of work offsite, your scalability doesn't theoretically have too many boundaries. We do the same city to live only in the Shanghai room inside the election, if it is more than live, may be the national or even the world can choose.

There is also a more difficult question, is how to ensure the security of data. Multi-live data may face multiple write points, may be confused, conflict, cyclic replication, data loop and other issues, in this case how to ensure consistency. If these are not considered well before, you are not able to live on the scheme, the risk of multiple-point writing to the test of the data is very large.

Considering that we have chosen to live in different places, we all need to overcome these problems, which means that we will face a lot of transformation.

How to solve the impact of the time delay on the business, including a variety of jitter and even the problem of network break. How to effectively distinguish the traffic we visit, to maximize the protection of the user's access to the correct room, this is a problem to be solved.

We have taken some measures:

1, as far as possible to make the business classified collection, so that a user's access to fall in the same place.

2, not all business has many live characteristics, but also can have the global use of the business (for example, user data), so need to do business type division.

How to define the boundary of the business call at the level of the Service division, and what we divide according to the traffic and the user, we use the geographical fence (POI) based on our business characteristic at present, the location of the user, merchant and rider is the basis of our entrance flow division.
Then look at the routing control this piece, in addition to the entrance flow of this layer to do the division of the room, but also in the internal use of virtual Shardingkey,shardingkey will be divided into parts of the national traffic, and POI tag binding, This can be logical shardingkey and physical location, switching time access can be diverted to different computer rooms with Shardingkey. Apirouter is to complete this work, it will be configured according to the rules of the shardingkey corresponding traffic to the corresponding room.
Dirty write prevention, in order to prevent data conflicts, we also need to do business with a number of live rules of the transformation, these rules on the business or some intrusive. Another soa-route is an internal access route that is invoked across the business. There is also a dal, which is the proxy layer of the database.
Business access is under our multiple layers of routing control, theoretically should be able to correctly route to the appropriate room, if beyond the rules or the accidental flow of the transformation of the rules to really penetrate the DAL layer when we are forced to refuse, because the bottom of the view that this access is an exception to the call, the flow of the wrong room, this time will be rejected. So we'd rather let him fail and not let the data outside the control come in, in order to guarantee the controllability of the rules and data.
Data consistency we have an important DRC data synchronization component, the database has some self control, and the DBA has developed a tool for data consistency verification (described later).
II. Multi-Living architecture

Once we've had some difficult points and our solutions, let's take a look at the whole structure of living. This above is our from the inlet flow, the shunt control, the data multiple engine room synchronization, including each important component's structure and so on. And there's Globol zone, which is what we just talked about, and the business that needs to be globally dependent will put it in Globol zone.

DRC This is a very important component of long live, mainly to solve the data in a number of computer rooms in the synchronous replication. The data we write from the Beijing engine room will be synchronized to the room in Shanghai. The data written in Shanghai's computer room will also be synchronized to the Beijing room by DRC this component. You can see that it actually contains three blocks of services, Replicator, applier and manager, one to collect change data, one to write change data to another, and one to do management control.

DB architecture This one we have two classes (and, exactly, a class that is more than a push, relatively few), the first category is Shardingzone, whether the data is written or access is the room to provide, the problem is only the flow of the switch, does not involve the bottom of the changes, this is really much alive structure.

There is a just said, some do not have the means to do business zoning, is the structure of globolzone, write is concentrated in a computer room, read in the local computer room to complete. You may think that globalzone this architecture will naturally face some of the problems of just talking about data latency, so this is our definition of some of the small amount of writing, the volume of traffic, the data latency is less sensitive to the business can be put here. Third, the database transformation

Click to have a surprise