Scaling out => loss-coupled Architecture

According to the learning plan, this week's learning Master/Slave Mode + read/write splitting, sharding ).

This part of content is also my first contact. if the content is incomplete or inaccurate, Which of the following is recommended.

Summary:The core issue is to relieve the read/write pressure on the database and ensure the availability of the service (availability). Applications that are sensitive to data consistency,

How to ensure consistency through a reasonable architecture ). There are two policies: the commercial database e.g. Oracle adopts scaling-up,

That is to say, by upgrading the single-point software and hardware to control the read/write pressure, the key is not only high O & M costs, but poor scalability. Scaling-out

Through MySQL tools and data partitioning policies, distributed storage, backup, and data recovery must solve the join operation and consistency issues.

For the latter, whether it is to solve the read pressure by using master-slave & memcache, or to solve the write pressure by Using Vertical/horizontal partitioning,

EssentiallyIt is a loosely coupled architecture.

1. Single form library. It is estimated that ver1.0 is the basis of this lamp architecture. It takes only one week to develop basic functions of Sina Weibo!

Read/write/transaction locks; spof; Disaster Tolerance and backup... all kinds of constraints on performance and service quality. It is indeed simple, any computer

People in related majors can design such databases, but they tend to become slow and slow after a period of time.

Index addition may be temporarily mitigated, but its controllable data records are just hundreds of millions of records, and the overhead of write requests

Higher than read requests.

2. Master-slave: the division of server roles is introduced here. In this way, the master does not have to process all read/write requests.

For applications in the Read Request set, the Read Request (select) is distributed to each slaves, while the Write Request (insert/update/delete)

It is processed centrally by the master. Here we need to solve master-slave data synchronization problems caused by network latency, deadlocks, power loss, and so on.

Although not all applications require real-time results, you need to read the latest data on the master to ensure data consistency.

Especially after the introduction of memcache, the cache consistency issue needs to be carefully considered. The principle is: while recognizing the network delay and operation delay,

Ensure "eventual consistency. At the same time, for write-heavy applications, slaves's data synchronization operations will be more intensive, but inefficient.

3. vertical partitioning: Based on application functions, data with low correlation and no join operation is needed to be distributed to independent servers.

For example, user images, blogs, and voting. Distribute some important tables to related nodes through redundancy, and the join operation is still

Available. At this time, there is still only one master.

4, of course, similar to the system requirements in 2, you can subdivide the local M-S relationship, the whole form a tree structure.

But failing to perform a M-S configuration will make the system more difficult to manage and possibly lose part of the join functionality. And

The data records are not balanced in each table, and a table may increase dramatically.

5. Horizontal partitioning (sharding): after all, the above four architectures are still in single-Table mode, subject to "write ".

The idea of horizontal partitioning (database/table sharding) first appeared at 1996, breaking through the read/write bottleneck, and is still the core idea of memcache.

To deploy sharding, there are two main problems: (1) What: that is, partition based on the attribute, 'sharding/partitioning key'

(2) How: Partition Algorithm, 'sharding/partitioning scheme '.

There are also some classic problems to solve: (1) Cross-zone query (2) Data Consistency (3) Partition balancing...

Of course, there are also many Solutions e.g. MySQL cluster, hivedb

References:

Database sharding at netlog, with MySQL and PHP

Speech by Yang Weihua, Sina Weibo architect

How friendfeed uses MySQL to store schema-less data

Scaling-out experience shared by Facebook engineers

Sharding description

MySQL master-slave mode configuration Manual

Next learning key/value storage, schema-less, nosql