MySQL's basic architecture continues

" problems introduced after data splitting "

The problem of data horizontal splitting is mainly only through sharding key to read and write operations, such as the Sharding key to the userid of the segmentation example, read the details of the UserID, you must first know the UserID, In order to figure out which cluster to query, assuming that I need to retrieve the user information by username, I need to introduce an additional reverse indexing mechanism (similar to the HBase two-level index), such as storing username->userid mappings on Redis, The example of username query becomes the first query Username->userid, and then through the UserID query the corresponding information.

In fact, this is very simple, but we do not overlook an additional hidden danger, that is the hidden danger of data inconsistency. The username->userid stored in Redis and the userid->username stored in MySQL must be consistent, which is a very difficult thing to guarantee, for example, for changing the username. You need to modify Redis and MySQL at the same time, these two things are difficult to do business assurance, such as MySQL operation success but Redis operation failed (Distributed transaction introduction cost is higher), for Internet applications, usability is the most important, consistency is second, So we can tolerate a small amount of inconsistency. After all, this type of inconsistency can be negligible (typically write updates also use MQ to ensure that the retry operation is not stopped until successful).

　　In this architecture, let's look at the bottleneck of data storage.
In this split idea to build up the structure, theoretically there is no bottleneck (sharding key can ensure that each cluster traffic is relatively balanced under the premise), but there is a disgusting thing, that is cluster expansion of the time to redo the cost of data, such as I originally had 3 cluster, But now that my data is growing faster and I need 6 cluster, then we need to split each cluster into two, and the general practice is
1. Pick a Slave, stop syncing,
2. To write record increment log (implementation can be the business side to write more than once write persisted MQ or MySQL master create trigger record write and so on)
3. Start data on static slave, split to two
4. Play back the incremental write until all the increments are caught, basically keeping pace with the original cluster
5. Write switch, switch from the original 3 cluster to 6cluster

There is no like the air refueling feeling, this is a dirty work, dirty, prone to problems, in order to avoid this, we generally at the beginning of time, design enough sharding cluster to prevent the possibility of cluster expansion this thing

Reference: http://www.cnblogs.com/Creator/p/3776110.html

MySQL's basic architecture continues