Building a highly available Web site Learning (iii)

Distributed construction

As a lifeline for Web site access (data access), of course, you can also use a distributed approach to reduce the access pressure of a single server. Previously there were Memcached distributed, but the Memcached servers were not communicating, so we also mentioned the master-slave distribution of redis. This article focuses on the basic ideas of distributed servers.

Master-Slave distribution of the database: Here I take MySQL For example, when MySQL needs to scale out the strategy is divided into three parts: copy, Split and data fragmentation, The main concern of master-slave distribution is the synchronization between the main library and the library. Principles such as:

　　

Describe the process roughly: from the server's IO thread, get the binary log from the primary server, save it locally as a trunk log, and then execute the contents of the relay log from the top through the SQL thread to keep it consistent from the library and the main library. The detailed process of master-slave synchronization is as follows:

1. The primary server verifies the connection.

2. The primary server opens a thread from the server.

3. shift the primary server log from the server to the primary server.

4. the primary server checks to see if the value is less than the current binary log bias shift.

5. if it is less than, notify the server to fetch the data.

6. from the server continues to fetch data from the primary server until the completion, at this time, from the server thread into sleep, the primary server thread into sleep.

7. when the primary server has an update, the primary server thread is activated and the binary log is pushed to the slave server, and the server thread is notified to enter the working state.

8. Executes the binary log from the server SQL thread and then goes to sleep.

Of course, a master repository can allocate read to multiple repositories in a small amount of write and bulk reads. In fact, this configuration also has a great use:

1. Use different repositories for different roles.

2. Use a standby as the main repository for use, with no additional data transfer in addition to replication.

3. Place a standby library in a remote data center for disaster recovery.

4. Delay one or more standby libraries for disaster recovery.

5. Use one of the repositories to use the server as a backup, training, development, or test.

6. Create a log server

In addition,Mysql supports other replication topologies, such as Master - master replication in active - Active mode , active - Master in passive mode - primary replication (similar to creating a hot backup, but can also perform read operations, backups, "offline" maintenance and upgrades, etc.); ring copy; Main Library, distribution main library, and standby library (on the Distribution main library table changes to blackhole storage Engine)

For Mysql extension problems can also use load balancing and other means to solve, the content will have to learn, and later also out of the blog in their own language to describe it.

Redisthe master-slave distribution: itself is also through consistencyHashfor sharding, the advantages:Mastercan have multipleSlave. does not blockMaster, when one or moreSlavewith theMasterwhen you synchronize data for the first time,Masterthe client's request can continue to be processed. Instead,Slavewhen the data is first synchronized, it is blocked so that the client's request cannot be processed. In theMasterdata persistence is prohibited on the server.

Master-Slave synchronization generally divided into two phases: the first phase:Slavethe server actively connects toMasterserver. SlaveServer SendSYNCcommand toMasterThe server requests synchronization. Masterthe server backs up the database toRdbfile. MasterputRdbFile Transfer toSlaveserver. SlaveThe server empties the database data andRdbThe file data is imported into the database. NextMasterthe UseAll operations are forwarded to theSlaveserver.

MongoDB Shard Technology:

Look at the auto-shard of MongoDB : After http://blog.fens.me/mongodb-shard/, there is a lot to learn about MongoDB's shard cluster.

A shard can also be considered when one of the following three conditions is met:

(a) The data set size is close to the storage capacity of a single node.

(b) The active data volume is close to the maximum memory capacity of the node.

(c) The write request speed of the node cannot meet the requirements. (The read request speed can not meet the requirements of the time by reading and writing separation or Replicset mode)

The components of the Shard are as follows:

　　　　

MONGOs, the entrance to the database cluster request, all requests are coordinated through MONGOs, do not need to add a route selector in the application, MONGOs itself is a request distribution center, it is responsible for the corresponding data request request forwarded to the corresponding Shard server. In a production environment there is usually more mongos as the entrance to the request, preventing one of the other MongoDB requests from being hung out of operation.

Config server, as its name implies, configures the configuration of servers, storing all database meta-information (routing, sharding). The mongos itself does not have a physical storage Shard server and data routing information, but is cached in memory, and the configuration server actually stores the data. MONGOs the first boot or shutdown reboot will load configuration information from config server, and if configuration server information changes will notify all MONGOs to update their status, so that MONGOs can continue to route accurately. In a production environment there are usually multiple config server configuration servers, because it stores the metadata of the Shard route, this can not be lost! Even if you hang one of them, as long as there is inventory, the MongoDB cluster will not hang off.

Shard, this is the legendary shard. The above mentioned a machine even if the ability to have a large ceiling, as the military war, a person can drink blood bottle also can not spell the other one's division. As the saying goes Three stooges the top of Zhuge Liang, this time the strength of the team is highlighted. In the Internet, too, a common machine can do more than one machine to do.

　　Summary : I still have a little bit of learning about MongoDB. After all, there are a lot of books and materials like MySQL research. My English level also needs to be strengthened, try to look at the official documents. Learn more about the essence. The blog about MongoDB will then be updated.

　　

Building a highly available Web site Learning (iii)