Read/write consistency of distributed systems from MongoDB Replica Set HA

Replica Set basic Replica Set is a decentralized backup mode provided by mongodb (mongodb also provides master-slave deployment and sharding mode). Each mongod can be a master, A primary is automatically selected in the replica set, and the others are currently seconary. After the primary fails, a new primary is automatically elected. All mongod In the replica set stores the complete set of data. The secondary node synchronizes data operations from primary to ensure that its data is up-to-date. The replica set has its own election mechanism, which is relatively simple. It selects an available node based on factors such as heartbeat and weight for primary.

Replica set

Only one primary in the replica set can be written to maintain strict consistency (primary is strictly consistent, and secondary is the final consistency ). Primary writes the log in the oplog. The secondary node asynchronously copies the primary oplog (for convenience, each oplog In the replica set can be obtained and copied to each other ), operation is applied to your own dataset. Operations on datasets in oplog are idempotent. When mongod is started, an oplog of the default size will be generated. To change the oplog size, you need to set it before startup, and to set it after startup, you need to go through another process, which is more troublesome. If the replica set is an even number, you can add an arbiter. If the replica set is not maintained, the replica set only participates in voting. Member has other states, such as delayed member and hidden memberclients. By default, the primary dataset is read, and the secondary dataset can be read. Since secondary asynchronously implements operation on its own dataset Based on oplog, there must be a delay. There are two ways to synchronize replica set data: 1. initialize synchronization: for new member without data or missing data sets with some historical records lost, copy the entire data of a member in the existing replica set. The whole process is to clone data, apply all changes, and finally create an index. 2. uninterrupted synchronization: Generally, secondary is synchronized from primay (timed ping primary to view state ). To ensure Durability, journaling is provided on a single thread. Each write needs to be written into journal first and then to the dataset (the journal is available only in 64-bit mongodb2.0 or later by default ). For multi-thread Replication, mongodb uses multi-thread batch writing to ensure concurrency. batch processing is grouped by namespace, and the concurrent writing in each namespace is sequential. When writing in batches, mongodb does not allow read. When Journaling is used, mongodb first stores the write operation records in the memory and implements them in journal. The default journal path is/data/db. When mongod is started, a certain size is pre-allocated, and the journal files in it are append-only. The maximum size of jfile can be set. When jfile is greater than 1 GB, a new jfile is created. The jfile path can also be specified to speed up reading and writing to other file systems. Once mongodb completes all write operations, these jfiles will be deleted. Briefly describe the process from memory to jfile and then to the disk dataset. Journalling provides three views: shared views, private views (read-only), and ?. They have different permissions. Mongodb uses batch processing to first fl write operations in memory to jfile in private view mode, and then flush the contents in jfile to the disk in share view mode, in this case, the dataset is updated to the latest version. Summary Mongodb improves availability by means of replica sets. On the one hand, replica sets are decentralized, which can automatically detect and elect new primary, synchronize data between nodes, and use mongos to route intersections, make sure that the data is transparent to the client. On the other hand, as long as you can tolerate its eventual consistency, the secondary node can also provide read. In Versions later than Mongodb2.0, journaling is added to enhance reliability and ensure that after your primary crashes, the previous modification operations are still recorded in journal, it can be restored or abolished to avoid data inconsistency or dirty data. Journaling is similar to the practice adopted in many distributed systems. It involves memory, which involves writing to journal first, and then executing to a real data set after writing successfully. This article tries to analyze the mongodb replica set to give a simple understanding of the read/write consistency problem of the distributed system. Please forgive me for any mistakes. (Full text)

MongoDB details: click here
MongoDB: click here

MongoDB backup and recovery

CentOS compilation and installation of MongoDB

CentOS compilation and installation of php extensions for MongoDB and mongoDB

CentOS 6 install MongoDB and server configuration using yum

Install MongoDB2.4.3 in Ubuntu 13.04

How to create a new database and set in MongoDB

MongoDB beginners must read (both concepts and practices)

MongoDB authoritative Guide (The Definitive Guide) in English [PDF]