Thoughts on MongoDB application practices and practices

Thoughts on MongoDB application practices and practices
I recently studied MongoDB and used it to quickly build a flexible no schema storage system. Through demonstration and analysis, this article uses MongoDB to quickly build a storage system with good performance and availability to meet the needs of hundreds of millions of users. In the selection of NoSQL databases, You need to integrate with your ownData Model,Access MethodAndCostTrade off ).
MongoDB (2.6.4) has the following features: Availability:
1. Supports highly available and flexible service cluster configuration, including Master/Slave, replica set, and automatic sharding mode.
2. document-based query, high performance, simple query of tens of thousands of QPS.
3. Full-text search is supported.

Consistency:
1. Supports the update mode in the document, which is efficient.
2. In the latest version 2.6, the read/write locks and write locks take precedence over the collection level.

Ease of use:
1. SQL usage similar to that of traditional relational databases.
2. Rich management and configuration tools.
3. Support the permission management system based on user roles.

Storage Mechanism:
Mmap file + memory index is used. The operating system is responsible for memory and Cache Management. When the data set size exceeds the upper limit, the performance decreases. In Linux, memory control can be implemented by configuring user ulimit-u.
Actual needs: 1. Flexible semi-structured data with fields needs to be stored. 2. 1 ~ The storage scale of 0.2 billion records, with an average of 20 k per record. 3. The read requirement is much greater than that of writing (calculated at). Writing is performed in batch. Read requires flexible and complex query methods. 4. Write Performance: qps read performance: qps
Because of MongoDB's flexible storage and access methods, as well as excellent query performance, scalability, and maintenance costs, I think of using MongoDB to store about 0.2 billion of the data volume. The requirements are analyzed as follows: 1. The total storage capacity is about 4 TB in 0.2 billion scale. Currently, server hard disks on the market have a maximum capacity of 2 TB. during deployment, you can consider using LVM to install 1 ~ 2 TB disks, which can be easily resized as needed when capacity increases. 2. If there is already 0.1 billion million data records inserted, and 4 indexes are written each time, the qps can reach 8000 to meet the write performance requirement of 5000 qps. 3. when a single thread reads data in a simple way, the qps can reach qps, and the overall performance of concurrent multi-thread reading is close to qps, which can meet the query performance of qps, if you want to increase the read performance in the future, you can consider enabling the read permission to share the read Pressure on the node. During the test, the read latency is less than 1 ms.
Based on the results of the demand demonstration, it is feasible to use MongoDB to store the 0.2 billion records. In actual deployment, MongoDB supports multiple methods, including Master/Slave, replica set, and shard. Compared with the master-slave mode, the replica set has the advantage of automatic failover, but it also brings the complexity and the disadvantage of increasing machine costs. Therefore, after comprehensive consideration, select the master-slave mode for deployment, and select the server configuration16 Physical cores + GB memory + 2 TB hard driveTo build the master and slave nodes. The read permission is enabled for the slave node. On the one hand, the application layer can initiate read requests to the slave node when the master node is not readable, and on the other hand, the front end can allocate some read requests to the slave Node Based on the load. Because data writing is performed offline, you only need to monitor the status of the master and slave nodes and restore the service status in time.
Through the above practices, we have completed the rapid construction of MongoDB to meet the needs of hundreds of millions of storage.

Mongodb learning materials and videos

You can buy this MongoDB authoritative guide. This is the only mongodb tutorial that can be translated into Chinese.

You can find the driver on www.mongodb.org. Mongodb supports drivers in many languages, but the documentation is in English. You must have a certain degree of English Reading Capability, because many problems may occur during the driver process, if you do not check the document, you must be blind.

Who has such video for MongoDB application development (basic, Development Guide, system management, cluster and system architecture?