MongoDB summary MongoDB structure division MongoDB C # driver tutorial)

Production Environment Best Practices
1. Linux:
1] disable the atime option of the file system/partition
VI/etc/fstab
Add noatime and nodiratime after the corresponding partition item
Label =/1/ext3 defaults 1 1
Label =/data1/Data ext4 defaults, noatime, nodiratime 1 2
2] set the file handle to 4 K +. Currently, this configuration has been integrated into the startup script.
VI/etc/security/limit. conf
* Soft nproc 65536
* Hard nproc 65536
* Soft nofile 65536
* Hard nofile 65536
3] Do not use large VM page (do not use the Large Memory Page option)
LinuxLarge Memory Page reference: http://linuxgazette.net/155/krishnakumar.html
4. Use dmesg to view host information.
2. Linux File System Selection:
MongoDB uses pre-allocated large files to store data. We recommend that you
1] ext4
2) XFS
3. kernel version:
The network is skeptical about the performance of 2.6.33-31 and 2.6.32, and it is strongly recommended that the 2.6.36. ext4 file system data loss bug affects multiple stable versions of the kernel.Ext4 users should avoid using Linux
3.4.14, 3.4.15, 3.5.7, 3.6.2, and 3.6.3.
4. Thread stack size
The default thread stack size is 10 m, adjusted to 1 m, and has been integrated into the startup script.
Summary and suggestions in the project process
1. Case sensitivity
MongoDB is case sensitive by default.MySQLThe same problem? (MySQL Zone
Case Sensitive, resulting inWindowsThe table name and field name in Linux are inconsistent ).
If it is not particularly useful, we recommend that you use a table name with all lowercase letters.
2. shorten the length of field names as much as possible
MongoDB schema free causes each data to store its key and attributes, which leads to
Massive data redundancy. Developers may consider that the key designed from Yi Yun is basically long, basically all
It is designed literally. This leads to a long key. The corresponding data storage occupies a lot of space.
When necessary, you can consider creating a map table of key and actual significance to minimize the length of the key.
Example:
// Basic Information
Static string _ id = "_ id ";
Static string status_code = "SC ";
// Buffer
Static string date = "date ";
Static string max_age = "Age ";
// Content
Static string content = "content ";
Static string content_type = "ctype ";
Static string content_length = "clen ";
Static string zip = "Zip ";
3. the maximum number of indexes in a single MongoDB table is 64.
The maximum data volume for non-index sorting is 4 MB. If the maximum data volume is exceeded, an error is returned and the system exits.
We recommend that you set the where condition on the index field as much as possible. You must create an index for the sorting field. The usage principle of the index is as follows:Oracle
Consistent with MySQL, minimize the number of indexes and the index length.
Only one index can be used for MongoDB queries at a time, and data queries are not executed concurrently.
For example, DB. Tab. Find ({'id' = 1, 'name' = 2}) if the 'id' and 'name' columns have indexes
It is of little significance for improving the query efficiency. If the index is ('id', 'name'), the efficiency is greatly improved.
4. Add fields to MongoDB
If a field is added with a default value, all data needs to be modified. This is also a consideration during the design phase.
Another solution to this problem is to make a judgment in the application code.
5. Password issues during the test
ForDatabasePassword verification should be added for the MongoDB used in the Code test phase. Currently
There will be issues with password verification (password verification is not performed if the cache is used ).
6. Data source connection method
Use the Connection Pool Mode to minimize extra performance consumption caused by Authentication
We recommend that you use the standard URI connection mode: MongoDB: // User: passwd @ host: Port, host: Port/DB
7. MongoDB log volume
Under normal circumstances, you do not need to enable the-V log option.
MongoDB-V logs are suitable for online deployment of debugging in the development environment. This parameter is not recommended currently.
When deployed, the-V log will have several GB of logs each day. Remove this parameter and perform operations related to data query.
No logs will be recorded, and important operations inside the database will still write logs.
8. Connections size settings
The MongoDB driver uses a connection pool to connect to the database. Currently, application 1 is observed.
When this function is enabled, all connections are established based on the variable settings and provided to the program. Once a connection is enabled
If the access to the database fails, the connection pool to the database is cleared and the connection is established again.
MongoDB is a lazy and passive cleanup method for the garbage collection of disconnected connections.
The number of established connections is too large. Once a reconnection occurs, a large amount of junk connection data will be accumulated on the Mongo end, resulting in
Host resources are exhausted.
Suggestion: Generally, the size of the Connection Pool driven by MongoDB should be controlled below 100. Generally, the value ranges from 30 to 50.
Supports application access.
9. Lock Problems
MongoDB locks all access to the database. If it is a query request, it sets a shared lock and a data modification request,
The global exclusive lock is set and is an instance-level exclusive lock. The write lock blocks read requests.
Write locks block read requests of the entire instance.
Deployment suggestions:
1] Generally, we recommend that you do not use a set of examples for different applications.
2] if resources are not met and need to be used together, an application with the same attributes should be used together with a set of instances.
For example, the Mongo applications under the Contract read more and write less, to prevent one write multiple applications from blocking read requests.
10. About MAP/reduce
MongoDB supports MAP/reduce in a single thread. We do not recommend using this function in the foreground.
It is implemented through MAP/reduce, and should be used with caution during the development process.
11. Security Issues
1] MongoDB runs on MongoDB users and prevents MongoDB users from logging on.
2] use the MongoDB built-in authentication method (adduser, auth) to restrict user access behavior
3. Place MongoDB in an Intranet environment
4] When MongoDB must be exposed to the Internet, use network-layer technologies such as iptables for protection.
5] The content on the network layer is plain text transmission. You can consider storing Encrypted documents, applications, and encryption and decryption.
12. Performance monitoring
MongoDB self-contained Performance Data Collection System
Mongostat collects multiple database indicators in real time and provides the HTTP Console port number as the application port number + 1000.
Main Performance Indicators of interest:
1] faults: displays the number of page faults on MongoDB per second. This is the ing from MongoDB to virtual address space,
Instead of physical memory, if this value is too high, it may mean that the machine does not have enough memory
Store Data and indexes.
2) flushes: How many fsync operations are performed per second, and how many times the data is refreshed into the disk
3] locked: Write lock
4] idx Miss: Index Miss percentage
5) QR | QW: length of the read/write lock Request queue.
6) Conn: number of established connections.
Other commands:
DB. Stat ()
DB. serverstatuse ()
DB. collection. Stats ()
13. Fragmentation issues
If the data of the MongoDB database is frequently modified, serious space fragmentation problems may occur, as shown in the disk
File expansion does not match the actual data volume, the memory is insufficient, the index hit rate is low, and the query efficiency is reduced.
Currently, the version we use is not very effective.
You can use dB. repairedatabase () to sort databases. This process is very slow.
In master-slave mode, the master-slave switchover is performed, and the slave database is created again.
In a replset architecture, you can stop the database, delete the data directory, and synchronize all data from the replication group,
In this case, the oplog size should be considered.
A general step:
1. Call Rs. Freeze (1200) first, and make every machine that does not want to make it a primary so that it cannot become a master within 1200 seconds.
Primary (you can skip this step)
2. Stepdown the primary so that the new primary will get up.
3. Kill the original primary.
4. Delete AllDataData (Calling Repair is very slow, so it is better to get rid of it again)
5. Restart the original primary process.
6. Rebuild the entire replication group cyclically.
14. SystemBackup:
Currently, MongoDB does not support online backup. Only offline backup is supported.
Our architecture is replset and master-slave.
Based on our current Architecture and Data Consistency requirements, we have not arranged relevant backup systems.
15. MongoDB connection problems in application code
Some applications may have the following two minor issues when using MongoDB:
1. During application startup, the application requires that all connections in the connection pool be established to make the application correct.
It is often started. This approach is not advisable because of network problems, MongoDB rejects connection, or MongoDB is suspended, such
If no external try catch is added for protection, the port cannot be started after constant restart.
2. Some applications have been configured with safe = true, W = 1 in the MongoDB connection pool.
When inserting data or updating data, MongoDB must write the updated data to the disk and return the update successful.
To the program. If the application is under high access pressure, MongoDB will be slow and may be suspended,
In this case, we need to assess the data consistency requirements and make appropriate adjustments. We generally recommend that you disable this option.
16. Added some development issues.
1] Skip + limit page flip, the more slow the back, some materials say that using the array element paging can solve, not yet
I tried it. The more reliable method is to first find the ID of the last time and do not use skip when turning pages:
Last_row_id = objectid ('....');
DB. activity_stream-> Find ({_ ID: {$ LT: last_row_id },
User_id: 20}). Sort ({_ ID:-1}). Limit (10 );
2]. select only the fields that are actually needed
3]. When updating a piece of data, it is found first and then updated to reduce the lock time.
4]. indexes are used only for queries that return few results. Otherwise, too much data is loaded, Which is slower than no index.
5]. When there are many attributes, establishing a hierarchical relationship can improve the query efficiency. Otherwise, each record will be retained once.
To find the desired property

MongoDB structure division

MongoDB C # driver tutorial)