Parameter Adjustment when deploying MongoDB

Parameter Adjustment when deploying MongoDB

Deploy the MongoDB production server and give the following suggestions:

• Use a virtualized environment;
• System Configuration

1) recommended RAID configuration

RAID (Redundant Array of Independent Disk, Redundant Array of Independent Disks) is a technology that allows us to use multiple disks as a separate Disk. You can use it to improve disk reliability or performance, or both. A group of disks using RAID technology are called RAID disk arrays.

Depending on the performance, RAID has multiple configuration methods, which usually take into account both speed and fault tolerance. The following are the most common configuration methods:

• RAID0
  Disk striping is used to combine multiple disks to improve performance. Each disk stores a part of data, similar to the sharding in mongodb. Because multiple underlying disks exist, a large amount of data can be written to the disk at the same time. However, if one of the disks fails and data is lost, the data will not be backed up, which will also lead to slow reading speed (especially on Amazon's Elastic Block Store service ),
  Because some data volumes may be slower than others.
• RAID1
  Use images to improve reliability. The same data copy is written to every member of the array. The performance of this method is lower than RAID0, because a slow member in the array slows down the write speed of the entire array. However, if one of the disks fails, you can find data copies on other members of the array.
• RAID5
  Based on the disk Partitioning technology, data verification information is stored to prevent data loss caused by server faults. In general, RAID5 can automatically handle a disk fault, and the user will not feel the fault. However, this also makes RAID 5 the most slow in these RAID configuration schemes because it needs to calculate the verification information when writing data. Mongodb is typically used for writing a small amount of data multiple times. Therefore, the cost of using RAID 5 is particularly high.
• RAID10
  RAID10 is a combination of RAID0 and RAID1: data is split to improve the speed, and images are copied to improve the reliability.

We recommend that you use RAID10, Which is safer than RAID0 and can solve the performance problem of raid1. RAID1 is a waste of resources on the basis of the replica set, so you can select RAID0.

Do not use RAID5. It will be very slow.

2) Virtualization

A. Disable excessive memory allocation.

The value of memory overcommitting determines the policy that should be adopted when the process requests excessive memory to the operating system. Based on this setting, the kernel may allocate memory for the process, even if the memory is currently unavailable (the expected result is that it has become available when the process uses this memory ). This kind of kernel promises non-existent memory to the process, which is called over-memory allocation. This feature makes Mongodb unable to operate well.

Vm. the overcomit_memory value may be 0 (let the kernel guess the Excessive allocation size), and may be 1 (meeting all memory allocation requests ), it may be 2 (the allocated virtual address space cannot exceed the sum of the swap space and a small part of the excessive allocation ). Setting this value to 2 is the most complex and best choice. Run the following command to set this value to 2:

$ Echo 2>/proc/sys/vm/overcommit_memory

After changing this setting, you do not need to restart Mongodb.

In this directory,/proc/sys/vm/overcommit_memory

This file specifies the kernel memory allocation policy. The value can be 0, 1, or 2.

0 indicates that the kernel will check whether there is enough available memory for use by the process. If there is enough available memory, the memory application will be allowed; otherwise, the memory application will fail, and return the error to the application process.

1 indicates that the kernel allows all physical memory allocation regardless of the current memory status.

2, indicating that the kernel can allocate more memory than the total physical memory and swap space (refer to overcommit_ratio ).

Default Value: 0.

3) in the document, the disk Performance Parameters in proc in Linux are involved as follows:

Mongodb's mmap does not call fsync to forcibly fl to the disk, and the operating system will also automatically fl to the disk. in linux, The dirty_writeback_centisecs parameter is used to define the time when dirty data stays in the memory (500 by default, that is, 5 seconds). After this timeout time, the system will be flushed to the disk. All IO operations will be blocked during the automatic refreshing process. If a large amount of data needs to be refreshed, some time-consuming operations may be generated, for example, some mmap programs may encounter time-out operations at intervals. The general optimization method is to modify the system parameter dirty_writeback_centisecs to speed up dirty page flushing and reduce the time consumption. Mongodb is a regular and strong brush, so this problem does not occur.

In this directory,/proc/sys/vm/dirty_writeback_centisecs

This parameter controls the run interval of the kernel's dirty data refresh process pdflush. Unit: 1/100 seconds. The default value is 500, that is, 5 seconds. If your system writes data continuously, it is better to reduce the value. In this way, you can split the peak write operation into multiple write operations. The setting method is as follows:

Echo "100">/proc/sys/vm/dirty_writeback_centisecs

If your system is a short-term peak write operation, and the data is not written (dozens of MB/time) and the memory is rich, you should increase the value:

Echo "1000">/proc/sys/vm/dirty_writeback_centisecs

4) logs

By default, mongodb sends logs to stdout (standard output, usually a terminal ). Most initialization scripts use the-logpath option to Send Logs to files. If multiple MongoDB instances (such as one mongod and one mongos) exist on the same machine, ensure that the logs of each instance are stored in a separate file. Make sure that you know the location where logs are stored and have the read and access permissions for the files.

Mongodb outputs a large number of log messages, but do not use the -- quiet option (this option hides some log information ). It is usually good to keep the log level as the default value. At this time, the log has enough information for Basic Debugging (such as the time consumed or the cause of startup exception), but the log space is not large. When debugging an application for a specific problem, you can use some options to obtain more information from the log.

First, when you restart Mongodb, you can add more "v" (namely-v,-vv,-vvv,-vvvv, or-vvvvv) to the parameter ), or run the following setParameter command to change the log level.

> Db. adminCommand ({"setParameter": 1, "logLevel": 3 });

Remember to reset the log level to 0, otherwise there will be too much unnecessary content in the log. You can increase the log level to 5. Then, mongod records almost all operations in the log, including the content processed by each request. Because mongod writes all the content to a log file, it can generate a large number of disk read/write operations (IO), thus slowing down a busy system. To view all ongoing operations in real time, it is better to open the analyzer:

By default, MongoDB records query information that takes more than 100 milliseconds. If 100 ms is not applicable to applications, you can use the setProfilingLevel command to change this threshold:

> // Only records query operations that take more than 500 milliseconds

> Db. setProfilingLevel (1,500)

{"Was": 0, "slowms": 100, "OK": 1}

> Db. setProfilingLevel (0)

{"Was": 1, "slowms": 500, "OK": 1}

The second command will disable analyzer, but the value in milliseconds in the First Command will continue to take effect as the threshold value for logging in all databases. You can also use the-slowms option to restart MongoDB to change this threshold.

Finally, set a scheduled task to split (rotate) log files every day or every week. For example, use the-logpath option to start MongoDB and send a SIGUSR1 signal to the process even if it separates logs. You can also use the logRotate command for the same purpose:

> Db. adminCommand ({"logRotate": 1 })

If MongoDB is not started using the -- logpath option, logs cannot be split.

5) Disable atime of database files

Disabling the system to update the file access time effectively improves the file reading performance. You can/Etc/fstabFile addedNoatimeParameters. For example:

/Dev/xvdb/data ext4 noatime 0 0

Modify the file and try again.MountYou can:

# Mount-o remount/data

6) Disable NUMA

In a multi-processor Linux System Using NUMA technology, you should disable NUMA. MongoDB running performance in the NUMA environment may sometimes slow down, especially when the process load is very high.

------------------------------------------ Split line ------------------------------------------

CentOS 6 install MongoDB and server configuration using yum

Install MongoDB2.4.3 in Ubuntu 13.04

MongoDB beginners must read (both concepts and practices)

MongoDB Installation Guide for Ubunu 14.04

MongoDB authoritative Guide (The Definitive Guide) in English [PDF]

Nagios monitoring MongoDB sharded cluster service practice

Build MongoDB Service Based on CentOS 6.5 Operating System

MongoDB details: click here
MongoDB: click here

This article permanently updates the link address: