Common Redis performance problems and solutions

1. when the Master writes a memory snapshot, the save command schedules the rdbSave function, which will block the work of the Master thread. When the snapshot is large, the performance will be greatly affected and the service will be suspended intermittently, therefore, it is recommended that the Master node not write memory snapshots. 2. MasterAOF persistence. If the AOF file is not overwritten, this persistence method has the least impact on the performance, but the AOF file will

1. when the Master writes a memory snapshot, the save command schedules the rdbSave function, which will block the work of the Master thread. When the snapshot is large, the performance will be greatly affected and the service will be suspended intermittently, therefore, it is recommended that the Master node not write memory snapshots. 2. Master AOF persistence. If the AOF file is not overwritten, this persistence method has the least impact on the performance, but the AOF file will

1. when the Master writes a memory snapshot, the save command schedules the rdbSave function, which will block the work of the Master thread. When the snapshot is large, the performance will be greatly affected and the service will be suspended intermittently, therefore, it is recommended that the Master node not write memory snapshots.

2. Master AOF persistence. If the AOF file is not overwritten, this persistence method has the smallest impact on the performance, but the AOF file will keep increasing. The AOF file will affect the recovery speed of Master restart after the Assembly.

1. when the Master writes a memory snapshot, the save command schedules the rdbSave function, which will block the work of the Master thread. When the snapshot is large, the performance will be greatly affected and the service will be suspended intermittently, therefore, it is recommended that the Master node not write memory snapshots.

2. Master AOF persistence. If the AOF file is not overwritten, this persistence method has the smallest impact on the performance, but the AOF file will keep increasing. The AOF file will affect the recovery speed of Master restart after the Assembly.

3. When the Master calls BGREWRITEAOF to rewrite the AOF file, AOF will occupy a large amount of CPU and memory resources during rewriting, resulting in service load being too high and service suspension temporarily.

The following is my actual project, which is probably like this: one Master, four Slave, no Sharding mechanism, only read/write splitting, the Master is responsible for write operations and AOF log backup. The AOF file is about 5 GB, And the Slave is responsible for read operations. When the Master calls BGREWRITEAOF, the load of the Master and Slave will suddenly increase sharply, the write requests of the Master node basically do not respond. It lasted for about five minutes, and Slave's read requests could not respond in a timely manner. The server load diagrams of the Master and Slave servers are as follows:

Master Server load:

Slave server load:

The above situation should not have happened because the previous Master's machine was Slave and there was a shell scheduled task on it that called BGREWRITEAOF to rewrite the AOF file at ten o'clock A.M. every day, later, because the Master machine was down, the Slave of the backup was cut into the Master machine, but this scheduled task was forgotten to be deleted, which led to the above tragedy, the reason was found only a few days later.

Setting the no-appendfsync-on-rewrite configuration to yes can alleviate this problem. Setting it to yes indicates that the new write operation is not fsync during rewrite and is temporarily in memory, wait until the rewrite is complete before writing. It is best not to enable the AOF backup function of the Master.

4. redis Master-Slave replication performance problems, the first implementation of the Slave synchronization to the Master is: slave sends a synchronization request to the Master, the Master first dump the rdb file, and then the rdb file is transmitted to the Slave, then the Master forwards the cached command to Slave, and the initial synchronization is completed. The second and later synchronization implementation is that the Master directly sends the snapshot of the variable to each Slave in real time. No matter what causes the server Load balancer and Master to be disconnected and reconnected, the above process will be repeated. Redis master-Slave replication is based on the persistence of memory snapshots. As long as there is a Slave, there will be a memory snapshot. Although Redis claims that Master-slave replication is not blocked, due to disk io restrictions, if the Master snapshot file is large, dump will take a long time and the Master may not be able to respond to the request during this process, that is to say, the service will be interrupted, which is terrible for key services.

The root cause of the above 1.2.3.4 problem is inseparable from the system I/O bottleneck, that is, the hard disk read/write speed is not fast enough, and the main process fsync ()/write () operation is blocked.

5. single point of failure (spof), because the current master-slave replication of Redis is not mature enough, there is an obvious single point of failure problem. Currently, only solutions can be made by yourself, such as active replication, proxy is used to replace the Master with Slave. This is also one of the priorities of Redis authors. The author's solution is simple and elegant. Details can be seen in Redis Sentinel design draft? Http://redis.io/topics/sentinel-spec.

Summary:

1. It is recommended that the Master do not perform any persistence work, including memory snapshots and AOF log files. In particular, do not enable memory snapshots for persistence.

2. if data is critical, an Slave enables AOF to back up data, and the policy is to synchronize data once per second.

3. For the speed of Master-Slave replication and the stability of the connection, Slave and Master should be in the same LAN.

4. Avoid adding slave databases to the master database with high stress.

5. for the stability of the Master node, do not use a graph structure for Master-slave replication. The one-way linked list structure is more stable, that is, the Master-slave relationship is: Master <-Slave1 <-Slave2 <-Slave3 ......., This structure can also easily solve the single point of failure (spof) Problem and implement Slave's replacement of the Master. That is, if the Master fails, you can immediately enable Slave1 for the Master, and the others remain unchanged.

Write so much for the time being today and try again later.

-----

Reprinted from: http://zhupan.iteye.com/blog/1576108

Original article address: Redis common performance problems and solutions. Thank you for sharing them with me.