Redis-cluster A single node is not available

Background:

The company's audio and video piracy detection business, using Redis-cluster as a key-value database, using the characteristics of the picture hash value as key. Because the amount of data is too large, the machine is limited, there is no slaver in the cluster

Problem description

Online Service error (SCard 4288594168) of pipeline caused error:moved 9239 10.xxx.xx.149:6390, this port is a node in the Redis cluster.

1 Use Redis-trib check 10.xxx.xx.149:6390 to report an error that cannot be connected

2 View 6390 node log, error error opening/setting AOF rewrite IPC pipes:numerical result out of range

Google, found this link linka:http://blog.csdn.net/syq521125/article/details/53006317 mentioned how to solve.

I restarted redis in accordance with his 2nd statement, my aof file was about 4.4G, about a few minutes after the service was up, the Redis-trib check 10.xxx.xx.149:6390 was re-executed, and the cluster already had access to 6390 nodes.

The memory consumption of the node I hung up has reached 14.5g, corresponding to the AoF file 4.4G. According to the problem of Linka, I need to expand the database nodes to reduce the amount of data in a single point.

I doubled the number of Redis instances in the cluster, the single point of memory consumption of about 7g, file 2G. I think it should be temporarily safe, in my experience, 10g of memory below, is also safe.

Their own psychological changes.

This is the first time I encountered online database hanging off, online problems, cautious, so hesitated for 1 hours, to do not restart, the middle also checked for a while, found that after the restart is OK, data recovery is very fast.

Redis-cluster A single node is not available