Segmentation of large Redis Database example _redis

The partnership feature on the Mint APP uses a large memory database Redis, and as the volume of data grows fast, Redis expands fast and is close to GB, all in a single Redis instance. A single huge Redis instance has the following disadvantages:

1. First, a machine with a large memory is needed. Redis is a memory database, it needs to put all the requirements in memory, need to install a GB of Redis instances, at least a GB of memory size of the machine, consider the reservation growth space, generally requires 12 * 1.5 GB of memory. There is also a factor to consider is that Redis hard disk data storage, the fork process needs to consume the same size of memory, so a 12GB Redis instance needs about GB of memory more appropriate, this machine put forward a very high demand, often difficult to meet.

2. Then, Redis easily becomes a performance bottleneck. Redis's concurrency model is a single process single-threaded, it does not take full advantage of multi-core CPUs, when the number of requests is high, or some request processing is slow (such as some large data sorting), may become the system performance bottleneck. There are ways to alleviate even this problem, is to create multiple Redis instances, through multiple redis connections to achieve.

3. In addition, a single large Redis instance will also increase the difficulty of data management, because such a large amount of data, whether replication, backup operations are relatively slow, easy to impact on the online system.

Therefore, it is very necessary to separate a huge redis instance into several small redis instances.

Using the Redis replication mechanism, the Redis instance segmentation can be smoothed on line, which will hardly affect the system greatly.

The specific operation ideas of the segmentation are as follows:

1. First, planning redis segmentation strategies, usually based on business division, such as the Mint partner is based on the business split into timeline, User_relationship, and other 3 Redis instances. After planning, you need to modify the Redis program code in your application based on your planning results, usually with a unified Redis link to multiple Redis connections, and different services using different connections.

2. Then, create multiple Redis replicas via Redis replication to allow different Redis connections to use a different Redis copy to delete the extra data in the Redis copy. To bulk delete the keys of a pattern, you can use the following shell command:

Copy Code code as follows:

REDIS-CLI KEYS "<pattern>" | Xargs redis-cli DEL

into the actual pattern, as

Copy Code code as follows:

REDIS-CLI KEYS "User:*:followers" | Xargs redis-cli DEL

Indicates that the user followers data is deleted.

Finally, a complete split Redis instance is reached by switching back and forth and restarting the Redis instance.