Redis Storage and disaster recovery strategy

Redis uses memory to play a high-performance read and write in many scenarios, but the Redis itself is still a stand-alone database, if the system is strongly dependent on it, then still must do the necessary disaster tolerance, for this problem, there are several strategies:

One, m/s switch

Because Redis is a stand-alone database, some disaster-tolerant scenarios for MySQL can also work well, such as when Redis crashes unexpectedly, you can instantly cut the request to the standby and quickly recover the data.

Second, AOF

Redis has two ways of persisting, namely snapshotting and Append-only File, whose principles and features can be referenced in the article " some ideas on Redis data Persistence ", in the sense that snapshots have less impact on performance, Only the required data is backed up, but the data in the middle of the two snapshots cannot be guaranteed to persist.

In contrast, AoF's granularity is finer, the persistence is better, similar to MySQL's Binlog, the disadvantage is that it will lose a part of performance, and will log unnecessary logs, which will be particularly prominent when the system runs for a long time, perhaps restore all the data originally only 1 hours, It could take 100 or even 1000 hours to do it.

Third, read data source Direct recovery

This scenario is related to the business scenario, where Redis has the ability to easily re-establish all of the Redis content by using MySQL data because it has been used to store the index in a previous project, where you can temporarily engage a trigger, read MySQL and write Redis, Using MySQL's sequential read and Redis high TPS features, in practice, tens of millions of data can be rebuilt in a matter of minutes.

currently Redis is typically still functionally used for caches, consider specific scenarios if you need to ensure the persistence of HA storage, and consider whether you can use native distributed memcached or upgrade hardware such as SSDs, Fusion-io) enhances the performance of the original DB.

In order to ensure high performance in my application, the data did not dump or use AOF. Because the failure to do dump is much lower than when the dump, even if the data is lost, the automatic repair script can immediately recover data. After all, the massive data Redis can only do data fragmentation, then the amount of data falling on each node is not much.

Redis Virtual Memory recommendations also do not use, with Redis is to achieve the abnormal performance, virtual memory, AOF looks a little chicken.

Now can not be separated from Redis, because its mget is now all the best performance in the DB, previously also considered using Tokyocabinet hash method to do mget, performance does not give force. Directly with Redis, basically a single redis node mget can reach 10w/s

http://lkf0217.iteye.com/blog/1076700

As you can see from this diagram:

1) for non-persistent way, read and write in the data volume reached 8 million when the performance degradation of several times, it is precisely when the memory 10g,redis began to swap out to disk. And since then there is no way to cheer up again, performance is much worse than MongoDB.

2) for aof persistence, the overall performance will not be much worse than without persistence, it is in the tens of millions of data, memory full after reading performance of only hundreds of.

3) for dump persistence mode, read and write performance fluctuations are relatively large, may be at that time the dump also has a relationship, and after reaching 14 million data volume, read and write performance of the bottom. In the dump, it will not be swapped out, and all the modified data is a new page created, memory consumption is much higher than usual, more than 15GB. And dump also compresses, taking up a lot of CPU. That is, at that time the memory, disk and CPU pressure are close to the limit, the performance is not bad only strange.

To summarize:

1) Redis is actually only suitable as a cache, not a database or storage. Its persistence method is suitable for saving emergency and what is not suitable for use as a normal function. For this version of Redis, it is not recommended to use any persistence mode. Otherwise it would be ugly to die. To be frank, Redis is expected to be an upgraded version of memcached with a variety of data structures, but do not expect Redis to compare with Mongodb/kt.

2) for VMS is also not recommended to turn on, although the open VM can let Redis save more than memory data, but if the hot and cold data is not very obvious performance will be very poor (my test is random query Key, hot and cold is not obvious). Of course, 200%-400% of memory can be set as VM space for both hot and cold, and it is not recommended to set up 10 times times as much memory as a VM (like my configuration).

3) Servicestack.redis client seems to have a few bugs, first of all redistypedclient Dispose actually did not implement, should be to call the client. Dispose (), followed by the Info property of Redisnativeclient not getting the latest value every time, third Pooledredisclientmanager Writepoolindex and Readpoolindex only see add did not see the place, do not know what this is to do, in fact, each time take the first not active client can be, Pooledredisclientmanager also did not forcibly recycle the active client that was used in the timeout (avoid forgetting that dispose consumes too many connections when used). For these points, I will try to contact the author of Servicestack.redis.

Redis Storage and disaster recovery strategy