Redis VS Memcached Reprint

Introduction:

In the big data age, there is always a need for a key-value storage mechanism, like hashmap in memory to handle a large number of (tens of millions of orders of magnitude) of key-value, in order to improve data lookup, modification speed.

So, we think of the two NoSQL databases of memcached and Redis (which, strictly speaking, cannot be counted as a database).

1, memcached is a cache mechanism, when the memory is not enough to use the LRU mechanism, replace the stale data, so he can not guarantee that our data as in the HashMap is not lost, and there is no data persistence mechanism;

2. Redis overcomes this shortcoming and adopts disk storage mechanism to achieve data persistence. However, when the amount of data reached about 10 million, because the memory can not store such a large number of data, frequent data exchange with the disk, resulting in a sharp decline in data query, storage performance, will lead to service unavailability.

Conclusion: At present, there is no good product can realize key-value guarantee data integrity, tens order of magnitude, efficient storage and query support products.

Appendix I: The following is the test data transferred from other netizens:

Appendix II: Comparison of memcached and Redis, and their respective uses

Appendix I:

You can guess that there will also be Redis 2.2.1 test, the same test environment, 1K of data, using the Servicestack.redis client for the following tests:

1) Set operation

2) Get operation

3) del operation

Each set of tests is tested using three configurations:

1) Green Line is to open dump mode of persistence, 5 minutes to persist once

2) Blue Line is to open aof mode of persistence, write to disk once per second

3) The red line is to turn off any persistence mode

Use the same additional configuration for each configuration:

1) Open VM Max memory 10GB (128 bytes one page) Start swapping out, VM space 160GB

2) use up to 15GB of memory to ensure sufficient memory is available at dump

3) Open compression, no master/slave configuration

Now let's take a look at the test results:

　　

As you can see from this diagram:

1) for non-persistent way, read and write in the data volume reached 8 million when the performance degradation of several times, it is precisely when the memory 10g,redis began to swap out to disk. And since then there is no way to cheer up again, performance is much worse than MongoDB.

2) for aof persistence, the overall performance will not be much worse than without persistence, it is in the tens of millions of data, memory full after reading performance of only hundreds of.

3) for dump persistence mode, read and write performance fluctuations are relatively large, may be at that time the dump also has a relationship, and after reaching 14 million data volume, read and write performance of the bottom. In the dump, it will not be swapped out, and all the modified data is a new page created, memory consumption is much higher than usual, more than 15GB. And dump also compresses, taking up a lot of CPU. That is, at that time the memory, disk and CPU pressure are close to the limit, the performance is not bad only strange.

To summarize:

1) Redis is actually only suitable as a cache, not a database or storage. Its persistence method is suitable for saving emergency and what is not suitable for use as a normal function. For this version of Redis, it is not recommended to use any persistence mode. Otherwise it would be ugly to die. To be frank, Redis is expected to be an upgraded version of memcached with a variety of data structures, but do not expect Redis to compare with Mongodb/kt.

2) for VMS is also not recommended to turn on, although the open VM can let Redis save more than memory data, but if the hot and cold data is not very obvious performance will be very poor (my test is random query key, hot and cold is not obvious). Of course, 200%-400% of memory can be set as VM space for both hot and cold, and it is not recommended to set up 10 times times as much memory as a VM (like my configuration).

3) Servicestack.redis client seems to have a few bugs, first of all redistypedclient Dispose actually did not implement, should be to call Client.dispose (), Next Redisnativeclient's Info property is not every time you get the latest value, and the third Pooledredisclientmanager Writepoolindex and Readpoolindex only see where the minus is added, Do not know what this is to do, in fact, each time to take the first is not active client, Pooledredisclientmanager also did not use the timeout active client forced recovery ( Avoid using it when you forget that dispose consumes too many connections). For these points, I will try to contact the author of Servicestack.redis.

Appendix II:

These two years of redis fire can be, Redis is often used as a memcached challenger was mentioned on the desktop. The comparison between Redis and memcached is ubiquitous. However, does Redis really go beyond memcached in terms of functionality, performance, and memory efficiency?  

The following is an answer from the Redis author on StackOverflow, with the question "is memcached a dinosaur in comparison to Redis?" (compared to Redis, Is memcached really out of date? )  

• There is no need to focus too much on performance, as both are already high enough to perform. Because Redis uses only single cores, and memcached can use multicore, the two are compared on average, and Redis has a higher performance than memcached when storing small data on each core. In more than 100k of data, memcached performance is higher than Redis. Although Redis has also recently been optimized for the performance of storing big data, it is slightly less than memcached. Having said so much, the conclusion is that no matter which one you use, the number of requests per second will not be a bottleneck.
• In terms of memory efficiency, memcached memory utilization is higher if simple key-value storage is used. If Redis uses a hash structure to do key-value storage, its memory utilization will be higher than memcached due to its combined compression. Of course, this is related to your application scenario and data characteristics.
• If you have requirements for data persistence and data synchronization, it is recommended that you choose Redis. because neither of these features is memcached. Choosing Redis is also wise, even if you just want the cached data to be not lost after upgrading or rebooting the system.
• And, of course, you have to talk about your specific application needs in the end. Redis has more data structures and supports richer data operations than memcached. Usually in memcached, you need to get the data to the client to make similar changes and set it back. This greatly increases the number of network IO and the volume of data. In Redis, these complex operations are often as efficient as the general get/set. So, if you need caching to support more complex structures and operations, Redis is a good choice.

Redis VS Memcached Reprint