The difference between "turn" Redis and Memcached
Problems with traditional MySQL + Memcached architectures
In fact, MySQL is suitable for massive data storage, through the Memcached to load hot data to the cache, speed up access, many companies have used such a structure, but with the increasing volume of business data, and the continued growth of traffic, we encounter a lot of problems:
1. MySQL needs to continue to disassemble the table, Memcached also need to continuously follow the expansion, expansion and maintenance work occupies a lot of development time.
2. Memcached and MySQL database data consistency issues.
3. Memcached data hit rate is low or down, a lot of access directly through to the DB, MySQL can not support.
4. Cross-room cache sync problem.
Many NoSQL blossom, how to choose
In recent years, the industry has been emerging many kinds of NoSQL products, so how to properly use these products, maximize their strengths, is the problem we need to study and think deeply, in fact, the most important thing is to understand the positioning of these products, and understand the advantages and disadvantages of each product, In the practical application to achieve weaknesses, in general, these NoSQL mainly used to solve the following problems:
1. Small amount of data storage, high-speed read and write access. This type of product ensures high-speed access through all in-memory of data, while providing the capability of data landing, which is actually the most important application scenario for Redis.
2. Massive data storage, distributed system support, data consistency guarantee, convenient cluster node Add/delete. The most representative of this is the Dynamo and BigTable 2 papers elaborated in the idea. The former is a completely non-center design, the node through the gossip way to transmit the cluster information, the data to ensure the final consistency; the latter is a centralized scheme design that ensures strong consistency through a similar distributed lock service, writes the data to write the memory first and redo log, and then periodically compat Merge to disk, and write optimizations to sequential writes to improve write performance.
3. Schema free, auto-sharding, etc. For example, some of the common document databases are support Schema-free, directly store data in JSON format, and support functions such as auto-sharding, such as MongoDB.
In the face of these different types of NoSQL products, we need to choose the most appropriate product based on our business scenario.
Redis application scenario, how to use it correctly
As already analyzed, Redis is best suited for all data in-memory scenarios, although Redis also provides persistence, but actually more of a disk-backed function, compared to the traditional sense of persistence there is a big difference, then you may have questions, it seems that Redis More like a Memcached, when to use Memcached, when to use Redis?
If you simply compare the differences between Redis and Memcached, most of them will get the following ideas:
1. Redis not only supports simple k/v types of data, but also provides storage of list, set, sorted set, hash and other data structures.
2. Redis supports backup of data, that is, Master-slave mode of data backup.
3. Redis support data persistence, can keep the in-memory data on disk, restart can be loaded again for use.
Aside from this, you can delve into the internal structure of Redis to see more essential differences and understand the design philosophy of Redis.
In Redis, not all data is stored in memory all the time. This is one of the biggest differences compared to Memcached. Redis only caches information for all keys, and if Redis finds that memory usage exceeds a certain threshold, the swap operation is triggered, and Redis calculates which keys are based on "swapability = Age * Log (Size_in_memory)" The corresponding value needs to be swap to disk, and then the value corresponding to these keys is persisted to disk and purged in memory. This feature allows Redis to maintain data that is larger than the memory size of its machine itself. Of course, the memory of the machine itself must be able to maintain all the keys, after all, the data will not be swap operations. Also, since Redis swaps the in-memory data to disk, the main thread that provides the service and the sub-thread that is doing the swap will share this memory, so if you update the data that needs swap, REDIS will block the operation until the sub-thread completes the swap operation before it can be modified.
Use the Redis-specific memory model before and after the case comparison:
VM off:300k keys, 4096 bytes values:1.3g used
VM on:300k keys, 4096 bytes values:73m used
VM off:1 million keys, bytes values:430.12m used
VM on:1 million keys, bytes values:160.09m used
VM on:1 million keys, values as large as you want, still:160.09m used
When reading data from Redis, if the value of the key being read is not in memory, then Redis needs to load the data from the swap file before returning it to the requester. There is a problem with the I/O thread pool. By default, Redis will become blocked, which will not respond until all swap files have been loaded. This strategy has a small number of clients and is appropriate for batch operations. However, if you apply Redis to a large web site application, this is obviously not sufficient for large concurrency scenarios. So Redis allows us to set the size of the I/O thread pool and concurrently operate on read requests that need to load the corresponding data from the swap file, reducing the time to block.
If you want to use Redis in an environment of massive data, I believe it is essential to understand the memory design and blocking scenarios of Redis.
==================================================
Complementary points of knowledge
Comparison of Memcached and Redis
1. Network IO Model
Memcached is a multi-threaded, non-blocking IO multiplexing network model, divided into the main thread and the worker sub-thread, listening thread listening network connection, after accepting the request, the connection description Word pipe to the worker thread, read/write IO, the network layer using Libevent encapsulated event Library. Multithreaded models can play a multi-core role, but the problem of cache coherency and locks is introduced. For example, Memcached most commonly used stats command, in fact, all the operations of Memcached to this global variable lock, count, etc., resulting in performance loss.
Redis uses a single-threaded IO multiplexing model, which encapsulates a simple Aeevent event processing framework that implements Epoll, Kqueue, and select, with a single thread that maximizes the speed advantage for simple IO operation requests, but Redis also Provides a number of simple calculation functions, such as sorting, aggregation, etc., for these operations, the single-threaded model will seriously affect the overall throughput, CPU calculation process, the entire IO schedule is blocked.
2. Memory management aspects
Memcached uses a pre-allocated memory pool to manage memory using slab and chunk of different sizes. Item is based on the size of the appropriate chunk storage, the way the memory pool can save the application/release of memory overhead, and can reduce the memory fragmentation, but this way will also bring a certain amount of space waste, and in memory still have a lot of space, the new data may also be rejected, for the reasons can be referenced Timyang's article: http://timyang.net/data/Memcached-lru-evictions/
Redis uses on-site memory storage to store data, and rarely uses free-list to optimize memory allocations to some extent with memory fragmentation. The Redis data store command parameter, which will be stored separately with the expiration time, and call them temporary data, non-temporary data will never be rejected, even if the physical memory is not enough, the swap will not eliminate any non-temporary data (but will try to exclude some temporary data), this is the Redis More suitable as storage rather than cache.
3. Data consistency issues
Memcached provides a CAS command that guarantees consistency of the same data for multiple concurrent access operations. Redis does not provide CAS commands, and this is not guaranteed, but Redis provides the functionality of a transaction that guarantees the atomicity of a sequence of commands and is not interrupted by any action.
4. Storage methods and other aspects
Memcached basically only supports simple key-value storage, does not support enumeration, and does not support persistence and replication functions. In addition to supporting Key/value, Redis supports list, set, sorted set, hash and many other data structures, provides the KEYS for enumeration operations, but cannot be used online, if you need to enumerate the data on line, Redis provides tools to directly scan its Dump file, which enumerates all the data. Redis also provides features such as persistence and replication.
5. Client support for different languages
Memcached and Redis have rich third-party clients to choose from for different language clients, but because Memcached has been developing for a longer period of time, many of Memcached's clients are more mature and stable in terms of client support, and Redis Because the protocol itself is more complex than Memcached, plus the author constantly add new features, and so on, the corresponding third-party client tracking speed may not be able to catch up, sometimes you may need to make some changes on the basis of third-party clients to better use.
According to the above comparisons it is not difficult to see that when we do not want the data to be kicked out, or need more data types other than Key/value, or need to use the landing function, using Redis is more appropriate than using Memcached.
Some of the peripheral features of Redis
In addition to being stored as storage, Redis also provides some other functions, such as aggregation calculation, pubsub, scripting, etc., for such functions need to understand its implementation principle, clearly understand its limitations, can be used correctly. For example, the PubSub function, this is not any persistent support, consumer connection flash or reconnect after the message will be all lost, and such as aggregation calculation and scripting and other functions are limited by the Redis single-threaded model, it is impossible to achieve high throughput, need to use caution.
In general, the Redis author is a very diligent developer who can often see that the author is experimenting with a variety of new ideas and ideas, and that the functionality of these areas requires that we need to know more about them before using them.
Summarize:
1. The best way to use Redis is to in-memory all data.
2. Redis more scenarios are used as substitutes for Memcached.
3. It is more appropriate to use Redis when more data type support is required other than key/value.
4. Using Redis is more appropriate when the stored data cannot be excluded.
The difference between Redis and Memcached
