Why use redis and its product positioning

Problems with traditional MySQL + memcached Architecture

MySQL is suitable for storing massive data. Many companies have used this architecture to load hotspot data to the cache through memcached to accelerate access. However, as the business data volume increases, and the continuous growth of access traffic, we have encountered many problems:

 

1. MySQL needs to constantly split databases and tables, and memcached also needs to be scaled up. resizing and maintenance take a lot of development time.
2. Data Consistency between memcached and MySQL databases.
3. Memcached has a low data hit rate or goes down, and a large number of accesses directly penetrate into the database, which is not supported by MySQL.
4. Cross-Data Center cache synchronization problems.

How to choose a large number of nosql Databases

In recent years, many nosql products have emerged in the industry, so how can we properly use these products and maximize their strengths is a problem that we need to study and think deeply, in the final analysis, the most important thing is to understand the positioning of these products, as well as the tradeoffs of each product. In practice, these nosql products are used to solve the following problems.

1. Small Data Storage, high-speed read/write access. This type of product ensures high-speed access through all the data in-momery and provides the data landing function. This is the main application scenario of redis.
2. Massive Data Storage, distributed system support, data consistency assurance, and convenient addition/deletion of cluster nodes.
3. The most representative of this is the ideas described in dynamo and bigtable. The former is a completely decentralized design, and cluster information is transmitted between nodes through the gossip method to ensure data consistency. The latter is a centralized solution design, similar to a distributed lock service to ensure strong consistency, data is written to the memory and redo log first, and then compat is periodically merged to the disk to optimize random write to sequential write, improving write performance.
4. Schema free, auto-sharding, etc. For example, some common document databases currently support schema-free, directly store JSON data, and support auto-sharding and other functions, such as MongoDB.

In the face of these different types of nosql products, we need to select the most appropriate product based on our business scenarios.

How to Use apsaradb for redis correctly

Previously, we have analyzed that redis is most suitable for all data in-momory scenarios. Although redis also provides persistence functions, it is actually a disk-backed function, there is a big difference from the persistence in the traditional sense, so you may be wondering, it seems that redis is more like an enhanced version of memcached, so when Will memcached be used and when will redis be used?

Comparison between redis and memcached

1. Network I/O model

   Memcached is a multi-thread, non-blocking I/O multiplexing network model. It is divided into the listening main line and the worker subthread. The listening thread listens to network connections. After receiving the request, it passes the connection description pipe to the worker thread, for read/write Io, the network layer uses the libevent encapsulated event library. The multi-threaded model can play a multi-core role, but introduces the cache coherency and lock issues, for example, memcached's most commonly used stats command, all actual memcached operations require locking and counting the global variable, resulting in performance loss.

   (Memcached network I/O model)

   Redis uses a single-thread Io multiplexing model and encapsulates a simple aeevent event processing framework, which mainly implements epoll, kqueue, and select. For Io operations alone, A single thread can maximize the speed advantage, but redis also provides some simple computing functions, such as sorting and aggregation. For these operations, the single-threaded model will seriously affect the overall throughput. During CPU computing, the entire Io scheduling is blocked.

2. Memory Management

   Memcached uses a pre-allocated memory pool. It uses slab and chunks of different sizes to manage the memory. Item selects the appropriate chunk storage based on the size, the memory pool method can save the overhead of applying for/releasing memory and reduce the generation of memory fragments. However, this method also results in a certain amount of space waste, in addition, new data may also be removed when there is still much space in the memory. For the reason, refer to timyang's article:Http://timyang.net/data/Memcached-lru-evictions/

   Redis uses on-site memory application to store data, and seldom uses free-list or other methods to optimize memory allocation. To some extent, memory fragments exist. redis and data storage Command Parameters, data with an expiration time is stored separately and called temporary data. Non-temporary data will never be deleted, even if the physical memory is insufficient, as a result, swap will not remove any non-temporary data (but will try to remove some temporary data). redis is more suitable for storage than cache.

3. Data Consistency

   Memcached provides CAS commands to ensure the consistency of multiple concurrent accesses to the same data copy. Redis does not provide CAS commands and cannot guarantee this. However, redis provides the transaction function to ensure the atomicity of a string of commands without any operation interruption.

4. Storage Methods and other aspects

   Memcached supports only simple key-value storage, enumeration, persistence, replication, and other functions.

   In addition to key/value, redis also supports multiple data structures such as list, set, sorted set, and hash, providing keys

   Enumeration, but not online. If you need to enumerate online data, redis provides a tool to directly scan its dump file and enumerate all the data, redis also provides persistence, replication, and other functions.

5. Client support in different languages

   For clients in different languages, memcached and redis both have a wide range of third-party clients to choose from. However, because memcached has been developing for a longer time, it is currently supported by clients, many clients of memcached are more mature and stable, while redis is more complicated than memcached because of its Protocol itself. In addition, due to the author's continuous addition of new features, the follow-up speed of the corresponding third-party clients may not be able to catch up, sometimes you may need to make some modifications on the basis of a third-party client for better use.

It is not difficult to see from the above that when we do not want data to be kicked out, or need more data types except key/value, or need to implement the function, using redis is more suitable than using memcached.

Some peripheral functions of redis

In addition to storage, redis also provides some other functions, such as aggregate computing, pubsub, and scripting. For such functions, you need to understand their implementation principles, you can use pubsub correctly only after you understand its limitations. For example, the pubsub function does not support persistence, all messages sent from the consumer's transient disconnection or reconnection will be lost. For example, functions such as aggregate computing and scripting are limited by the redis single-thread model, it is impossible to achieve a high throughput, so use it with caution.

In general, the author of redis is a very diligent developer. We can often see that the author is trying various new ideas and ideas, to address these features, we need to have a deep understanding before using them.

Summary:

1. The best way to use redis is to use all data in-memory.
2. Redis is used as a replacement for memcached in more scenarios.
3. When more data types except key/value are required, it is more appropriate to use redis.
4. When the stored data cannot be removed, it is more appropriate to use redis.

Subsequent redis article plans:

1. Redis data type and capacity planning.
2. How to build a stable, reliable, and scalable redis cluster based on business scenarios.
3. Redis parameters, code optimization, and secondary development practices.