A ramble on using Redis as a cache

What business scenarios Redis fits
In general business System database access, the ratio of read and write operations is generally 7/3 to 9/1, that is, read operations far more than write operations, so the high concurrency system design, through the NoSQL technology to the hot data (short-term change probability of small data) into memory to reduce the DB pressure, improve data access speed purposes, Redis and MongoDB are the most widely used NoSQL products, of course, if the system writes mostly, and there is no need to use the cache, so Redis is mainly used to solve the problem of access performance and concurrency capability. In addition to the role of pure data caching, Redis is often used to provide distributed lock solutions thanks to its ultra-fast responsiveness.

What design ideas ensure Redis high performance
The processing of requests by a single Redis server is based on the single-threaded work model, but because it is pure memory operations, and the single-threaded mode of operation avoids the additional overhead of thread context switching, using the NIO multiplexing mechanism (single-threaded to maintain the state of multiple I/O sockets, Socket Event handler is unified for event distribution and notifies each event listener), so even the performance of a single Redis server is very fast and can support a set operation of 110,000 times per second, and a get operation of 81,000 times per second.

In high concurrency systems, sometimes a single Redis server does not meet the performance requirements, the Redis 3.0 approach is Redis Sentinel, multiple nodes are serving at the same time, and each node holds a full amount of data, and Redis 3.0 introduces Redis Cluster. Only partial data (16,384 slots in total) is retained on each node through data sharding for high availability.

The Redis cluster is implemented without a central node, the client connects directly to each node of the Redis cluster, and after the client requests reach the node, using a unified hashing algorithm, CRC16 (key)%16384, calculates the slot for the key, and then from the Redis Cluster locates the specific server, the specific data sharding, and eventually returns it to the user. However, since Redis transactions can only solve the acid problem on a single server, the common problem with multiple servers is the consistency of the operations of the same key with zookeeper, which requires the use of distributed locking mechanisms to solve the problem of consistency and sequential operations.

Redis supports dynamic addition and deletion of nodes, dynamic migration and rebalance slots, dynamically re-electing leader and fail-over; Each node saves a portion of the slot, but saves a copy of the same Data sharding mapping table, This table records the host distribution locations of all 16,384 slots and periodically updates this table information between nodes, so that the design ensures that only a small amount of information is required between the Redis cluster internal nodes to synchronize information with each other. When a client accesses a Redis cluster, a host:port is specified in the cluster, and if the action key is not on the current host, the host will tell the client the correct host:port based on the Data sharding mapping table. If the access Host:port is offline, the client's link is automatically transferred to the corresponding master or slave node.

Usage scenarios for various data types in Redis
Redis's data storage is implemented primarily through Key/value, where key is a string type, and value has five types defined for different scenarios:
#1 String type: can contain any data (JPS pictures or serialized objects, a single key can store up to 512M of data), with global statistics function data, such as Global ID generator, cluster configuration information;
#2 Hash Type: Data used to store the structure of an object, multiple fields bound to a key (the advantage of using a string type to store an object is that the hash type can directly update the value of a specific field without affecting the other field), such as implementing SSO, The cookie is key, the user information is value, and the expiry time is specified;
#3 list type: Used to store series of data that need to be based on queue or stack operations, such as Message Queuing;
#4 set type: Used for storage needs to maintain a global non-repeating collection, such as service registration discovery, can achieve global deduplication functions, such as access to Web pages of independent IP, common friends, etc.;
#5 zset type: Used to store a list that needs to maintain a globally distinct but weighted sort can use sorted SET, such as an integral leaderboard, a message queue with weights.

Similar operations can be performed on the above field types,
Set a value: [Set|hmset|lpush|sadd] key value
Get a value: [Get|hget] Key
Delete a value: [Del|hdel] Key
Set a value with expiration: [Setex] Key time value
Set this value if the value does not exist: [SETNX] key value
Find keys or Pattern:[keys/scan in Redis] key
Determine if a value exists: [Exists|hexists] Key
Set expiration time for the specified value: [Expire] key seconds
Adds the value of the specified key to 1| minus 1:[INCR|DECR] key
Migrate a key\value to the specified Server:[migrate] host port key dest-db timeout [copy] [replace]

Hyperloglog is used to do cardinality statistics (based on the set type of encapsulation, only according to the number of independent elements of input statistics, not the elements themselves), can guarantee that the number of input elements or the volume is very large to ensure that the statistical space required to be fixed to 12KB (maximum 2^64 elements).
Add cardinality to the specified key: [Pfadd] key value
Count the number of different cardinality in the specified key: [Pfcount] Key
Merge Sourcekey technology into Destkey's cardinality statistics: [Pfmerge] Destkey Sourcekey

The pub/sub is used to publish the message (based on a list-type wrapper, encapsulating the message as a list node).
Create a message to receive Channel:[subscribe] Channel
Sends a message to the specified channel: [Publish] Channel message

The execution of a single Redis command is atomic, and for multiple commands Redis provides the underlying transaction mechanism, but does not guarantee atomicity of multiple command executions, a typical Redis transaction is as follows:
Open a transaction: [Multi]
You can then schedule multiple Redis commands, but you will not perform
Submit and execute all previous commands: [exec]
Start execution of the previously scheduled Redis command, if one of the commands fails, does not affect the execution of other commands
To cancel all planned Redis commands within a transaction block: [Discard]
Monitor one or more Key:[watch] keys
Indicates that the current transaction is discard before exec executes if the key is modified by a command before the transaction.

Redis data expiration policy and memory recycling policy
A policy that combines periodic deletion and deferred deletion for data that has expired, but both are flawed; Due to periodic checking that all keys are out of date will cause performance problems, so the periodic deletion of the policy using random spot checks, and before the operation of key will determine whether it has expired, such as expired immediately delete , such a strategy would cause some of the expired keys to accumulate in memory, making Redis server memory consumption high, so it needs to be used in conjunction with the Maxmemory-policy configuration in redis.conf, when Redis Memory reclamation policy when the server has insufficient memory to write new data.
#1 noeviction: Indicates a direct error;
#2 Allkeys-lur: Indicates that key is deleted in all keys according to LRU;
#3 Allkeys-random: Indicates that key is deleted randomly in all keys;
#4 Volatile-lru/volatile-random/volatile-ttl is used when Redis server acts as both a cache and a db, indicating that it is deleted in the keys with expire date set. The TTL indicates the deletion of keys with earlier expiration time.

Troubleshoot Redis cache penetration and cache avalanche issues
Cache penetration and avalanche can be seen as a problem, but the severity is different; When a request arrives at Redis, it finds no corresponding cache data, then sends a data request to DB, and if it gets to the data, the problem stays on the cache penetration, and the db gets the data cached on the Redis. If there is no corresponding data in the DB, and when such a request reaches an order of magnitude and consumes all of the DB resources, there is a cache avalanche problem resulting in a DB connection exception.
There are several ways to solve the cache penetration problem, regardless of whether the corresponding value is found in the db (no value is null), a cache record is recorded in the Redis, a bitmap is maintained at the DAO layer, a bit is used to record if the corresponding key has a corresponding value, thereby avoiding redundant DB operations The background thread is dedicated to updating Redis data that is about to expire, thus avoiding cache penetration.
The idea of solving the cache avalanche problem is to add a mutex on the DB connection, so that when a large number of cache requests are invalidated, it is necessary to queue up to DB to request data, set a random value for the data with the same expiration time, and avoid the collective failure of the data. Using either a dual cache or a multi-tier cache strategy, Need to mate with cache warming.

A ramble on using Redis as a cache