1,redis technical Briefs and questions redis is an open source, memory-stored data structure server. Can be used as databases, caches, and message queues. He supports data types such as strings, hash tables, lists, collections, ordered collections, and so on. can achieve the database master-slave replication, data persistence, memory expansion and other means to ensure redis stability and capacity expansion. Questions, Redis is a pure memory architecture system, the first thing to consider if the initial memory structure is not enough, how to expand? Then, if Redis is off or down, how does the data in memory recover, or backup? Again, Redis as a caching system, processing so many user requests, if it is used in conjunction with the relational DB, how to prevent the occurrence of cache avalanche problem? 2,redis master-Slave database to prevent cache avalanche redis the master-slave replication function is very powerful, a master can have multiple slave, and a slave can have more than slave, so go on, Form a powerful multi-level server cluster architecture. Can avoid Redis single point of failure, to build a read-write separation architecture, to meet the number of read and write less application scenarios. Several important aspects of Redis replication feature 1. A master can have multiple slave;2. Redis uses asynchronous replication. Starting with 2.8, the slave will periodically (once per second) initiate an ACK acknowledgement of the replication stream (replication stream) being processed; 3. Not only the master server can have from the server, from the server can also have its own from the server, multiple from the server can constitute a graph structure; 4. Replication is in non-blocking mode on the master side, which means that even if multiple slave perform the first synchronization, master can still provide the query service; 5. Replication is also non-blocking on the slave side: if you set up in redis.conf, slave can still provide queries using the old dataset while performing the first synchronization; You can also configure the slave to return an error message to the client when master loses contact with slave; 6. When slave deletes the old data set and reloads the new version of the data, slave blocks the connection request (typically in the recovery phase after disconnecting from master); 7. Replication can be used purely for data redundancy (dataredundancy), or it can improve extensibility by having multiple requests for read-only commands from the server (scalability): For example, a heavy SORT command can be handed to a subordinate node to run. 8. You can modify the master side of the RediS.config to avoid persistent operations on the master side (Save), which is persisted by the slave side.
3,redis Data partitioning technology to expand sharding (partitioning) is the process of splitting your data across multiple Redis instances, so that each instance will contain only a subset of all the keys. redis's shards assume two main goals: allowing a larger database to be supported by using the sum of the memory of many computers. Without sharding, you are limited to the amount of memory that can be supported by a single machine. Allows scalable computing power to multi-core or multi-server, scaling network bandwidth to multiple servers or multiple network adapters. The different implementations of shards can be assumed by different parts of the software stack. Client side partitioning means that the client chooses the correct node directly to write to and read the specified key. Many Redis clients implement client-side sharding. Proxy assisted partitioning means that our client sends a request to an agent that understands the Redis protocol, rather than sending the request directly to the Redis instance. The agent will ensure that our request is forwarded to the correct Redis instance and return the response to the client, based on the configured shard pattern. The proxy twemproxy for Redis and Memcached implements proxy-assisted sharding. Query routing means that you can send your query to a random instance that will ensure that your query is forwarded to the correct node. With the help of the client, the Redis cluster implements a mixed form of query routing (the request is not forwarded directly from the Redis instance to the other, but the client receives the redirect to the correct node). Disadvantages of Sharding Some of the features of Redis are not very good at playing with shards: operations involving multiple keys are not usually supported. For example, you cannot intersect a key that is mapped on two different Redis instances (in fact there are ways to do it, but not directly). Transactions involving multiple keys are not available. The granularity of a shard (granularity) is a key, so you cannot use a large key to fragment a dataset, such as a large, ordered set. When using shards, data processing becomes more complex, for example, you need to process multiple rdb/aof files, and you need to aggregate multiple instances and host persistence files when backing up data. Adding and removing capacity is also complex. For example, a Redis cluster has the ability to dynamically add and remove nodes at runtime to support the transparent rebalancing of data, but other ways, such as client shards and proxies, do not support this feature. However, there is a technique called pre-sharding (presharding) that can be helpful at this point. Pre-sharding we already know a problem with sharding, unless weUsing Redis as a cache, adding and removing nodes is a tricky task, and using fixed key and instance mappings is much simpler. However, the demand for data storage may have been changing. Today I can accept 10 Redis nodes (instances), but tomorrow I may need 50 nodes. Since Redis has only a fairly small footprint (footprint) and is lightweight (an idle instance is only using 1MB of memory), a simple solution is to open many instances from the beginning. Even if you start out with just one server, you can decide to live in a distributed world on the first day, using shards to run multiple Redis instances on one server. You can select a number of instances from the outset. For example, 32 or 64 instances can satisfy most users and provide ample space for future growth. This way, when your data storage needs to grow, you need more Redis servers, and all you have to do is simply move the instances from one server to another. When you add the first server, you need to move half of the Redis instances from the first server to the second one, and so on. With Redis replication, you can complete mobile data in small or no downtime: Start an empty instance on your new server. Move the data and configure the new instance as the slave for the source instance. Stop your client. Updates the server IP address configuration for the moved instance. Send the slaveof NO one command to the slave node on the new server. Start your client with a new update configuration. Finally, shut down instances that are no longer used on the old server. 4,redis how data in memory is persisted and data recovery There are two main types, RDB and aof, in general, the RDB is every time, or there is a trigger point, generate a snapshot of all the data, save it. AoF is that every time you make a change to the memory, you record the command, and then, if you need to recover, you know what data you have when you follow the instructions. As follows, refer to website https://my.oschina.net/davehe/blog/174662
Advantages of rdb: The RDB is a very compact file that holds the data set of the Redis at a certain point in time. This file is ideal for backup: For example, you can back up an RDB file every hour within the last 24 hours, and also back up an RDB file every day of the month. In this case, you can restore the dataset to a different version at any time, even if you encounter problems. The RDB is ideal for disaster recovery (disaster recovery): It has only one file, and the content is very compact and can be transferred (after encryption) to another datacenter, or Amazon S3. An RDB maximizes the performance of Redis: The only thing the parent process has to do when saving an Rdb file is to fork out a child process, and the child process will handle all subsequent save work, and the parent process does not have to perform any disk I/O operations. An RDB recovers a large data set faster than AOF. rdb disadvantage: If you need to avoid data loss when the server fails, the RDB is not for you. Although Redis allows you to set different savepoint (save point) to control how often an RDB file is saved, it is not an easy operation because the Rdb file needs to preserve the state of the entire dataset. Therefore, you may be able to save the RDB file at least 5 minutes. In this case, you may lose several minutes of data in the event of a failure. Each time you save an RDB, Redis will fork () out a subprocess and perform the actual persistence work by the child process. When the dataset is large, fork () can be time consuming, causing the server to stop processing the client in such milliseconds, and if the data set is huge and CPU time is very tight, the stop time may even be a full second. Although the AOF rewrite also requires fork (), there is no loss of data durability regardless of the length of the AOF rewrite execution interval. Advantages of aof: Using AOF persistence makes Redis very durable (much more durable): You can set different fsync policies, such as no fsync, one fsync per second, or every time you write a command fsy NC. The default policy for AOF is Fsync once per second, in which Redis can still maintain good performance and, even if a failure occurs, it will only lose one second of data (Fsync is executed in the background thread, so the main thread can continueTry to handle the command request). The AOF file is an append-only log file (append only), so the write to the AOF file does not require seek, even if the log contains incomplete commands for some reason (such as a write-in disk full, write-down, etc.), The Redis-check-aof tool can also easily fix this problem. Redis can automatically override AOF in the background when the AOF file size becomes too large: The rewritten new AOF file contains the minimum set of commands required to restore the current dataset. The entire rewrite operation is absolutely secure, as Redis will continue to append commands to the existing AOF file during the creation of the new AOF file, and the existing AOF files will not be lost even if there is an outage during the rewrite. Once the new AOF file is created, Redis switches from the old AOF file to the new AOF file and begins appending to the new AOF file. The AOF file preserves all write operations performed on the database in an orderly manner, and these writes are saved in the format of the Redis protocol, so the contents of the AOF file are easily readable and parsing (parse) is easy. The export AOF file is also very simple: for example, if you accidentally executed the Flushall command, but as long as the AOF file is not rewritten, just stop the server, remove the Flushall command at the end of the AOF file, and restart Redis, you can The data set reverts to the state before Flushall execution. aof disadvantage: For the same data set, the size of the AOF file is usually larger than the size of the RDB file. Depending on the Fsync policy you are using, the AOF may be slower than the RDB. In general, the performance of Fsync per second is still very high, while closing fsync can make AOF as fast as an RDB, even under high load. However, the RDB can provide a more guaranteed maximum delay time (latency) when handling large write loads. AOF has been a bug in the past: Because of the reason of individual commands, the AOF file cannot be restored as it was when the data set was reloaded. (for example, a blocking command Brpoplpush has caused such a bug.) The test suite adds tests to this situation: they automatically generate random, complex datasets, and re-load the data to make sure everything is fine. Although this bug is not in the AOF fileCommon, but in contrast, it is almost impossible for an RDB to come up with this bug.
Redis Learning a rough overview
