Redis QuickStart-Advanced knowledge

Preface

We've learned the data types of Redis in the first place, and then we'll simply learn about Redis's transactions, sorting, pipelines, optimizing storage, and management. Transactions

The concept of business is not to be mentioned here, learning the principles of the database should be known.

Redis transactions: Send a command that belongs to a transaction to Redis, and then let Redis execute the commands in turn:

MULTI  //Start a transaction
//transaction command
EXEC//Execute transaction

exec tells Redis that all commands in the queue of transactions waiting to be executed (that is, all the commands that returned queued) are executed sequentially in the order in which they were sent. Error Handling

1. Syntax error: The command does not exist or the number of command arguments is incorrect.

As long as a command has a syntax error, REDIS returns an error directly after Exec executes, and those commands that are syntactically correct will not execute.

2. Run ERROR: An error occurred while the command was executing, such as using a hash type command to manipulate the key of the collection type.

If a command runs out of error, the other commands in the transaction continue to execute. the Redis transaction does not have rollback capability. Watch Command

The result of each command in the transaction is returned last together, so we cannot use the result of the previous command as a parameter to the next command.

For example, if we are going to implement the INCR command ourselves, we can determine if the key already exists based on the return value of the Get command, and then do the corresponding set operation. To prevent race, transactions can be used, but the transaction has the problem mentioned above, so the race problem cannot be resolved.

Let's take a look at the watch command: The watch command can monitor one or more keys, and once a key has been modified (or deleted), subsequent transactions will not be executed, and monitoring continues until the EXEC command is executed.

Example:

> Set key 1
OK
> WATCH key
OK
> Set key 2
OK
> MULTI
OK
> SET key 3
QUE UED
> EXEC
(nil)
> GET key
"2"

As you can see, after we point to the Watch command, the value of key is modified before the transaction executes, so the last transaction is not executed and exec returns an empty result.

So with watch you can implement your own incr function through transactions:

def incr ($key)
    WATCH $key
    $value = GET $key
    if not $value $value
        = 0
    $value = $value + 1
    MUL TI
    SET $key, $value
    result = EXEC
    return result[0]

Three points to be aware of:

1.EXEC return value is a multi-line string type

The purpose of the 2.WATCH command is to block the execution of a transaction after the monitored key value has been modified, and not to guarantee that the other client will not modify the key value. Therefore, if Exec execution fails, we need to re-execute the entire function.

The monitoring of all keys is canceled after 3.EXEC execution, and we can use Unwatch to cancel the monitoring. Survival Time

In Redis, you can use the expire command to set the lifetime of a key in seconds:

> SET foo bar
OK
> EXPIRE foo
(integer) 1
> TTL foo
(integer)
> TTL foo
   (integer) 7
> TTL foo
(integer)-1

As you can see, the Foo Key's survival time is gradually reduced over time, and Foo is deleted after 20 seconds, and when the key does not exist, the TTL returns-1, and when the lifetime is not set, the default is persistent, and the TTL returns-1.

Use the persist command to cancel the lifetime of the key:

PERSIST Foo

Also, assigning a key using the set or Getset command clears the lifetime of the key at the same time.

The expire parameter must be an integer, and if you want to be more precise, you can use Pexpire, whose units are milliseconds. If you use watch to monitor a time-to-live key, the key time expiration is automatically deleted and will not be considered altered by the watch command. Cache

Redis can be used as a cache, on the one hand we can periodically delete the cache by giving the key a time to live, on the other hand, if the memory has reached the upper limit, Redis will retire the unwanted cache key according to certain rules (common is the least recently used lru-algorithm).

Maximum available memory size (in bytes): Configure MaxMemory;
Retire policy: Configure Maxmemory-policy. Sort

1. Using an ordered set

2. Use the sort command (with a by, get, store and other parameters can be selected)
The time complexity is O (N+MLOGM), n represents the number of elements in the list to sort, and M indicates the number of elements to return. Message Notification

Message notifications can use the task queue, the queue for delivery tasks, which is the common producer/consumer model, where the producer puts the tasks that need to be processed into the task queue, and the consumer continuously reads the task information from the task queue and executes it.

Pros: Loose coupling: Producers and consumers do not need to know each other's implementation details; Easy to scale: consumers can have multiple, and can be distributed across different servers. Implementing a task queue using Redis

Producer: Use the Lpush command for the task if it is in a key;
Consumer: Constantly adapting the Rpop command to remove tasks from the key.

The following is a pseudo-code for consumers:

# infinite Loop reads the contents of the task Queue
loop
    $task = Rpop queue
    if $task
        execute ($task)
    else
        wait 1 second

The above constant polling, inefficient, we can use Brpop to optimize:

Loop
    #如果队列中没有新任务, Brpop will always block
    $task = Brpop queue, 0
    #返回值是一个数组, the second element of the array is the task we need to
    execute ($task [1])

Brpop receives two parameters: Key name and Time-out (in seconds, 0 for infinite wait).

Brpop returns two values: Key name and element value. Priority Queue

Here's the case: let's say your site has push features, that is, when you post a new article, you'll be emailed to the followers, and your site will need to be verified when you sign up. If you do not use a priority queue, then when you post a new article, the thread that sent the mailbox is occupied, so that the registered user does not get a timely response. Therefore, it is necessary to make the registered mail priority be processed to send.

Loop
    $task = 
        brpop queue:confirmation.email,
              queue:notification.eamil,
              0
    execute ($task [ 1])  

Brpop can receive multiple keys at the same time, and if more than one key has elements, it takes one element from the first key in order from left to right. Publish/Subscribe mode

In addition to implementing the task queue, Redis also provides a set of commands that enable developers to implement the Publish/subscribe (publish/subscribe) mode:
Subscribers can subscribe to one or several channels (channel), and publishers can send messages to the specified channel, so subscribers subscribing to this channel will receive this message.

To publish a message:

PUBLISH Channel Message

To subscribe to a message:

SUBSCRIBE Channel [Channel ...]

After executing the subscribe command, the client enters the subscription state and cannot use a command other than the 4 "Publish/Subscribe" mode of the subscribe/unsubscribe/psubscribe/punsubscribe, otherwise it will error.

The Psubscribe command can subscribe to the specified rules, such as:

>psubscribe channel.? *

It can match channel.1 and channel.10, etc., and will not match the channel. Piping

Redis is a TCP server and supports the request/response protocol. The following steps are required for a REDIS request to complete:

The client sends a query to the server and waits for the server to respond.

The server processes the command and returns the response to the client.

With pipelines, clients can send multiple requests to the server without waiting for a reply. That is, the pipeline can send multiple commands at once and return the results one time after execution. Space Saving

1. Thin key names and key values

2. Internal coding optimization

View internal code:

OBJECT ENCODING foo

Each key value for Redis is saved using a redisobject structure:

typedef struct REDISOBJECT {
    unsigned type:4;
    unsigned notused:2; /*not used*/
    unsigned encoding:4;
    Ussigned lru:22; /*LRU time*/
    int refcount;
    void *ptr;
} RobJ;

The Type field is the data type of the key value:

#define REDIS_STRING 0
#define REDIS_LIST 1
#define REDIS_SET 2
#define REDIS_ZSET 3
#define Redis_ HASH 4

The encoding field represents the internal encoding of the key value:

String type encoding
#define REDIS_ENCODING_RAW 0
//String type encoding
#define REDIS_ENCODING_INT 1
//hash type, collection type encoding
#define REDIS_ENCODING_HT 2
#define REDIS_ENCODING_ZIPMAP 3
//list type encoding
#define Redis_ Encodinglinkedlist 4
//hash type, list type, ordered collection type encoding
#define REDIS_ENCODING_ZIPLIST 5
//collection type encoding
#define Redis_encoding_intset 6
//Ordered collection type encoding
#define REDIS_ENCODING_SKIPLIST 7

For each type of optimization, see the Redis Getting Started Guide (Li Zi). Script

Redis uses the LUA interpreter to calculate scripts. It is built-in from the 2.6.0 version, using a script with the eval command.

> EVAL Script numkeys key [key ...] arg [arg ...]

The scripting feature allows developers to write scripts into Redis using the Lua script, and most Redis commands can be called in Lua scripts.

Benefits of using scripting: Reduce network overhead: code that would otherwise require multiple requests, using the Scripting feature requires only one request. Atomic manipulation: Redis executes the entire script as a whole. Multiplexing: Scripts sent by clients are permanently stored in Redis. Management Persistence of

Redis data is stored in memory, so that Redis can still ensure that data is not lost after a reboot, and that the data needs to be synchronized from memory to the hard disk in some form, a process called persistence.
Redis supports two persistence modes: RDB mode and AOF mode. RDB Mode

The Rdb method is done via snapshots (snapshotting), and Redis automatically snaps all of the data in memory to the hard disk when certain conditions are met.

The conditions for the snapshot can be customized by the user in the configuration file, consisting of two parameters: time (in seconds) and number of keys changed.

A snapshot is taken when the number of keys changed within the specified time is greater than the specified number.

The RDB is the default persistence mode, and the configuration file has 3 conditions preset:

Save 1
Save
60 10000

The relationship between multiple snapshot conditions is "or". To disable a snapshot, simply delete all of the save parameters.

RDB persistence The default generated file name is Dump.rdb, which can be configured by configuring Dir and Dbfilename to specify the path and file name of the store.

The snapshot process is as follows:

1.redis copy a copy of the current process using fork;
2. The parent process continues to receive and process commands from the client, and the child process begins to write the data in memory to a temporary file on the hard disk;
3. When the child process finishes writing all the data, it replaces the old Rdb file with the temporary file.

As you can see, through temporary files, RDIs guarantees that the RDB files are complete at any time.

The Rdb file is a compressed binary format that occupies little space.

In addition to automatic snapshots, you can manually send save or bgsave to have Redis perform snapshots, which take snapshots of the primary process, which fork the child process.

In addition, you may wonder how the server knows how much I have made changes.

There is a dirty counter and a lastsave timestamp in the server.

When the server executes a database modification command, the dirty counter will be updated, the number of times the command modifies the database, dirty will increase how much, such as: "Set msg Hello" modified one, then dirty add one, if "Mset msg word name Nihao age 20 "then dirty increases by three.

The Lastsave property records the last time the server performed a save operation, which is a Unix timestamp, which, through these two properties, can be a very simple distance from the last time it was saved, and how many times the database was modified, and once the above three conditions are met, the Bgsave command is called automatically. Updates both the Lastsave property and the dirty property to zero.

As for checking whether the save condition satisfies this work, it is performed by the Redis server periodic operation function Servercron the default interval of 100 milliseconds to perform a check, this function is used in many places, note that this function is to maintain the running server function, It is mentioned in the Redis event (Redis server is an event driver, what is event driver?). Is the time when the event will move, or it will be the same as the death, the event driver is divided into file events and time events, Servercron is a kind of time-driven, as for the file driver, is actually the client sent a command, the server will go to execute, and then return the results to the client)

Finally, the Redis itself comes with an Rdb file Checker Redis-check-dump, which can be used to check the Rdb file for completeness. AoF Way

Redis does not turn on aof (append only file) persistence by default and can be turned on by the following parameters:

AppendOnly Yes

After AoF is turned on, each command that changes data in Redis will be written to the AoF file on the hard disk, with the same save location as the RDB, and the file name defaults to appendonly.aof.

The aof file is a plain text file whose content is the content of the original communication protocol that the Redis client sends to Redis.

Although each modification will be recorded in the AoF file, but due to the operating system caching mechanism, the data is not actually written to the hard disk, but into the system's hard disk cache, the default is every 30 seconds to perform a synchronization operation, so the system unexpectedly exits when the unsynchronized data loss. To solve this problem, Redis needs to proactively ask the system to synchronize after writing the AoF file:

#appendfsync always
appendfsync everysec
#appendfsync No

Copy

To avoid a single point of failure, it is a good idea to replicate multiple copies of the database to be deployed on different servers, and Redis provides the process of replicating (replication) functionality that automates synchronization. Configuration

The primary database (master) can read and write operations and automatically synchronizes data to the slave database (slave) When a write operation occurs.

From a database is generally read-only, it can only have one primary database.

The configuration is simple:

The primary database does not require any configuration, only the "slaveof primary database IP Primary database port" is added to the configuration file from the database. principle

After booting from the database, the Sync command is sent to the primary database, and the master database starts to save the snapshot (even if the RDB persistence is turned off) after the Sync command has been received, during which time all commands to the primary database are cached and when the snapshot is saved, The primary database sends the snapshot and cached commands together from the database, saves the snapshot files from the primary database from the database, and sequentially executes the cache command sent from the primary database. During synchronization, from the database is not blocked, it defaults to using the data before the synchronization to continue to respond to the commands sent by the client.

After that, any data changes to the master data are synchronized to the slave database via TCP. read/write separation

Replication can be used to build multiple from the database, the primary database only write operations, from the database responsible for read operations to improve the server load capacity. persisting from a database

To improve performance, you can create a database from the replication feature and enable persistence from the database, while the primary database disables persistence.

When the database crashes, restarting the base database will synchronize the data without worrying about data loss;

When the primary database crashes, the use of the slaveof NO one command from the database will promote the primary database continuation service, and the data will be synchronized back by using the slaveof command to set it to the new primary database from the database after the original primary database restarts. Safety

The Redis database can set a password, and any related client needs to authenticate before executing the command.

The password can be set through the Requirepass of the configuration file.

Example

127.0.0.1:6379> CONFIG get requirepass
1) "Requirepass"
2) ""

By default, this property is blank, indicating that no password is set. You can change this property by executing the following command

127.0.0.1:6379> config set requirepass "jiange"
OK
127.0.0.1:6379> CONFIG get requirepass
1) " Requirepass "
2)" Jiange "

The client needs to authenticate using the AUTH command.

127.0.0.1:6379> AUTH Password

In addition, Redis can modify the bind parameter to restrict access to the address. Conclusion

Well, here's what the Redis QuickStart is all about, and the next thing to do is to use it skillfully in practice.

Later if there is a need, there is time, I will learn the Redis source code implementation (￣▽￣) ".