Redis Notes 03

RDB Persistence

1. rdb files are used to save and restore all key-value pairs of data in all databases of the Redis server.

2. The save command is executed directly by the server process, so the command blocks the server.

3. Bgsave enables a child process to perform a save operation, so the command does not block the server.

4. The server status saves all save conditions set with the Save option, and the server automatically executes the Bgsave command when any of the save conditions are met.

5. The Rdb file is a compressed binary file that consists of several parts.

6. For different types of key-value pairs, the Rdb file will save them in different ways.

7. Redis is an in-memory database that stores its own database state in memory. The RDB persistence feature saves Redis in-memory database state to disk and avoids accidental data loss.

8. Redis does not have a dedicated command to load an RDB file, and it automatically loads the Rdb file whenever the Redis server detects an RDB file at startup.

9. Typically, the AoF file is updated more frequently than the Rdb file, so if the server turns on the AOF persistence feature, the server takes precedence over the aof file to restore the database state. Only in the AOF persistence feature

When it is turned off, the server uses the Rdb file to restore the database state.

10. During the execution of the BGSAVE command, the new Save,bgsave command is rejected and the bgrewriteaof command is deferred. If the bgrewriteaof is executed, then the Bgsave command is rejected.

11. The server will remain blocked during the loading of the Rdb file until the loading work is complete.

12. Used to set multiple save conditions through the Save option, but the server executes the Bgsave command whenever any of these conditions are met. For example, if we provide a configuration to the server:

Save 1         10000  //server within 60 seconds, make at least 10,000 modifications to the database

When the modification frequency is low, the changes are guaranteed to be saved. When the modification frequency is high, save it in time.

The server periodic operation function of Redis Servercron is executed every 100 milliseconds by default, which is used to maintain the server being executed, and one of its tasks is to check the save condition set by the Save option.

Whether to satisfy, if satisfied, execute the bgsave command.

. rdb file consists of "REDIS" string, db_version (database version), databases (database contents), EOF (end flag), check_sum (checksum, determining whether the file is complete)

With the Rdb file Retrieval tool Redis-check-dump, Redis also finds many tools to process RDB files on the web.

AOF Persistence

1. The aof file records the database state of the server by saving all write command requests that modify the database.

2. All commands in the AoF file are saved in the format of the Redis command request protocol.

3. The command request is saved to the AOF buffer and then periodically written and synchronized to the AoF file.

4. The different values of the Appendfsync option have a significant impact on the security of the AOF persistence feature and the performance of the Redis server.

5. The server can restore the original state of the database simply by loading and re-executing the commands stored in the AoF file.

6. AoF rewrite can produce a new aof file, the new AoF file and the original aof copy of the saved database state, but smaller size.

7. AoF Rewrite is an ambiguous name that is implemented by reading a key-value pair in the database, and the program does not need to perform any read, parse, or write operations on the existing AOF file.

8. When executing the bgrewriteaof command, the Redis server maintains a aof rewrite buffer that records all write commands executed by the server during the creation of a new aof file for the child process. When a child process finishes creating a new AoF file,

The server will overwrite all of the contents in the buffer to the end of the new aof file, making the database state saved by the old and new two aof files. Finally, the server replaces the AoF file with the new AoF file to complete the aof file rewrite operation.

9. Rdb persistence by saving key-value pairs in the database to record the state of the database, AOF persistence records the state of the database by saving the write commands that the Redis server executes.

10. When the server starts, you can restore the database state before the server shuts down by loading and executing the Save command in the AoF file.

AoF the implementation of the persistence function can be divided into three steps: command append, file write, file synchronization.

The value of the Appendfysync option, the default value is everysec:

Always: Writes and synchronizes all contents of the AOF_BUF buffer to the aof file.

Everysec: Writes all the contents of the AOF_BUF buffer to the aof file, and if the time distance for the last synchronization of the AoF file is now more than one second, then the AoF file is synchronized, and the synchronization operation is performed by a single thread.

No: Writes all contents of the AOF_BUF buffer to the aof file, but does not synchronize the AoF file, and when synchronization is determined by the operating system.

The detailed steps for Redis to read the aof file to restore the database state are as follows:

1). Create a pseudo-client without a network connection.

2). A write command is parsed and read from the AoF file.

3). Use pseudo-client to execute the read write command.

4). Follow steps 2 and 3 to know that all the commands in the AoF file have been processed.

Redis puts AOF overrides into child processes, which can achieve two purposes at the same time:

1). During aof rewrite of the child process, the server process (the parent process) can continue to process the command request

2). The child process has a copy of the data from the server process, using a child process instead of a thread, to ensure the security of the data in the case of avoiding it.

15. During the entire AOF background rewrite process, only the signal handler function executes when the server process (parent process) is blocked.

Event

1. The Redis server is an event driver, and the events handled by the server are divided into two categories: Time events (timed events) and file events (interacting with clients).

2. The file event handler is a network communication program implemented based on the reactor mode. The file event handler listens to multiple sockets at the same time. Keeps the simplicity of the Redis internal single-threaded design.

3. A file event is an abstraction of a socket operation: each time a socket becomes a acceptable, writable (writable), or readable (readable), the corresponding file event is generated.

4. File events are divided into two categories: the Ae_readable event (read event) and the Ae_writable event (write event).

5. Time events are divided into timed events and periodic events: Timed events arrive only once at a specified time, while periodic events arrive at intervals.

6. In general, the server only performs a servercron function for a time event, and this event is a recurring event.

7. There is a partnership between file events and time events, and the server takes turns to handle both events, and the process of processing time is not preempted.

8. Because a time event executes after a file event, and there is no preemption between events, the actual processing time of the time event is usually slightly later than the time-to-event set time.

9. Time Event Application Example: Servercron function, mainly includes the following work:

1). Update the server's various statistics, such as time, memory usage, database occupancy, etc.

2). Clean up key-value pairs in the database.

3). Close and clean the client that is failing the connection.

4). Try the AOF and RDB persistence operations

5). If the server is the primary server, synchronize periodically from the server.

6). If you are in cluster mode, the cluster is scheduled for periodic synchronization and connectivity testing.

The Redis server runs the Servercron function as a recurring event, and every once during the service runs, Servercron executes once until the server shuts down.

11. Starting with Redis2.8, the user can modify the Hz option to adjust the number of executions per second of the Servercron. In version 2.6, the default Servercron is 10 times per second.

Redis avoids each poll by taking the fastest executed event when traversing the time-event list.

Redis writes multiple processors for file events that are used to implement the network communication requirements that are not needed.

14. The server places all time events in an unordered list, and whenever the time event executor runs, he traverses the entire list, finds all the time events that have arrived, and invokes the appropriate event handler.

Client:

1. The server State structure uses the clients linked list to connect multiple client states, and the newly added customer order status is placed at the end of the list.

2. The Flags property of the client state uses a different flag to represent the role of the client and the current state of the client.

3. The input buffer records the command request sent by the client, and the buffer size cannot exceed 1G.

4. The command parameters and the number of parameters are recorded in the argv and ARGC properties of the client state, while the CMD attribute records the implementation function of the client to execute the command.

5. The client is available from both fixed-size buffers and variable-size buffers, where the maximum size of a fixed-size buffer is 16KB, and the maximum size of a variable-size buffer cannot exceed the hard limit set by the server.

6. There are two input buffer limits, if the output buffer size exceeds the hard limit set by the server, then the client will be shut down immediately, in addition, if the client for a certain period of time, has exceeded the server settings

Soft limit, the client will also be shut down.

7. When a client connects to the server over the network, the server creates the appropriate client state for the client. A network connection is turned off, a command request is sent in a non-protocol format, a target for the client kill command,

The idle time timeout and the output buffer size exceed the limit, which causes the client to be shut down.

8. The pseudo-client that handles the Lua script is created when the server is initialized, and the client persists until the server shuts down.

9. The pseudo-client used when loading the aof file is created dynamically at the start of loading work and is closed after loading.

10. By default, a client that is connected to the server does not have a name. Use the client SetName command to set a name for clients to make the client's identity clearer.

The. PubSub command and the scriptload command do not modify the database, but the server state is modified, so you need to force this command to be written to the AoF file. Also, in order for both the primary server and the slave server to load the script load command correctly

Specifies the script that the server needs to use the REDIS_FORCE_REPL flag to force the script load command to replicate to all slave servers.

12. You can use the CLIENT-OUTPUT-BUFFER-LIMIT option to set different soft and hard limits for normal clients, from server-side clients, and from clients that perform publishing and subscription functions. The format is:

Client-output-buffer-limit <class>

Redis server uses single-threaded single-process processing for command requests and network communication with multiple clients.

14. When a master-slave server makes a copy operation, the primary server becomes the client from the server, and the slave server becomes the client of the primary server.

15. The client authenticated property is used to record whether the client has passed authentication. Used only when the server has the authentication feature enabled.

Redis Notes 03