Redis Notes 04

Server

1. A command request from send to finish mainly consists of the following steps:

1). The client sends a command request to the server

2). The server reads the command request and parses the command parameters

3). The command executor looks up the implementation function of the command according to the parameters, then executes the implementation function and draws the command reply

4). The server returns the command reply to the client

2. The Servercron function is executed every 100 milliseconds by default, and its work mainly includes updating the server state information, processing the sigterm signal received by the server, managing client resources and database status, checking and performing persistent operations, and so on.

3. The server from boot to grace to handle the client's command request needs to perform the following steps:

1). Initialize the server state structure. Sets the default value for each property in the structure.

2). Load server configuration options.

3). Initialize the server data structure. This includes creating shared objects, such as some commonly used strings ("OK", etc.), Integer 1 to 10000 string objects, and so on.

4). Restore the state of the database, loading the RDB or aof file.

5). Execute the event loop to begin accepting connection requests from the client.

4. Main operations performed by the Servercron function:

1). Update the server time cache. The system time can be cached each time the system current time is too resource-intensive. Unimportant events use cache time, and important events still get the current system time.

2). Update the LRU clock. One of the server time caches. Each Redis object has an LRU property that holds the time the object was last accessed by a command.

3). The update server executes the number of commands per second. Can be viewed through the instantaneous_ops_per_sec domain of the info Stats command.

4). Update the server memory peak record. The Used_memory_peak and Used_memory_peak_human two domains of the INFO memory command record the server's peak memory in two formats, respectively.

5). Process the sigterm signal. After the server shuts down the signal and receives the signal, the server will persist the RDB before shutting itself down.

6). Manage client resources. Close, reassign input buffers, and so on.

7). Manage database resources. A subset of the database in the server is checked, the expiration key is deleted, the field is shrunk, and so on.

8). Execute the deferred bgrewriteaof.

9). Check the health status of the persistence operation. Checks whether the new Rdb file is new complete, checks whether the aof rewrite condition is met, and so on.

10). Writes the contents of the AOF buffer to the aof file

11). Close the async client. Close those clients whose output buffer size exceeds the limit.

12). Increase the value of the Cronloopse counter. This is also the number of times that the Servercron function executes.

Copy

1. Redis2.8 the previous replication function did not efficiently handle the re-replication after disconnection, but Redis2.8 the newly added partial resynchronization feature to resolve the problem.

2. Partial resynchronization is achieved by copying offsets, copying the backlog buffers, and three parts of the server run ID.

3. At the beginning of the copy operation, the slave server becomes the client of the primary server, and the replication steps are performed by sending a command request to the primary server, and the master and subordinate servers become clients of each other later in the copy operation.

4. The primary server updates the state from the server by propagating commands to the slave server, keeping the master-slave server consistent, while the slave server sends commands to the primary server for heartbeat detection and command loss detection.

5. In Redis, users can replicate another server (the primary server) by executing the slaveof command or setting the SLAVEOF option to have one server (from the server).

6. Implementation of the Legacy replication feature:

1). Sync

A. Sending sync commands from the server to the primary server

B. The primary server executes the Bgsave command, generates an RDB file in the background, and uses the buffer to record all the write commands that are executed from now on

C. The primary server sends the RDB file to the slave server, loading the Rdb file from the server

D. The primary server sends the command of the buffer to the slave server

2). Command propagation: In order for the master and slave servers to remain consistent, the primary server is required to perform command propagation operations from the server: the master server sends its own write commands to execute from the server.

7. Defects in legacy replication: Wire Break replication will re-replicate everything on the master server.

8. With the implementation of the new version of replication, Redis starts with the 2.8 version and uses the Psync command instead of the Sync command to perform the synchronous operation at copy time. The Psync command has both full and partial resynchronization modes:

1). Full resynchronization: The steps performed are basically the same as the steps of the Sync command.

2). Partial resynchronization: Refers to the operation after the synchronization is broken.

9. Partial resynchronization implementation:

1). Copy offset: The master-slave server maintains a copy offset, respectively. By comparing the copy offset of the master-slave server, the program can easily know whether the master and slave servers are in a consistent state.

2). Replication Backlog Buffer: The replication backlog is a fixed-length FIFO queue maintained by the primary server with a default size of 1M. A portion of the most recently propagated write command is stored in the buffer, and the replication backlog buffers the queue

The corresponding copy offset is recorded for each byte of the

3). Server run ID: On initial connection, the primary server passes the run ID to the slave server and saves the ID from the server. When you disconnect a re-connection, you determine whether a replication relationship was preceded by this ID.

10. It is important to correctly estimate and set the size of the replication backlog buffer. Estimating formula: Second*write_size_per_second, second is the average time required for the connection. The write_size_per_second indicates that the primary server generates per second

The amount of write command data. For example: 1m,5 seconds per second to connect, the buffer size can not be less than 5M. Security can be set to twice times.

The implementation of the Psync command:

There are two ways to invoke the Psync command:

1). PSYNC? -1: Not copied before

2). PSYNC <runid> <offset>, Runid Primary server run ID, offset offsets

Home Server reply:

1). Full re-sync

2). Partial synchronization

3). Cannot identify Psync,redis version less than 2.8

12. Implementation of replication

1). Set the address and port of the primary server: slaveof <master_ip> <master_port>

2). Establishing a Socket connection

3). Send Ping command

4). Authentication

5). Send port information that will be notified to the primary server from the server listening port.

6). Sync

7). Command propagation

13. Heartbeat detection, in the command propagation phase, from the server by default at a frequency per second, to the primary server to send commands: replconf ACK <replication_offset>

1). Detect network connection status from the server

2). Auxiliary implementation min-slaves configuration options. Prevents the primary server from executing the Write command in the event of an unsafe situation. For example, we provide the following settings to the primary server:

Min-slaves-to-write 3

Min-slaves-max-lag 10

The primary server rejects the write command when the number of servers from the server is less than 3, or if the three latency (lag) value from the server is greater than or equal to 10 seconds.

3). The detection command is missing. By comparing the offsets, the lost data can be detected when the network is disconnected. Re-sent to the slave server.

The replconf ACK command and replication backlog are all new in Redis 2.8, so it's best to use 2.8 or later to ensure data consistency at the master-slave server during replication

Redis Notes 04