Redis Analysis Series: Starting the loading process

Starting with this article (called The Redis Analysis series), you will explore the internal implementation of Redis by analyzing its source code (with the Redis 2.2.0 RC1 as the standard). This article mainly introduces the Redis boot loading process, which can be divided into the following steps in general:

1. Initializing the global server configuration

2. Load the configuration file (if a profile is specified, the default configuration is used)

3. Initializing the server

4. Loading the database

5. Network Monitoring

The entire boot load process is shown in the following diagram:

Here are some examples of the steps in the previous illustration, some of which, such as database loading, network snooping, are described in detail later in a separate article. initializing the global server configuration

Initializing the global server configuration is done through the initserverconfig () function, primarily the initialization of the server variable, which is a redisserver structure type:

struct Redisserver server;

The contents of the initialization include the following:

1. Network monitoring related, such as binding address, TCP port, etc.
2. Virtual memory related, such as swap file, page size, etc.
3. Save mechanism, how many times there are updates to save
4. Replication related, such as whether it is a slave,master address, port
5. Hash Related Settings
6. Initialize the command table

As in the save mechanism, the server initialization policy is:

1 Update
Appendserversaveparams (60*60,1) within 1 hours;

100 updates in 5 minutes
appendserversaveparams (300,100);

10,000 updates in 1 minutes
appendserversaveparams (60,10000);

If you specify a configuration file when you start the server, some of these server configurations are changed in the load profile step below, depending on the profile content. Load configuration file

If a configuration file is specified, Redis uses the Loadserverconfig () function to load the configuration file, and the whole process is nothing more than to open the configuration file using the standard I/O library, loop through each line and overwrite the default configuration of the previous step.

One thing to note here is that there is a default profile in the code package after the Redis download, and if you start the Redis server without specifying a profile, Redis does not use the configuration of this default file, but instead uses the configuration in the previous step, "initializing the global server configuration." The configuration item provided in the default profile is not the same as the default initialization configuration in the previous step, so if you do not specify a configuration file, you cannot assume that the behavior of Redis will follow the default profile, the most typical example of which is the data retention policy in the default configuration file:

# 1 updates in 15 minutes
save 1

# 100 updates within 5 minutes
save

# 1 updates in 10,000 minutes
save 60 10000

The data saving policy in the global configuration is initialized by default:

1 Update
Appendserversaveparams (60*60,1) within 1 hours;

100 updates in 5 minutes
appendserversaveparams (300,100);

10,000 updates in 1 minutes
appendserversaveparams (60,10000);

Initializing the server

The initialization server works in the Initserver () function, primarily to complete the unfinished work, to continue initializing the server variables, such as setting up signal processing, creating clients, slaves lists, creating Pub/sub channel lists, and creating shared objects:

Shared.crlf = CreateObject (Redis_string,sdsnew ("\ r \ n"));
Shared.ok = CreateObject (Redis_string,sdsnew ("+ok\r\n"));
Shared.err = CreateObject (Redis_string,sdsnew ("-err\r\n"));
Shared.emptybulk = CreateObject (Redis_string,sdsnew ("$0\r\n\r\n"));

Finally, if you enable the virtual memory mechanism, you also need to initialize the virtual memory-related, such as thread I/O, and so on. Load Database

After all of the initialization work has been done, Redis starts loading data into memory, if AppendOnly is enabled (see the config file first if you don't know what this parameter does), Redis loads the data from appendfile, otherwise it loads the data from DBFile.

1. Load data from AppendFile: Loadappendonlyfile () function

Before we do this, let's take a look at what's stored in appendfile, like I did the following two commands (remember to turn on appendonly in the config file):

redis> SET mykey001 myvalue001
ok
redis> GET mykey001
"myvalue001"

Use the cat command to view the contents of the Appendonly.aof file:

$ cat appendonly.aof
*
$6
SELECT
$
0
for
SET
$8
mykey001
$
myvalue001

Well, I believe everyone can see (see this article first), in the appendonly.aof file it is the request command sent from the client, and you can see that for the Get command, it is not saved. Since all the request commands for writing data are saved in appendonly.aof, it is only possible to re-execute the commands when the data is loaded.

In fact, that's exactly what Redis did, temporarily shutting down AppendOnly before starting the load and then Redis creating a fake Redis client:

server.appendonly = 0;

Fakeclient = Createfakeclient ();
Startloading (FP);

The commands in the appendonly.aof file are then read, executed in the context of the fake Redis client, and the server does not respond to the client any more:

FAKECLIENT->ARGC = argc;
FAKECLIENT->ARGV = argv;
Cmd->proc (fakeclient);

/* The fake client should not has a reply *
/redisassert (Fakeclient->bufpos = = 0 && listlength (fakeclient ->reply) = = 0);

If physical memory is not available during the loading process, and Redis opens the VM, the swap operation is also required, and the APPENDONLY flag is reset after the load is complete.

2. Load data from DBFile: Rdbload () function

If Redis does not turn on appendonly, you need to load the data from the database file into memory, the basic steps are as follows:

A. Handling the Select command, which is selecting a database
B. Read key
C. Read value
D. Detect if key is out of date
E. Adding a new object to a hash table
F. Setting the Expiration time (if required)
G. If the VM is turned on, handle the swap Operation Network monitoring

After the initialization configuration and data loading have been completed, Redis initiates the listener. The Redis network library does not use libevent or Libev, but rather a very lightweight library implemented by the author itself (mainly implemented in the Ae.c file), which is written separately when analyzing the network library of Redis later.

Through this article, introduced the Redis start loading process, hoped to have the help to everybody.