Redis Implementation Client

Client

Redis server is a typical one-to-many server program: a server can establish a network connection with multiple clients, each client can send a command request to the server, and the server receives and processes a command request sent by the client and returns a command reply to the client. Using the file event handler implemented by the I/O multiplexing technology, the REDIS Server processes command requests in a single-threaded single process and communicates with multiple clients on the network

For each client connected to the server, the server establishes a corresponding Redis.h/redisclient structure (client state) for these clients, which holds the current state information of the client and the data structures needed to perform the relevant functions, including:

• Socket descriptor for Client
• The name of the client
• Flag value for the client (flag)
• Point to the database pointer that the client is using, and the number of the database
• The command that the client is currently executing, the arguments to the command, the number of command arguments, and pointers to the command implementation function
• Input buffers and output buffers for clients
• Replication state information for the client, and the data structures required for replication
• The data structure used by the client to execute a list of Brpop, Blpop, etc. blocking commands
• The client's state of affairs and the data structure used to execute the watch command
• Data structures used by clients to perform publishing and subscription functions
• Authentication identity of the client
• Client creation time, client and server last communication time, and client's output flushing area size exceeds soft limit time

The clients property of the Redis server state structure is a linked list that holds the state structure of all clients connected to the server, performs bulk operations on the client, or finds a specified client, which can be done by traversing the clients list:

Redis.h

struct redisServer {
     ...
// A linked list that holds all client state
     list * clients;
     ...
};

Figure 1-1 Client and server

As an example, figure 1-1 shows a server connected to three clients, and figure 1-2 shows an example of this server clients list

Figure 1-2 Clients linked list

Client Properties

The client state contains attributes that can be divided into two categories:

• A class is a more general-purpose property that is rarely related to a particular feature, and is used by clients regardless of what they do.
• The other is properties related to a particular function, such as the DB attribute and the Dictid attribute needed to manipulate the database, the master attribute to be used to perform the transaction, and the Watched_keys attribute to be used to execute the watch command.

Socket descriptor

The FD attribute of the client state records the socket descriptor that the client is using

Redis.h

typedef struct redisClient {
    int fd;
    ……
} redisClient;

Depending on the client type, the value of the FD attribute can be either 1 or an integer greater than-1:

• The value of the FD property of the pseudo-client (fake clients) is-1: The pseudo-client handles the command request from the AoF file or the Lua script instead of the network, so the client does not need a socket connection and naturally does not need to record the socket descriptor. The Redis server now uses pseudo-clients in two places, one for loading the aof file and restoring the database state, and the other for performing the Redis commands contained in the LUA script
• The value of the FD property of the normal client is an integer greater than-1: The normal client uses the socket to communicate with the server, so the server uses the FD attribute to record the client socket descriptor. Because a valid socket descriptor cannot be-1, the value of the socket descriptor for the normal client must be an integer greater than-1

The Execute client List command lists all current normal clients connected to the server, and the FD domain in the command output shows the socket descriptor used by the server connection client:

127.0.0.1:6379> CLIENT list
id=3 addr=127.0.0.1:57522 fd=9 name= age=76021 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=32768 obl=0 oll=0 omem=0 events=r cmd=client

Name

By default, a client that is connected to the server does not have a name. For example, the Name field is blank in the client List command above. However, we can use the client setname command to set a name for clients to make the client's identity clearer. The following shows a list of clients after the client executes the client setname command

127.0.0.1:6379> CLIENT SETNAME message_quque
OK
127.0.0.1:6379> CLIENT list
id=3 addr=127.0.0.1:57522 fd=9 name=message_quque age=76399 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=32768 obl=0 oll=0 omem=0 events=r cmd=client

The name of the client is recorded in the Name property of the client state:

Redis.h

typedef struct redisClient {
    ……
    robj *name;             
    ……
} redisClient;


	

If the client does not set a name for itself, the Name property of the corresponding client state points to a null pointer; Conversely, if the client sets a name for itself, the name property points to a string object, and the object holds the client's name. Figure 1-3 shows an example of the client status, shown by the Name property, with the client name "Message_queue"

Figure 1-3 Name Property Example

Sign

Flag property of the client flags records the role of the client and the current state of the client

Redis.h

typedef struct redisClient {
    ……
    int flags;           
    ……
} redisClient;

The value of the Flags property can be a single flag: flags = <FLAG>, or it can be binary with multiple flags: flags = <flags1> | <flags2> | ......

Each flag is represented by a constant, and a subset of the flags record the role of the client:

• When a master-slave server makes a copy operation, the master-slave server becomes the client from the server, and from the server becomes the client of the primary server. The Redis_master flag represents a primary server represented by the client, and the Redis_slave flag indicates that the client represents a slave server
• The REDIS_PRE_PSYNC flag indicates that the client represents a version of a slave server that is less than Redis2.8, and the primary server cannot synchronize with this from the server using the Psync command. This flag can only be used when the REDIS_SLAVE flag is open
• The Redis_lua_client identity indicates that the client is a pseudo-client dedicated to processing Redis commands contained within a LUA script

Another part of the flag records the current state of the client:

• The Redis_monitor flag indicates that the client is executing the monitor command

• The Redis_unix_socket flag indicates that the server uses a UNIX socket to connect to the client

• The redis_blocked flag indicates that the client is being blocked by commands such as Brpop, Blpop, etc.

• The redis_unblocked flag indicates that the client has been detached from the blocking state represented by the redis_blocked flag and is no longer blocked. The redis_unblocked flag can only be used if the redis_blocked flag is already open

• The REDIS_MULTI flag indicates that the client is executing a transaction

• The REDIS_DIRTY_CAS flag indicates that the database key monitored by the transaction using the Watch command has been modified, and the REDIS_DIRTY_EXEC flag indicates that the transaction encountered an error while the command was queued, and that the above two flags indicate that the security of the transaction has been compromised. As soon as any of these two tokens is opened, the EXEC command will inevitably fail. These two flags can only be used if the client has the REDIS_MULTI flag turned on

• The REDIS_CLOSE_ASAP flag indicates that the client's output buffer size exceeds the scope allowed by the server, and the server shuts down the client the next time the Servercron function is executed, so that the stability of the server is not affected by this client. All content that accumulates in the output buffer is released directly and is not returned to the client

• The REDIS_CLOSE_AFTER_REPLY flag indicates that a user has executed the Client_kill command on the client, or the command request sent to the server by the client contains the wrong protocol content. The server sends all the content that the client accumulates in the output buffer to the client and then shuts down the client

• The redis_asking flag indicates that the client sent the asking command to the cluster node (the server running in cluster mode)

• The REDIS_FORCE_AOF flag forces the server to write the currently executing command to the AoF file, and the REDIS_FORCE_REPL flag forces the primary server to replicate the currently executing command to all slave servers. Executing the pubsub command causes the client to open the REDIS_FORCE_AOF flag, and executing the script_load command causes the client to open the REDIS_FORCE_AOF flag and the REDIS_FORCE_REPL flag

• During the command propagation of the master-slave server, from the server need to send the Replicationack command to the primary server, before sending this command, from the server must open the primary server corresponding to the client's redis_master_force_reply flag, Otherwise, the send operation will be denied execution

All the flags mentioned above are defined in the Redis.h file

The particularity of the PubSub command and the script load command

Typically, Redis writes only those commands that have been modified on the database to the AoF file and to the individual slave servers. If a command does not make any modifications to the database, it is considered a read-only command, which is not written to the AoF file and is not copied to the slave server

The above rules apply to most Redis commands, but the pubsub command and the script Load command are exceptions. The pubsub command does not modify the database, but the PUBSUB command sends a message to all subscribers to the channel. This behavior has side effects, and the status of all clients receiving the message will change because of this command. Therefore, the server needs to use the REDIS_FORCE_AOF flag to force the command to be written to the AoF file so that the server can execute the same pubsub command again and produce the same side effects when loading the AoF file in the future. The script load command is similar to the PubSub command: Although the script load command did not modify the database, it modifies the server state, so it is a command with side effects, and the server needs to use the REDIS_FORCE_AOF flag. Force this command to be written to the AoF file so that the server can have the same side effects when loading the AoF file in the future

In addition, in order for both the primary server and the slave server to properly load the script specified by the script load command, the server needs to use the REDIS_FORCE_REPL flag to force replication of the script load command to all slave servers

Here are some examples of flags properties

# Client is a master server
REDIS_MASTER
# Client is being blocked by list command
REDIS_BLOCKED
# The client is executing a transaction, but the security of the transaction has been broken
REDIS_MULTI | REDIS_DIRTY_CAS
# The client is a slave and the version is lower than Redis 2.8
REDIS_SLAVE | REDIS_PRE_PSYNC
# This is a pseudo-client specifically for executing Redis commands contained in Lua scripts
# It forces the server to write the currently executed command to the AOF file and copy it to the slave server
REDIS_LUA_CLIENT | REDIS_FORCE_AOF | REDIS_FORCE_REPL

Input buffers

The input buffer of the client state is used to hold the command request sent by the client

Redis.h

typedef struct redisClient {
    ……
    sds querybuf;
    ……
} redisClient;

Raise a chestnut if the client sends the following command request to the server:

SET Key value

Then the Querybuf property of the client state will be a SDS value that contains the following:

*3\r\n$3\r\nSET\r\n$3\r\nkey\r\n$5\r\nvalue\r\n

Figure 1-4 shows the SDS value and the appearance of the Querybuf attribute

Figure 1-4 Querybuf Property Example

Command and command parameters

After the server saves a command request sent by the client to the Querybuf property of the client state, the server parses the contents of the command request and saves the resulting command parameters and the number of command arguments to the argv and ARGC properties of the client state, respectively:

Redis.h

typedef struct redisClient {
    ……
    int argc;
    robj **argv;
    ……
} redisClient;

The argv property is an array in which each item in the array is a string object, where Argv[0] is the command to execute, and then the other items are arguments passed to the command. The ARGC property is responsible for recording the length of the argv array

For a chestnut, for the querybuf attribute shown in Figure 1-4, the server parses and creates the Argv property and ARGC property shown in Figure 1-5.

Figure 1-5 argv Properties and ARGC property examples

Note that in the client state shown in Figure 1-5, the value of the ARGC property is 3, not 2, because the name "SET" of the command is itself a parameter

Implementation function of the command

After the server parses and obtains the value of the Argv property and the Argc property from the protocol content, the server finds the command implementation function for the command in the command table based on the value of the item argv[0]. Figure 1-6 shows an example of a command, the table is a dictionary, the Dictionary key is an SDS structure, save the name of the command, the value of the dictionary is the Rediscommand structure of the command, the structure holds the command implementation function, command flag, command should be given the number of arguments, Statistics for the total number of executions and total consumption time of the command

Figure 1-6 Command table

When the program successfully finds the Rediscommand struct for argv[0] in the command table, it points the cmd pointer of the client state to the struct:

Redis.h

typedef struct redisClient {
    ……
    struct redisCommand *cmd;
    ……
} redisClient;

The server can then use the Rediscommand structure that the cmd attribute points to, as well as the command parameter information saved in the argv, ARGC properties, invoke the command implementation function, execute the command specified by the client

Figure 1-7 illustrates the entire process of locating the command table and pointing the cmd pointer of the client state to the target Rediscommand structure when the server argv[0] is "SET"

Figure 1-7 Finding a command and setting the Cmd property

The lookup operation for the command table does not differentiate the case of the input letters, so the lookup results are the same whether argv[0] is "set", "Set" or "set", etc.

Output buffers

The command reply that executes the command is saved in the output buffer of the client state, each client has two output buffers available, the size of one buffer is fixed, and the size of the other buffer is variable.

• Fixed-size buffers are used to hold smaller replies, such as OK, short string values, integer values, error replies, and so on.
• Variable-size buffers are used to hold larger replies, such as a very long string value, a list of many items, a collection containing many elements, and so on.

The client's fixed size buffers are combined by BUF and Bufpos two properties:

Redis.h

typedef struct redisClient {
    ……
    int bufpos;
    char buf[REDIS_REPLY_CHUNK_BYTES];
} redisClient;

BUF is a byte array of size redis_reply_chunk_bytes bytes, and the Bufpos property records the number of bytes currently used by the BUF array. The default value for the Redis_reply_chunk_bytes constant is 16*1024, which means that the default size of the BUF array is 16KB

Figure 1-8 shows an example of using a fixed size buffer to hold the return value +ok\r\n

Figure 1-8 Example of a fixed size buffer

When the space for the BUF array is exhausted, or if the reply is too large to fit into the BUF array, the server starts using a variable size buffer. A variable size buffer consists of a reply linked list and one or more string objects:

Redis.h

typedef struct redisClient {
    ……
    list *reply;
    ……
} redisClient;

By using a linked list to concatenate multiple string objects, the server can save a very long command reply to the client without being constrained by the size of the fixed size buffer 16KB. Figure 1-9 shows a reply linked list with three string objects

Figure 1-9 Example of a variable size buffer

Identity verification

The authenticated property of the client state is used to record whether the client is authenticated:

Redis.h

typedef struct redisClient {
    ……
    int authenticated;      /* when requirepass is non-NULL */
    ……
} redisClient;

If the value of authenticated is 0, the client has not yet been authenticated, and if the value of authenticated is 1, the client is authenticated. For a chestnut, for a client that is not yet process-authenticated, the client state is shown in the Authenticated property 1-10

Figure 1-10 Client status when no identity is verified

When the client authenticated property has a value of 0 o'clock, all other commands sent by the client are rejected by the server except for the AUTH command:

127.0.0.1:6379> PING
(error) NOAUTH Authentication required.
127.0.0.1:6379> SET msg "hello world"
(error) NOAUTH Authentication required.

When the client successfully authenticates through the AUTH command, the value of the client state authenticated property changes from 0 to 1, 1-11, and the client can send a command request to the server as usual:

127.0.0.1:6379> AUTH 123456
OK
127.0.0.1:6379> PING
PONG
127.0.0.1:6379> SET msg "hello world"
OK

Figure 1-11 Client status that has been authenticated

The Authenticated property is used only when the server has authentication enabled, and if the server does not have authentication enabled, the server does not deny command requests from unauthenticated clients even if the value of the authenticated property is 0 (the default). For more information about server authentication, refer to the configuration file for instructions on the Requirepass option

Time

Finally, the client also has several time-related properties:

Redis.h

typedef struct redisClient {
    ……
    time_t ctime;          
    time_t lastinteraction; 
    time_t obuf_soft_limit_reached_time;
    ……
} redisClient;

The CTime property records the time the client was created, which can be used to calculate how many seconds the client and server have been connected, and the age domain of the client List command records the number of seconds

127.0.0.1:6379> CLIENT list
…… age=87315 ……
…… age=535 ……

The Lastinteraction property records the last time the client interacts with the server (interaction), where the interaction can be either a client sending a command request to the server or a server sending a command reply to the client. The Lastinteraction property can be used to calculate the idle time of the client, that is, how many seconds have elapsed since the client's last interaction with the server, and the idle field of the client List command records this number of seconds

127.0.0.1:6379> CLIENT list
…… idle=10916 ……
…… idle=0 ……

The Obuf_soft_limit_reached_time property records the time that the output buffer first reached the soft limit (soft times)

Client creation and shutdown

The server uses different ways to create and close different types of clients, and this section describes how the server creates and shuts down clients

Create a normal client

If the client is a normal client connecting over a network connection and a server, the server invokes the connection event handler to create the appropriate client state for the client when the client uses the Connect function to connect to the service side. and add this client state to the end of the clients linked list in the server state structure body

Take a chestnut, assuming that there are currently C1 and C2 two ordinary clients are connecting to the server, then when a new normal client C3 connected to the server, the server will c3 the corresponding client state added to the end of the clients list, 1-12, Which is surrounded by a dashed line is the server for C3 the newly created client state

Figure 1-12 Clients linked list of server state structure

Close the normal client

A normal client can be shut down for a variety of reasons:

• If the client process exits or is killed, the network connection between the client and the server is closed, causing the client to be shut down
• If a client sends a command request to the server with a non-conforming protocol format, the client is also shut down by the server
• If the client becomes the target of the Clientkill command, it will also be closed
• If the user sets the timeout configuration option for the server, the client will be shut down when the client's idling time exceeds the value set by the timeout option. However, there are some exceptions to the timeout option: If the client is the primary server (the Redis_master flag is turned on), from the server (the Redis_slave flag is turned on), is blocked by commands such as Blpop (the redis_blocked flag is turned on), Or you are performing a subscription command such as subscribe, psubscribe, or even if the client is idling longer than the timeout option, the client will not be shut down by the server
• If the size of the command request sent by the client exceeds the limit of the input buffer (by default, 1GB), the client is shut down by the server
• If the size of the command reply that you want to send to the client exceeds the limit size of the output buffer, the client is shut down by the server

As mentioned earlier in the output buffer, a variable size buffer consists of a list and any number of string objects, in theory, this buffer can hold any length of command reply. However, in order to avoid client replies being too large and consuming too many server resources, the server will always check the size of the client's output buffer and perform the appropriate throttling action when the buffer size is out of range. The server uses two modes to limit the size of the client output buffer:

• Hard limit: If the output buffer size exceeds the size set by the hard limit, the server shuts down the client immediately
• Soft limit (soft limit): If the output buffer size exceeds the size set by the soft limit, but does not exceed the hard limit, then the server will use the server state structure Obuf_soft_limit_reached_time Property records the start time that the client arrives at the soft limit, after which the server continues to monitor the client, and if the output buffer size has been extended beyond the soft limit and lasts longer than the server set, the server shuts down the client, and conversely, if the client no longer exceeds the soft limit within the specified time, Then the client will not be shut down and Obuf_soft_limit_reached_time will be zeroed.

You can use Client-output-buffer-limit to set different soft and hard limits for normal clients, from server clients, from clients that perform publishing and subscription functions, in the following formats:

client-output-buffer-limit <class> <hard limit> <soft limit> <soft seconds>


Here are three example settings:

client-output-buffer-limit normal 0 0 0 #Do not limit the output buffer size
client-output-buffer-limit slave 256mb 64mb 60
client-output-buffer-limit pubsub 32mb 8mb 60 #The client that performs the publish and subscribe functions has a hard limit of 32mb, a soft limit of 8mb, and a soft limit duration of 60 seconds


The first line sets the hard and soft limits of the common client to 0, which means that the output buffer size of the client is not limited

The second line sets the hard limit from the server client to 256MB, the soft limit to 64MB, and the soft limit to 60 seconds.

The third line sets the hard limit of the client that performs the publish and subscribe function to 32MB, the soft limit to 8MB, and the soft limit to 60 seconds.

For more usage of the client-output-buffer-limit option, refer to the sample configuration file redis.conf

Lua script pseudo client

The server will create a pseudo client responsible for executing the Redis command contained in the Lua script during initialization, and associate this pseudo client in the lua_client attribute of the server structure:

redis.h

struct redisServer {
    ...
    redisClient * lua_client;
    ...
};


The lua_client pseudo-client will exist throughout the life cycle of the server. This client will only be closed when the server is shut down.

AOF file pseudo client

When the server loads the AOF file, it creates a pseudo client for executing the Redis command contained in the AOF file, and closes the pseudo client after loading is completed

 

Redis client