The principle analysis of Redis Publish/subscribe mechanism

Redis implements publishing and subscription functionality through commands such as PUBLISH, SUBSCRIBE, and Psubscribe.

These commands are widely used to build instant messaging applications such as network chat rooms (chatroom) and live broadcasts, real-time reminders, and so on.

This article is to deepen our understanding of Redis by analyzing the PUBSUB.C files in the Redis source code to understand the underlying implementation of the publish and subscribe mechanism.

Subscribe, Post and Unsubscribe

Before you start studying the source code, you might want to review how several related commands are used.

The PUBLISH command is used to send information to a given channel, and the return value is the number of subscribers who receive the message:

Redis> PUBLISH treehole "Top secret here ..." (integer) 0 redis> PUBLISH chatroom "Hi?" (integer) 1

The SUBSCRIBE command subscribes to one or more channels given:

Redis> SUBSCRIBE Chatroom Reading messages ... (Press Ctrl-c to quit) 1) "Subscribe" # Subscription Feedback 2) "Chatroom" # subscribed Channel 3) (integer) 1 # The number of channels/modes currently subscribed to by the client 1) "Message" # Information 2) "Chatroom" # Channel to send messages 3) "Hi?" # Information Content

The return value of SUBSCRIBE, 1) for SUBSCRIBE is the feedback of the subscription, and 1 for the message is the information sent by the subscribed channel.

SUBSCRIBE can also subscribe to multiple channels so that the information it receives may come from multiple channels:

Redis> SUBSCRIBE Chatroom Talk-to-jack Reading messages ... (Press Ctrl-c to quit) 1) "Subscribe" # Subscribe to chatroom feedback 2) "Chatroom" 3) (integer) 1 1) "Subscribe" # Subscribe to Talk-to-jack Feedback 2) "Talk-to-jack" 3) (integer) 2 1) "Message" # Messages from chatroom 2) "Chatroom" 3) "Yahoo!" 1) "Message" # Messages from Talk-to-peter 2) "Talk-to-jack" 3) "GoodMorning, Peter."

Psubscribe provides a way to subscribe to all channels that match a given pattern, for example, by using it.* as input, you can subscribe to all it. The beginning of the channel, such as It.news, It.blog, it.tweets, etc.:

Redis> Psubscribe it.* Reading messages ... (Press Ctrl-c to quit) 1) "Psubscribe" 2) "it.*" 3) (integer) 1 1) "Pmessage" 2) "it.*" # matching pattern 3) "It.news" # source channel of the message 4) "Redis 2.6RC5 release" # Message content 1) "Pmessage" 2) "it.*" 3) "It.blog" 4) "Why NoSQL Matters" 1) "Pmessage" 2) "it.*" 3) "It.tweet" 4) "@redis: When would the 2.6 stable release?"

Of course, Psubscribe can also accept multiple parameters to match multiple patterns.

Finally, the unsubscribe command and Punsubscribe are responsible for retiring the given channel or mode.

Publishing and subscription mechanisms

When a client sends a message to a subscriber via the PUBLISH command, we call the client Publisher.

When a client uses the SUBSCRIBE or Psubscribe command to receive information, we call the client the Subscriber (Subscriber).

To understand the relationship between a decoupled publisher (publisher) and a Subscriber (subscriber), Redis uses the channel as an intermediary for both-the publisher publishes the information directly to the channel, and the channel is responsible for sending the information to the appropriate subscribers. There is no correlation between the publisher and the Subscriber, nor does it know the other person's presence:

　　

Click to enlarge

Now that we know the mechanism for publishing and subscribing, we can begin to look at the specific implementation, starting with the Redis subscription command.

Implementation of the SUBSCRIBE command

As mentioned earlier, Redis takes all the tasks of receiving and transmitting information to the channel, and all channel information is stored in the REDISSERVER structure:

struct Redisserver { Omitted... Dict *pubsub_channels; Map channels to list of subscribed clients Omitted... };

Pubsub_channels is a dictionary, the key of the dictionary is a channel, and the dictionary value is a linked list, the list of all subscriptions to the channel to save the client.

For example, if you have a channel called news in a Redisserver instance, and the channel is subscribed to by both Client_123 and client_456 two clients, the redisserver structure should look like this:

　　

Click to enlarge

As you can see, the key to implementing the SUBSCRIBE command is to add the client to the subscription list for a given channel.

The function pubsubsubscribechannel is the underlying implementation of the SUBSCRIBE command, which completes the task of adding the client to the subscription list:

Subscribe to designated channels The subscription successfully returns 1, if already subscribed, returns 0 int Pubsubsubscribechannel (redisclient *c, RobJ *channel) { struct Dictentry *de; List *clients = NULL; int retval = 0; /* ADD the channel to the client, channels hash Table */ Dictadd returns DICT_OK when a new element is added successfully So this sentence indicates that if the new subscription channel succeeds, then ... if (Dictadd (c->pubsub_channels,channel,null) = = DICT_OK) { retval = 1; Incrrefcount (channel); /* Add the client to the channel-list of clients hash table */ Adding a client to a linked list that subscribes to a given channel This list is the value of a hash table, the key of the hash table is the given channel This hash table is stored in the server.pubsub_channels. de = Dictfind (Server.pubsub_channels,channel); if (de = = NULL) { If the de equals NULL Indicates that the client is the first client to subscribe to this channel Then create a new list and add it to the hash table Clients = Listcreate (); Dictadd (server.pubsub_channels,channel,clients); Incrrefcount (channel); } else { If the de is not empty, remove the clients list Clients = Dictgetval (DE); } Joining a client to a linked list Listaddnodetail (CLIENTS,C); } /* Notify the client */ Addreply (C,shared.mbulkhdr[3]); Addreply (C,shared.subscribebulk); Back to subscribed channels Addreplybulk (C,channel); Returns the sum of the number of channels and modes currently subscribed by the client Addreplylonglong (C,dictsize (c->pubsub_channels) +listlength (c->pubsub_patterns)); return retval; }

Implementation of the Psubscribe command

In addition to subscribing directly to a given channel, you can subscribe to a pattern using psubscribe, and subscribe to a pattern that is equivalent to subscribing to all channel matching this pattern.

Like the Redisserver.pubsub_channels property, the Redisserver.pubsub_patterns property is used to save all subscribed patterns, unlike Pubsub_channels, which pubsub_ Patterns is a linked list (not a dictionary):

struct Redisserver { Omitted... List *pubsub_patterns; A List of Pubsub_patterns Omitted... };

Each node of the pubsub_patterns is an instance of a Pubsubpattern structure that holds the subscribed schema and the client clients that subscribe to the pattern:

typedef struct PUBSUBPATTERN { Redisclient *client; RobJ *pattern; } Pubsubpattern;

For example, suppose that in a redisserver instance, a pattern called news.* is subscribed by both the client client_789 and client_999, then this redisserver structure should look like this:

　　

Click to enlarge

It is now possible to know that the key to implementing the Psubscribe command is to add the client and subscription patterns to the redisserver.pubsub_patterns.

Pubsubsubscribepattern is the underlying implementation of Psubscribe, which adds the client and the subscribed schema to the Redisserver.pubsub_patterns:

Subscribe to a specified pattern The subscription successfully returns 1, if already subscribed, returns 0 int Pubsubsubscribepattern (redisclient *c, RobJ *pattern) { int retval = 0; To find the specified pattern in C->pubsub_patterns If the return value is NULL, it means that the pattern has not been subscribed to by this client. if (Listsearchkey (c->pubsub_patterns,pattern) = = NULL) { retval = 1; Add pattern to client Pubsub_patterns Listaddnodetail (C->pubsub_patterns,pattern); Incrrefcount (pattern); Adding the pattern to the server Pubsubpattern *pat; Pat = Zmalloc (sizeof (*PAT)); Pat->pattern = Getdecodedobject (pattern); Pat->client = C; Listaddnodetail (SERVER.PUBSUB_PATTERNS,PAT); } /* Notify the client */ Addreply (C,shared.mbulkhdr[3]); Addreply (C,shared.psubscribebulk); Returns the subscribed mode Addreplybulk (C,pattern); Returns the sum of the number of channels and modes currently subscribed by the client Addreplylonglong (C,dictsize (c->pubsub_channels) +listlength (c->pubsub_patterns)); return retval; }

Implementation of the PUBLISH command

To use the PUBLISH command to send a message to subscribers, you need to perform the following two steps:

1) using the given channel as the key, find the list of all the clients that subscribed to the channel in the Redisserver.pubsub_channels dictionary, traverse the list, and publish the message to all Subscribers.

2) traverse the redisserver.pubsub_patterns linked list, match the schema in the list with the given channel, and if the match succeeds, publish the message to the client in the appropriate mode.

For example, if you have two clients subscribing to It.news Channel and it.* mode respectively, when executing the command publish it.news "Hello Moto", the subscriber of It.news channel will receive the message at Step 1, and when publish proceed to step 2 , subscribers to the it.* mode will also receive information.

The actual implementation of the PUBLISH command is done by the Pubsubpublishmessage function, which is fully defined as follows:

Send Message int Pubsubpublishmessage (RobJ *channel, RobJ *message) { int receivers = 0; struct Dictentry *de; ListNode *ln; Listiter Li; /* Send to clients listening for that channel */ Send a message to subscribers on all channels de = Dictfind (Server.pubsub_channels,channel); if (DE) { List *list = Dictgetval (DE); Remove all Subscribers ListNode *ln; Listiter Li; Traverse all Subscribers and send messages to them Listrewind (List,&li); while ((ln = listnext (&li)) = NULL) { Redisclient *c = ln->value; Addreply (C,shared.mbulkhdr[3]); Addreply (C,shared.messagebulk); Addreplybulk (C,channel); Print Channel name Addreplybulk (C,message); Print messages receivers++; Update the number of recipients } } /* Send to clients listening to matching channels */ Send a message to all subscribers who are matched to the pattern if (Listlength (server.pubsub_patterns)) { Listrewind (Server.pubsub_patterns,&li); Remove all modes Channel = Getdecodedobject (channel); while ((ln = listnext (&li)) = NULL) { Pubsubpattern *pat = ln->value; Remove mode If the pattern matches the channel, Send a message to subscribers in this mode if (Stringmatchlen (char*) pat->pattern->ptr, Sdslen (PAT->PATTERN->PTR), (char*) Channel->ptr, Sdslen (Channel->ptr), 0)) { Addreply (Pat->client,shared.mbulkhdr[4]); Addreply (Pat->client,shared.pmessagebulk); Addreplybulk (Pat->client,pat->pattern); Print the pattern that is matched Addreplybulk (Pat->client,channel); Print Channel name Addreplybulk (Pat->client,message); Print messages receivers++; Update the number of recipients } } Decrrefcount (channel); Releasing The Used channel } return receivers; Returns the number of recipients }

Implementation of UNSUBSCRIBE and Punsubscribe

Unsubscribe and Punsubscribe respectively are SUBSCRIBE and psubscribe anti-operation, if you understand the SUBSCRIBE and psubscribe working mechanism, it should not be difficult to understand the two anti-operation principle, So here we omit the detailed analysis, interested can directly see the code.

Section

The analysis of the pubsub mechanism of Redis is over, as usual, the full PUBSUB.C file with comments can be found on my GITHUB: Https://github.com/huangz1990/reading_redis_source

The principle analysis of Redis Publish/subscribe mechanism