Implementation of the Redis Message notification system

Recently, I was busy using Redis to implement a Message notification system. Today I will summarize the technical details. If the Demo code does not have any special instructions, it will be implemented using PhpRedis extensions. For example, if you want to push a global message to all users, it will occupy a lot of memory. In fact, no matter how high the stickiness of the Product

Recently, I was busy using Redis to implement a Message notification system. Today I will summarize the technical details. If the Demo code does not have any special instructions, it will be implemented using PhpRedis extensions. For example, if you want to push a global message to all users, it will occupy a lot of memory. In fact, no matter how high the stickiness of the Product

Recently, I was busy using Redis to implement a Message notification system. Today I will summarize the technical details. If the Demo code does not have any special instructions, it will be implemented using PhpRedis extensions.

Memory

For example, if you want to push a global message to all users, it will occupy a lot of memory. In fact, no matter how high the stickiness of the product, active users are much smaller than all users. Therefore, if you only process logon users, it is quite cost-effective at least in terms of memory consumption, it can be postponed until the user logs on again. If the user does not log on, it will be a hundred times.

Queue

When a large number of users log on at the same time, if all of them are processed in real time, it is easy to crash. At this time, a queue can be used to store the login users to be processed, which means the response is slow at most, but it won't crash.

The LIST Data Type of Redis can naturally create a queue. The Code is as follows:

connect('/tmp/redis.sock');$redis->lPush('usr',);while ($usr = $redis->rPop('usr')) {    var_dump($usr);}?>

For a similar reason, we also need a queue to store the messages to be processed. Of course, you can also use LIST, but LIST can only implement queues in the FIFO or LIFO format according to the inserted sequence. However, messages actually have a priority. For example, personal messages have a high priority, low priority of global messages. In this case, you can use ZSET to implement it. The score concept Naturally realizes the priority.

However, ZSET does not have a native POP operation, so we need to simulate the implementation. The Code is as follows:

zsetPop($zset, self::POSITION_FIRST);    }    public function zRevPop($zset)    {        return $this->zsetPop($zset, self::POSITION_LAST);    }    private function zsetPop($zset, $position)    {        $this->watch($zset);        $element = $this->zRange($zset, $position, $position);        if (!isset($element[0])) {            return false;        }        if ($this->multi()->zRem($zset, $element[0])->exec()) {            return $element[0];        }        return $this->zsetPop($zset, $position);    }}?>

After the POP operation is simulated, we can use ZSET to implement the queue. The Code is as follows:

connect('/tmp/redis.sock');$redis->zAdd('msg',,);while ($msg = $redis->zRevPop('msg')) {    var_dump($msg);}?>

Push and pull

In the past, the question of pushing and pulling in Weibo architecture has been discussed many times. In fact, the messaging and notification system is similar to Weibo, and there are also questions about the push and pull selection. The same answer is similar, that is, the combination of push and pull. Specifically, when a login user obtains a message, it is a process of pulling the message. When sending the message to the login user, it is a process of pushing the message.

Speed

If you want to push 1 million messages, the most straightforward implementation is continuous insertion. The Code is as follows:

Note: Here I use the SET data type. Of course, you can change it to LIST or ZSET as needed.

Redis is fast, but it will be faster with PIPELINE. The Code is as follows:

 exec();}?>

Note: the so-called PIPELINE is to omit unnecessary re-run and package the command to the server for unified processing.

In my test, the speed of using PIPELINE is about ten times faster than that without PIPELINE.

Query

We use the Redis command line to demonstrate how users query messages.

Insert three messages first.

redis> HMSET msg:1 title title1 content content1redis> HMSET msg:2 title title2 content content2redis> HMSET msg:3 title title3 content content3

Then, send the three messages to a user.

redis> SADD usr:123:msg 1redis> SADD usr:123:msg 2redis> SADD usr:123:msg 3

In this case, you can only find some messages that the user has.

redis> SMEMBERS usr:123:msg1) "1"2) "2"3) "3"

If you still need to use

redis> SORT usr:123:msg GET msg:*->title1) "title1"2) "title2"3) "title3"redis> SORT usr:123:msg GET msg:*->content1) "content1"2) "content2"3) "content3"

The disadvantage of SORT is that it can only GET data of the string type. If you want multiple data, you need to GET it multiple times:

redis> SORT usr:123:msg GET msg:*->title GET msg:*->content1) "title1"2) "content1"3) "title2"4) "content2"5) "title3"6) "content3"

In many cases, this is not flexible enough. Fortunately, we can use other methods to balance the advantages and disadvantages. For example, we can add a new field, store the serialization of the complete message in redundancy, and then only GET this field to be OK.

When the query interface is actually exposed, PHP and other programs will not be used for encapsulation, because it will multiply the RPS, we recommend using Webdis, which is a Redis Web proxy, the efficiency is not worth mentioning.

...

Recently, Tumblr published a similar article: Staircar: Redis-powered notifications, introducing some of their usage of the Redis Message notification system. If you are interested, take a look.

Original article address: Implementation of the Redis Message notification system. Thank you for sharing it with the original author.

Related Articles:

Message notification mechanism in HTML 5

Web Notification System Design Problems

Message notification plug-in based on HTML5 Notifications API