Implementation of the redis Message notification system

Recently, I was busy using redis to implement a Message notification system. Today I will summarize the technical details, which demonstratesCodeUnless otherwise specified, all phpredis extensions are used.

Memory

For example, if you want to push a global message to all users, it will occupy a lot of memory. In fact, no matter how high the stickiness of the product, active users are much smaller than all users. Therefore, if you only process logon users, it is quite cost-effective at least in terms of memory consumption, it can be postponed until the user logs on again. If the user does not log on, it will be a hundred times.

Queue

When a large number of users log on at the same time, if all of them are processed in real time, it is easy to crash. At this time, a queue can be used to store the login users to be processed, which means the response is slow at most, but it won't crash.

The List Data Type of redis can naturally create a queue. The Code is as follows:

 
<? PHP $ redis = new redis; $ redis-> connect ('/tmp/redis. sock '); $ redis-> lpush ('usr', <usrid>); While ($ USR = $ redis-> rpop ('usr ')) {var_dump ($ USR) ;}?>
 

For a similar reason, we also need a queue to store the messages to be processed. Of course, you can also use list, but list can only implement queues in the FIFO or LIFO format according to the inserted sequence. However, messages actually have a priority. For example, personal messages have a high priority, low priority of global messages. In this case, you can use zset to implement it. The score concept Naturally realizes the priority.

However, zset does not have a native pop operation, so we need to simulate the implementation. The Code is as follows:

<? Phpclass redisclient extends redis {const position_first = 0; const position_last =-1; Public Function zpop ($ zset) {return $ this-> zsetpop ($ zset, self :: position_first);} public function zrevpop ($ zset) {return $ this-> zsetpop ($ zset, self: position_last);} private function zsetpop ($ zset, $ position) {$ this-> Watch ($ zset); $ element = $ this-> zrange ($ zset, $ position, $ position); If (! Isset ($ element [0]) {return false;} if ($ this-> multi ()-> zrem ($ zset, $ element [0]) -> exec () {return $ element [0];} return $ this-> zsetpop ($ zset, $ position) ;}}?>
 

After the pop operation is simulated, we can use zset to implement the queue. The Code is as follows:

 
<? PHP $ redis = new redisclient; $ redis-> connect ('/tmp/redis. sock '); $ redis-> zadd ('msg', <priority>, <msgid>); While ($ MSG = $ redis-> zrevpop ('msg ')) {var_dump ($ MSG) ;}?>
 

Push and pull

In the past, the question of pushing and pulling in Weibo architecture has been discussed many times. In fact, the messaging and notification system is similar to Weibo, and there are also questions about the push and pull selection. The same answer is similar, that is, the combination of push and pull. Specifically, when a login user obtains a message, it is a process of pulling the message. When sending the message to the login user, it is a process of pushing the message.

Speed

If you want to push 1 million messages, the most straightforward implementation is continuous insertion. The Code is as follows:

 
<? Phpfor ($ msgid = 1; $ msgid <= 1000000; $ msgid ++) {$ redis-> sadd ('usr: <usrid>: MSG ', $ msgid) ;}?>
 

Note: Here I use the set data type. Of course, you can change it to list or zset as needed.

Redis is fast, but it will be faster with pipeline. The Code is as follows:

 
<? Phpfor ($ I = 1; $ I <= 100; $ I ++) {$ redis-> multi (redis: Pipeline); For ($ j = 1; $ j <= 10000; $ J ++) {$ msgid = ($ I-1) * 10000 + $ J; $ redis-> sadd ('usr: <usrid>: MSG ', $ msgid) ;}$ redis-> exec () ;}?>
 

Note: the so-called pipeline is to omit unnecessary re-run and package the command to the server for unified processing.

In my test, the speed of using pipeline is about ten times faster than that without pipeline.

Query

We use the redis command line to demonstrate how users query messages.

Insert three messages, whose <msgid> values are 1, 2, and 3:

Redis> hmset MSG: 1 title title1 content content1redis> hmset MSG: 2 Title title2 content content2redis> hmset MSG: 3 title title3 content content3
 

Then, send the three messages to a user, whose <usrid> is 123:

 
Redis> sadd USR: 123: MSG 1 redis> sadd USR: 123: MSG 2 redis> sadd USR: 123: MSG 3
 

In this case, you can only find some <msgid>:

 
Redis> smembers USR: 123: msg1) "1" 2) "2" 3) "3"
 

If you want to useProgramAccording to <msgid>, the next query is undoubtedly inefficient. Fortunately, the built-in sort command of redis can get twice the result with half the effort. In fact, it is similar to the join in SQL:

 
Redis> sort USR: 123: MSG get MSG: *-> title1) "title1" 2) "title2" 3) "title3" redis> sort USR: 123: MSG get MSG: *-> content1) "content1" 2) "content2" 3) "content3"
 

The disadvantage of sort is that it can only get data of the string type. If you want multiple data, you need to get it multiple times:

Redis> sort USR: 123: MSG get MSG: *-> title get MSG: *-> content1) "title1" 2) "content1" 3) "title2" 4) "content2" 5) "title3" 6) "content3"
 

In many cases, this is not flexible enough. Fortunately, we can use other methods to balance the advantages and disadvantages. For example, we can add a new field, store the serialization of the complete message in redundancy, and then only get this field to be OK.

When the query interface is actually exposed, PHP and other programs will not be used for encapsulation, because it will multiply the RPS, we recommend using webdis, which is a redis web proxy, the efficiency is not worth mentioning.

Go to: http://huoding.com/2012/02/29/146