Disque:redis's father new open source distributed memory job queue

Disque is a distributed memory message agent Salvatore Sanfilippo, the father of Redis, is a new open source. It adapts to the "Redis as Job queue" scenario, but employs a dedicated, standalone, extensible, fault-tolerant design that combines the simplicity and performance of Redis with the C language as a non-blocking network server.

Redis the author Salvatore Sanfilippo (Network Name Antirez ) published a blog post that describes a new project that he has developed in the evenings and on weekends over the past few months. --disque .

Development intent

antirez The reason why development disque redis to process the queue , but the advantages and disadvantages of this are obvious: redis fast, Easy-to-use and a lot of infrastructure already in use, but redis high availability Span style= "font-family: Microsoft Jas Black" >/ Cluster features are designed to be completely biased toward variable data structures, which are very different from immutable messages and are not the best solution.

the important function of message mediation is to ensure that messages are sent at least once or at most once, and that the former is more important. Antirez began to want to make a small number of modifications to Redis , but a few days later found the client algorithm too complex. Redis already has a lot of features, and adding functionality is not a good idea. Moreover, Message Queuing operates in a different way than Redis .

So, is it possible to develop a new message queue?

There are already many message queues in the world, is it worthwhile to make a new one? Antirez think that since so many people use Redis to handle Message Queuing, the existing scheme looks either too simple or too complex, and there must be a chance, so he does.

Development process

For the first time, he did not write the code directly, but spent several weeks thinking about the design and trying to understand what message queues from the user's perspective would make people feel better. The main usage scenario has not changed: delay the job. disque is a generic system, but the main issue is to send messages that may be processed by the job. If anything goes against this scenario, it will be killed.

design has, antirez direct from redis code start. Fortunately redis part is to write c A framework for distributed systems. Protocol, network library, client-side processing, node-to-node message bus already has, no need to write again. But he did not want to affect the redis redis branch, then redis

Fortunately, there are C # of the Client to operate the disque:https://github.com/ziyasal/disque.net.

RELATED links:

Disque:redis's father new open source distributed memory job queue

Disque use Tutorial (disquebook.com)

Disque:redis's father new open source distributed memory job queue