On the routing model of message bus

Recently writing a RABBITMQ-based message bus . Although RABBITMQ provides a mechanism for plugin to extend it, due to the fact that the Erlang language is not familiar, given the high cost of getting started, the implementation of plugin is instead based on the smart client + tree topology routing model. This, of course, greatly reduces the flexibility of our functionality, and I'll take a moment to get back to the new article and talk about the limitations of smart client.

Pre-knowledge

RABBITMQ provides only a few very simple api:channel#basicpublish;channel#basicconsume (channel#basicget) for message communication. The API is very simple and can almost be seen as providing only a communication model of Produce/consume. But Rabbitmq's flexibility lies in its combination of multiple exchange. If you do not know the type of exchange, please visit the official website's documentation.

There are several principles: RABBITMQ all messages are sent to exchange, and all messages are fetched from the queue. As for how messages are routed from Exchange to queue, this depends on the type of Exchange (+exchange) combined with Exchange or exchange and queue (but only if the queue can only be used as a leaf node and cannot be combined between queues).

Tree-shaped topology

The entire architecture of the message bus takes the form of a smartclient + server-side tree topology. This is a very typical proxy topology, the advantages of this topology are: easy to hide internal details, convenient unified control, easy to achieve interception and so on. Although it is a tree-like topology, but for the implementation of its details there is still a great deal, the following discussion in the implementation process of some personal ideas.

The tradeoff of the app in the topology

The initial implementation is simple and supports only the point-to-point Produce/consume model, which is also the native model of RABBITMQ. At the time, part of the focus was on the log that it could record all through the bus at very small cost. Its tree topology is this:

(Note: The rectangle in the diagram represents Exchange, the circle is the queue, the same as below)

After the implementation is complete, the granularity of these queues is found to be too coarse, almost at the app level. This does not satisfy the need for an app to have multiple queues, a requirement that is common after an app is split into distributed components. Therefore, the granularity of the queue must be further refined and almost refined into a queue "service" (yes, you can understand that each queue, whether in the production of messages, or in the consumption of messages, as if providing a service). The app is the ancestor of these "service queues". So these apps are referred to as the last level of exchange to be the organizer of these service queues, and the queues are at the same height, which is also easy to control. So the topology diagram becomes this:

After continuing to think about it, the only benefit of moving apps from the queue to the last level of exchange is the ability to see the ownership of a queue at a glance on the console and to control all the queue under an app. In fact, to do this, just need to maintain a good app and queue logical relationship (through the control console interface to manipulate the database), there is no need to maintain the app becomes real exchange also makes the whole route more layer, but affect performance. So decided to remove the app Exchange layer.

Tradeoffs of request/response implementation

The message bus currently offers four types of communication models:

• Produce/consume
• Request/response
  
• Publish/subscribe
  
• Broadcast
  

From the topology diagram above, it can be seen that there are mainly two message route exchange:pubsub,business. Where the message in business is point-to-point, a message can only be consumed once and has only one consumer. The PUBSUB message is one to many, which is not reusable from the implementation of the same set of systems. Back to the above, we said that we provided a four-way communication model, and we have only two messages to route exchange, which means that the communication model multiplexed the routing exchange, and their correspondence is:

• Produce/consume-Business
  
• Request/response-Business
  
• Publish/subscribe-PubSub
  
• Broadcast->pubsub
  

Multiplexing routing of Exchange results in the reuse of queues, which brings a little trouble to the implementation of Request/response. The following two questions are mainly:

1. The queue's responsibilities are unclear: produce messages are mixed with request messages in the same queue and the API for consuming messages is differentiated, which causes a consuming API to consume messages it "doesn't want" to consume (such as response encounters produce messages). At this time these it "do not want" to consume the news, there is no good way to deal with.
   
2. Infinite wait: This is mainly related to the implementation mechanism of Request/response. It is a type of communication that is blocking synchronous waiting for a response. If it is multiplexed with the Produce/consume mode, then if there is a large number of messages sent by produce in the queue that have not been consumed, then the message sent by request will not be consumed in time, resulting in response's almost impossible to respond quickly. That is, the request/response pattern may not really be realized at all.
   

The crux of the problem is that different types of messages reuse the same queue, but the APIs that consume different types of messages are differentiated, and the order of the different types of messages in the queue is not measurable.

There are two ways to solve this problem:

1. Multiplexing queue, reusing consumption API, specifying different processing logic for different messages;
   
2. Don't use queues to differentiate between consumer APIs.
   

I almost immediately discharged the first idea, because the distinction between the communication model is to facilitate the programming of the client, the purpose of encapsulation is for a higher level of abstraction, and the first train of thought certainly can not achieve request/response this model, because this model is characterized by real-time nature, This is not guaranteed by the multiplexing queue. The tradeoff of publish/subscribe implementation from what you can see PubSub the type of Exchange is Fanout (Fanout represents diffusion, that is, exchange and queue that are bound to it can receive messages). In fact, the implementation of publish/subscribe on the basis of RABBITMQ, there are two ways to achieve:

1. PubSub exchange for fanout, client filtering
2. PubSub exchange for topic, service-side precision push
   

To explain a little bit, first you need to understand: this is a generic exchange, that is, a queue that is similar to the Publish/subscirbe type of communication is bound underneath it.
For the first way: if it is fanout, then the client only needs to call publish to push a message, and when the message is routed to PubSub, all the bound queues below will receive the message (that is, how the Fanout is actually broadcast). So how do you get a queue that isn't subscribed to the channel without receiving the message? In fact, when the client calls subscribe, the received message is filtered inside the client (the client connects to a control center and can get the subscription relationship).
For the second way: just as opposed to the first implementation, it defines pubsub Exchange as topic. When a client invokes publish to push a message, it is sent to the queue of each subscription under PubSub by acquiring a subscription relationship, so the actual number of sends here is consistent with the number of subscriptions for that channel, and in this case, the subscribe consumes without filtering, Because it can be determined that the message sent to its queue is definitely sent by the channel it subscribes to!
There are pros and cons to both of these models:

• The first way: The number of sends is low, the server is efficient (fanout almost no need for a specific route), but each pubsub under the queue will receive messages, and if some of the queue's consumers are offline, these messages will occupy the server resources, and increase the speed of client consumption, because the need to filter
  

• The second way: many times, how many subscribers to send the number of times, the efficiency of the service is lower than the first, but this sequential push can effectively save the resources of the message server, and this mode of client consumption efficiency is high

The specific implementation should depend on the application scenario. This is a trade-off between performance and resources. At present, the implementation has been switched from the first mode to the second one-in order to be consistent with the other two models. Because the other two models are producer authentication rights, consumers just consume. The tradeoff of broadcast implementation our awareness of broadcasting should be a message that can be received in real time by all communication participants (sometimes broadcast messages may even be considered to be non-rejecting, but this depends on the scenario). Because consume and response should also be able to receive broadcast messages. Therefore, it should not be mixed with publish/subscribe (although they are very similar from the technical implementation, you can think that broadcast is a special case of publish/subscribe). If you do not mix, it means that for broadcast, you have to separately allocate a queue for each communication participant? That is, the number of communication participants on the system will have the corresponding broadcast message queue.

From the previous discussion, if you want to reduce the number of queues, you have to reuse the queue, and if we reuse the queue, we must merge the API to consume the different types of messages in the queue and assume the risk of receiving the same type of message is not timely. So I tried the following topology diagram structure, which is also the current topology diagram:

It enables each queue to be reused, and receives broadcast messages in addition to receiving messages related to its own duties. The advantage of this design is that it saves the resources of the server. It re-uses the queue with ordinary business messages, which causes it to "follow" calls only when the consuming API is called. And some of the pure message senders mentioned above, such as produce,publish, are usually not queued and therefore cannot receive broadcast messages, which is also a tradeoff design.

On the routing model of message bus