Why should I choose RABBITMQ, RABBITMQ profile, various MQ selection comparisons

Objective:

What MQ is. What the queue is, MQ we can understand as Message Queuing, the queues we can understand as pipelines. The message is delivered in a pipe way.

Scene:

1. In fact, when we are in the double 11, when we have a large number of seconds in the morning to kill and buy merchandise, and then go to settle, we will find that the interface will remind us, let us wait, as well as some friendly picture text reminders. Instead of the times of previous years, the page is stuck and the error is presented to the user.

In this business scenario, we can use the queue mechanism to deal with, because at the same time the settlement can only achieve so much.

2. In our usual supermarket shopping is the same, when we are in the settlement, and will not swarm the same influx of cash registers, but queuing to settle. This is also the queue mechanism.

Yes, it's a line. One after the other, you can't jump in the queue.
RABBITMQ Introduction

AMQP, Advanced message Queuing Protocol, is an open standard for application-layer protocols designed for message-oriented middleware. Message middleware is mainly used for decoupling between components, the sender of the message does not need to know the existence of the message consumer, and vice versa. The main features of AMQP are message-oriented, queue, routing (including point-to-point and publish/subscribe), reliability, and security. RABBITMQ is an open-source AMQP implementation that is written in Erlang and supported by a variety of clients such as Python, Ruby,. NET, Java, JMS, C, PHP, ActionScript, XMPP, stomp, etc., and support Ajax. It is used to store and forward messages in distributed system, which is very good in ease of use, extensibility, high availability and so on. The following will focus on some of the basic concepts in RABBITMQ, and understand these concepts as the basis for using good RABBITMQ. connectionfactory, Connection, Channel

ConnectionFactory, Connection, and channel are the most basic objects in the API RABBITMQ provided externally. Connection is the socket link for RABBITMQ, which encapsulates some of the logic associated with the socket protocol. ConnectionFactory is a manufacturing facility for connection. Channel is one of the most important interfaces we have with RABBITMQ, and most of our business operations are done in the channel interface, including defining queue, defining exchange, binding queue and exchange, publishing messages, and so on. Queue

A queue is an internal object of RABBITMQ that is used to store messages, as shown in the following illustration.

The messages in the RABBITMQ can only be stored in the queue, and the producer (p in the image below) produces the message and is eventually posted to the queue, and the consumer (c in the image below) can get the message from the queue and consume it.

Multiple consumers can subscribe to the same queue, and messages in the queue are evenly distributed to multiple consumers, rather than every consumer receiving all the messages and processing. Message Acknowledgment

In real-world applications, consumers may receive messages in the queue, but they do not complete the performance outage (or other surprises), which can cause messages to be lost. To prevent this from happening, we can ask the consumer to send a receipt after consuming the message to RABBITMQ,RABBITMQ to receive the message receipt (msg acknowledgment) before removing the message from the queue If RABBITMQ does not receive a receipt and detects that the consumer's RABBITMQ connection is broken, RABBITMQ will send the message to other consumers (if multiple consumers exist) for processing. There is no concept of timeout here, and a consumer processing the message longer will not cause the message to be sent to other consumers unless its RABBITMQ connection is broken. There is another problem here, if our developers forget to send receipts to RABBITMQ after they have processed the business logic, this will result in more and more messages piling up in serious bug--queue, and consumers will repeat consuming these messages and repeating the business logic after they restart ...

In addition the pub message is no ACK. Message Durability

If we want to not lose the message even when the RABBITMQ service is restarted, we can set the queue and message to be persisted (durable), which ensures that our RABBITMQ messages will not be lost in most cases. But still can not solve the small probability of the loss of events (such as the RABBITMQ server has received the message of the producer, but not enough time to persist the message when the RABBITMQ server is powered off), if we need to manage this small probability event, then we have to use the transaction. Since this is a simple introduction to RABBITMQ, RABBITMQ related transactions will not be explained here. Prefetch Count

We said earlier that if more than one consumer subscribes to a message in the same queue, the message in the queue is split to multiple consumers. At this point, if the processing time of each message is different, it may cause some consumers to be busy, while others quickly finish the work at hand and always idle. We can set Prefetchcount to limit the number of messages each queue sends to each consumer, such as when we set up prefetchcount=1, and the queue sends a message to each consumer every time The queue will send a message to the consumer after the consumer finishes processing the message.

Exchange

In the previous section we saw that the producer delivered the message to the queue, which in fact never happened in RABBITMQ. The real situation is that the producer sends the message to exchange (the switch, x in the image below), which is routed by Exchange to one or more of the queue (or discarded).

Exchange is the logic by which messages are routed to the queue. This will be described in the binding section. There are four types of exchange in RABBITMQ, and different types have different routing policies, which are described in the Exchange Types section. Routing Key

When a producer sends a message to exchange, a routing key is typically specified to specify the routing rules for the message, and the routing key needs to be used in conjunction with Exchange type and binding key to finally take effect. In the case where Exchange type is fixed with the binding key (which is usually fixed in normal use), our producer can determine where the message flows by specifying routing key when sending a message to exchange. The length limit set for routing key is limited to 255 bytes rabbitmq. Binding

Exchange is associated with the queue through binding in RABBITMQ so that RABBITMQ knows how to properly route messages to the specified queue. Binding Key

At the same time as binding (binding) Exchange and queue, a binding key is typically specified; When a consumer sends a message to exchange, a routing key is typically specified; When the binding key and the routing When the key matches, the message is routed to the corresponding queue. This will be illustrated in the Exchange Types section, which will list the actual examples. These bindings allow the same binding key to be used when binding multiple queues to the same exchange. The binding key does not take effect in all cases, it relies on exchange type, for example, exchange of type fanout ignores the binding key, but instead routes the message to all the queue bound to that exchange. Exchange Types

RABBITMQ Common Exchange type has fanout, direct, topic, headers Four (the AMQP specification also mentions two types of exchange, respectively, system and custom, not described here), The following are described separately. fanout

The fanout type of Exchange routing rule is simple, and it routes all messages sent to that Exchange to all the queues it binds to.

In the image above, all messages sent to Exchange (X) by the producer (P) are routed to the two queue in the graph and ultimately consumed by two consumers (C1 and C2). Direct

The direct type of Exchange routing rule is also simple, which routes messages to a queue whose binding key exactly matches the routing key.

As an example of the configuration of the above diagram, we send a message to exchange with routingkey= "error", the message is routed to Queue1 (amqp.gen-s9b ..., This is the queue name automatically generated by RABBITMQ) and Queue2 (Amqp.gen-agl ... If we send a message with routingkey= "info" or routingkey= "warning", the message will only be routed to Queue2. If we send a message with another routingkey, the message is not routed to the two queue. Topic

The preceding Exchange routing rules for the direct type are exactly the same as the binding key and routing key, but this strict matching method does not meet the actual business requirements in many cases. Topic type of exchange has been extended on matching rules, similar to the exchage of the direct type, and to the queue where the message is routed to the binding key that matches the routing key, but there are some differences in the matching rules, which contract: Routing key is a period number ". "Delimited string (we will be the period number".) "Each separated string of independent strings is called a word", such as "Stock.usd.nyse", "NYSE.VMW", "quick.orange.rabbit" binding key and routing key are also the period number ". "Delimited string binding key can have two special characters" * "and" # "for fuzzy Matching, where" * "is used to match a word," # "is used to match multiple words (can be 0)

As an example of the configuration in the diagram above, messages that routingkey= "Quick.orange.rabbit" are routed to Q1 and q2,routingkey= "Lazy.orange.fox" messages are routed to Q1 and q2,routingkey= " Lazy.brown.fox "messages that are routed to q2,routingkey=" Lazy.pink.rabbit "will be routed to Q2 (only sent to Q2 once, although this routingkey matches the Q2 of two Bindingkey) , routingkey= "Quick.brown.fox", routingkey= "Orange", routingkey= "quick.orange.male.rabbit" messages will be discarded, Because they don't match any bindingkey. Headers

The headers type of exchange does not rely on the matching rules of routing key and the binding key to route messages, but rather matches the headers attributes in the message content sent. Specifies a set of key-value pairs when a queue is bound to exchange, and when a message is sent to Exchange, RABBITMQ takes the headers of the message (which is also a key-value pair). Compares whether the key-value pair exactly matches the key-value pair specified by the queue with Exchange bindings, or if the message is routed to the queue in an exact match, otherwise it is not routed to the queue. This type of exchange is not used (but should also be useful), so don't introduce it. RPC

MQ itself is asynchronous-based message processing, and all the Producers (P) in the previous example send the message to RABBITMQ without knowing that the consumer (C) is processing success or failure (even if there is no consumer to handle the message). However, in the actual application scenario, we will probably need some synchronous processing and need to wait for the server to finish processing my message before proceeding to the next step. This is equivalent to the RPC (remote Procedure call, which is called remotely). RPC is also supported in RABBITMQ.

The mechanism for implementing RPC in RabbitMQ is that when a client sends a request (message), the properties of the message (Messageproperties, the properties in the AMQP protocol are defined in 14, which are sent along with the message) set two values ReplyTo (a queue name that tells the server to send a message to the queue when it's finished processing) and Correlationid (the identification number of this request, which needs to be returned after the server has finished processing.) Based on this ID, the client will know which request was successfully executed or failed to execute. After the server receives the message and processes the server-side processing of the message, a reply message is generated to the queue specified by ReplyTo, with the Correlationid property client previously subscribed ReplyTo Specifies the queue from which the server's reply message is received, based on the Correlationid attribute in which the request was executed, followed by a subsequent business process summary based on the results of the execution

This article introduces the most important concepts in RABBITMQ, and makes full use of the capabilities provided by RABBITMQ to handle most of our asynchronous operations. RabbitMQ selection and comparison 1. From the Community activity level

According to the current information on the network, RabbitMQ, Activem, ZeroMQ, in general, RabbitMQ is the first choice.
2. Persistent message comparison

ZeroMq not supported, ACTIVEMQ and RABBITMQ are supported. Persistent message is mainly refers to our machine in the case of force majeure and other circumstances hang out, the message will not be lost mechanism. 3. Integrated Technology Implementation

Reliability, flexible routing, clustering, transactions, highly available queues, message sequencing, problem tracking, visual management tools, plug-in systems, and more.

Rabbitmq/kafka Best, ActiveMq second, ZeroMq worst. Of course zeromq can also do, but they have to manually write code implementation, the amount of code is not small. Especially in reliability: persistence, delivery Confirmation, publisher verification, and high availability. 4. High concurrency

Undoubtedly, RabbitMQ is the highest because its implementation language is inherently highly concurrent and highly available in the Erlang language.
5. Comparison of comparative concerns, RabbitMQ and Kafka

RabbitMq than Kafka mature, in usability, stability, reliability, RabbitMq better than Kafka (theoretically).

In addition, the location of Kafka mainly in the log, and so on, because the original intention of Kafka design is to deal with the log, can be seen as a log (message) system an important component, targeted very strong, so if the business is still recommended to choose RabbitMq.

Also, Kafka's performance (throughput, TPS) is much higher than RABBITMQ. Selection Final Summary:

If we already have a choice in our system,