[Turn] RABBITMQ Introduction

Transferred from: http://lynnkong.iteye.com/blog/1699684

1 What is RABBITMQ?

RABBITMQ is one of the messaging middleware implementations of the AMQP (Advanced Message Queuing Protocol) that originated in the financial system and is used to store and forward messages in distributed systems, with good performance in terms of ease of use, scalability, and high availability. Message middleware is mainly used for decoupling between components, the sender of the message does not need to know the existence of the message consumer, and vice versa:

Unidirectional decoupling

Bidirectional decoupling (for example, RPC)

For example, a log system, easy to use RABBITMQ to simplify the workload, one consumer can do the normal processing of messages, the other consumer is responsible for logging messages, As long as the queue of two consumer is specified in the program to be bound to the same exchange in the same way, the remaining message distribution work is done by RABBITMQ.

Using RABBITMQ server requires:

1. Erlang language Pack;

2. RABBITMQ installation package;

The RABBITMQ also provides a Java client (a jar package).

2 Concepts and features 2.1 switch (Exchange):

1. Receive the message, forward the message to the bound queue. Four types: Direct, topic, headers and fanout

Direct: Forwards the message to the queue specified by Routigkey

Topic: Forwarding Messages by rules (most flexible)

Headers: (This hasn't come into contact yet)

Fanout: Forwarding messages to all bound queues

2. If no queue is bound on the switch, the message sent to the switch is lost.

3. A switch can bind multiple queues, and one queue can be bound by multiple switches.

4. The topic type Exchanger parses the Routing-key property of the message through pattern matching. It divides the strings of routing-key and binding-key into words. These words are separated by dots. It will also recognize two wildcard characters: #匹配0个或者多个单词, * match one word. For example, binding Key:*.stock. #匹配routing Key:usd.stcok and eur.stock.db, but does not match Stock.nana.

There are other types of switches, such as header, failover, System, and so on, that are not implemented in the current RABBITMQ version.

5. Because the exchanger is a named entity, declaring an existing exchanger, but trying to give different types is an error. The client needs to delete the existing switch and then re-declare and give the new type.

6. Properties of the Exchanger:

-Persistence: If enabled, the switch will be valid until the server restarts.

-Automatic deletion: If enabled, the switch will automatically delete itself after its bound queue has been deleted.

-Inertia: If the switch is not declared, it will cause an exception when executed to use, and will not be actively declared.

2.2 Queues (Queue):

1. The queue is a RABBITMQ internal object that stores messages. A queue of the same property can be defined repeatedly.

2. Temporary queue. Channel.queuedeclare (), sometimes you do not need to specify the name of the queue, and you want to delete the queue when disconnecting.

3. Properties of the queue:

-Persistence: If enabled, the queue will be valid until the server restarts.

-Automatic deletion: If enabled, the queue will automatically delete itself after all consumers have stopped using it.

-Inertia: If no queue is declared, it will cause an exception when executed to use, and will not be actively declared.

-Exclusivity: If enabled, the queue can only be used by consumers who declare it.

These properties can be used to create transient or private queues such as exclusive and self-deleting. This queue will be automatically deleted after all the clients linked to it are disconnected. They are only briefly connected to the server, but can be used to implement peer communication such as RPC or on AMQ. 4. RPC is used like this: The RPC client declares a reply queue, uniquely named (for example, with a UUID), and is self-deleted and exclusive. It then sends a request to some exchanger, which contains the name of the previously declared reply queue in the Reply-to field of the message. The RPC server will answer these requests, using the reply-to of the message as routing key (the default binder binds all queues to the default switch, the name "amp. Exchanger type name") to the default switch. Note that this is only customary and can be based on the contract with the RPC server, which can interpret any property of the message (even the data body) to decide who to reply to.

2.3 Message Passing:

1. Messages are saved in the queue, and the message is sent to the consumer listening to the message queue in a poll, which can dynamically increase the consumer to improve the processing power of the message.

2. In order to achieve load balancing, you can notify RABBITMQ on the consumer side, and the next message will not be accepted until a message is processed.

Channel.basic_qos (Prefetch_count=1)

Note: To prevent a message in the queue from accumulating if all consumers are processing it.

3. The message has 14 properties, the most commonly used:

DeliveryMode: Persistent Properties

ContentType: Encoding

ReplyTo: Specifying a callback queue

Correlationid: Message ID

Instance code:

4. The message producer can choose whether to be notified when the message is sent to the exchanger and not delivered to the queue (no binder exists) and/or if no consumer can handle it immediately. By setting the mandatory and/or immediate properties of the message to true, the ability to deliver the guarantee mechanism has been enhanced.

5. In addition, a producer can set the message's persistent property to true. As a result, the server will attempt to store these messages in a stable location until the server crashes. Of course, these messages will certainly not be delivered to a non-durable queue.

2.4 High Availability (HA):

1. Message ack, notifies the RABBITMQ that the message has been processed and can be removed from memory. If the consumer does not send an ACK (unlike ACTIVEMQ, in RABBITMQ, where the message does not expire) due to downtime or link failure, RABBITMQ will resend the message to another consumer listening to the queue.

Channel.basicconsume (QueueName, Noack=false, consumer);

2. Persistence of messages and queues. You can specify the persistence properties of a queue when you define a queue (Q: How do I delete a persistent queue?). ）

Channel.queuedeclare (QueueName, Durable=true, False, false, NULL);

You can specify a message persistence property when sending a message:

Channel.basicpublish (Exchangename, Routingkey,

Messageproperties.persistent_text_plain,

Message.getbytes ());

This way, the queues and messages are not lost even if the RABBITMQ server is restarted.

3. Publisher confirms

4. Master/slave mechanism, with mirrored Queue, in which case Publisher will normally send messages and receive messages confirm, but for subscriber, you need to receive consumer cancellation notifications to get notification of the failure of the master node and then re-consume the from the queue, which requires the client to have the ability to process duplicate messages. Note: If the queue adds a slave to a newly joined node, there is no information on the previous queue on slave (there is no synchronization mechanism at this time).

(from the command line or the management plug-in, you can see which slave are synchronized:

Rabbitmqctl list_queues name Slave_pids synchronised_slave_pids)

When a slave is re-added to the mirrored-queue, if the queue is durable, it will be emptied.

2.5 clusters (Cluster):

1. Cross-network segments are not supported (if required, shovel or federation plugins are required)

2. Can dynamically increase or decrease, start or stop nodes, allow node failure

3. The cluster is divided into RAM node and disk node, and a cluster should have at least one disk node to save the state of the cluster.

4. The configuration of the cluster can be through the command line, or through the configuration file, command line priority.

3 using 3.1 easy-to-use process
The use of 3.2 rabbitmq in OpenStack

In OpenStack, the use of rabbitmq between components is essentially a "Remote Procedure Calls" approach. Each Nova service (such as compute services, storage services, etc.) is initialized with two queues, one named "Node-type." Node-id ", another named" Node-type ", node-type refers to the type of service, Node-id refers to the node name.

At an abstraction level, the use of RABBITMQ components is similar to the following:

Each service binds two queues to exchange of the same topic type, receiving different types of messages from different queues. If the sender of the message cares about the return value of the message, it listens to another queue, which is bound to exchange in a direct type. When the recipient receives the message and processes it, the return of the message is sent to this exchange.

In OpenStack, if you do not care about message return, the flowchart of the message is as follows:

If you care about the message return value, the flowchart is as follows:

3.3 Why use RABBITMQ?

Once a person has done a test (http://www.cnblogs.com/amityat/archive/2011/08/31/2160293.html), send 1 million concurrent messages, the performance has a high demand, So the author compared the RABBITMQ, MSMQ, ActiveMQ, Zeromqueue, the whole process produced a total of 1 million 1K messages. The test was performed on a Windows Vista with the following test results:

Although ZEROMQ performance is high, but this product does not provide message persistence, need to implement audit and data recovery, so in the ease of use and HA is not satisfactory, through the test results can be seen, RABBITMQ performance is really good.

I have also done some tests on this machine, but my test is based on the native configuration of the component, without any configuration optimization, so the total sleep is not reliable. I only tested RABBITMQ and ACTIVEMQ two products, although the internet said ACTIVEMQ performance than the former, but in all fairness, ACTIVEMQ provides a lot of configuration, there is a large tuning space, perhaps modify a configuration parameter will make the performance of the component has a qualitative leap.

[Turn] RABBITMQ Introduction