Use Python to learn about RabbitMQ and pythonrabbitmq

Use Python to learn about RabbitMQ and pythonrabbitmq

RabbitmqBasic management commands:

Start the Erlang node and Rabbit applications in one step: sudo rabbitmq-server

Start Rabbit node: sudo rabbitmq-server-detached in the background

Disable the entire node (including applications): sudo rabbitmqctl stop

Add_user <UserName> <Password>
Delete_user <UserName>
Change_password <UserName> <NewPassword>
List_users
Add_vhost <VHostPath>
Delete_vhost <VHostPath>
List_vhosts
Set_permissions [-p <VHostPath>] <UserName> <Regexp>
Clear_permissions [-p <VHostPath>] <UserName>
List_permissions [-p <VHostPath>]
List_user_permissions <UserName>
List_queues [-p <VHostPath>] [<QueueInfoItem>...]
List_exchanges [-p <VHostPath>] [<ExchangeInfoItem>...]
List_bindings [-p <VHostPath>]
List_connections [<ConnectionInfoItem>...]

 

 

Demo:

Producer. py

1 #! /Usr/bin/env python 2 #-*-coding: utf_8-*-3 # Date: July 4 # Author: blue Line 5 # blog http://www.cnblogs.com/duanv/ 6 7 import pika 8 import sys 9 10 # create connection to localhost11 con = pika. blockingConnection (pika. connectionParameters ('localhost') 12 # create a virtual connection channel13 cha = con. channel () 14 # create a queue anheng. When the durable parameter is true, the queue will be persistent; when the exclusive parameter is true, a temporary queue will be created 15 result = cha. queue_declare (queue = 'anheng', durable = True, exclusive = False) 16 # create an exchange with the name yanfa and the type fanout. Other types include direct and topic. If durable is specified to be True, exchange will persist 17 cha. exchange_declare (durable = False, 18 exchange = 'yanfa ', 19 type = 'direct',) 20 # bind exchange and queue, result. method. queue obtains the queue name 21 cha. queue_bind (exchange = 'yanfa ', 22 queue = result. method. queue, 23 routing_key = '',) 24 # fair distribution, so that each consumer can process at most one message at a time. No new message25 cha will be allocated before ack is received. basic_qos (prefetch_count = 1) 26 # send messages to the queue 'anheng' 27 message = ''. join (sys. argv [1:]) 28 # specify delivery_mode = 2; 29 cha for message persistence. basic_publish (exchange = '', 30 routing_key = 'anheng', 31 body = message, 32 properties = pika. basicProperties (33 delivery_mode =) 35 print '[x] Sent % R' % (message,) 36 # Close connection 37 con. close ()

 

Consumer. py

1 #! /Usr/bin/env python 2 #-*-coding: utf_8-*-3 # Date: July 4 # Author: blue Line 5 # blog http://www.cnblogs.com/duanv/ 6 import pika 7 8 # establish a connection to localhost 9 con = pika. blockingConnection (pika. connectionParameters ('localhost') 10 # create a virtual connection channel11 cha = con. channel () 12 # create a queue anheng13 result = cha. queue_declare (queue = 'anheng', durable = True) 14 # create a vswitch named yanfa with the fanout type. Other types include direct and topic15 cha. exchange_declare (durable = False, 16 exchange = 'yanfa ', 17 type = 'direct',) 18 # bind exchange and queue, result. method. the queue name obtained by queue is 19 cha. queue_bind (exchange = 'yanfa ', 20 queue = result. method. queue, 21 routing_key = '',) 22 # fair distribution, so that each consumer can process at most one message at a time. No new message23 cha will be allocated before ack is received. basic_qos (prefetch_count = 1) 24 print '[*] Waiting for messages. to exit press CTRL + C' 25 # define the callback function 26 def callback (ch, method, properties, body): 27 print "[x] Received % r" % (body ,) 28 ch. basic_ack (delivery_tag = method. delivery_tag) 29 30 cha. basic_consume (callback, 31 queue = 'anheng', 32 no_ack = False,) 33 34 cha. start_consuming ()

 

I. concept:

Connection: A TCP connection. Both Producer and Consumer are connected to the RabbitMQ Server through TCP. The TCP connection is established at the beginning of the program.

Channels: Virtual Connection. Established in the preceding TCP connection. Data Flow is performed in the Channel. Generally, the program starts to establish a TCP connection, and the second step is to establish this Channel.

 

Ii. queue:

First, establish a Connection, then establish Channels, and create a queue on the channel.

If the durable parameter is set to true, the queue will be persistent. If the exclusive parameter is set to true, the queue is a temporary queue. After the consumer is disabled, the queue no longer exists. Generally, the queue name is not specified when a temporary queue is created, rabbitmq is named randomly through result. method. to get the queue Name:

Result = channel. queue_declare (exclusive = True)

Result. method. queue

Differences: Durable indicates whether the queue is persistent or not. If it is true, the queue will still exist after the rabbitmq service is restarted. If it is false, the rabbitmq service will not disappear before it is restarted, and it has nothing to do with whether the consumer is disabled or not;

Exclusive creates a temporary queue. When the consumer is disabled, the queue will be deleted.

 

Iii. exchange and bind

The durable parameter in Exchange specifies whether exchange is persistent, the exchange parameter specifies the exchange name, And the type specifies the exchange type. Exchange types include direct, fanout, and topic.

Bind associates exchange with queue. The exchange and queue Parameters specify the exchange and queue for bind, and the routing_key parameter is optional.

Three Modes of Exchange:

Direct:

Any sentDirect ExchangeWill be forwardedRouting_keySpecified inQueue

1. Generally, you can use the Exchange: "(the Exchange name is an empty string) that comes with rabbitMQ );

2. In this mode, you do not need to bind Exchange;

3. A "routing_key" is required for message transmission, which can be simply understood as the name of the queue to be sent;

4. If the queue name specified in routing_key does not exist in the vhost, the message will be discarded.

Although a bind with exchange = 'yanfa 'and queue = 'anheng' is declared in the Demo, the exchange and bind are not used when a message is sent, instead of specifying exchange, the direct mode is used. Instead, the routing_key is specified as the queue name, and the message is sent to the specified queue.

If an exchange statement is direct and the routing_key is specified in the bind, you must specify the exchange and routing_key when sending the message.

Fanout:

Any sentFanout ExchangeWill be forwarded toExchangeBind(Binding)AllQueueUpper

1. It can be understood as the route table mode.

2. This mode does not require routing_key

3. In this mode, you need to bind Exchange and Queue in advance. One Exchange can bind multiple Queue and one Queue can be bound to multiple exchanges.

4. If the Exchange receiving the message is not bound to any Queue, the message will be discarded.

In the Demo, a bind of the fanout type is created for one exchange and one queue. but it is not used when sending information. If you want to use it, you only need to specify the name of the exchange when sending the message, the exchange will send the message to all Queues with its bind. In fanout mode, the specified routing_key is invalid.

Topic:

Any sentTopic ExchangeWill be forwarded to all concernedRouting_keySpecified topicQueueUpper

1. this mode is complicated. In short, each queue has a topic of interest, and all messages have a "title" (routing_key ), exchange will forward messages to all Queues with a topic that can be fuzzy matched with routing_key.

2. This mode requires routing_key. You may need to bind Exchange and Queue in advance.

3. when binding, you must provide a topic that the queue cares about, such as "#. log. # "indicates that the queue cares about all messages involving logs (one routing_key is" MQ. log. the error message is forwarded to the queue ).

4. "#" indicates 0 or several keywords, and "*" indicates a keyword. For example, "log. *" can match "log. warn" and cannot match "log. warn. timeout", but "log. #" can match the two.

5. Similarly, if Exchange does not find a Queue that can match the routing_key, the message will be discarded.

 

Iv. Task Distribution

1. Rabbitmq tasks are cyclically distributed. If two consumers are enabled, the information sent by the producer is sent to two consume in turn.

2. use cha. basic_publish () to send messages. The body parameter is the message to be sent. properties = pika. basicProperties (delivery_mode = 2,) Enable message persistence to prevent data loss caused by RabbitMQ Server restart or crash.

3. use cha. basic_consume () infinite loop listener. If the no-ack parameter is set to true, the RabbitMQ Server immediately marks the Message as complete no matter whether the no-ack parameter is set to true, then deleted from the queue. To ensure that data is not lost, RabbitMQ supports message confirmation, that isAcknowledgments. To ensure that the data can be correctly processed and not only received by Consumer, we cannot use no-ack. Instead, an ack should be sent after the data is processed.

The ack sent after processing the data tells RabbitMQ that the data has been received and processed completely. RabbitMQ can safely delete the data. If the Consumer exits but does not send ack, RabbitMQ will send the Message to the next Consumer. This ensures that data will not be lost when the Consumer exits abnormally.

The timeout mechanism is not used here. RabbitMQ only confirms that the Message is not correctly handled through connection interruption of Consumer. That is to say, RabbitMQ gives Consumer enough time to process data.

In the Demo callback method, ch. basic_ack (delivery_tag = method. delivery_tag) tells rabbitmq that the message has been correctly processed. If this code is not provided, the Message will be resold when the Consumer exits. Then RabbitMQ will occupy more and more memory. Because RabbitMQ will run for a long time, this "Memory leakage" is fatal. To debug this error, run the following command to print un-acked Messages:

Sudo rabbitmqctl list_queues name messages_ready messages_unacknowledged

4. Fair distribution: Set cha. basic_qos (prefetch_count = 1). In this way, RabbitMQ enables each Consumer to process up to one Message at a time point. In other words, it will not distribute the new Message to the Consumer before receiving the ack of the Consumer.

 

5. Note:

Both producers and consumers should declare the establishment of a queue. In the online tutorial, if the parameter for the second creation is different from that for the first time, the operation is successful, but the attributes of the queue will not be modified.

Due to version issues, if the attributes of the created queue are not exactly the same as those of the first time in my test, an error similar to this 406 error will be reported, "PRECONDITION_FAILED-parameters for queue 'anheng' in vhost'/'not equivalent"

If the attributes are different for the Second Creation of exchange, this error 406 will be reported, "PRECONDITION_FAILED-cannot redeclare exchange 'yanfa 'in vhost'/' with different type, durable, internal or autodelete value"

If this error occurs when a queue is declared for the first time, it indicates that a queue with the same name already exists and some attributes of this Declaration are different from those previously declared. You can run the sudo rabbitmqctl list_queues command to check which queues are currently in use. The solution is to establish another queue or delete the original queue. If the original queue is not persistent, restart the rabbitmq service to delete the original queue. If the original queue is persistent, you can only delete the vhost where it is located, re-create the vhost, and then set the vhost permission (ConfirmVhostNo other useful queues in).

Sudo rabbitmqctl delete_vhost/
Sudo rabbitmqctl add_vhost/
Sudo rabbitmqctl set_permissions-p/username '.*''.*''.*'