Use Python to learn RabbitMQ message queues,

Use Python to learn RabbitMQ message queues,

RabbitMQ can be used as a message proxy. The core principle of RabbitMQ is very simple: receiving and sending messages. You can think of it as a post office: We put the mail into the mailbox, the postman will deliver the mail to your recipient. RabbitMQ is a combination of email, post office, and courier functions. The whole process is: email receiving, post office forwarding, the courier delivers the mail to the recipient.

The main difference between RabbitMQ and the post office is that RabbitMQ receives, stores, and sends binary data-a message.

Basic rabbitmq management commands:

Start the Erlang node and Rabbit applications in one step: sudo rabbitmq-server

Start Rabbit node: sudo rabbitmq-server-detached in the background

Disable the entire node (including applications): sudo rabbitmqctl stop

add_user <UserName> <Password>delete_user <UserName>change_password <UserName> <NewPassword>list_usersadd_vhost <VHostPath>delete_vhost <VHostPath>list_vhostsset_permissions [-p <VHostPath>] <UserName> <Regexp> <Regexp> <Regexp>clear_permissions [-p <VHostPath>] <UserName>list_permissions [-p <VHostPath>]list_user_permissions <UserName>list_queues [-p <VHostPath>] [<QueueInfoItem> ...]list_exchanges [-p <VHostPath>] [<ExchangeInfoItem> ...]list_bindings [-p <VHostPath>]list_connections [<ConnectionInfoItem> ...]

Demo:

Producer. py

#! /Usr/bin/env python #-*-coding: utf _-*-# Date: year, month, and day # Author: blue Line # blog http://www.cnblogs.com/duanv/ import pika import sys # create connection to localhost con = pika. blockingConnection (pika. connectionParameters ('localhost') # create a virtual connection channel cha = con. channel () # create a queue anheng. When the durable parameter is true, the queue will be persistent. When the exclusive parameter is true, the result of creating a temporary queue is cha. queue_declare (queue = 'anheng', durable = True, exclusive = False) # create an exchange named yanfa and of the fanout type. Other types include direct and topic. If durable is specified to be True, exchange will persist cha. exchange_declare (durable = False, exchange = 'yanfa ', type = 'direct',) # bind exchange and queue, result. method. the queue Name cha is obtained by queue. queue_bind (exchange = 'yanfa ', queue = result. method. queue, routing_key = '',) # fair distribution, so that each consumer can process at most one message at a time. No new message cha will be allocated before ack is received. basic_qos (prefetch_count =) # send messages to the queue 'anheng' message = ''. join (sys. argv [:]) # specify delivery_mode =; cha. basic_publish (exchange = '', routing_key = 'anheng', body = message, properties = pika. basicProperties (delivery_mode =,) print '[x] Sent % R' % (message,) # Close the connection con. close ()

Consumer. py

#! /Usr/bin/env python #-*-coding: utf _-*-# Date: year, month, and day # Author: blue Line # blog http://www.cnblogs.com/duanv/ import pika # establish connection to localhost con = pika. blockingConnection (pika. connectionParameters ('localhost') # create a virtual connection channel cha = con. channel () # create a queue anheng result = cha. queue_declare (queue = 'anheng', durable = True) # create a vswitch named yanfa with the fanout type. Other types include direct and topic cha. exchange_declare (durable = False, exchange = 'yanfa ', type = 'direct',) # bind exchange and queue, result. method. the queue Name cha is obtained by queue. queue_bind (exchange = 'yanfa ', queue = result. method. queue, routing_key = '',) # fair distribution, so that each consumer can process at most one message at a time. No new message cha will be allocated before ack is received. basic_qos (prefetch_count =) print '[*] Waiting for messages. to exit press CTRL + C' # define the callback function def callback (ch, method, properties, body): print "[x] Received % r" % (body,) ch. basic_ack (delivery_tag = method. delivery_tag) cha. basic_consume (callback, queue = 'anheng', no_ack = False,) cha. start_consuming ()

I. concept:

Connection: a TCP Connection. Both Producer and Consumer are connected to the RabbitMQ Server through TCP. The TCP connection is established at the beginning of the program.

Channels: virtual connection. Established in the preceding TCP connection. Data Flow is performed in the Channel. Generally, the program starts to establish a TCP connection, and the second step is to establish this Channel.

Ii. queue:

First, establish a Connection, then establish Channels, and create a queue on the channel.

If the durable parameter is set to true, the queue will be persistent. If the exclusive parameter is set to true, the queue is a temporary queue. After the consumer is disabled, the queue no longer exists. Generally, the queue name is not specified when a temporary queue is created, rabbitmq is named randomly through result. method. to get the queue Name:

Result = channel. queue_declare (exclusive = True)

Result. method. queue

Difference: durable indicates whether the queue is persistent or not. If it is true, the queue will still exist after the rabbitmq service is restarted. If it is false, the rabbitmq service will not disappear before it is restarted, and it has nothing to do with whether the consumer is disabled or not;

Exclusive creates a temporary queue. When the consumer is disabled, the queue will be deleted.

Iii. exchange and bind

The durable parameter in Exchange specifies whether exchange is persistent, the exchange parameter specifies the exchange name, And the type specifies the exchange type. Exchange types include direct, fanout, and topic.

Bind associates exchange with queue. The exchange and queue Parameters specify the exchange and queue for bind, and the routing_key parameter is optional.

Three Modes of Exchange:

Direct:

Any message sent to Direct Exchange will be forwarded to the specified Queue in routing_key.

1. Generally, you can use the Exchange: "(the Exchange name is an empty string) that comes with rabbitMQ );

2. In this mode, you do not need to bind Exchange;

3. A "routing_key" is required for message transmission, which can be simply understood as the name of the queue to be sent;

4. If the queue name specified in routing_key does not exist in the vhost, the message will be discarded.

Although a bind with exchange = 'yanfa 'and queue = 'anheng' is declared in the Demo, the exchange and bind are not used when a message is sent, instead of specifying exchange, the direct mode is used. Instead, the routing_key is specified as the queue name, and the message is sent to the specified queue.

If an exchange statement is direct and the routing_key is specified in the bind, you must specify the exchange and routing_key when sending the message.

Fanout:

Any message sent to Fanout Exchange will be forwarded to all Queue bound to the Exchange (Binding ).

1. It can be understood as the route table mode.

2. This mode does not require routing_key

3. In this mode, you need to bind Exchange and Queue in advance. One Exchange can bind multiple Queue and one Queue can be bound to multiple exchanges.

4. If the Exchange receiving the message is not bound to any Queue, the message will be discarded.

In the Demo, a bind of the fanout type is created for one exchange and one queue. but it is not used when sending information. If you want to use it, you only need to specify the name of the exchange when sending the message, the exchange will send the message to all Queues with its bind. In fanout mode, the specified routing_key is invalid.

Topic:

Any messages sent to Topic Exchange will be forwarded to all the Queue that cares about the specified Topic in routing_key.

1. this mode is complicated. In short, each queue has a topic of interest, and all messages have a "title" (routing_key ), exchange will forward messages to all Queues with a topic that can be fuzzy matched with routing_key.

2. This mode requires routing_key. You may need to bind Exchange and Queue in advance.

3. when binding, you must provide a topic that the queue cares about, such as "#. log. # "indicates that the queue cares about all messages involving logs (one routing_key is" MQ. log. the error message is forwarded to the queue ).

4. "#" indicates 0 or several keywords, and "*" indicates a keyword. For example, "log. *" can match "log. warn" and cannot match "log. warn. timeout", but "log. #" can match the two.

5. Similarly, if Exchange does not find a Queue that can match the routing_key, the message will be discarded.

Iv. Task Distribution

1. Rabbitmq tasks are cyclically distributed. If two consumers are enabled, the information sent by the producer is sent to two consume in turn.

2. use cha. basic_publish () to send messages. The body parameter is the message to be sent. properties = pika. basicProperties (delivery_mode = 2,) Enable message persistence to prevent data loss caused by RabbitMQ Server restart or crash.

3. use cha. basic_consume () infinite loop listener. If the no-ack parameter is set to true, the RabbitMQ Server immediately marks the Message as complete no matter whether the no-ack parameter is set to true, then deleted from the queue. To prevent data loss, RabbitMQ supports the message validation mechanism, namely, acknowledgments. To ensure that the data can be correctly processed and not only received by Consumer, we cannot use no-ack. Instead, an ack should be sent after the data is processed.

The ack sent after processing the data tells RabbitMQ that the data has been received and processed completely. RabbitMQ can safely delete the data. If the Consumer exits but does not send ack, RabbitMQ will send the Message to the next Consumer. This ensures that data will not be lost when the Consumer exits abnormally.

The timeout mechanism is not used here. RabbitMQ only confirms that the Message is not correctly handled through connection interruption of Consumer. That is to say, RabbitMQ gives Consumer enough time to process data.

In the Demo callback method, ch. basic_ack (delivery_tag = method. delivery_tag) tells rabbitmq that the message has been correctly processed. If this code is not provided, the Message will be resold when the Consumer exits. Then RabbitMQ will occupy more and more memory. Because RabbitMQ will run for a long time, this "Memory leakage" is fatal. To debug this error, run the following command to print un-acked Messages:

Sudo rabbitmqctl list_queues name messages_ready messages_unacknowledged

4. Fair distribution: Set cha. basic_qos (prefetch_count = 1). In this way, RabbitMQ enables each Consumer to process up to one Message at a time point. In other words, it will not distribute the new Message to the Consumer before receiving the ack of the Consumer.

5. Note:

Both producers and consumers should declare the establishment of a queue. In the online tutorial, if the parameter for the second creation is different from that for the first time, the operation is successful, but the attributes of the queue will not be modified.

Due to version issues, if the attributes of the created queue are not exactly the same as those of the first time in my test, an error similar to this 406 error will be reported, "PRECONDITION_FAILED-parameters for queue 'anheng' in vhost'/'not equivalent"

If the attributes are different for the Second Creation of exchange, this error 406 will be reported, "PRECONDITION_FAILED-cannot redeclare exchange 'yanfa 'in vhost'/' with different type, durable, internal or autodelete value"

If this error occurs when a queue is declared for the first time, it indicates that a queue with the same name already exists and some attributes of this Declaration are different from those previously declared. You can run the sudo rabbitmqctl list_queues command to check which queues are currently in use. The solution is to establish another queue or delete the original queue. If the original queue is not persistent, restart the rabbitmq service to delete the original queue. If the original queue is persistent, you can only delete the vhost where the vhost is located, re-create the vhost, and set the vhost permission (first confirm that there are no other useful queues in the vhost ).

sudo rabbitmqctl delete_vhost /sudo rabbitmqctl add_vhost /sudo rabbitmqctl set_permissions -p / username '.*' '.*' '.*'

The above content is a small Editor to introduce you to the use of Python to learn RabbitMQ message queue, I hope you like it.

Articles you may be interested in:

• Message Queue in php Memcache
• How to operate a Linux message queue in PHP to complete inter-process communication
• In-depth Message Queue explanation for inter-process communication
• Android development Tutorial: using loose to process message queues
• Python uses rabbitmq to implement web crawler examples
• Message Queue case sharing using PHP + memcache
• Details about how to copy a file using Python
• Python: pygame-based ball elasticity effect (source code)