Learning RABBITMQ Message Queuing with Python

RABBITMQ can be used as a message agent, its core principle is very simple: to receive and send messages, you can think of it as a post office: We put the letter into the mailbox, the postman will send the letter to your recipient, RABBITMQ is a mailbox, post office, couriers functional complex, The whole process is: The mailbox receives the letter, the Post Office forwards the letter, the couriers delivers the letter arrives at the addressee.

The main difference between RABBITMQ and the post office is that RABBITMQ receives, stores, and sends binary data----messages.

RABBITMQ Basic Management commands:

One-step activation of Erlang node and rabbit application: sudo rabbitmq-server

Start Rabbit Node:sudo rabbitmq-server-detached in the background

Close the entire node (including apps): sudo rabbitmqctl stop

Add_user
  
  
   Delete_user
   
    Change_password
     
     
      List_usersadd_vhost
      
       Delete_vhost
       
        
          list_vhostsset_permissions [-P 
         
          ] 
           
            
             
             
               clear_permissions [-P 
              
               ] 
               
                 list_permissions [-P 
                
                  ]list_user_permissions 
                 
                   list_queues [-P 
                  
                   ] [ 
                   
                     ...] list_exchanges [-P 
                    
                     ] [
                     
                       ...] list_bindings [-P 
                      
                       ]list_connections [
                       
                         ...] 
                        
                       
                      
                      
                   
                   
                  
                 
                
               /regexp> 
              
             
           
           
          
        
       
      
     
    
   
  
 

Demo:

producer.py

 #!/usr/bin/env python #-*-coding:utf_-*-# Date: Month Day # Author: Blue Line # blog Http://www.cnblogs.com/duanv/import pika impo RT sys #创建连接connection到localhost con = pika. Blockingconnection (Pika. Connectionparameters (' localhost ')) #创建虚拟连接channel cha = Con.channel () #创建队列anheng, the queue is persisted when the durable parameter is true, and when exclusive is true, Create a temporary queue Result=cha.queue_declare (queue= ' Anheng ', Durable=true,exclusive=false) #创建名为yanfa, exchange of type fanout, Other types are direct and topic, and if you specify durable as true, Exchange will persist Cha.exchange_declare (Durable=false, exchange= ' Yanfa ', Ty Pe= ' direct ',) #绑定exchange和queue, Result.method.queue gets the queue name Cha.queue_bind (exchange= ' Yanfa ', Queue=result.method. Queue, routing_key= ',) #公平分发 so that each consumer handles at most one message at a time, and no new message Cha.basic_qos (prefetch_count=) is assigned until the ACK is received #发 Send message to queue ' anheng ' message = '. Join (sys.argv[:]) #消息持久化指定delivery_mode =; Cha.basic_publish (exchange= ", Routing_key = ' Anheng ', Body=message, Properties=pika. Basicproperties (Delivery_mode = ,)) print ' [x] Sent%r '% (message,) #关闭连接 Con.close () 

consumer.py

 #!/usr/bin/env python #-*-coding:utf_-*-# Date: Month Day # Author: Blue Line # Blog http://www.cnblogs.co M/duanv/import Pika #建立连接connection到localhost con = pika. Blockingconnection (Pika. Connectionparameters (' localhost ')) #创建虚拟连接channel cha = Con.channel () #创建队列anheng result=cha.queue_declare (queue= ' Anheng ', durable=true) #创建名为yanfa, switches of type fanout, other types are direct and topic cha.exchange_declare (Durable=false, exchange= ' Yanfa ', type= ' direct ',) #绑定exchange和queue, Result.method.queue gets the queue name Cha.queue_bind (exchange= ' Yanfa ', q Ueue=result.method.queue, routing_key= ',) #公平分发 so that each consumer handles at most one message at a time, and no new message is assigned until the ACK is received Cha.basic_qos (prefetch_count=) print ' [*] waiting for messages. To exit Press CTRL + C ' #定义回调函数 def callback (ch, method, properties, body): print "[x] Received%r"% (body,) ch.basic_ ACK (Delivery_tag = Method.delivery_tag) Cha.basic_consume (callback, queue= ' Anheng ', No_ack=false,) cha.s Tart_consuming () 
 

First, the concept:

Connection: A TCP connection. Both producer and consumer are connected to RABBITMQ server through TCP. The starting point of the program is to establish this TCP connection.

Channels: Virtual connection. is established in the above TCP connection. The flow of data is carried out in the channel. The general scenario is that the program starts with a TCP connection, and the second step is to establish the channel.

Second, the queue:

First build a connection, then build channels, set up a queue on the channel

When established, specifies that the durable parameter is true, that the queue is persisted, that exclusive is true, that the queue is a temporary queue, that the queue will no longer exist after the consumer is closed, that the temporary queue is not specified in general, and that RABBITMQ will be randomly named. To get the queue name by Result.method.queue:

result = Channel.queue_declare (exclusive=true)

Result.method.queue

Difference: Durable is the queue persisted or not, if true, the queue will still exist after the RABBITMQ service restarts, if False, the RABBITMQ service will not disappear before restarting, and consumer shutdown is irrelevant;

The exclusive is a temporary queue, and when consumer is closed, the queue is deleted

Iii. Exchange and bind

The durable parameter in exchange Specifies whether Exchange is persisted, the Exchange parameter specifies Exchange name, and type specifies the exchange type. Exchange types are direct,fanout and topic.

Bind is associating exchange with a queue, and the Exchange and queue parameters specify that Exchange and Queue,routing_key are optional parameters for bind.

Three modes of exchange:

Direct:

Any messages sent to direct Exchange will be forwarded to the queue specified in Routing_key

1. In general, you can use RABBITMQ's own Exchange: "" (The Exchange name is an empty string);

2. This mode does not require any bind (BIND) operation of Exchange;

3. Message delivery requires a "Routing_key", can be easily understood as the name of the queue to be sent to;

4. If the queue name specified in the Routing_key does not exist in the Vhost, the message is discarded.

Although the demo declares a exchange= ' Yanfa ' and queue= ' Anheng ' Bind, but instead of using the exchange and bind at the end of the message, it takes the direct mode, does not specify Exchange, and instead specifies the name of the Routing_key as the queue name, and the message is sent to the specified queue.

If an exchange is declared as direct and Routing_key is specified in bind, it is necessary to indicate both exchange and Routing_key when sending the message.

Fanout:

Any messages sent to Fanout Exchange will be forwarded to all the queue with that Exchange binding (binding)

1. Patterns that can be understood as routing tables

2. This mode does not require Routing_key

3. This mode requires Exchange to be bound with the queue in advance, one exchange can bind multiple queues, and one queue can bind to multiple exchange.

4. If exchange that receives the message is not bound to any queue, the message is discarded.

The demo creates a bind that fanout a type of exchange and a queue. But it is not used when sending the message, and if it is to be used, the Exchange sends the message to all and its bind queues whenever it is sent with the name of the exchange specified. In fanout mode, the specified Routing_key is not valid.

Topic:

Any messages sent to topic Exchange will be forwarded to all queues that are concerned with the specified topic in Routing_key

1. This mode is more complex, simply put, is that each queue has its own topic of interest, all messages with a "title" (Routing_key), Exchange will forward the message to all the topics of interest can match the Routing_key fuzzy queue.

2. This mode requires Routing_key, perhaps to bind exchange and queue in advance.

3. When binding, to provide a topic that the queue cares about, such as "#.log.#" means that the queue is concerned with all messages involving log (a message Routing_key "MQ.log.error" is forwarded to the queue).

4. "#" means 0 or more keywords, and "*" denotes a keyword. such as "log.*" can Match "Log.warn", cannot match "log.warn.timeout", but "log.#" can match the above two.

5. Similarly, if Exchange does not find a queue that can match Routing_key, this message is discarded.

Iv. Distribution of tasks

The task of 1.RABBITMQ is to circulate, if the information sent by two Consumer,producer is turned on, it is sent to two consume in turn.

2. Use Cha.basic_publish () to send a message on the producer side, where the body parameter is the message to be sent, Properties=pika. Basicproperties (Delivery_mode = 2,) enables message persistence to prevent RABBITMQ Server from restarting or crash data loss.

3. The receiver uses cha.basic_consume () infinite loop monitoring, if the set no-ack parameter is true, each time consumer receives the data, regardless of whether the processing is completed, RabbitMQ server immediately marks the message as complete. It is then removed from the queue. To ensure that data is not lost, RABBITMQ supports the message acknowledgement mechanism, which is acknowledgments. In order to ensure that the data can be handled correctly and not only received by consumer, we cannot use No-ack. Instead, send an ACK after processing the data.

After processing the data sent by the ACK, is to tell RABBITMQ data has been received, processing completed, RABBITMQ can go to safely delete it. If consumer exits but does not send an ACK, then RABBITMQ sends the message to the next consumer. This guarantees that the data will not be lost in the event of an abnormal exit of the consumer.

The timeout mechanism is not used here. RABBITMQ only confirms that the message was not handled correctly by consumer connection interruption. In other words, RABBITMQ gave consumer enough time to do the data processing.

Ch.basic_ack (Delivery_tag = Method.delivery_tag) in the callback method of the demo tells the RABBITMQ that the message has been handled correctly. If this code is not available, the message is redistributed when consumer exits. The RABBITMQ then consumes more and more memory, and because RABBITMQ is running for a long time, this "memory leak" is fatal. To debug this error, you can print un-acked Messages by command:

sudo rabbitmqctl list_queues name Messages_ready messages_unacknowledged

4. Fair distribution: Set Cha.basic_qos (prefetch_count=1) so that RABBITMQ will make each consumer at most one message at the same point in time. In other words, before receiving the ACK of the consumer, he will not distribute the new message to it.

Five, note:

Both the producer and the consumer should declare a queue, and the online tutorial says the second creation if the parameter is not the same as the first time, the operation succeeds, but the queue's properties are not modified.

Probably because of the version issue, in my test if the second declaration establishes a queue property that is not exactly the same as the first time, it will be reported similar to this error 406, "Precondition_failed-parameters for queue ' Anheng ' in vhost '/' not Equivalent "

If it is different from the second time that the exchange is created, the error 406 will be reported, "Precondition_failed-cannot redeclare exchange ' Yanfa ' in Vhost '/' with different type, Durable, internal or autodelete value "

If this error occurs when the first declaration is made, stating that a queue with the same name exists before and some properties of this declaration are different from the previous declaration, you can see which queues are currently available by command sudo rabbitmqctl list_queues. The workaround is to declare a queue or delete the original queue, if the original queue is non-persistent, you can delete the original queue by restarting the RABBITMQ service, if the original queue is persisted, you can only delete the vhost it is in, and then rebuild the vhost. Set the Vhost permissions again (verify that there are no other useful queues in the vhost).

sudo rabbitmqctl delete_vhost/sudo rabbitmqctl add_vhost/sudo rabbitmqctl set_permissions-p/username '. * '. * '.

The above content is small to introduce you to the use of Python learning rabbitmq message queue, I hope you like.

