Python Operation Rabbitmq

Introduced

RABBITMQ is a complete, reusable enterprise messaging system based on AMQP. He follows the Mozilla Public License open source agreement.

MQ is all called the message queue, and Message Queuing (MQ) is an application-to-application communication method. Applications communicate by reading and writing messages to and from the queue (data for the application), without requiring a dedicated connection to link them. Message passing refers to the process of communicating between programs by sending data in a message, rather than by directly invoking each other, and directly invoking techniques such as remote procedure calls. Queuing refers to an application communicating through a queue. The use of queues removes the requirement that both the receiving and sending applications execute concurrently.

installation

Rabbitmq:http://www.rabbitmq.com/install-standalone-mac.html

Installation configuration Epel source   $ rpm-ivh http://dl.fedoraproject.org/pub/epel/6/i386/epel-release-6-8.noarch.rpm install Erlang   $ yum -Y install Erlang installation rabbitmq   $ yum-y installs Rabbitmq-server

Use

Official Use document: http://www.rabbitmq.com/getstarted.html

1. Achieve the simplest queue communication

#!/usr/bin/env python# A simple send-data send-side, one-mode (producer) Import pikaconnection = Pika. Blockingconnection (Pika. Connectionparameters (host= ' localhost ')) #初始化连接, connected to Rabbitmqchannel = Connection.channel () #生成一个连接管道channel. Queue_ Declare (queue= ' hello ') #声明一个队列名
#生产消息channel. Basic_publish (exchange= ",                      routing_key= ' Hello ', #要发送到那个队列中                      body= ' Hello world! ' #发送的内容  ) Print ("[x] Sent ' Hello world! '") Connection.close () #关闭RabbitMQ的连接

#!/usr/bin/env python# A simple recvive-side, one-mode (consumer) Import pikaconnection = Pika. Blockingconnection (Pika. Connectionparameters (host= ' localhost ')) #初始化连接, connected to Rabbitmqchannel = Connection.channel () #生成一个连接管道channel. Queue_ Declare (queue= ' hello ') #这里也要声明一下队列的名称, if the client starts, the server does not start, no statement will error DEF callback (ch, method, properties, body): # Defines a callback function after receiving the message    print ("[x] Received%r"% body) #消费消息channel. Basic_consume (Callback, #当接收到一个消息之后回调CALLBACk函数处理                      queue= ' Hello ', #定义那个队列中去取, message                      no_ack=true# This means that it does not return confirmation, the message processing is completed after the confirmation is returned, RABBITMQ removed from the queue, generally not used  ) print (' [*] Waiting for messages. To exit Press CTRL + C ') channel.start_consuming () #启动连接, if no data is received it will wait, will not stop

2. Work Queues

In this mode, RABBITMQ will distribute P-sent messages to individual consumers (c) By default, similar to load balancing

RABBITMQ Persistence and fair distribution

1) acknowledgment message does not lose the method

Effective methods: Channel.basic_consume (Consumer_callback, queue, No_ack=false, Exclusive=false, Consumer_tag=none, arguments= None)
That is, No_ack=false (the default is False, which means you must have a confirmation ID), in the callback function Consumer_callback, the acknowledgement ID is not received, and RABBITMQ will re-add the task to the queue. (Consumer side)

2) Persistence of messages

Although there is a message feedback mechanism, if RABBITMQ itself hangs, then the task will still be lost. So you need to persist the task to store it. Declaring persistent storage

Channel.queue_declare (queue= ' Hello ', durable=true) # declares queue persistence

Ps: However, the program executes an error because the ' Hello ' queue already exists and is non-persistent, andRABBITMQ does not allow the use of different parameters to redefine the existing queue . So you need to redefine a queue

Channel.queue_declare (queue= ' test_queue ', durable=true) # declaring queue persistence

Note : If only the queue persistence is set, only the queue itself can remain after the rabbit-server is down, the information in the queue is still lost, and if you want to keep the information or tasks in the queue, you need to do the following:

Channel.basic_publish (exchange= ",                      routing_key=" Test_queue ",                      body=message,                      Properties=pika. Basicproperties (                         Delivery_mode = 2, # to persist the message or task)                      )  Message Queuing persistence consists of 3 parts:    (1) Exchange persistence, specifying durable when declaring = > 1    (2) queue persistence, specifying durable = + 1    (3) Message persistence on declaration, specifying delivery_mode=> 2 (1 non-persistent)    on delivery If both Exchange and queue are persisted, then the binding between them is persistent. If there is a persistence between Exchange and queue, a non-persisted, binding is not allowed.

3) Fair distribution of messages

If rabbit in order to send the message to the individual consumers, regardless of consumer load, it is likely to occur, a machine is not high-profile consumers piled up a lot of message processing, while the high-profile consumers have been very easy. To solve this problem, you can configure perfetch=1 on each consumer side, which means telling rabbitmq not to give it to me when the current consumer message is not finished.

Channel.basic_qos (prefetch_count=1) means who is to fetch, no longer in odd even order

Full code with message Persistence + fair distribution

Producer side:

#!/usr/bin/env pythonimport pikaimport SYS connection = Pika. Blockingconnection (Pika. Connectionparameters (        host= ' localhost ')) channel = Connection.channel () channel.queue_declare (queue= ' task_ Queue ', durable=true) message = '. Join (sys.argv[1:]) or "Hello world!" Channel.basic_publish (exchange= ',                      routing_key= ' Task_queue ',                      body=message,                      Properties=pika. Basicproperties (                         Delivery_mode = 2, # make message persistent                      )) print ("[x] Sent%r"% message) Connection.close ()

Consumer side:

#!/usr/bin/env pythonimport pikaimport Time connection = Pika. Blockingconnection (Pika. Connectionparameters (        host= ' localhost ')) channel = Connection.channel () channel.queue_declare (queue= ' task_ Queue ', durable=true) print (' [*] waiting for messages. To exit Press CTRL + C ') def callback (ch, method, properties, body):    print ("[x] Received%r"% body)    Time.sleep (bod Y.count (b '. ')) #模拟处理操作    Print ("[x] Done")    ch.basic_ack (Delivery_tag = Method.delivery_tag) Channel.basic_qos (Prefetch_ count=1) #消息公平分发, configure the perfetch=1, meaning to tell rabbitmq in my current consumer news has not finished processing time don't send me new news Channel.basic_consume (callback,                      Queue= ' Task_queue ') channel.start_consuming ()

3. Publish a Subscription

The difference between a publish subscription and a simple message queue is that a publish subscription sends a message to all subscribers, and the data in the message queue disappears once it is consumed. Therefore, when RABBITMQ implements the publication and subscription, a queue is created for each subscriber, and when the publisher publishes the message, the message is placed in all related queues.

Exchange_type = Fanout

Published by:

#!/usr/bin/env pythonimport pikaimport sysconnection = Pika. Blockingconnection (Pika. Connectionparameters (        host= ' localhost ')) channel = Connection.channel () channel.exchange_declare (exchange= ' Logs ',                         exchange_type= ' fanout ') message = '. Join (sys.argv[1:]) or "Info:hello world!" Channel.basic_publish (exchange= ' logs ',                      routing_key= ',                      body=message) print ("[x] Sent%r"% message) Connection.close ()

Subscribers:

Import pikaconnection = Pika. Blockingconnection (Pika. Connectionparameters (host= ' localhost ')) channel = Connection.channel () channel.exchange_declare (exchange= ' logs ',                         exchange_type= ' fanout ') result = Channel.queue_declare (exclusive = True)    #不指定queue名字, rabbit randomly assigns a name, Exclusive=true will automatically delete the queue after the consumer using this queue is disconnected queue_name = result.method.queuechannel.queue_bind (Exchange = ' logs ',       #绑定                   queue = queue_name) print (' [*] waiting for logs. To exit Press CTRL + C ') print (' Randome queuename: ', queue_name) def callback (ch, method, properties, body):    print ("[x] %r "% body" channel.basic_consume (callback,                      queue = queue_name,                      no_ack = True                      ) channel.start_consuming ()

4. Selective acceptance message (keyword send)

RABBITMQ also supports the sending of keywords according to the keyword, that is: the queue binding keyword, the sender sends the data according to the keyword to the message exchange,exchange according to the keyword decision should send the data to the specified queue.

Exchange_type = Direct

Publisher

Import Pikaimport sysconnection = Pika. Blockingconnection (Pika. Connectionparameters (                                                    host = ' localhost ')) channel = Connection.channel () channel.exchange_declare (Exchange = ' Direct_logs ',                         exchange_type = ' direct ') severity = sys.argv[1] If len (SYS.ARGV) > 1 Else ' info ' message = '. Join (SY S.argv[2:]) or ' Hello world! ' Channel.basic_publish (Exchange = ' direct_logs ',                      routing_key = severity,                      BODY = message) print ("[x] Sent%r:%r"% (severity, message)) Connection.close ()

subscriber:

import pikaimport sysconnection = Pika. Blockingconnection (Pika. Connectionparameters (host = ' localhost ')) channel = Connection.chan Nel () channel.exchange_declare (Exchange = ' direct_logs ', Exchange_type = ' direct ') result = Channel. Queue_declare (exclusive = True) queue_name = Result.method.queueseverities = Sys.argv[1:]if not SEVERITIES:SYS.STDERR.W Rite ("Usage:%s [INFO] [WARNING] [error]\n"% sys.argv[0]) sys.exit (1) for severity in Severities:channel.queue_bind (Exchange = ' direct_logs ', queue = queue_name, Routing_key = severity) print (' [*] Waiting for logs. To exit Press CTRL + C ') def callback (ch, method, properties, body): Print ("[x]%r:%r"% (Method.routing_key, body)) Chann El.basic_consume (callback, queue = queue_name, No_ack = True) Channel.start_consu Ming () 

5. More granular message filtering (fuzzy matching)

Exchange_type = Topic

Under the topic type, you can have the queue bind several fuzzy keywords, and the sender sends the data to Exchange,exchange to match the incoming route value and keyword, and the match succeeds, sending the data to the specified queue.

• # indicates that 0 or more words can be matched
• * Indicates that only one word can be matched

              error.info.warning    Error in the sender route value queue. *    //mismatch error.info.warning    error. #     //Match

Consumers:

import pikaimport sysconnection = Pika. Blockingconnection (Pika. Connectionparameters (host = ' localhost ')) channel = Connection.chan Nel () channel.exchange_declare (Exchange = ' topic_logs ', Exchange_type = ' topic ') result = Channel.qu Eue_declare (exclusive = True) queue_name = Result.method.queuebinding_keys = Sys.argv[1:]if not Binding_keys:sys.stderr . Write ("Usage:%s [binding_key]...\n"% sys.argv[0]) sys.exit (1) for Binding_key in Binding_keys:channel.queue_bind (  Exchange = ' topic_logs ', queue = queue_name, Routing_key = binding_key) print (' [*] Waiting for logs. To exit Press CTRL + C ') def callback (ch, method, properties, body): Print ("[x]%r:%r"% (Method.routing_key, body)) Chann El.basic_consume (callback, queue = queue_name, No_ack = True) Channel.start_consu Ming () 

Producers:

Import Pikaimport sysconnection = Pika. Blockingconnection (Pika. Connectionparameters (                                                    host = ' localhost ')) channel = Connection.channel () channel.exchange_declare (Exchange = '  Topic_logs ',                         exchange_type = ' topic ') Routing_key = sys.argv[1] If len (SYS.ARGV) > 1 Else ' anonymous.info ' message = ". Join (sys.argv[2:]) or ' Hello world! ' Channel.basic_publish (Exchange = ' topic_logs ',                      routing_key = Routing_key,                      BODY = message) print ("[x] Sent%r:%r "% (routing_key, message)) Connection.close ()

Other

#Add an accountsudo rabbitmqctl add_user musker 123. com#set user to Administrator rolesudo rabbitmqctl set_user_tags musker Administrator#Set Permissionssudo rabbitmqctl set_permissions-p"/"Musker'."'."'.'#then restart the RABBIMQ servicesudo/etc/init.d/rabbitmq-Server Restart#You can use a user to connect remotely to RABBITMQ server. Credentials= Pika. Plaincredentials ("Musker","123.com") Connection= Pika. Blockingconnection (Pika. Connectionparameters ('192.168.14.47', credentials=credentials))#View the number of queues and messagesRabbitmqctl list_queues

View Code

Python Operation Rabbitmq