Python-Operation RABBITMQ

Introduced

RABBITMQ is a complete, reusable enterprise messaging system based on AMQP. He follows the Mozilla Public License open source agreement.
MQ is all called the message queue, and Message Queuing (MQ) is an application-to-application communication method. Applications communicate by reading and writing messages to and from the queue (data for the application), without requiring a dedicated connection to link them. Message passing refers to the process of communicating between programs by sending data in a message, rather than by directly invoking each other, and directly invoking techniques such as remote procedure calls. Queuing refers to an application communicating through a queue. The use of queues removes the requirement that both the receiving and sending applications execute concurrently.

Application Scenarios:

RABBITMQ is undoubtedly one of the most popular message queues in the world, and support for various locales is also plentiful, and it is necessary to learn and understand this tool as a. NET developer. There are about 3 usage scenarios for Message Queuing:

1, system integration, the design of distributed systems. Various subsystems are connected via messages, and this solution has evolved into an architectural style, the "architecture through messaging".

2, when the synchronization process in the system seriously affect the throughput, such as logging. If you need to record all the user behavior logs in the system, if the log is bound to affect the response speed of the system, when we send the log message to the message queue, the logging subsystem will consume the log message asynchronously.

3, the high availability of the system, such as the e-commerce of the second kill scene. A system outage occurs when the application server or database server receives a large number of requests at a time. If the ability to forward requests to Message Queuing, and then the server to consume these messages will make the request smooth, improve system availability.

Installation and Installation RABBITMQ

Basic Environment: Kernel 3.10.0-327.el7.x86_64 system version CentOS Linux release 7.2.1511 (CORE) installation configuration Epel Source # RPM-IVH http://mirrors.neusoft.edu.cn /EPEL/7/X86_64/E/EPEL-RELEASE-7-7.NOARCH.RPM installation erlang# yum install Erlang download RABBITMQ 3.6.1# wget/  http www.rabbitmq.com/releases/rabbitmq-server/v3.6.1/rabbitmq-server-3.6.1-1.noarch.rpm Installing rabbitmq-server# RPM-IVH  rabbitmq-server-3.6.1-1.noarch.rpm build config file # Cp/usr/share/doc/rabbitmq-server-3.6.1/rabbitmq.config.example/ Etc/rabbitmq/rabbitmq.config starting rabbitmq# rabbitmq-server start

Installing the Python API

# PIP3 Install pikaor# Easy_install pika

Python Operation RABBITMQ

For RABBITMQ, production and consumption no longer target a queue object in memory, but rather a message queue implemented by RABBITMQ server on a single server.

1. Producer Code

#!/usr/bin/env python#-*-coding:utf-8-*-# auth:pangguopingimport pika# ######################### producer ############### ######### #credentials = Pika. Plaincredentials (' admin ', ' admin ') #链接rabbit服务器 (localhost is native, if it is another server please modify the IP address) connection = Pika. Blockingconnection (Pika. Connectionparameters (' 192.168.1.103 ', 5672, '/', credentials)) #创建频道channel = Connection.channel () # declares a message queue, The message will be passed in this queue. If a message is sent to a queue that does not exist, RABBITMQ will automatically clear the messages. If the queue does not exist, create a channel.queue_declare (queue= ' hello ') #exchange-it allows us to specify exactly which queue the message should go to. #向队列插入数值 Routing_key is the queue name body is the content to be inserted channel.basic_publish (exchange= ",                  routing_key= ' Hello ',                  body= ' Hello world! ') Print ("Start queue") #缓冲区已经flush而且消息已经确认发送到了RabbitMQ中, close link connection.close ()

2. Consumer Code

#!/usr/bin/env python#-*-coding:utf-8-*-# auth:pangguopingimport pika# ########################## Consumer ############## ########### #credentials = Pika. Plaincredentials (' admin ', ' admin ') # connected to RABBITMQ server connection = Pika. Blockingconnection (Pika. Connectionparameters (' 192.168.1.103 ', 5672, '/', credentials)) channel = Connection.channel () # declares Message Queuing, and messages are passed in this queue. If the queue does not exist, create channel.queue_declare (queue= ' Wzg ') # to define a callback function to handle, the callback function here is to print out the information. DEF callback (ch, method, properties, body):    print ("[x] Received%r"% body) # tell RABBITMQ to use callback to receive information Channel.basic _consume (callback,                      queue= ' Hello ',                      no_ack=true) # No_ack=true indicates that there is no need to send a confirmation print in the callback function (' [*] waiting for Messages. To exit Press CTRL + C ') # starts receiving information and enters the blocking state, and the message in the queue is called callback for processing. Press CTRL + C to exit. Channel.start_consuming ()

RABBITMQ Persistence 1, acknowledgment message does not lose the method

Effective methods: Channel.basic_consume (Consumer_callback, queue, No_ack=false, Exclusive=false, Consumer_tag=none, arguments= None)

That is, No_ack=false (the default is False, which means you must have a confirmation ID), in the callback function Consumer_callback, the acknowledgement ID is not received, and RABBITMQ will re-add the task to the queue.

Producer Code

#!/usr/bin/env python#-*-coding:utf-8-*-# auth:pangguopingimport pika# ######################### producer ############### ######### #credentials = Pika. Plaincredentials (' admin ', ' admin ') #链接rabbit服务器 (localhost is native, if it is another server please modify the IP address) connection = Pika. Blockingconnection (Pika. Connectionparameters (' 192.168.1.103 ', 5672, '/', credentials)) #创建频道channel = Connection.channel () # declares a message queue, The message will be passed in this queue. If a message is sent to a queue that does not exist, RABBITMQ will automatically clear the messages. If the queue does not exist, create a channel.queue_declare (queue= ' hello ') #exchange-it allows us to specify exactly which queue the message should go to. #向队列插入数值 Routing_key is the queue name body is the content to be inserted channel.basic_publish (exchange= ",                  routing_key= ' Hello ',                  body= ' Hello world! ') Print ("Start queue") #缓冲区已经flush而且消息已经确认发送到了RabbitMQ中, close link connection.close ()

Consumer Code:

Import pikacredentials = Pika. Plaincredentials (' admin ', ' admin ') # link rabbitconnection = pika. Blockingconnection (Pika. Connectionparameters (' 192.168.1.103 ', 5672, '/', credentials) # Create channels channel = Connection.channel () # If the producer does not run the Create queue, Then the consumer creates the queue Channel.queue_declare (queue= ' Hello ') def callback (ch, method, properties, body):    print ("[x] Received%r" % body)    import time    time.sleep (Ten)    print    ' OK '    ch.basic_ack (Delivery_tag=method.delivery_tag)  # Mainly use this code Channel.basic_consume (callback,                      queue= ' Hello ',                      no_ack=false) print (' [*] waiting for messages. To exit Press CTRL + C ') channel.start_consuming ()

2. Message Persistence Store (msg durability)

Although there is a message feedback mechanism, if RABBITMQ itself hangs, then the task will still be lost. So you need to persist the task to store it. Declaring persistent storage

Channel.queue_declare (queue= ' Wzg ', durable=true) # declaring queue persistence

Ps: But this program executes the error because the ' WZG ' queue already exists and is non-persistent, andRABBITMQ does not allow the use of different parameters to redefine the existing queue . So you need to redefine a queue

    Channel.queue_declare (queue= ' test_queue ', durable=true) # declaring queue persistence

Note : If only the queue persistence is set, only the queue itself can remain after the rabbit-server is down, the information in the queue is still lost, and if you want to keep the information or tasks in the queue, you need to do the following:

Channel.basic_publish (exchange= ",                      routing_key=" Test_queue ",                      body=message,                      Properties=pika. Basicproperties (                         Delivery_mode = 2, # makes the message or task persist                      ))

    Message Queuing persistence consists of 3 parts:    (1) Exchange persistence, specifying durable + 1    (2) Queue persistence on declaration, specifying durable + 1    (3) Message persistence at declaration time, Specify Delivery_mode=> 2 (1 non-persistent) on delivery    if both Exchange and queue are persisted, the binding between them is persistent. If there is a persistence between Exchange and queue, a non-persisted, binding is not allowed.

Publish and subscribe

The release and subscription of RABBITMQ is implemented with the help of switches (Exchange).

How the Switch works: The message sender sends the message to the switch, the switch sends the message to the bound message queue, and each receiving end (consumer) receives the message from its own message queue.

Exchange has three modes of operation:fanout, Direct, Topic

Mode 1 fanout

Any messages sent to Fanout Exchange will be forwarded to all the queue with that Exchange binding (binding)
1. Patterns that can be understood as routing tables
2. This mode does not require Routing_key (even if specified, is not valid)
3. This mode requires Exchange to be bound with the queue in advance, one exchange can bind multiple queues, and one queue can bind to multiple exchange.
4. If exchange that receives the message is not bound to any queue, the message is discarded.

　　Note : This is the time to start the consumer, the Subscriber. Because the random queue is randomly generated when the consumer is started, and is bound. The producer is only sent to exchange and does not communicate directly with the random queue.

Producer Code

#!/usr/bin/env python#-*-coding:utf-8-*-# auth:pangguoping# rabbitmq Publisher Import pikacredentials = Pika. Plaincredentials (' admin ', ' admin ') #链接rabbit服务器 (localhost is native, if it is another server please modify the IP address) connection = Pika. Blockingconnection (Pika. Connectionparameters (' 192.168.1.103 ', 5672, '/', credentials)) channel = Connection.channel () # Defines the switch, Exchange represents the switch name, Type expression channel.exchange_declare (exchange= ' logs_fanout ',                         type= ' fanout ') message = ' Hello Python ' # Send the message to switch channel.basic_publish (exchange= ' logs_fanout ',  # Specify Exchange                      routing_key= ',  # fanout does not need to be configured, Configured also does not take effect                      body=message) print ("[x] Sent%r"% message) Connection.close ()

Consumer Code

 #!/usr/bin/env python#-*-coding:utf-8-*-# auth:pangguoping# rabbitmq subscribers Import pikacredentials = Pika. Plaincredentials (' admin ', ' admin ') #链接rabbit服务器 (localhost is native, if it is another server please modify the IP address) connection = Pika. Blockingconnection (Pika. Connectionparameters (' 192.168.1.103 ', 5672, '/', credentials)) channel = Connection.channel () # define the switch, make exchange claims, Exchange represents the switch name, type Channel.exchange_declare (exchange= ' logs_fanout ', type= ' fanout ') # randomly creates a queue res Ult = Channel.queue_declare (exclusive=true) # exclusive=true indicates that a temporary queue is established, and when consumer is closed, the queue is deleted queue_name =  result.method.queue# binds the queue to Exchange Channel.queue_bind (exchange= ' logs_fanout ', queue=queue_name) print (' [*] Waiting for logs. To exit Press CTRL + C ') def callback (ch, method, properties, body): Print ("[x]%r"% body) # Get information from the queue Channel.basic_consume (Callback, Queue=queue_name, No_ack=true) channel.start_consuming () 

Mode 2 Direct

How the routing key works: Each receiving end of the message queue when binding the switch, you can set the corresponding routing key. When the sender sends information through the switch, it can indicate the routing key, and the switch sends the message to the corresponding message queue according to the routing key, so that the receiving end can receive the message.

Any messages sent to direct Exchange will be forwarded to the queue specified in Routing_key:
1. In general, you can use RABBITMQ's own Exchange: "" (The Exchange name is an empty string), or you can customize the Exchange
2. No binding (BIND) operation is required for Exchange in this mode. Of course, you can also bind. Different routing_key can be bound to different queue, and different queues are bound to different exchange
3. Message delivery requires a "Routing_key"
4. If the queue name that is bound in Routing_key does not exist in the message, the message is discarded.
If an exchange is declared as direct and Routing_key is specified in bind, it is necessary to indicate both exchange and Routing_key when sending the message.

Consumer Code

#!/usr/bin/env python#-*-coding:utf-8-*-# auth:pangguoping# consumer Import pikacredentials = Pika. Plaincredentials (' admin ', ' admin ') #链接rabbit服务器 (localhost is native, if it is another server please modify the IP address) connection = Pika. Blockingconnection (Pika. Connectionparameters (' 192.168.1.103 ', 5672, '/', credentials)) channel = Connection.channel () # Define Exchange and Type Channel.exchange_declare (exchange= ' direct_test ', type= ' direct ') # Generate random Queue result = Chann El.queue_declare (exclusive=true) queue_name = result.method.queueseverities = [' Error ',]# will be random queue with Routing_ Key keyword and exchange to bind for severity in Severities:channel.queue_bind (exchange= ' direct_test ', queue= Queue_name, routing_key=severity) print (' [*] waiting for logs. To exit Press CTRL + C ') def callback (ch, method, properties, body): Print ("[x]%r:%r"% (Method.routing_key, body)) # Receive Channel.basic_consume (Callback, Queue=queue_name, No_ack=true) Channel.start_con Suming ()

Producers

#!/usr/bin/env python#-*-coding:utf-8-*-# auth:pangguoping# Publisher Import pikacredentials = Pika. Plaincredentials (' admin ', ' admin ') #链接rabbit服务器 (localhost is native, if it is another server please modify the IP address) connection = Pika. Blockingconnection (Pika. Connectionparameters (' 192.168.1.103 ', 5672, '/', credentials)) channel = Connection.channel () # Define switch name and type Channel.exchange_declare (exchange= ' direct_test ',                         type= ' direct ') severity = ' Info ' message = ' 123 ' # Publish message to switch Direct_test, and the published message carries the keyword Routing_key is infochannel.basic_publish (exchange= ' direct_test ',                      routing_key= Severity,                      body=message) print ("[x] Sent%r:%r"% (severity, message)) Connection.close ()

When the receive is correct at run time, you can use Rabbitmqctl list_bindings to view the binding situation.

Mode 3 Topic

routing key fuzzy matching is actually the extension of the routing key (Routing_key), that is, the regular expression can be used, and the usual regular expressions are different, here the words "#" means all, all the meaning; "*" matches only one word.

Any messages sent to topic Exchange will be forwarded to all queues that are concerned with the specified topic in Routing_key
1. This mode is more complex, simply put, is that each queue has its own topic of interest, all messages with a "title" (Routing_key), Exchange will forward the message to all the topics of interest can match the Routing_key fuzzy queue.
2. This mode requires Routing_key, perhaps to bind exchange and queue in advance.
3. When binding, to provide a topic that the queue cares about, such as "#.log.#" means that the queue is concerned with all messages involving log (a message Routing_key "MQ.log.error" is forwarded to the queue).
4. "#" means 0 or more keywords, and "*" denotes a keyword. such as "log.*" can Match "Log.warn", cannot match "log.warn.timeout", but "log.#" can match the above two.
5. Similarly, if Exchange does not find a queue that can match Routing_key, this message is discarded.

The specific code here is not redundant write, referring to the second mode can be, the only place to change is the declaration of exchange type, as well as binding and sending when the Routing_key use regular mode.

At this point, the Python operation Rabbitmq, along with some simple usage, is described here.

Reference Http://www.cnblogs.com/jishuweiwang

? Previous: Redis's aof function
? Next: Python operation MySQL-that is Pymysql/sqlalchemy usage posted @2016-07-30 07:37 waiting_for_you_you Read (322) Comments (0) Edit Collection

Refresh Comments Refresh page back to top

Python-Operation RABBITMQ