Python Operation RABBITMQ Remote result return for server Message Queuing

RABBITMQ is an MQ-based server, Python can be manipulated through the Pika library, where we will explain the Python operation rabbitmq The remote result of the server message queue:

First of all, I have the test environment here: Ubuntu14.04 + Python 2.7.4
RABBITMQ Server

sudo apt-get install Rabbitmq-server

Python uses RABBITMQ requires Pika library

sudo pip install Pika

Remote result return
The message sender sends a message out and no result is returned. If you just send a message, of course, there is no problem, but in practice, it is often necessary for the receiving side to process the received message, returned to the sender.

Processing method Description: The sending side generates a temporary queue for receiving messages before sending the message, which is used to receive the returned results. In fact, here the receiver, the sender of the concept has been blurred, because the sender also to receive messages, the receiving side also to send messages, so here I use another example to demonstrate this process.

Example content: Suppose there is a control center and a compute node, the control center sends a natural number n to the compute node, and the compute node returns the N value to the control center after 1. The center.py Analog control center is used here to compute.py the compute nodes.

compute.py Code Analysis

#!/usr/bin/env python#coding=utf8import Pika #连接rabbitmq服务器connection = Pika. Blockingconnection (Pika. Connectionparameters (    host= ' localhost ')) channel = Connection.channel () #定义队列channel. Queue_declare (queue= ' Compute_queue ') print ' [*] waiting for n ' #将n值加1def increase (n):  return n + 1 #定义接收到消息的处理方法def request (ch, method, pro Perties, body):  print "[.] Increase (%s) "% (body,)   response = increase (int (body))   #将计算结果发送回控制中心  ch.basic_publish (exchange=",           routing_key=properties.reply_to,           body=str (response))  ch.basic_ack (Delivery_tag = Method.delivery_ TAG) Channel.basic_qos (prefetch_count=1) channel.basic_consume (Request, queue= ' Compute_queue ') Channel.start_ Consuming ()

The code of the compute node is relatively simple, it is worth mentioning that the original receiving method is to print the message directly, this side of the Gay calculation, and send the results back to the control center.

center.py Code Analysis

#!/usr/bin/env Python#coding=utf8import Pika Class Center (object): Def __init__ (self): Self.connection = Pika. Blockingconnection (Pika. Connectionparameters (host= ' localhost ')) Self.channel = Self.connection.channel () #定义接收返回消息的队列 Resul t = Self.channel.queue_declare (exclusive=true) self.callback_queue = Result.method.queue Self.channel.basic_consume (Self.on_response, No_ack=true, Queue=self.callback_queue) #定义接收到返回消息的处理方法 def on_res Ponse (self, ch, method, props, body): Self.response = Body def request (self, n): Self.response = None #发送计 Self.channel.basic_publish (exchange= ', routing_key= ' compute_queue ', prop Erties=pika.    Basicproperties (reply_to = Self.callback_queue,), BODY=STR (n)) #接收返回的数据 while Self.response is None:self.connection.process_data_events () return int (self.response) CENter = Center () print "[x] requesting increase (" response = Center.request () print "[.] Got%r "% (response,)

The code above defines the queue and processing method that receives the returned data, and assigns the queue to reply_to when the request is sent, which is used in the compute node code to get the return queue.

Open two terminals, one running code python compute.py, another terminal running center.py, if successful, you should be able to see the effect.

The author in the test, there are some small problems, that is, when the center.py sent the message did not indicate the return queue, the results compute.py over there in the calculation results to send back the data times wrong, the hint Routing_key does not exist, again run error. Using Rabbitmqctl list_queues to view the queue, the Compute_queue queue was found to have 1 data, each time the compute.py re-run, the data will be re-processed. Later use/etc/init.d/rabbitmq-server restart reboot under RABBITMQ OK.

Correlation number Correlation ID
The previous demo shows the example of a remote result return, but there is one that is not mentioned, that is, correlation ID, what is this?

Assuming there are multiple compute nodes, the control center turns on multiple threads, sends a number to those compute nodes, requires a calculation and returns, but the control center only opens a queue where all the threads are getting messages from this queue, and how does each thread determine that the message received is the corresponding thread? This is the use of correlation ID. Correlation translation into Chinese is related to each other and also expresses this meaning.

Correlation ID Operation principle: The control center sends a calculation request when the correlation ID is set, and the compute node returns the result, along with the received correlation ID, so that the control center can identify the request through the correlation ID. In fact, the correlation ID can also be understood as a unique identification code for the request.

Example content: Control center opens multiple threads, each line routine to initiate a calculation request, with the correlation ID, each thread can receive the corresponding calculation results accurately.

compute.py Code Analysis

Compared with the above, just modify a place: When the results are sent back to the control center, add the parameter correlation_id setting, the value of the parameter is actually sent from the control center, here just send back again. The code is as follows:

#!/usr/bin/env python#coding=utf8import Pika #连接rabbitmq服务器connection = Pika. Blockingconnection (Pika. Connectionparameters (    host= ' localhost ')) channel = Connection.channel () #定义队列channel. Queue_declare (queue= ' Compute_queue ') print ' [*] waiting for n ' #将n值加1def increase (n):  return n + 1 #定义接收到消息的处理方法def request (ch, method, pro PS, body):  print "[.] Increase (%s) "% (body,)   response = increase (int (body))   #将计算结果发送回控制中心, increase correlation_id settings  ch.basic_ Publish (exchange= ",           routing_key=props.reply_to,           Properties=pika. Basicproperties (correlation_id = \                           props.correlation_id),           body=str (response))  Ch.basic_ack (delivery _tag = Method.delivery_tag) Channel.basic_qos (prefetch_count=1) channel.basic_consume (Request, queue= ' Compute_ Queue ') channel.start_consuming ()

center.py Code Analysis

The control center code is a little more complicated, with three more key areas:

Use Python's uuid to produce a unique correlation_id.
When sending a calculation request, set the parameter correlation_id.
Defines a dictionary to hold the returned data, and the key value is the correlation_id generated by the corresponding thread.
The code is as follows:

#!/usr/bin/env Python#coding=utf8import Pika, threading, uuid #自定义线程类, inherit threading. ThreadClass MyThread (Threading. Thread): Def __init__ (self, func, num): Super (MyThread, self). __init__ () Self.func = func Self.num = num def r Un (self): print "[x] requesting increase (%d)"% self.num response = Self.func (self.num) print "[.] Increase (%d) =%d "% (self.num, response) #控制中心类class Center (object): Def __init__ (self): Self.connection = Pika. Blockingconnection (Pika. Connectionparameters (host= ' localhost ')) Self.channel = Self.connection.channel () #定义接收返回消息的队列 result = Self.channel.queue_declare (exclusive=true) self.callback_queue = Result.method.queue Self.channel.basic_consume (SE Lf.on_response, No_ack=true, queue=self.callback_queue) #返回的结果都会存储在该字典里 Self.resp    Onse = {} #定义接收到返回消息的处理方法 def on_response (self, ch, method, props, body): self.response[props.correlation_id] = body def request (self, n): Corr_id = str (UUID.UUID4 ()) self.response[corr_id] = None #发送计算请求, and sets the return queue and correlation_id Self.channel.basic_publi SH (exchange= ', routing_key= ' Compute_queue ', Properties=pika.                     Basicproperties (reply_to = self.callback_queue, correlation_id = corr_id, ), BODY=STR (n)) #接收返回的数据 while self.response[corr_id] is NONE:SELF.CONNECTION.PR Ocess_data_events () return int (self.response[corr_id]) center = center () #发起5次计算请求nums = [Ten, +, +, 50]threads = [] For Num in Nums:threads.append (MyThread (center.request, num)) for the thread in the Threads:thread.start () for the thread in threads : Thread.Join ()

The author opened two terminals, to run compute.py, open a terminal to run center.py, the final result output is as follows:

You can see that although the results obtained are not sequential outputs, the results are corresponding to the source data.

The example here is to create a queue that uses the correlation ID to identify each request. It is also possible to do this without using the correlation ID, which is to create a temporary queue every time a request is made, but it is too expensive and is not recommended by the authorities.