RabbitMQ quick start python tutorial

HelloWorld introduction RabbitMQ: it can be regarded as a "post office" to receive and re-transmit messages ". The sender and receiver interact through the queue. The size of the queue can be viewed as infinite. multiple senders can be sent to one queue, and multiple recipients can also receive messages from one queue. Introduction to co... HelloWorld

RabbitMQ: receives a message and then transmits the message. it can be considered as a "Post Office ". The sender and receiver interact through the queue. The size of the queue can be viewed as infinite. multiple senders can be sent to one queue, and multiple recipients can also receive messages from one queue.

Code

The protocol used by rabbitmq is amqp, and the recommended client for python is pika.

pip install pika -i https://pypi.douban.com/simple/

Send. py

# Coding: utf8import pika # Create a connection = pika. BlockingConnection (pika. ConnectionParameters ('localhost') # connect to the local RabbitMQ server channel = connection. channel () # obtain the channel

The link here is local. if you want to connect to a server on another machine, you only need to fill in the address or host name.

Next we start to send the message. make sure that the queue that receives the message exists. otherwise, rabbitmq discards the message.

Channel. queue_declare (queue = 'Hello') # create the queue channel hello in RabbitMQ. basic_publish (exchange = '', # use the default exchange to send messages to the queue routing_key = 'hello', # send to the queue hello body = 'Hello World! ') # Message content connection. close () # close flush at the same time

By default, RabbitMQ requires 1 GB of free disk space. Otherwise, the message fails to be sent.

At this time, a message has been stored in the local queue hello. if you use rabbitmqctl list_queues, you can see

hello 1

A hello queue stores a message.

Receive. py

# coding: utf8import pikaconnection = pika.BlockingConnection(pika.ConnectionParameters(               'localhost'))channel = connection.channel()

Link to the server first, which is the same as before

Channel. queue_declare (queue = 'Hello') # This is declared to ensure that the queue hello exists. it can be declared multiple times. this is mainly to prevent the receipt of program first run errors def callback (ch, method, properties, body): # The callback print ("[x] Received ED % r" % body) channel after receiving the message. basic_consume (callback, queue = 'hello', # receive the message no_ack = True for the specified queue hello) # Do not send ack to the server channel after processing the message. start_consuming () # Start message acceptance. this will enter an endless loop.

Work queue (task queue)

A work queue is used to distribute time-consuming tasks to multiple worker processes. Do not do the resource-consuming tasks immediately (wait for these tasks to be completed), but schedule these tasks and execute them. For example, we send a task as a message to the queue, start the worker process to accept and finally execute the task, and start multiple worker processes to work. This applies to web applications, that is, they should not complete complex tasks within the processing window of an http request.

Channel. basic_publish (exchange = '', routing_key = 'task _ queue ', body = message, properties = pika. BasicProperties (delivery_mode = 2, # message Persistence ))

The message allocation method is polling, that is, each worker receives the same number of messages.

Message ack

If a message is assigned to a worker, but the worker crashes after it is not processed, the message may be lost because once rabbitmq delivers a message to the worker, it deletes the message.

To prevent message loss, rabbitmq provides ack, that is, after receiving and processing a message, the worker sends an ack to rabbitmq, informing rabbitmq that the message can be deleted from the queue. If the working process crashes and rabbitmq does not receive the ack, the message will be re-delivered to other working processes. You do not need to set timeout. this task can be processed even if it takes a long time.

Ack is enabled by default. Previously, no_ack = True was specified in the worker process.

Channel. basic_consume (callback, queue = 'Hello') # ack is enabled.

Callback with ack:

Def callback (ch, method, properties, body): print "[x] Received % r" % (body,) time. sleep (body. count ('. ') print "[x] Done" ch. basic_ack (delivery_tag = method. delivery_tag) # send ack

Message Persistence

However, sometimes RabbitMQ restarts and messages are lost. You can set persistence when creating a queue:
(The nature of the queue cannot be changed once determined)

channel.queue_declare(queue='task_queue', durable=True)

At the same time, you must set the persistence attribute of the message when sending the message:

Channel. basic_publish (exchange = '',

                  routing_key="task_queue",                  body=message,                  properties=pika.BasicProperties(                     delivery_mode = 2, # make message persistent                  ))

However, if a message has not been stored in RabbitMQ, the message will still be lost. At the same time, RabbitMQ does not accept that every message is stored on the disk. Use publisher confirm if you need more comprehensive assurance.

Fair message distribution

The message distribution in polling mode may be unfair. for example, if an odd number of messages are heavy tasks, some processes will continue to run complex tasks. Even if a worker process has a backlog of unprocessed messages, for example, many do not issue ack, RabbitMQ still sends messages to it in order. You can add the following settings to the acceptance process:

channel.basic_qos(prefetch_count=1)

Inform RabbitMQ, so that no messages will be distributed to a working process if ack is not returned.

Group sending

Generally, a message is sent to a worker process and then completed. sometimes you want to send a message to multiple processes at the same time:

Exchange

The sender does not directly send a message to the queue. In fact, the sender does not know that the message will be sent to that queue, and the sender can only send the message to exchange. Exchange collects messages from producers and pushes messages to queues. As exchange, it needs to know what it needs to do when receiving a message, whether to add it to a special queue or put it in many queues, or discard it. Exchange has direct, topic, headers, fanout and other types, while fanout is widely used. When publishing a message, if the exchange value is '', default exchange is used.

Channel. exchange_declare (exchange = 'logs', type = 'fanout') # this exchange sends messages to all queues it knows.

Temporary queue

Result = channel. queue_declare () # Create a random queue result = channel. queue_declare (exclusive = True) # Create a random queue and destroy it after no receiver connects to the queue. method. queue

In this way, result. method. queue is the queue name, which can be used for sending or receiving.

Bind exchange and queue

channel.queue_bind(exchange='logs',               queue='hello')

Logs also sends a message to hello.

When sending a message, use the created logs exchange

   channel.basic_publish(exchange='logs',                  routing_key='',                  body=message)

Routing

You have used bind before, that is, to establish the relationship between exchange and queue (this queue is interested in messages from this exchange). you can specify the routing_key option in bind.

Use direct exchange

Send messages of the corresponding routing key to the queue bound with the same routing key.

channel.exchange_declare(exchange='direct_logs',                     type='direct')

Send function to publish messages of different severity:

channel.basic_publish(exchange='direct_logs',                  routing_key=severity,                  body=message)

Bind the corresponding severity to the acceptance function:

channel.queue_bind(exchange='direct_logs',                   queue=queue_name,                   routing_key=severity)

Use topic exchange

The previous direct exchange can only bind one routing key. you can use this topic exchange that can take and separate the routing key. for example:

"stock.usd.nyse" "nyse.vmw"

Like direct exchange, the key bound to the receiver is the same as the routing key specified during sending. In addition, some special values are as follows:

* 1 word #0 or more words

If the routing key sent by the sender is in three parts, for example, celerity. color. species.

Q1 :*. orange. * The color in the middle is orange Q2 :*. *. rabbit corresponds to the last species of rabbit lazy. # The first part corresponds to lazy.

Both qucik. orange. rabbit Q1 Q2 can be received, and quick. orange. fox can only be received by Q1. for lazy. pink. rabbit, although it matches Q2 twice, it will only be sent once. If you bind # directly when binding, you will receive all.

RPC

Run a function on a remote machine and obtain the result.

1. when the client starts, a temporary queue is set for receiving the callback and binding the queue

    self.connection = pika.BlockingConnection(pika.ConnectionParameters(            host='localhost'))    self.channel = self.connection.channel()    result = self.channel.queue_declare(exclusive=True)    self.callback_queue = result.method.queue    self.channel.basic_consume(self.on_response, no_ack=True,                               queue=self.callback_queue)

2. the client sends an rpc request with the corresponding callback queue reply_to. correlation_id is set to the unique id of each request (although a callback queue can be created for each RPC request, however, this is not efficient. if a client uses only one queue, correlation_id must be used to match the request.) then, the request is blocked in the callback queue until a reply is received.

Note: If you receive an invalid correlation_id, you can directly discard it, because in this case, the server has sent a response but no ack has been sent, after the server is restarted, the task will be re-processed and sent again, but the request has been processed.

Channel. basic_publish (exchange = '', routing_key = 'rpc _ queue ', properties = pika. basicProperties (reply_to = self. callback_queue, correlation_id = self. consumed _id,), body = str (n) # call while self. response is None: # This is equivalent to blocking self. connection. process_data_events () # view the callback queue return int (self. response)

3. the request will be sent to the rpc_queue queue
4. the RPC server extracts the data from rpc_queue, runs the command, and sends a reply.

Channel. basic_consume (on_request, queue = 'rpc _ queue ') # bind to wait for the request # After processing: ch. basic_publish (exchange = '', routing_key = props. reply_to, properties = pika. basicProperties (correlation_id = \ props. correlation_id), body = str (response) # send a reply to the callback queue ch. basic_ack (delivery_tag = method. delivery_tag) # send ack

5. the client extracts data from the callback queue, checks correlation_id, and performs corresponding operations.

if self.corr_id == props.correlation_id:        self.response = body

The above is the details of the RabbitMQ quick start python tutorial. For more information, see other related articles in the first PHP community!