Python development [Article 10]: RabbitMQ queue, pythonrabbitmq

Python development [Article 10]: RabbitMQ queue, pythonrabbitmq
Introduction

RabbitMQ is a popular open-source message queue system developed in erlang. RabbitMQ is the standard implementation of AMQP (Advanced Message Queue Protocol.

Install

First install the erlang environment.

Official Website: http://www.erlang.org/

Http://erlang.org/download/otp_win64_20.0.exe for Windows

Linux: yum Installation

Windows Installation Steps

Step 1

Step 2

Step 3

Step 4

Step 5

Erlang installation is complete.

Then install RabbitMQ. First download the Windows version of RabbitMQ.

Official Website: http://www.rabbitmq.com/

Http://www.rabbitmq.com/releases/rabbitmq-server/v3.6.10/rabbitmq-server-3.6.10.exe for Windows

Open the installer and follow the steps below to install it.

RabbitMQ installation is complete.

Go to the management tools page.

Run commands

Check whether the RabbitMQ service is started.

So far, all installation is complete.

Linux Installation Steps

Install erlang.

Install RabbitMQ.

RabbitMQ installation fails. the following error is returned.

The reason is that the erlang version installed by yum is too low, the RabbitMQ provided here is the latest version 3.6.10, and the minimum erlang version required is the R16B-03, otherwise the compilation will fail, that is, the above error.

Reinstall erlang.

Reinstall erlang.

Run erlang.

Install socat.

Reinstall RabbitMQ.

The preceding error message shows that the installation failed because of the dependency of rabbitMQ. To ignore the dependency, run the following command.

The installation is successful.

Start and Stop RabbitMQ.

 

Use RabbitMQ

Implement the simplest queue Communication

Send end (producer)

Receive (consumer)

No_ack Analysis

The no_ack attribute is an important parameter that can be set when the Basic. Consume method is called. No_ack is used to ensure that the message is successfully processed by the consumer. The success consciousness here is that when no_ack = false is set, as long as the consumer manually responds to Basic. Ack, it will be processed successfully.

No_ack = true (automatic response at this time)

In this case, consumer immediately replies to Ack after receiving Basic. Deliver + Content-Header + Content-Body, which is an Ack in TCP. The Ack reply does not care whether the consumer processes the received data or the time required to process the data.

No_ack = False (manual response at this time)

In this case, the consumer must reply to Ack after processing the received Basic. Deliver + Content-Header + Content-Body, which is the Basic. Ack in the amqp protocol. The Ack reply is related to the business processing, so the specific response time should depend on the time consumed by the business processing.

Summary

Basic. Ack is sent to RabbitMQ to notify you that the corresponding message can be removed from the RabbitMQ message cache.

Basic. an exception occurred before the Ack was sent to RabbitMQ by the consumer. RabbitMQ found that the connection with the consumer was disconnected, send the message to other consumers in polling mode (multiple consumers need to subscribe to the same queue ).

When no_ack = true, RabbitMQ considers that the message has been confirmed once it is deliver, so the message in the cache will be deleted immediately, therefore, message loss occurs when a consumer exception occurs.

The Basic. Ack from consumer has no direct relationship with the Basic. Ack sent to the Producer.

Message persistence acknowledgment message persistence

No-ack = False. If the consumer fails, RabbitMQ adds the task to the queue again.

Callback Function

Basic_consume

Receive (consumer)

Durable message persistence

The producer fails when sending a message and the consumer fails when receiving the message. The following method causes RabbitMQ to add the message to the queue again.

Callback Function

Basic_consume

Add parameters to basic_publish

Add parameters to channel. queue_declare

Send end (producer)

The receive end (consumer) is the same as the acknowledgment end (consumer) in message persistence.

Message Distribution

By default, data in the message queue is distributed to consumers in order. However, in most cases, the processing capabilities of the backend consumer servers of the message queue are different, in this case, some servers are idle for a long time and resources are wasted. Then, we need to change the order in which the default message queue is obtained. You can configure prefetch_count = 1 on each consumer to tell RabbitMQ not to send new messages when the current message of this consumer is not processed.

Consumer

The producer end remains unchanged.

Publish and subscribe messages (publish \ subscribe)

The difference between publishing and subscription is that publishing and subscription will send messages to all subscribers, and the data in the message queue will disappear once it is consumed. Therefore, when RabbitMQ implements publishing and subscription, a queue is created for each subscriber, and when the publisher publishes a message, the message is placed in all related queues. Similar to the broadcast effect, exchange is required at this time. Exchange has a type when defined to determine which Queue meets the conditions and can receive messages.

Fanout: All bind queues to this exchange can receive messages.

Direct: the unique queue determined by routingKey and exchange to receive messages.

Topic: All queues that match the routingKey (which can be an expression) can receive messages.

Expression symbol description

#: One or more characters

*: Any character

For example, #. a matches a. a, aa. a, aaa. a, and so on.

*. A matches a. a, B. a, c. a, and so on.

Note: When RoutingKey is # And Exchange Type is topic, fanout is used.

Heaers: Uses headers to determine which queue the message is sent.

Publisher

Subscriber

Keyword sending (echange type = direct)

When a message is sent, a queue is explicitly specified and a message is sent to it. RabbitMQ also supports sending Based on the keyword, that is, the queue is bound with a keyword. The sender sends the data to the message exchange based on the keyword, exchange determines the queue to which data is sent based on the keyword.

Publisher

Subscriber

Start subscriber1

Start subscriber2

Start publisher1

Start publisher2

Start publisher3

Result

Fuzzy match (exchange type = topic)

Under the topic type, you can bind the queue with several fuzzy keywords. The sender sends the data to exchange, and exchange will pass in the "route value" and "keyword" for matching, if the matching is successful, the data is sent to the specified queue.

*: Match any character

#: Match any character

Publisher

Subscriber

Test

Remote Procedure Call (RPC)

Remote Procedure Call Protocol (RPC. In a large company, the system consists of services, large and small. Different teams maintain different codes and deploy them on different servers. However, when developing, you often need to use the methods of other teams, because implementation is already in place. However, if these services are deployed on different servers, network communication is required to call them. These codes are complex and complex, which may be inefficient if you are not careful. The PRC Protocol defines the plan, and other companies provide different implementations. For example, Microsoft's wcf and WebApi.

The implementation of RPC in RabbitMQ is very simple and efficient. Currently, both the client and server are message publishers and message recipients.

First, the client sends a request to the server through RPC. Correlation_id: Request ID, erply_to: Return queue. (Here I have some data that you need to process for me. correlation_id is the ID of my request. After processing, you will return the result to the erply_to queue)

The server receives the request, starts processing, and returns it. Correlation_id: client request ID. (Correlation_id: This is your request identifier and is returned to you. In this case, the client compares its own correlation_id with the correlation_id returned by the server, and receives the same result .)

Rpc_server

Rpc_client