C # Use RabbitMQ,

C # Use RabbitMQ,
1. Description

In the enterprise application system field, communication, integration and integration between different systems will be faced, especially when facing heterogeneous systems, such distributed calls and communication become more and more important. Second, the system usually has many places that do not have high real-time requirements but are time-consuming, such as sending text messages, email reminders, updating the article reading count, and recording user operation logs, if real-time processing is performed, the system is under great pressure when the user traffic is large.

In the face of these problems, we usually put these requests in Message Queue MQ for processing. heterogeneous systems use messages for communication.

MQ is called Message Queue. MQ is a communication method for applications. Applications communicate by reading and writing messages in and out of the queue (for Application Data) without dedicated connections. Message transmission refers to the communication between programs by sending data in messages, rather than by directly calling each other. Direct calls are usually used for such remote process calls. Queuing means that applications communicate through queues. The use of the queue removes the need to receive and send applications simultaneously.

MQ is a typical consumer-producer model. One End continuously writes messages to the message queue, while the other end can read or subscribe to messages in the queue.

RabbitMQ is a complete and reusable enterprise Message System Based on AMQP. He complies with the Mozilla Public License open-source protocol.

Compared with file transfer and Remote Procedure Call (RPC), message transmission seems superior because it has better platform independence and supports concurrent and asynchronous calls. Therefore, if the following situations occur in the system:

• The requirements for real-time operations are not high, and the tasks to be executed are extremely time-consuming;
• Integration between heterogeneous systems;

Generally, you can consider introducing message queues. In the first case, message queues are often used to process tasks that have been executed for a long time. The introduced Message Queue becomes the buffer for message processing. The asynchronous communication mechanism introduced by message queue enables both the sender and receiver to continue executing the following code without waiting for the other party to return a successful message, thus improving the data processing capability. Especially when traffic and data traffic are large, you can combine message queues and background tasks to process big data by avoiding peak periods, this effectively reduces the load on data processing in the database.

This article briefly introduces the message proxy tool RabbitMQ and how to use RabbitMQ in. NET.

2. Build the environment

2.1 install the Erlang Language Runtime Environment

Because RabbitMQ is written in Erlang, the Erlang runtime environment is installed first. Step-by-Step blog: configure the Erlang environment in windows

2.2 install the RabbitMQ Server

Address http://www.rabbitmq.com/

Download and install.

Run RabbitMQ in the background as a Windows Service: Open cmd and switch to the sbin directory for execution.

rabbitmq-service installrabbitmq-service enablerabbitmq-service start

Now the server of RabbitMQ is started.

To view and control the status of the RabbitMQ server, use the rabbitmqctl script.

For example, view the status:

rabbitmqctl status

　　

If the node is not connected, go to the C: \ Windows directory and. erlang. copy the cookie file to the user directory C: \ Users \ {user name}. This is the Cookie file of Erlang and can interact with Erlang.

Run the following command to view the user:

rabbitmqctl list_users

RabbitMQ creates the default username guest and password guest for us, and guest has all the permissions of RabbitMQ by default.

Generally, we need to create a new user, set the password, grant permissions, and set it as an administrator. You can use the following command to perform this operation:

Rabbitmqctl add_user JC JayChou // create user JC with the password JayChourabbitmqctl set_permissions JC ". *"". *"". * "// grant the JC permission to read and write all message queues. rabbitmqctl set_user_tags JC administrator // assign user groups.

Change the JC password to 123:

rabbitmqctl change_password JC  123

Delete user JC:

rabbitmqctl delete_user  JC

You can also enable the rabbitmq_management plug-in to view and manage the RabbitMQ Service on the web interface.

rabbitmq-plugins enable rabbitmq_management  

 

2.3 download Client dll of RabbitMQ

: Http://www.rabbitmq.com/releases/rabbitmq-dotnet-client/

I downloaded this rabbitmq-dotnet-client-3.6.6-dotnet-4.5.zip

Decompress the package. We need this file, which will be referenced in the vs project in the future:

3. Use

3.1 Before using RabitMQ, describe several concepts first

　　

RabbitMQ is a message proxy. He receives a message from the producer and sends the message to the consumer (consumer) between sending and receiving. He can route, cache, and persist the message according to the set rules.

RabbitMQ and messages are generally mentioned in some proprietary terms.

• Production means sending. The program that sends messages is a producer ). We generally use "P" to represent:

• The queue is the name of the mailbox. Messages are transmitted through your application and RabbitMQ. They can only be stored in the queue. There is no limit on the queue capacity. You can store any number of messages-basically an infinite buffer. Multiple producers can send messages to the same queue. Likewise, multiple consumers can also obtain data from the same queue. The queue can be drawn as follows (the queue name is shown in the figure ):

• Consumption is the same as obtaining a message. A consumer is a program waiting to get messages. Let's draw "C ":

Generally, the message producer, consumer, and proxy are not on the same machine.

3.2 Hello Word

The following shows how to use RabbitMQ:

3.2.1 create a console project named ProjectSend and reference RabbitMQ. Client. dll. This program is used as the Producer to send data:

Static void Main (string [] args) {var factory = new ConnectionFactory (); factory. hostName = "localhost"; // The RabbitMQ service runs factory locally. userName = "guest"; // user name factory. password = "guest"; // Password using (var connection = factory. createConnection () {using (var channel = connection. createModel () {channel. queueDeclare ("hello", false, null); // create a message queue named hello string message = "Hello World "; // The transmitted message content var body = Encoding. UTF8.GetBytes (message); channel. basicPublish ("", "hello", null, body); // start to pass the Console. writeLine ("sent: {0}", message );
Console. ReadLine ();}}}

　　

First, create a ConnectionFactory and set the target. Because it is on the local machine, set it to localhost. If RabbitMQ is not on the local machine, you only need to set the IP address or machine name of the target machine, set the username and password created earlier.

Create a Channel. If you want to send a message, create a queue and publish the message to the queue. When a queue is created, it is created only when the queue does not exist on RabbitMQ. Messages are transmitted as binary arrays. Therefore, if a message is an object, it needs to be serialized and then converted to a binary array.

Now the code sent by the client has been written. After running the code, the message will be published to the RabbitMQ message queue. Now, you need to write the server code to connect to RabbitMQ to obtain the message.

3.2.2 create a console project named ProjectReceive and reference RabbitMQ. Client. dll. As a Consumer, it is used to receive data:

Static void Main (string [] args) {var factory = new ConnectionFactory (); factory. hostName = "localhost"; factory. userName = "guest"; factory. password = "guest"; using (var connection = factory. createConnection () {using (var channel = connection. createModel () {channel. queueDeclare ("hello", false, null); var consumer = new EventingBasicConsumer (channel); channel. basicConsume ("hello", false, consumer); consumer. stored ed + = (model, ea) =>{ var body = ea. body; var message = Encoding. UTF8.GetString (body); Console. writeLine ("Received: {0}", message) ;}; Console. readLine ();}}}

Like sending, you must first define the connection and then declare the message queue. To receive a message, you need to define a Consume and process the data in the event of receiving the message.

3.2.3 now all the sending and receiving clients have been written. Let's compile and execute them.

Send message:

Now, a message is sent in the hello message queue. This message is stored on the RabbitMQ server. You can use list_queues of rabbitmqctl to view all message queues and the number of messages in them. Currently, there is only one message queue in Rabbitmq, with only one message in it:

You can also view the related information of this queue on the web management interface:

 

 

Receive message:

Now that the message has been received, let's take a look at the queue content:

It can be seen that the content in the message has been deleted after receiving it.

3.3 working queue

The preceding example shows how to send and receive messages in a specified message queue.

Now we create a work queue to distribute time-consuming tasks to multiple workers ):

The main idea of a work queue (work queue) is to avoid immediate execution and wait for operations that consume a large amount of resources and time to complete. The Task is sent to the queue as a message and will be processed later. A worker process running in the background will take out the task and process it. When running multiple workers, tasks are shared among them.

This is very useful in network applications. It can process complex tasks in a short HTTP request. In some places with low real-time requirements, we can process other unimportant operations, such as writing logs, in the form of messages after processing the main operations.

Preparation

In the first part, a message containing "Hello World!" is sent !" . Send some strings as complex tasks. Here we use the time. sleep () function to simulate time-consuming tasks. Adding the dot (.) to the string indicates the complexity of the task. A dot (.) takes 1 second. For example, "Hello..." takes 3 seconds.

Make some simple adjustments to the sending. cs in the previous example so that you can send arbitrary messages. This program sends tasks to our work queue as planned.

static void Main(string[] args){    var factory = new ConnectionFactory();    factory.HostName = "localhost";    factory.UserName = "yy";    factory.Password = "hello!";    using (var connection = factory.CreateConnection())    {        using (var channel = connection.CreateModel())        {            channel.QueueDeclare("hello", false, false, false, null);            string message = GetMessage(args);            var properties = channel.CreateBasicProperties();            properties.DeliveryMode = 2;            var body = Encoding.UTF8.GetBytes(message);            channel.BasicPublish("", "hello", properties, body);            Console.WriteLine(" set {0}", message);        }    }    Console.ReadKey();}private static string GetMessage(string[] args){    return ((args.Length > 0) ? string.Join(" ", args) : "Hello World!");}

 

Next, we modify the receiver so that he can Sleep the corresponding number of seconds based on the number of funny points in the message:

Press Ctrl + C to copy the code Press Ctrl + C to copy the code

 

Round Robin Distribution

One advantage of working queue is that it can process queues in parallel. If many tasks are accumulated, we only need to add more workers. The extension is very simple.

Now, we first start two receivers, wait for receiving messages, and then start a sender to start sending messages.

 

Five messages are sent under the cmd condition. the comma after each message indicates the duration of the message to be executed to simulate time-consuming operations.

Then we can see that the two receivers receive the sent messages in sequence:

 

By default, RabbitMQ distributes each message to the next consumer in sequence. Therefore, the number of messages received by each consumer is roughly average. This method of message distribution is called round robin ).

3.4 message response

When processing a time-consuming task, you may want to know if the consumer is half running. In the current Code, when RabbitMQ sends a message to the consumer (consumers), the message will be immediately removed from the queue. At this time, if the worker that processes the message is stopped, the message being processed will be lost. At the same time, all unprocessed messages sent to this worker will be lost.

We do not want to lose any task message. If a worker crashes, we hope the message will be sent to other workers again ).

To prevent message loss, RabbitMQ provides a message response (acknowledgments)Mechanism. The consumer will use an ack (response) to tell RabbitMQ that a message has been received and processed, and then RabbitMQ will release and delete the message.

If the consumer fails and no response is sent, RabbitMQ considers that the message is not completely processed and resends it to other consumers ). In this way, even if workers occasionally fails, messages will not be lost.

There is no timeout concept for a message. When a worker is disconnected from the message, RabbitMQ resends the message. In this way, there will be no problems when processing a very long message task.

Message response is enabled by default. In the previous example, no_ack = True is used to disable it. It is time to remove this identifier. When the worker completes the task, it sends a response.

channel.BasicConsume("hello", false, consumer);while (true){    var ea = (BasicDeliverEventArgs)consumer.Queue.Dequeue();    var body = ea.Body;    var message = Encoding.UTF8.GetString(body);    int dots = message.Split('.').Length - 1;    Thread.Sleep(dots * 1000);    Console.WriteLine("Received {0}", message);    Console.WriteLine("Done");    channel.BasicAck(ea.DeliveryTag, false);}

 

Now, we can ensure that even if the worker that is processing the message is stopped, the messages will not be lost, and all the messages that are not answered will be resold to other workers.

A common mistake is to forget the BasicAck method, which is common but has serious consequences. when the client exits, the messages to be processed will be re-distributed, but RabitMQ will consume more and more memory, because these messages that are not answered cannot be released. To debug this case, you can use rabbitmqct to print the messages_unacknoledged field.

rabbitmqctl list_queues name messages_ready messages_unacknowledgedListing queues ...hello    0       0...done.

 

3.5 message persistence

The preceding figure shows that even if the consumer goes down, the task will not be lost. However, if the RabbitMQ Server stops, the messages will still be lost.

When the RabbitMQ Server is disabled or crashed, the queues and messages stored in the Server are not saved by default. If you want RabbitMQ to save messages, you need to set them in two places at the same time: ensure that the queue and messages are both persistent.

First, make sure that RabbitMQ does not lose the queue, so do the following settings:

bool durable = true;channel.QueueDeclare("hello", durable, false, false, null);

 

Although the syntax is correct, it is incorrect in the current phase because we have defined a non-persistent hello queue. RabbitMQ does not allow us to use different parameters to redefine an existing queue with the same name. If this is done, an error is returned. Now, define another queue with different names:

bool durable = true;channel.queueDeclare("task_queue", durable, false, false, null);

 

QueueDeclare must be set at both the sender and receiver.

Now it is ensured that the Message Queue task_queue will not be lost even after the RabbitMQ Server is restarted. Then you need to ensure that the message is also persistent, which can be achieved by setting IBasicProperties. SetPersistent to true:

var properties = channel.CreateBasicProperties();properties.SetPersistent(true);

 

It should be noted that setting the message to persistence does not completely ensure that the message is not lost. Although he told RabbitMQ to save the message to the disk, there is still a small time window between receiving the message from RabbitMQ and saving it to the disk. RabbitMQ may only save the message to the cache and does not write it to the disk. Persistence cannot be guaranteed, but it is sufficient for a simple task queue. To ensure message queue persistence, you can use publisher confirms

3.6 fair distribution

You may notice that the distribution of messages may not be as fair as we want. For example, for two workers. When tasks with an odd number of messages are heavy but even number of message tasks are light, the odd number of workers always process the busy state, while the even number of workers always process the idle state. But RabbitMQ does not know this, and it still evenly distributes messages.

To change this state, we can use the basicQos method to set perfetchCount = 1. This tells RabbitMQ not to send more than one message to a worker at the same time, or in other words. Do not distribute new messages to a worker before the worker processes the message and does not respond to the message. Instead, send the new message to the next less busy worker.

channel.BasicQos(0, 1, false); 

 

3.7 complete instance

Now put all these together:

The sending code is as follows:

static void Main(string[] args){    var factory = new ConnectionFactory();    factory.HostName = "localhost";    factory.UserName = "yy";    factory.Password = "hello!";    using (var connection = factory.CreateConnection())    {        using (var channel = connection.CreateModel())        {                               bool durable = true;            channel.QueueDeclare("task_queue", durable, false, false, null);                                string message = GetMessage(args);            var properties = channel.CreateBasicProperties();            properties.SetPersistent(true);                              var body = Encoding.UTF8.GetBytes(message);            channel.BasicPublish("", "task_queue", properties, body);            Console.WriteLine(" set {0}", message);        }    }    Console.ReadKey();}private static string GetMessage(string[] args){    return ((args.Length > 0) ? string.Join(" ", args) : "Hello World!");}

 

The acceptor code is as follows:

static void Main(string[] args){    var factory = new ConnectionFactory();    factory.HostName = "localhost";    factory.UserName = "yy";    factory.Password = "hello!";    using (var connection = factory.CreateConnection())    {        using (var channel = connection.CreateModel())        {            bool durable = true;            channel.QueueDeclare("task_queue", durable, false, false, null);            channel.BasicQos(0, 1, false);            var consumer = new QueueingBasicConsumer(channel);            channel.BasicConsume("task_queue", false, consumer);            while (true)            {                var ea = (BasicDeliverEventArgs)consumer.Queue.Dequeue();                var body = ea.Body;                var message = Encoding.UTF8.GetString(body);                int dots = message.Split('.').Length - 1;                Thread.Sleep(dots * 1000);                Console.WriteLine("Received {0}", message);                Console.WriteLine("Done");                channel.BasicAck(ea.DeliveryTag, false);            }        }    }}

 

4. Management Interface

The RabbitMQ management interface allows you to view the current status of the RabbitMQ Server. This interface is provided in the form of a plug-in and has provided this plug-in when installing RabbitMQ. You need to enable this plug-in the RabbitMQ console interface. The command is as follows:

rabbitmq-plugins enable rabbitmq_management

Enter http: // in the browser ://Server-name: 15672/server-name is changed to the machine address or domain name. If it is local, use localhost directly (the port number of the version earlier than RabbitMQ 3.0 is 55672). After the input, the logon interface is displayed, use the user we created before to log on.

.

You can view all the statuses of the current RabbitMQServer.

5. Summary

This article briefly introduces the concepts of message queue, and introduces the basic principles of RabbitMQ message proxy, as well as how to install RabbitMQ on Windows and how to use RabbitMQ in. NET. Message Queue plays an important role in building a distributed system and improving system scalability and responsiveness. I hope this article will help you understand message queues and how to use RabbitMQ.