Using RABBITMQ in a. NET environment

In the field of enterprise application system, the communication, integration and integration between different systems will be faced, especially when faced with heterogeneous systems, this kind of distributed call and communication becomes more and more important. Secondly, the system generally will have a lot of real-time requirements are not high but the implementation of more time-consuming place, such as sending text messages, e-mail reminders, update article reading count, record user operation log and so on, if the real-time processing, in the case of a large number of users access, the system pressure is larger.

In the face of these problems, we will generally put these requests in the Message queue processing, heterogeneous systems use messages to communicate. Message delivery seems to be better than file delivery and remote procedure call (RPC) because it has better platform independence and is well-supported for concurrent and asynchronous calls. So if the following occurs in the system:

• The real-time requirements of the operation are not high, and the tasks that need to be performed are very time consuming;
• There is integration between heterogeneous systems;

It is generally possible to consider introducing Message Queuing. In the first case, Message Queuing is often selected to handle long-performing tasks. The incoming message queue becomes a buffer for message processing. The asynchronous communication mechanism introduced by Message Queuing allows both the sender and the receiver to continue executing the following code without waiting for the other party to return a successful message, thereby improving the ability to process data. Especially when the traffic volume and data flow are large, it can reduce the load of database processing data by combining the message queue with the background task, and processing the big data by avoiding the peak times.

In a previous article explaining the CQRS pattern, all changes to the state of the system are done through events, typically storing events in the message queue and then processing them uniformly.

This article briefly describes the message Agent tool in RABBITMQ, as well as in the. NET, how to use RABBITMQ.

An environment construction

First, because RABBITMQ is written using Erlang and needs to run on the Erlang runtime environment, you need to install the Erlang Runtime environment before installing RABBITMQ server, and you can download the installation files for the corresponding platform on the Erlang website. If you do not install the runtime environment, you will be prompted to install the Erlang environment before installing RABBITMQ server. After the installation is complete, ensure that the installation path for Erlang is registered to the environment variables of the system. After Erlang is installed, this environment is automatically set, if not, in the administrator environment under the console input, you can also set:

Setx ERLANG_HOME “D: \ Program Files (x86) \ erl6.3 ″
Then, go to the RabbitMQ official website to download the RabbitMQ Server server program and select the appropriate platform version to download. After the installation is complete, you can start using it.

Now you can configure RabbitMQ Server.

First, switch to the installation directory of RabbitMQ Server:

There are many batch files under sbin to control RabbitMQ Server. Of course, you can also perform the corresponding operations directly in the installation start menu:

The easiest way is to make RabbitMQ run in the background in the form of Windows Service, so we need to open cmd with administrator privileges, then switch to the sbin directory and execute these three commands:

rabbitmq-service install
rabbitmq-service enable
rabbitmq-service start
Now the RabbitMQ server has been started.

You can use the rabbitmqctl.bat script under the sbin directory to view and control the server status. Run rabbitmqctl status directly in cmd. If you see the following result:

It shows that the node is not connected, you need to go to the C: \ Windows directory, copy the .erlang.cookie file to the user directory C: \ Users \ {user name}, this is the Erlang cookie file, allowing interaction with Erlang, Now repeat the previous command and you will get the following information:

The RabbitMQ Server also has the user concept. After installation, use the rabbitmqctl list_users command to see the current users above:

As you can see, there is only one user named guest with the role of administrator. This is created by RabbitMQ by default. He has all the permissions of RabbitMQ. Generally, we need to create a new user, set a password, and grant Permissions, and set it as an administrator, you can use the following command to perform this operation:

rabbitmqctl add_user yy hello!
rabbitmqctl set_permissions yy ". *" ". *" ". *"
rabbitmqctl set_user_tags yy administrator
The above command added a user named yy and set the password hello! The following command grants user yy configuration, read, and write permissions for all message queues, respectively.

Now we can delete the default guest user, just use the following command:

rabbitmqctl delete_user guest
If you want to change the password, you can use the following command:

rabbitmqctl change_password {username} {newpassowrd}
2 Get started
To use RabbitMQ in .NET, you need to download the RabbitMQ client assembly. You can download it from the official website. After downloading and decompressing, you can get RabbitMQ.Client.dll, which is the RabbitMQ client.

Before using RabitMQ, you need to explain the following basic concepts:

RabbitMQ is a message broker. He receives messages from message producers and sends them to message consumers. Between sending and receiving, he can route, cache and persist according to the set rules.

Generally speaking, RabbitMQ and news use some proper nouns.

Producing means sending. The program that sends the message is a producer. We generally use "P" to express:
The queue is the name of the mailbox. Messages are transmitted through your application and RabbitMQ, and they can only be stored in a queue. There is no limit to the capacity of the queue. You can store as many messages as you like-basically an infinite buffer. Multiple producers can send messages to the same queue. Similarly, multiple consumers can also get data from the same queue. The queue can be drawn like this (the name of the queue on the picture):
Consuming is the same as getting news. A consumer is a program that waits for messages. We draw it as "C":
Usually, the message producer, message consumer and message broker are not on the same machine.

2.1 Hello World
To demonstrate the basic usage of RabbitMQ, we send a HelloWorld message, and then receive and process it.

First create a console program to send messages to RabbitMQ's message queue, the code is as follows:

    static void Main (string [] args)
    {
        var factory = new ConnectionFactory ();
        factory.HostName = "localhost";
        factory.UserName = "yy";
        factory.Password = "hello!";

        using (var connection = factory.CreateConnection ())
        {
            using (var channel = connection.CreateModel ())
            {
                channel.QueueDeclare ("hello", false, false, false, null);
                string message = "Hello World";
                var body = Encoding.UTF8.GetBytes (message);
                channel.BasicPublish ("", "hello", null, body);
                Console.WriteLine ("set {0}", message);
            }
        }
    }
First, you need to create a ConnectionFactory and set the target. Because it is on the local machine, set it to localhost. If RabbitMQ is not on the local machine, you only need to set the IP address or machine name of the target machine, and then set the user name yy and Password hello! .

Next, you need to create a Channel. If you want to send a message, you need to create a queue and then publish the message to this queue. When creating a queue, it will only be created if the queue does not exist on RabbitMQ. The message is transmitted in the form of a binary array, so if the message is a physical object, it needs to be serialized and then converted into a binary array.

Now the client sending code has been written. After running, the message will be published to RabbitMQ's message queue. Now we need to write the server code to connect to RabbitMQ to get these messages.

Similarly, create a server console application called Receive, the server code is as follows:

static void Main (string [] args)
{
    var factory = new ConnectionFactory ();
    factory.HostName = "localhost";
    factory.UserName = "yy";
    factory.Password = "hello!";

    using (var connection = factory.CreateConnection ())
    {
        using (var channel = connection.CreateModel ())
        {
            channel.QueueDeclare ("hello", false, false, false, null);

            var consumer = new QueueingBasicConsumer (channel);
            channel.BasicConsume ("hello", true, consumer);

            Console.WriteLine ("waiting for message.");
            while (true)
            {
                var ea = (BasicDeliverEventArgs) consumer.Queue.Dequeue ();

                var body = ea.Body;
                var message = Encoding.UTF8.GetString (body);
                Console.WriteLine ("Received {0}", message);

            }
        }
    }
}
As with sending, you first need to define the connection and then declare the message queue. To receive messages, you need to define a Consume, and then continuously Dequeue messages from the message queue and then process them.

Now the code of the sender and the receiver is written, run the sender, and send the message:

Now, a message is sent to the message queue named hello. This message is stored on the RabbitMQ server. Use rabbitmqctl's list_queues to view all the message queues and the number of messages in it. You can see that there is currently only one message queue on Rabbitmq and only one message:

D: \ Program Files \ RabbitMQ Server \ rabbitmq_server-3.4.2 \ sbin> rabbitmqctl list_queues
Listing queues ...
hello 1
Now run the receiver program as follows:

You can see that the Hello World sent by the client has been accepted. Now let's look at the message queue information on RabitMQ:

D: \ Program Files \ RabbitMQ Server \ rabbitmq_server-3.4.2 \ sbin> rabbitmqctl list_queues
Listing queues ...
hello 0
It can be seen that the number of message queues in the hello queue is 0, which means that when the receiving end receives the message, RabbitMQ deletes the message.

2.2 Work queue
The previous example shows how to send and receive messages to a specified message queue. Now we create a work queue to distribute some time-consuming tasks to multiple workers:

The main idea of work queues (also called task queues) is to avoid immediate execution and wait for some operations that take up a lot of resources and time to complete. Instead, the task is sent to the queue as a message and processed later. A worker process running in the background will take out the task and process it. When running multiple workers, tasks are shared among them.

This is very useful in network applications, it can handle some complex tasks in a short HTTP request. In some places where real-time requirements are not too high, we can process other insignificant operations, such as writing logs, etc., after processing main operations.

ready

In the first part, a string message containing "Hello World!" Was sent. Now send some strings and treat them as complex tasks. The time.sleep () function is used here to simulate time-consuming tasks. Add a dot (.) To the string to indicate the complexity of the task. A dot (.) Will take 1 second. For example, "Hello ..." will take 3 seconds.

Make some simple adjustments to send.cs in the previous example so that you can send random messages. This program will send tasks to our work queue as planned.

static void Main (string [] args)
{
    var factory = new ConnectionFactory ();
    factory.HostName = "localhost";
    factory.UserName = "yy";
    factory.Password = "hello!";

    using (var connection = factory.CreateConnection ())
    {
        using (var channel = connection.CreateModel ())
        {
            channel.QueueDeclare ("hello", false, false, false, null);
            string message = GetMessage (args);
            var properties = channel.CreateBasicProperties ();
            properties.DeliveryMode = 2;

            var body = Encoding.UTF8.GetBytes (message);
            channel.BasicPublish ("", "hello", properties, body);
            Console.WriteLine ("set {0}", message);
        }
    }

    Console.ReadKey ();
}

private static string GetMessage (string [] args)
{
    return ((args.Length> 0)? string.Join ("", args): "Hello World!");
}
The bold part has been revised.

Then we modify the receiving end, so that he can sleep corresponding to the number of seconds according to the number of commas in the message:

static void Main (string [] args)
{
    var factory = new ConnectionFactory ();
    factory.HostName = "localhost";
    factory.UserName = "yy";
    factory.Password = "hello!";

    using (var connection = factory.CreateConnection ())
    {
        using (var channel = connection.CreateModel ())
        {
            channel.QueueDeclare ("hello", false, false, false, null);

            var consumer = new QueueingBasicConsumer (channel);
            channel.BasicConsume ("hello", true, consumer);

            while (true)
            {
                var ea = (BasicDeliverEventArgs) consumer.Queue.Dequeue ();

                var body = ea.Body;
                var message = Encoding.UTF8.GetString (body);

                int dots = message.Split (‘.‘). Length-1;
                Thread.Sleep (dots * 1000);
                        
                Console.WriteLine ("Received {0}", message);
                Console.WriteLine ("Done");
            }
        }
    }
}
Polling distribution

One benefit of using a work queue is that it can process queues in parallel. If there are many tasks stacked, we only need to add more workers (workers), the expansion is very simple.

Now, we start two receiving ends, wait for receiving messages, and then start a sending end to start sending messages.

 

Under the cmd condition, 5 messages were sent. The comma after each message indicates the length of time the message needs to be executed to simulate a time-consuming operation.

Then you can see that the two receiving ends received the sent messages in sequence:

 

By default, RabbitMQ will distribute each message to the next consumer in order. Therefore, the number of messages received by each consumer is roughly average. This method of message distribution is called round-robin.

2.3 Message response
When dealing with a time-consuming task, you might want to know whether the consumer (consumers) halted halfway through. In the current code, when RabbitMQ sends a message to consumers, it will immediately remove the message from the queue. At this time, if the worker processing the message is stopped, the message being processed will be lost. At the same time, all unsent messages sent to this worker will be lost.

We don't want to lose any task messages. If a worker dies, we hope that the message will be resent to other workers.

In order to prevent the loss of messages, RabbitMQ provides a message response (acknowledgments) mechanism. The consumer will tell RabbitMQ that a message has been received and processed through an ack (response), and then RabbitMQ will release and delete the message.

If the consumer hangs up and does not send a response, RabbitMQ will think that the message has not been completely processed, and then resend it to other consumers. In this way, even if the workers (workers) hang up occasionally, the message will not be lost.

The message does not have the concept of timeout; when the worker disconnects from it, RabbitMQ will resend the message. In this way, there will be no problem when processing a very long message task.

Message response is enabled by default. In the previous example, no_ack = True was used to turn it off. It's time to remove this flag, and when the worker completes the task, a response is sent.

channel.BasicConsume ("hello", false, consumer);

while (true)
{
    var ea = (BasicDeliverEventArgs) consumer.Queue.Dequeue ();

    var body = ea.Body;
    var message = Encoding.UTF8.GetString (body);

    int dots = message.Split (‘.‘). Length-1;
    Thread.Sleep (dots * 1000);

    Console.WriteLine ("Received {0}", message);
    Console.WriteLine ("Done");

    channel.BasicAck (ea.DeliveryTag, false);
}
Now, it can be guaranteed that even if the worker who is processing the message is stopped, these messages will not be lost, and all unanswered messages will be resent to other workers.

A very common mistake is to forget the BasicAck method. This error is very common, but the consequences are very serious. When the client exits, the pending messages will be redistributed, but RabitMQ will consume more and more memory, because These unanswered messages cannot be released. To debug this case, you can use rabbitmqct to print the messages_unacknoledged field.

rabbitmqctl list_queues name messages_ready messages_unacknowledged
Listing queues ...
hello 0 0
... done.
2.4 Message Persistence
The task has not been lost even if the consumer is down, but if the RabbitMQ Server is stopped, these messages will still be lost.

When RabbitMQ Server shuts down or crashes, the queues and messages stored in it will not be saved by default. If you want RabbitMQ to save the message, you need to set it in two places at the same time: you need to ensure that the queue and the message are persistent.

First of all, to ensure that RabbitMQ will not lose the queue, so do the following settings:

bool durable = true;
channel.QueueDeclare ("hello", durable, false, false, null);
Although it is syntactically correct, it is incorrect at this stage because we have previously defined a non-persistent hello queue. RabbitMQ does not allow us to redefine an existing queue of the same name with different parameters. If you do so, you will get an error. Now, define another queue with a different name:

bool durable = true;
channel.queueDeclare ("task_queue", durable, false, false, null);
queueDeclare This change needs to be set on both the sending end and the receiving end.

It is now guaranteed that the task_queue message queue will not be lost even after the RabbitMQ Server restarts. Then you need to ensure that the message is also persistent, which can be achieved by setting IBasicProperties.SetPersistent to true:

var properties = channel.CreateBasicProperties ();
properties.SetPersistent (true);
It should be noted that setting the message to persistent does not completely guarantee that the message will not be lost. Although he told RabbitMQ to save the message to disk, there is still a small time window between when RabbitMQ receives the message and saves it to disk. RabbitMQ may just save the message to the cache, and not write it to disk. Persistence is not guaranteed, but it is sufficient for a simple task queue. If you need a strong guarantee for message queue persistence, you can use publisher confirms

2.5 Fair distribution
You may notice that the distribution of messages may not be distributed fairly as we want. For example, for two workers. When the task of odd number of messages is heavy, but the task of even number of messages is light, the odd number of workers always handle the busy state, and the even number of workers always handle the idle state. But RabbitMQ doesn't know this, he will still distribute the messages in order.

To change this state, we can use the basicQos method and set perfetchCount = 1. This tells RabbitMQ not to send more than one message to a worker at the same time, or in other words. Do not distribute new messages to a worker until the worker is still processing the message and there is no response to the message. Instead, send this new message to the next less busy worker.

channel.BasicQos (0, 1, false);
2.6 Complete example
Now put all these together:

The sender code is as follows:

static void Main (string [] args)
{
    var factory = new ConnectionFactory ();
    factory.HostName = "localhost";
    factory.UserName = "yy";
    factory.Password = "hello!";

    using (var connection = factory.CreateConnection ())
    {
        using (var channel = connection.CreateModel ())
        {
bool durable = true;
            channel.QueueDeclare ("task_queue", durable, false, false, null);
                    
            string message = GetMessage (args);
            var properties = channel.CreateBasicProperties ();
            properties.SetPersistent (true);
                  

            var body = Encoding.UTF8.GetBytes (message);
            channel.BasicPublish ("", "task_queue", properties, body);
            Console.WriteLine ("set {0}", message);
        }
    }

    Console.ReadKey ();
}

private static string GetMessage (string [] args)
{
    return ((args.Length> 0)? string.Join ("", args): "Hello World!");
}
The receiving end code is as follows:

static void Main (string [] args)
{
    var factory = new ConnectionFactory ();
    factory.HostName = "localhost";
    factory.UserName = "yy";
    factory.Password = "hello!";

    using (var connection = factory.CreateConnection ())
    {
        using (var channel = connection.CreateModel ())
        {
            bool durable = true;
            channel.QueueDeclare ("task_queue", durable, false, false, null);
            channel.BasicQos (0, 1, false);

            var consumer = new QueueingBasicConsumer (channel);
            channel.BasicConsume ("task_queue", false, consumer);

            while (true)
            {
                var ea = (BasicDeliverEventArgs) consumer.Queue.Dequeue ();

                var body = ea.Body;
                var message = Encoding.UTF8.GetString (body);

                int dots = message.Split (‘.‘). Length-1;
                Thread.Sleep (dots * 1000);

                Console.WriteLine ("Received {0}", message);
                Console.WriteLine ("Done");

                channel.BasicAck (ea.DeliveryTag, false);
            }
        }
    }
}
Three management interface
RabbitMQ also has a management interface through which you can view the current status of RabbitMQ Server. This interface is provided in the form of a plug-in, and the plug-in is already provided when RabbitMQ is installed. What needs to be done is to enable the plugin in the RabbitMQ console interface, the command is as follows:

rabbitmq-plugins enable rabbitmq_management
Now, enter http: // server-name: 15672 / server-name into the machine address or domain name in the browser. If it is local, use localhost directly (RabbitMQ 3.0 version port number is 55672). After input, pop up Login interface, log in with the user we created earlier.

 .

You can see all the current status of RabbitMQServer on this interface.

Four summary
This article briefly introduces the related concepts of message queue, and introduces the basic principle of RabbitMQ message broker and how to install RabbitMQ on Windows and how to use RabbitMQ in .NET. Message queuing plays an important role in building a distributed system and improving the scalability and responsiveness of the system. I hope this article will help you understand message queuing and how to use RabbitMQ.

5 References
http://www.infoq.com/cn/articles/message-based-distributed-architecture
http://www.rabbitmq.com/getstarted.html
http://www.codethinked.com/using-rabbitmq-with-c-and-net
http://www.infoq.com/cn/articles/AMQP-RabbitMQ
http://www.infoq.com/cn/articles/ebay-scalability-best-practices
Using RabbitMQ in .NET environment