RABBITMQ Guide (C #) (ii) Work queue

In the previous section, we implemented sending and receiving messages to the specified queue. In this section, we mainly talk about work queues, which are used to configure real-time tasks between multiple consumers.

The work queue approach is primarily to prevent the execution of a resource-intensive task by waiting for it to end before other things can be handled. We delay the execution of the task, encapsulate it as a message, and send it to a queue. Run a program in the background to remove the message from the queue and then perform the task. If you have multiple consumers, you can also share tasks.

For Web applications, this makes it possible to use short requests for HTTP to handle complex business.

Get ready

We send a string to represent complex tasks, and then use Thread.Sleep () to simulate time-consuming operations, using the number of points to represent the complexity of a task, a point that represents a second kind of work, such as Hello ... Represents a three-second task.

Modify the code for Send.cs in the previous section example, allowing you to enter any message from the command and send it to the work queue, which we named NewTask.cs.

var message = GetMessage (args); var body = Encoding.UTF8.GetBytes (message); var properties = Channel. Createbasicproperties ();p roperties. Setpersistent (true"",                     "task_queue"  ,                     basicproperties:properties,                     body:body);

GetMessage method gets the input message

Private Static string GetMessage (string[] args) {    return0string. Join ("" "Hello world! " );}

Modify the Receive.cs in the previous section to represent one second per point to simulate time-consuming tasks, process messages from RABBITMQ and perform tasks, and we'll name them Worker.cs

varConsumer =NewEventingbasicconsumer (channel); consumer. Received+ = (model, ea) = ={    varBODY =ea.    Body; varMessage =Encoding.UTF8.GetString (body); Console.WriteLine ("[x] Received {0}", message); intDots = message. Split ('.'). Length-1; Thread.Sleep (Dots* +); Console.WriteLine ("[x] done");}; Channel. Basicconsume (Queue:"Task_queue", Noack:true, Consumer:consumer);

Simulate when a task executes

int dots = message. Split ('. ' 1  );

Compile run

Cyclic scheduling

The benefit of using a work queue is that it is easy to achieve split work. If there is a backlog of work to be done, you just need to run multiple workers.

First, run two console programs to execute Worker.cs, and they all fetch messages from the queue. These two console programs are the message, respectively, C1,C2.

Run a console program to perform the NewTask.cs release task. After starting the consumer, in the control input the following message content is sent to the queue:

shell3$ NewTask.exe First message.shell3$ NewTask.exe Second message: shell3$ NewTask.exe Third message...shell3$ NewTask.exe fourth message....shell3$ NewTask.exe Fifth message .....

Then take a look at what the job is receiving:

C1

 for messages. To exit Press CTRL+'firstmessage. '  'third message ... '  'Fifth message ..... '

C2

 for messages. To exit Press CTRL+'Second message . '  'Fourth message .... '

RABBITMQ By default sends the message sequentially to each consumer, each consumer averages the same number of messages, and this distributed message is called round robin scheduling. You can also add more consumers.

Message Confirmation

It takes some time to perform a task, so you might consider what to do if a consumer is performing a time-consuming task that suddenly crashes. Our current code, once RABBITMQ sends a message to the consumer, the consumption will be deleted immediately. In this case, if a worker is aborted, the message being processed is lost, and the message that the current consumer has received but not yet processed is lost.

But we don't want to lose any tasks, and if one worker stops, we want the task to be sent to other workers.

To ensure that messages are not lost, RABBITMQ provides a message acknowledgement mechanism (acknowledgments). When the message is received and processed, the consumer sends an ACK to RABBITMQ, telling it that it is free to delete the message.

If the consumer stops and does not send an ACK acknowledgement, then RABBITMQ will think the message has not been processed and it will re-queue the consumption. If there are other consumers, it will immediately send the message to another consumer to deal with. This will not cause the message to be lost even if the consumer stops occasionally.

None of the messages will time out, and if the consumer crashes, RABBITMQ will resend the message, even if the message processing takes a long time to send.

The message acknowledgement mechanism is on by default. The preceding example closes the message acknowledgement by setting Noack to True. Now set Noack to false, and the worker sends a confirmation message when the task execution is complete.

varConsumer =NewEventingbasicconsumer (channel); consumer. Received+ = (model, ea) = ={    varBODY =ea.    Body; varMessage =Encoding.UTF8.GetString (body); Console.WriteLine ("[x] Received {0}", message); intDots = message. Split ('.'). Length-1; Thread.Sleep (Dots* +); Console.WriteLine ("[x] done"); Channel. Basicack (Deliverytag:ea. Deliverytag, multiple:false);}; Channel. Basicconsume (Queue:"Task_queue", Noack:false, Consumer:consumer);

The upper code ensures that the worker is crashing when the message is being processed, and that no messages will be lost. All unacknowledged messages will be resent.

Message persistence

We already know how to confirm that when a worker crashes, the task is not lost. But if RABBITMQ also stops, the task will also be lost.

If RABBITMQ exits or crashes, the queue and messages are cleared by default. To ensure that messages are not lost, you need to configure both queues and messages to persist.

First, ensure that the RABBITMQ does not lose the queue, which requires declaring the queue as a persistent queue

" Hello " ,                      true ,                      false ,                      false ,                      null);

Syntactically speaking, there is nothing wrong with the above code, but this does not work because we have previously declared a queue named Hello, and RABBITMQ does not allow a different parameter to redefine the existing queue, which throws an exception. So change the queue name, like Task_queue

" Task_queue " ,                      true ,                      false ,                      false ,                      null);

Queuedeclare modifications are to be configured on both sides of the producer and consumer.

After you set up the queue persistence, you need to set the message to persisted.

var properties = Channel. Createbasicproperties ();p roperties. Setpersistent (true);

Balanced scheduling

You may have noticed that the current distribution mechanism is still unsatisfactory. For example, if there are two workers, when an odd number of messages is complex, and even messages are simple, one worker is very busy, and another worker has little to do. But RABBITMQ does not know this, and will still distribute the task evenly.

Because whenever a message enters the queue, RABBITMQ will distribute the message. It does not check the number of unacknowledged messages for each consumer. It simply sends the nth message to the nth consumer in a blind.

To prevent this, use the Basicqos method and set the value of the Prefetchcount parameter to 1. This way, RABBITMQ will not send multiple messages to the same worker at the same time. In other words, new messages are not distributed to workers until the worker processes and confirms the previous message. It sends the message to the next worker who is not busy.

Channel. Basicqos (01false);

Full code

NewTask.cs Code

usingSystem;usingrabbitmq.client;usingSystem.Text;classnewtask{ Public Static voidMain (string[] args) {        varFactory =NewConnectionFactory () {HostName ="localhost" }; using(varConnection =Factory. CreateConnection ())using(varChannel =connection. Createmodel ()) {channel. Queuedeclare (Queue:"Task_queue", Durable:true, Exclusive:false, Autodelete:false, arguments:NULL); varMessage =GetMessage (args); varBODY =Encoding.UTF8.GetBytes (message); varProperties =Channel.            Createbasicproperties (); Properties. Setpersistent (true); Channel. Basicpublish (Exchange:"", Routingkey:"Task_queue", Basicproperties:properties, body:body); Console.WriteLine ("[x] Sent {0}", message); } Console.WriteLine ("Press [Enter] to exit.");    Console.ReadLine (); }    Private Static stringGetMessage (string[] args) {        return(args. Length >0) ?string. Join (" ", args):"Hello world!"); }}

Worker.cs Code

usingSystem;usingrabbitmq.client;usingRabbitMQ.Client.Events;usingSystem.Text;usingSystem.Threading;classworker{ Public Static voidMain () {varFactory =NewConnectionFactory () {HostName ="localhost" }; using(varConnection =Factory. CreateConnection ())using(varChannel =connection. Createmodel ()) {channel. Queuedeclare (Queue:"Task_queue", Durable:true, Exclusive:false, Autodelete:false, arguments:NULL); Channel. Basicqos (prefetchsize:0, Prefetchcount:1,Global:false); Console.WriteLine ("[*] waiting for messages."); varConsumer =NewEventingbasicconsumer (channel); Consumer. Received+ = (model, ea) = =            {                varBODY =ea.                Body; varMessage =Encoding.UTF8.GetString (body); Console.WriteLine ("[x] Received {0}", message); intDots = message. Split ('.'). Length-1; Thread.Sleep (Dots* +); Console.WriteLine ("[x] done"); Channel. Basicack (Deliverytag:ea. Deliverytag, multiple:false);            }; Channel. Basicconsume (Queue:"Task_queue", Noack:false, Consumer:consumer); Console.WriteLine ("Press [Enter] to exit.");        Console.ReadLine (); }    }}

The work queue can be established using the message acknowledgement mechanism and basicqos, and the persistence option guarantees that the task will not be lost even if the RABBITMQ is restarted.

RABBITMQ Guide (C #) (ii) Work queue