RABBITMQ Example Tutorial: Working with Java to get work queues done

In the previous section, we learned to send and receive a well-named queue programmatically. In this section, we will use the Task force to distribute tasks among multiple workers.

The core idea of the task force is to avoid immediate processing of tasks that must wait for the high density to be completed. It takes a task-based approach, encapsulates a task into a message and puts it in a queue. A worker process running in the background will then pop it up and execute it so that the tasks in the task queue will be executed by the worker process share.

Work queues apply to scenarios in a web app that handle complex tasks in a short HTTP request.

In the previous section, we sent a "Hello world! "String message. Sending multiple string messages now represents complex tasks. We now like the picture reset size, rendering the PDF file such a real task, but we use Thread.Sleep () pretend to be busy with us. We use the number of points in the string as their complexity, and each point takes 1 seconds to "work". For example, a dummy task that contains "..." will take three seconds.

650) this.width=650; "src=" Http://s3.51cto.com/wyfs02/M01/74/41/wKioL1YXaKyzqoSBAAHnTX0wnj0610.jpg "title=" Workqueue.png "alt=" Wkiol1yxakyzqosbaahntx0wnj0610.jpg "/>

　　Newtask.java

package com.favccxx.favrabbit;import com.rabbitmq.client.channel;import  com.rabbitmq.client.connection;import com.rabbitmq.client.connectionfactory;import  Com.rabbitmq.client.messageproperties;public class newtask {  private static  final String TASK_QUEUE_NAME =  "Task_queue";   public static void  main (STRING[]&NBSP;ARGV)  throws exception {    connectionfactory  factory = new connectionfactory ();     factory.sethost ("localhost");     connection connection = factory.newconnection ();     channel channel = connection.createchannel ();     channel.queuedeclare (task_queue_name, true, false, false, null);     string[] args  = {"Shuai ge", "AI", "Meinv", "..."};&NBSP;&NBSP;&NBSP;&NBsp String message = getmessage (args);     channel.basicpublish ("",  TASK_ queue_name,        messageproperties.persistent_text_plain,         message.getbytes ("UTF-8"));     System.out.println (" [x] Sent " " + message + " ");         for (int i=0;i<10;i++) {    channel.basicpublish ("",  task_queue_name,             messageproperties.persistent_text_plain,              (message+i) getBytes ("UTF-8");     system.out.println ("Sent message:"  +  message+i);     }    channel.close ();     Connection.close ();   }  private static&nbSp String getmessage (string[] strings)  {    if  (strings.length <  1)       return  "hello world!";     return joinstrings (strings,  " ");  }  private  Static string joinstrings (String[] strings, string delimiter)  {     int length = strings.length;    if  (length == 0)  return  "";     stringbuilder words = new stringbuilder ( Strings[0]);    for  (int i = 1; i < length; i++ )  {      words.append (delimiter). Append (Strings[i]);     }    return words.tostring ();   }}

　　Console output

[x] Sent ' Shuai Ge ai meinv ... ' Sent Message:shuai Ge ai Meinv ... 0 Sent Message:shuai Ge ai Meinv ... 1 Sent Message:shuai Ge ai Meinv ... 2 Sent Message:shuai Ge ai Meinv ... 3 Sent Message:shuai Ge ai Meinv ... 4 Sent Message:shuai Ge ai Meinv ... 5 Sent Message:shuai Ge ai Meinv ... 6 Sent Message:shuai Ge ai Meinv ... 7 Sent Message:shuai Ge ai Meinv ... 8 Sent Message:shuai Ge ai Meinv ... 9

　　Worker.java

/strong>

Package com.favccxx.favrabbit;import java.io.ioexception;import java.text.dateformat;import  java.text.SimpleDateFormat;import java.util.Date;import com.rabbitmq.client.AMQP;import  com.rabbitmq.client.channel;import com.rabbitmq.client.connection;import  com.rabbitmq.client.connectionfactory;import com.rabbitmq.client.consumer;import  Com.rabbitmq.client.defaultconsumer;import com.rabbitmq.client.envelope;public class worker  {private static final String TASK_QUEUE_NAME =  "Task_queue";p rivate  Static dateformat df = new simpledateformat ("Yyyy-mm-dd hh:mm:ss");p ublic  Static void main (STRING[] ARGV)  throws exception {connectionfactory factory  = new connectionfactory (); Factory.sethost ("localhost"); final connection connection  = factory.newconnection (); final channel channel&Nbsp;= connection.createchannel (); Channel.queuedeclare (Task_queue_name, true, false, false ,  null); System.out.println (" [*] waiting for messages. to exit press ctrl+c"); Channel.basicqos (1); Final consumer consumer = new defaultconsumer (channel)  {@ Overridepublic void handledelivery (STRING CONSUMERTAG, ENVELOPE ENVELOPE, AMQP. Basicproperties properties,byte[] body)  throws IOException {String message  = new string (body,  "UTF-8"); System.out.println (Df.format (New date ())  +  " [x] Received " " + message  +  "'"); try {dowork (message);}  finally {system.out.println (" [x] done"); Channel.basicack (Envelope.getdeliverytag (),  false);}}; Channel.basicconsume (Task_queue_name, false, consumer);} Private static void dowork (string tAsk)  {for  (Char ch : task.tochararray ())  {if  (ch ==  '. ')  {try {thread.sleep (1000);}  catch  (interruptedexception _ignored)  {thread.currentthread (). interrupt ();}}}}

　　Console output

[*] Waiting for messages. To exit Press CTRL + C 2015-10-08 15:41:36 [x] Received ' Shuai Ge ai meinv ... ' [x] Done 2015-10-08 15:41:39 [x] Received ' Shuai Ge ai meinv ... 0 ' [x] Done 2015-10-08 15:41:42 [x] Received ' Shuai Ge ai meinv ... 1 ' [x] Done 2015-10-08 15:41:45 [x] Received ' Shuai Ge ai meinv ... 2 ' [x] Done 2015-10-08 15:41:48 [x] Received ' Shuai Ge ai meinv ... 3 ' [x] Done 2015-10-08 15:41:51 [x] Received ' Shuai Ge ai meinv ... 4 ' [x] Done 2015-10-08 15:41:54 [x] Received ' Shuai Ge ai meinv ... 5 ' [x] Done 2015-10-08 15:41:57 [x] Received ' Shuai Ge ai meinv ... 6 ' [x] Done 2015-10-08 15:42:00 [x] Received ' Shuai Ge ai meinv ... 7 ' [x] Done 2015-10-08 15:42:03 [x] Received ' Shuai Ge ai meinv ... 8 ' [x] Done 2015-10-08 15:42:06 [x] Received ' Shuai Ge ai meinv ... 9 ' [x] Done 2015-10-08 15:42:46 [x] Received ' Shuai Ge ai meinv ... ' [x] Done 2015-10-08 15:42:49 [x] Received ' Shuai Ge ai meinv ... 0 ' [x] Done 2015-10-08 15:42:52 [x] Received ' Shuai Ge ai meinv ... 1 ' [x] Done 2015-10-08 15:42:55 [x] Received ' Shuai Ge ai meinv ... 2 ' [x] Done 2015-10-08 15:42:58 [x] Received ' Shuai Ge ai meinv ... 3 ' [x] Done 2015-10-08 15:43:01 [x] Received ' Shuai Ge ai meinv ... 4 ' [x] Done 2015-10-08 15:43:04 [x] Received ' Shuai Ge ai meinv ... 5 ' [x] Done 2015-10-08 15:43:07 [x] Received ' Shuai Ge ai meinv ... 6 ' [x] Done 2015-10-08 15:43:10 [x] Received ' Shuai Ge ai meinv ... 7 ' [x] Done 2015-10-08 15:43:13 [x] Received ' Shuai Ge ai meinv ... 8 ' [x] Done 2015-10-08 15:43:16 [x] Received ' Shuai Ge ai meinv ... 9 ' [x] Done

　　Cyclic distribution messages (Round-robin dispatching)

One advantage of using the task queue is that it is easy to handle parallel work if we have a backlog of work that can be done by adding more workers.

First, there are now two worker instances working at the same time, and they all read messages from the queue. Then do this:

(1) Run the NewTask class, send 10 message queues, the console output the following content:

[x] Sent ' Shuai ge ai meinv ... ' Sent message:shuai ge ai meinv ... 0Sent Message:shuai Ge ai Meinv ... 1Sent Message:shuai Ge ai Meinv ... 2Sent Message:shuai Ge ai Meinv ... 3Sent Message:shuai Ge ai Meinv ... 4Sent Message:shuai Ge ai Meinv ... 5Sent Message:shuai Ge ai Meinv ... 6Sent Message:shuai Ge ai Meinv ... 7Sent Message:shuai Ge ai Meinv ... 8Sent Message:shuai Ge ai Meinv ... 9

(2) Start a worker instance with the following output:

2015-10-08 15:53:45 [x] Received ' Shuai Ge ai meinv ... ' [x] done2015-10-08 15:53:48 [x] Received ' Shuai Ge ai meinv ... 1 ' [x] done2015-10-08 15:53:51 [x] Received ' Shuai Ge ai meinv ... 3 ' [x] done2015-10-08 15:53:54 [x] Received ' Shuai Ge ai meinv ... 5 ' [x] done2015-10-08 15:53:57 [x] Received ' Shuai Ge ai meinv ... 7 ' [x] done2015-10-08 15:54:00 [x] Received ' Shuai Ge ai meinv ... 9 ' [x] Done

(3) Start another worker instance with the following output:

2015-10-08 15:53:45 [x] Received ' Shuai Ge ai meinv ... 0 ' [x] done2015-10-08 15:53:48 [x] Received ' Shuai Ge ai meinv ... 2 ' [x] done2015-10-08 15:53:51 [x] Received ' Shuai Ge ai meinv ... 4 ' [x] done2015-10-08 15:53:54 [x] Received ' Shuai Ge ai meinv ... 6 ' [x] done2015-10-08 15:53:57 [x] Received ' Shuai Ge ai meinv ... 8 ' [x] Done

RABBITMQ may appear as shown in the following queue change chart

650) this.width=650; "src=" Http://s3.51cto.com/wyfs02/M02/74/44/wKiom1YXcj3zEONoAAHSUcEuN5s109.jpg "title=" Rountertaskqueue.png "alt=" Wkiom1yxcj3zeonoaahsuceun5s109.jpg "/>

By default, RABBITMQ sends messages to the next consumer sequentially, with each consumer having the same amount of information that is not related to the duration of the message. This pattern of distributing messages is circular distribution (round-robin).

　　Message reply mode (msg acknowledgment)

Each task takes a few seconds to execute, and if a task starts for a long time and dies for some reason, but only part of the task is completed, what should I do? In the Round-robin mode above, once the RABBITMQ distributes the message to a consumer, it is immediately removed from memory. In this case, if the worker process is killed, the message being processed is lost, and the unhandled messages distributed to the worker are lost, of course.

But we don't want to lose any tasks. If a worker process dies, we want to distribute the task to other worker processes.

To solve the above problem, the RABBITMQ support answer mode lets the consumer tell RABBITMQ whether a particular message has been received and processed, and removes it from memory if it is processed.

If a message consumer does not answer, RABBITMQ will assume that the message is not processed and forwarded to other consumers. This ensures that the message is not lost, even if the worker process dies unexpectedly.

Message not timed out the message is only re-delivered when the worker process is dead, RABBITMQ said. Even a message that takes a long, long time to process is not going to go wrong.

The message answering mode is turned on by default, and in the previous example we explicitly closed it by autoack=true. Now set this property to True.

　　Messaging Persistence (Message durability)

Above we know how to deal with the problem of consumer crash of message, but if the RABBITMQ server down?

When RABBITMQ exits or crashes, it forgets the queue and message unless you remind it. If you want the message not to be lost, you must make both the queue and the message persistent.

If you want the RABBITMQ not to lose the queue, you can declare it as persisted in the following way:

Boolean durable = True;channel.queuedeclare ("Hello", durable, false, false, NULL);

Although the code above is correct, it will not work because we have defined a non-persisted "Hello" queue. RABBITMQ does not allow the use of different parameters to redefine an existing queue, which would return an error. We can use it as a solution by declaring it a different queue name, such as:

Boolean durable = True;channel.queuedeclare ("Task_queue", durable, false, false, NULL);

When a queue declaration is changed, it needs to be applied to both the message producer and the message consumer.

At this point, we can ensure that the Task_queue queue is not lost after RABBITMQ restarts. You now need to mark the message as persisted by setting the Messageproperties property value to Persistent_text_plain.

Import Com.rabbitmq.client.messageproperties;channel.basicpublish ("", "Task_queue", Messageproperties.persistent_ Text_plain, Message.getbytes ());

　　Fair distribution of messages (Fair Dispatch)

You may notice that distribution is sometimes not as we might expect, for example, when there are two messages that have one side of the message that are complex and time-consuming, while the other side of the message is simple and fast, the queue is often busy, and the other queue is very easy. RABBITMQ does not know that these are still evenly distributed messages.

The reason for this is that RABBITMQ only forwards messages when the message arrives at the exit of the queue, and it does not care about the number of messages that have not reached the message consumer. It just blindly sends odd messages to a consumer, even to another consumer.

The way to solve the problem is to set Prefetchcount = 1, which is like telling RABBITMQ each to give a message to the worker process only. In other words, it is not sent to a new message until the worker process finishes processing and answering the message, and it sends its message to other idle worker processes.

int prefetchcount = 1;channel.basicqos (Prefetchcount);

This article is from the "This person's IT World" blog, be sure to keep this source http://favccxx.blog.51cto.com/2890523/1701253

RABBITMQ Example Tutorial: Working with Java to get work queues done