Message Queuing based on HBase: HQueue

1. HQueue Profile

HQueue is a set of distributed, persistent message queues based on HBase developed by the search Web crawl offline Systems team. It uses htable to store message data, HBase coprocessor to store the original keyvalue data in the message data format, and encapsulates the HBase client API for message access based on the HQueue client API.

HQueue can be used effectively in the need to store time series data, as MapReduce job and istream input, output for the upstream and downstream sharing data.

2. HQueue characteristic

Since HQueue is based on hbase for message access, it stands on the shoulders of HDFs and hbase, making it the following features:

(1) Support multiple partitions, can set the queue size according to the demand, support high concurrent access (HBase of many region);

(2) Support automatic failover, any machine down, partition can automatically migrate to other machines (hbase failover mechanism);

(3) Support dynamic load balancing, partition can be dynamically dispatched to the most reasonable machine (HBase loadbalance mechanism can be dynamically adjusted);

(4) Using HBase for persistent message storage without losing data (HBase Hlog and HDFs Append);

(5) The Read and write mode of the queue and hbase storage characteristics of natural, with good concurrent read and write performance (the latest message stored in the Memstore, write messages directly written to Memstore, usually under the scene are memory-level operation);

(6) Support message classified access by topic (Qualifier in HBase);

(7) Support message TTL, automatic cleanup of expired messages (HBase support keyvalue TTL);

(8) HQueue = htable Schema design + hqueue coprocessor + HBase Client wrappers, fully expanded development, no hack work, can automatically upgrade with HBase;

(9) The HQueue Client API is simple encapsulated based on HBase client wrappers, HBase Thriftserver enables it to support multilanguage APIs, so hqueue can easily encapsulate multilingual APIs; (a) HQueue The Client API can naturally support the InputFormat mechanism of the Hadoop MapReduce job and IStream, using the locality feature to dispatch computations to the nearest machine;

(one) HQueue supports the message subscription mechanism (HQueue 0.3 and subsequent versions).

3. HQueue system design and Process 3.1. HQUEUE System Structure

The HQUEUE system structure is shown in figure (1):

Figure (1): HQueue system structure

(1) Each queue corresponds to a htable, creating a queue can be created by presharding table, which is beneficial to load balancing.

(2) Each queue can have multiple partitions (HBase regions) that are evenly distributed across HBase region in servers clusters.

(3) Each partition can be dynamically migrated in multiple region servers of the HBase cluster. Any region server hangs, HQueue partition running on it can be migrated automatically to other region servers, and data is not lost. When the cluster load is unbalanced, the HQueue partition is automatically migrated Hmaster to the region Server with low load.

(4) Each message corresponds to a hbase keyvalue Pair, which is stored in HBase region, in MessageID chronological order. The MessageID is composed of timestamp and the SequenceID of the same timestamp, see the message Storage Structure section for details.

3.2. Message storage structure

The message storage structure is shown in figure (2):

Figure (2): Message storage structure

which：

(1) Rowkey: composed of PartitionID and MessageID.

PartitionID: A queue can have multiple partitions, currently supports short.max_value partitions. The Partition ID can be specified not when the message object is created, but when it is sent, or it is not specified and a random Partition ID is used. MessageID: The message ID, which consists of two parts, timestamp and SequenceID. Timestamp is the timestamp, in milliseconds, when the message is written to HQueue. SequenceID is the sequential number of messages under the same timestamp and currently supports the same timestamp Short.max_value messages.

(2) Column: composed of column accessibility and message topic.

column family:hbase column accessibility, here is the fixed value "message". Message topic:hbase Column Qualifier, Messages Topic name. Users can store messages under different topics as needed, or they can get interesting topics message data from the queue.

(3) Value: the message content.

3.3. HQueue message Write and coprocessor process

HQueue uses the HQueue Client API to write message data, in order to ensure that the message is unique and orderly, HQueue uses coprocessor to process the MessageID that the user writes the message, and then immediately puts it into the hbase memstore so that it can be accessed, Last Hlog. The specific processing logic is shown in Figure (3):

Figure (3): Data write and coprocessor process

which：

(1) HQueue encapsulates the HQueue Client API, which allows users to write messages to HQueue using their put methods.

(2) HQueue client uses Message.makekeyvaluerow () to complete the conversion of the message data structure to HBase Rowkey. The Rowkey format required by HQueue can participate in the above content.

(3) When the HQueue client completes the Rowkey conversion, it invokes the Htable put method to complete the write of the message according to the HBase standard write process.

(4) HQueue on the register has hqueuecoprocessor, it expands from Baseregionobserver. Hregion calls the Hqueuecoprocessor Prebatchmutate method, which is used primarily to adjust the messageid, to ensure that the MessageID is unique and orderly before the message data is actually written.

(5) in the Hqueuecoprocessor Prebatchmutate method, the durability is also adjusted to Skip_wal so that HBase will not actively persist the message data into Hlog.

(6) After writing the message data, Hregion invokes the Hqueuecoprocessor Postbatchmutate method, which mainly completes the function of persisting the message data into the Hlog.

3.4. HQueue Scan Process

To facilitate the scan of data from the queue, HQueue encapsulates Clientscanner, providing partitionscanner such as Queuescanner, Combinedpartitionscanner and scanner, for different scenarios. The specific processing flow of the HQueue scan is shown in figure (4):

Figure (4): HQueue Scan Process

which：

(1) Users can obtain the required queue Scanner from HQueue client as needed, and currently offer three kinds of Scanner mainly:

Queuescanner: Data for all partitions in the scan queue; Partitionscanner: Data for scan specified in partition queue Combinedpartitionscanner: Data for several specified partitions in the scan queue.

(2) After the user obtains the scanner, can iterate the scanner next method to take out the message data, until no data returns, the scan end. After the scan is over, users should voluntarily close the scanner in order to release resources in time. (3) When users no longer use the previously created queue object, they should voluntarily shut down the queue to release resources in a timely manner.

3.5. HQueue Subscription Process 3.5.1. Overall process

HQueue provides subscription functionality since version 0.3, one subscriber can subscribe to multiple partitions, multiple topics for a queue. Compared with the way that the user uses scanner active scan message data, the subscription method has (1) the message data will be pushed to subscribers when it writes to the queue, and the message service is more timely; (2) The subscriber passively receives the new message, It can save the hqueue of no new message data and reduce the cost of the system scan.

HQueue subscription process processing logic is shown in figure (5):

Figure (5): HQueue Subscription process processing logic

which：

(1) HQueue subscriptions are mainly composed of subscriber, zookeeper and coprocessor. which：

Subscrier: Subscriber. It mainly completes functions such as writing subscription information to Zoeokeeper, initiating listening, receiving new messages and callback the message handler function (MessageListener) registered on it. Zookeeper: Used to hold subscriber-submitted subscription information, mainly including subscriber-subscribed queue, partitions, and topics, subscriber address and checkpoint information, more details see follow-up description. Coprocessor: Primarily completes features such as obtaining subscription information from zookeeper, using Internalscanner to scan the latest messages from the queue, sending new messages to subscribers, and updating the current checkpoint to zookeeper.

(2) The main processing process of coprocessor is as follows: Step 1: Create subscriber, add subscription information and message processing functions, write subscription information to zookeeper, start listening and wait for new messages. The subscription information written in Zookeeper includes:

the queue name of the Subscriber subscription, Queuee partitions for the Subscriber subscription, and the start ID of the message on each partition. A subscriber can subscribe to multiple partitions, and if not specified, subscribe to all partitions of that queue. The Subscriber subscribes to the message topics. A subscriber can subscribe to multiple topics and, if unspecified, subscribe to all topics on that queue. Subscriber's addresss/hostname and listening port. When a user creates a subscriber, it can specify a listening port and, if not specified, randomly select a currently available port as the listening port.

Step 2:coprocessor obtains the subscription information from the zookeeper and registers the watcher with zookeeper, so that zookeeper can notify zookeeper in time when the subscription information changes in coprocessor. After obtaining the subscription information, Coprocessor creates worker threads such as subscriptionworker to scan messages from HQueue partition and send messages to Subscriber.

Step 3:coprocessor Scan new messages from HQueue partition.

Step 4:coprocessor sends a new message to Subscriber.

When the step 5:subscriber receives a new message, it recalls the callback function that is registered on it.

Step 6: After the new message is sent successfully, coprocessor updates the checkpoint of the message to zookeeper for subsequent use.

Step 7:subscriber Unsubscribe and removes the necessary subscription information from the zookeeper.

Step 8:zookeeper will notify the change of Subscriber subscription information to Coprocessor,coprocessor according to the change of subscription information by registering the Watcher on it. Pause Subscriptionworker such as worker threads.

3.5.2. HQueue Subscriber

The HQueue subscriber structure and main processing logic are shown in figure (6):

Figure (6): HQUEUE subscriber structure and main processing logic

which：

(1) Subscriber mainly consists of two parts: Subscriberzookeeper and Thrift Server. Among them, Subscriberzookeeper mainly completes several operations related to the zookeeper, including writes the subscription information, deletes the subscription information and so on. The communication between coprocessor and Subscriber is done through thrift, subscriber starts thrift Server, listens for the specified port, waits for new messages sent by coprocessor.

(2) subscriber receives a new message through thrift server, it recalls the callback function (messagelisteners) registered on it and returns the status code to coprocessor. (3) Multiple messagelisteners can be registered on a subscriber, and multiple messagelisteners will be called sequentially.

3.5.3. HQueue coprocessor

The HQUEUE coprocessor structure and main processing logic are shown in figure (7):

Figure (7): HQUEUE coprocessor structure and main processing logic

which：

(1) Coprocessor: Mainly composed of two parts subscriptionzookeeper and Subscriptionworker.

Subscriptionzookeeper: Mainly completes zookeeper related work, including obtains the subscription information from the zookeeper and registers the related watcher, Subscriptionworker will update checkpoint to zookeeper operations. Subscriptionworker mainly includes Messagescanner and Messagesender, mainly completes scan new message, sends the message to the Subscriber and updates the checkpoint and so on the operation.

(2) Messagescanner mainly completes the creation Internalscanner, scan the new message from the queue partition, and puts it into the buffer queue medium operation.

when there is no free space in the buffer queue, Messagescanner waits until the message in the buffer queue is consumed by the messagesender, freeing up the remaining space. When there is no new message in the queue partition, Messagescanner actively sleep, and when a new message is written, Coprocessor wakes Messagescanner through Subscriptionworker. Start a new round of scan.

(3) Messagesender mainly completes the new message from the buffer queue, sends it to the subscriber, and waits for the subscriber to send back the response. When there is no new message in the buffer queue, Messagesender waits until a new message arrives. (4) Checkpointupdater in Messagesender will periodically write the current checkpoint to the relevant subscription node in zookeeper for subsequent use.

3.5.4. Subscription information Hierarchy

HQueue related subscription information is stored in the hierarchy of subscription information in Zookeeper,zookeeper as shown in figure (8):

Figure (8): Subscription information hierarchy

which：

(1) The Subscriber node (subscriber_x) is logged checkpoint on the queue partition. The checkpoint is written by subscriber when it initiates a subscription and is updated by Checkpointupdater in Subscriptionworker Messagesender. (2) There will be two temporary nodes under the Subscriber node: address and topics, which hold the subscriber's IP Address/hostname:port and the subject of the subscription, respectively. These temporary nodes are deleted when the Subscriber actively cancels the subscription, and when the Subscriber quits unexpectedly, zookeeper deletes the temporary node after the session expires.

3.5.5. Subscriber Thrift Service

The HQueue subscription feature uses thrift to simplify support for multilingual clients. Subscriber starts the thrift Server, listens on the specified port, receives the message, and recalls Messagelisteners to process the message. The interface definition used to describe the services provided by HQueue Subscriber is as follows:

namespace Java com.etao.hadoop.hbase.queue.thrift.generated/*** hqueue messageid*/struct TMessageID {1:i64 Timestamp,2:i16 sequenceid}/*** hqueue message*/struct tmessage {1:optional tmessageid id,2:optional i16 partitionID,3: Binary topic,4:binary value}/*** hqueue subscriber service*/service hqueuesubscriberservice {i32 consumeMessages (1: list<tmessage> messages)}

namespace Java com. Etao. Hadoop. HBase. Queue. Thrift. Generated

/**

* HQueue MessageID

*/

struct Tmessageid {

1:i64 timestamp,

2:i16 SequenceID

}

/**

* HQueue Message

*/

struct Tmessage {

1:optional Tmessageid ID,

2:optional i16 PartitionID,

3:binary topic,

4:binary value

}

/**

* HQueue Subscriber Service

*/

Service Hqueuesubscriberservice {

I32 consumemessages (1:list < tmessage > Messages)

}

4. HQueue uses 4.1. HQueue Toolkit

For easy user use, HQueue encapsulates the HQueue Client API for accessing message data. Since the HQueue 0.3 version, the HQueue logging tools are integrated into the hqueue shell, forming HQueue Toolkit, providing a one-stop service for users to manage queue and queue subscribers.

Similar to the HBASE shell, users can enter the HQueue shell command line tool using the $ ${hbase_home}/bin/hqueue shell. It is to be noted that users need to ensure that hqueue Toolkit are deployed before using HQueue Toolkit.

HQueue Toolkit includes commands for creating queue, Disable queue, Enable queue, delete queue, and purge queue. Use the example below:

(1) Create queues

Usage:create ' queue_name ', Partition_count, TTL, [revisit Dictionary]

Descriptions:queue_name: The name of the HQueue to be created, the required arguments.

Partition_count: Number of partition HQueue to be created, required parameters.

TTL: Expiration time, required parameters.

revisit dictonary: Optional configuration parameters. Currently supported configuration parameters are: (1) hbase.hqueue.partitionsPerRegion; (2) Hbase.hregion.memstore.flush.size (3) hbase.hregion.majorcompaction; (4) Hbase.hstore.compaction.min (5) Hbase.hstore.compaction.max; (6) Hbase.hqueue.compression (7) Hbase.hstore.blockingStoreFiles etc.

Examples:

hqueue&gt; create ' q1′, 86400hqueue&gt; create ' q1′, 86400, {' hbase.hqueue.partitionsPerRegion ' =&gt; ' 4′ ' Hbase.hstore.compaction.min ' =&gt; ' 16′, ' Hbase.hstore.compaction.max ' =&gt; ' 32′}

(2) Empty queue

usage:truncate_queue ' queue_name ' descriptions:

Usage:truncate_queue ' queue_name '

Descriptions:

Queue_name: queue name to be emptied, required parameters.

Examples:

HQueue (Main):013:0&gt; truncate_queue ' Replication_dev_2_test_queue '

HQueue (Main): 013:0 > Truncate_queue ' replication_dev_2_test_queue '

Note that this command differs from the truncate in the HBase shell, which deletes only the data in the queue and retains the presharding information of the queue.

More actions See: http://searchwiki.taobao.ali.com/index.php/HQueue_Toolkit#Queue.E7.AE.A1.E7.90.86

(3) New subscribers

Usage:add_subscriber ' queue_name ', ' subscriber_name '

Descriptions:

Queue_name: Queue name, required parameters.

Subscriber_name: Subscriber name, required parameters.

Examples:

add_subscriber ' replication_dev_2_test_queue ', ' subscriber_1 '

Add_subscriber ' Replication_dev_2_test_queue ', ' subscriber_1 '

(4) Delete subscribers

Usage:delete_subscriber ' Subscriber_name ', ' queue_name '

Descriptions:

Queue_name: The name of the queue that the Subscriber subscribes to, required parameters.

Subscriber_name: Subscriber name, required parameters.

Examples:

HQueue (Main):040:0&gt; delete_subscriber ' replication_dev_2_test_queue ', ' subscriber_1 '

HQueue (Main): 040:0 > Delete_subscriber ' replication_dev_2_test_queue ', ' subscriber_1 '

More information can be found in: http://searchwiki.taobao.ali.com/index.php/HQueue_Toolkit#.E8.AE.A2.E9.98.85.E8.80.85.E7.AE.A1.E7.90.86

4.2. Put

The put-related operation in the HQueue client API completes the writing of user message data to HQueue, which supports bulk operations, using the following examples:

HQueue queue = new HQueue (queuename); String topic1 = "crawler"; String value1 = "http://www.360test.com"; writes a single message data without specifying the partition ID. Without specifying the partition ID, one is randomly selected in all partitions of the queue. Message Message1 = new Message (Bytes.tobytes (TOPIC1), Bytes.tobytes (value1)); explicitly specifies PartitionID. Short PartitionID = 10;queue.put (PartitionID, message1); list&lt;message&gt; messages = new arraylist&lt;message&gt; (); Messages.add (Message1); String topic2 = "Dump"; String value2 = "http://www.jd.com"; Message Message2 = new messages (Bytes.tobytes (TOPIC2), Bytes.tobytes (value2)), Messages.add (Message2); Do not specify a partition ID. Queue.put (Messages)//write multiple message data, specifying the partition ID. Queue.put (PartitionID, messages); Queue.close ();

HQueue queue = new HQueue (queuename);

String topic1 = "crawler";

String value1 = "http://www.360test.com";

Writes a single message data without specifying the partition ID. Without specifying the partition ID, one is randomly selected in all partitions of the queue.

Message Message1 = new Message (Bytes tobytes (TOPIC1), Bytes. Tobytes (value1));

Queue. put (message);

explicitly specifies the PartitionID when writing the message.

Short PartitionID = 10;

Queue. Put (PartitionID, message1);

List < message > messages = new ArrayList < message > ();

Messages. Add (Message1);

String topic2 = "Dump";

String value2 = "http://www.jd.com";

Message Message2 = new Message (Bytes tobytes (topic2), Bytes. Tobytes (value2));

Messages. Add (Message2);

Writes multiple message data without specifying the partition ID.

Queue. put (messages);

Writes multiple message data, specifying the partition ID.

Queue. Put (PartitionID, messages);

Queue. Close ();

4.3. Scan

To facilitate user scan of message data from queue, the HQueue Client API provides three custom scanner: Queuescanner, Partitionscanner, and Combinedpartitionscanner , use the following example:

String queuename = "Subscription_queue"; Queue queue = new HQueue (queuename);//start timestamp long Currenttimestamp = System.currenttimemillis (); MessageID Startmessageid = new MessageID (currentTimestamp-6000); MessageID Stopmessageid = new MessageID (currenttimestamp); Scan Scan = new Scan (Startmessageid, Stopmessageid);//Add Theme Scan.addtopic (Bytes.tobytes ("Topic1")); Scan.addtopic ( Bytes.tobytes ("Topic2")); Message message = null;//uses Queuescanner to scan data in all partitions under queue Queuescanner Queuescanner = Queue.getqueuescanner ( Scan) while (message = Queuescanner.next ())!= null) {//No-op}queuescanner.close (), Short partitionID1 = 1;// Using Partitionscanner, scans the partition data Partitionscanner specified in the queue Partitionscanner = Queue.getpartitionscanner ( PartitionID1, scan) while (message = Partitionscanner.next ())!= null) {//No-op}partitionscanner.close (); PartitionID2 = 2; Map&lt;short, scan&gt; partitions = new Hashmap&lt;short, scan&gt; ();//Add multiple Partitionspartitions.put (partitionID1, Scan);p Artitions.put (partitionID2, scan); Combinedpartitionscanner Combinedscanner = Queue.getcombinedpartitionscanner (partitions); Combinedscanner.next ())!= null) {//No-op}combinedscanner.close (); Queue.close ();

String queuename = "Subscription_queue";

Queue queue = new HQueue (queuename);

Start timestamp

Long Currenttimestamp = System. Currenttimemillis ();

MessageID Startmessageid = new MessageID (currentTimestamp-6000);

MessageID Stopmessageid = new MessageID (currenttimestamp);

Scan Scan = new Scan (Startmessageid, Stopmessageid);

Add a topic

Scan. Addtopic (Bytes tobytes ("Topic1"));

Scan. Addtopic (Bytes tobytes ("Topic2"));

Message message = NULL;

Use Queuescanner to scan data in all partitions under queue

Queuescanner Queuescanner = queue. Getqueuescanner (scan);

while (message = Queuescanner. Next ())!= null) {

No-op

}

Queuescanner. Close ();

Short partitionID1 = 1;

Use Partitionscanner to scan the partition data specified in the queue

Partitionscanner Partitionscanner = queue. Getpartitionscanner (partitionID1, scan);

while (message = Partitionscanner. Next ())!= null) {

No-op

}

Partitionscanner. Close ();

Short partitionID2 = 2;

Map < short, Scan > partitions = new HashMap < short, Scan > ();

Add multiple partitions

Partitions. Put (partitionID1, scan);

Partitions. Put (partitionID2, scan);

Combinedpartitionscanner Combinedscanner = queue. Getcombinedpartitionscanner (partitions);

while (message = Combinedscanner. Next ())!= null) {

No-op

}

Combinedscanner. Close ();

Queue. Close ();

4.4. Subscribe to Message

HQueue has been available for subscription since version 0.3, as shown in the following examples:

HQueue queue = null; Hqueuesubscriber subscriber = null;try {String queuename = "Subscription_queue"; queue = new HQueue (queuename); Set&lt;pair&lt;short, messageid&gt;&gt; partitions = new Hashset&lt;pair&lt;short, messageid&gt;&gt; (); Add the subscribed partitions pair&lt;short, messageid&gt; partition1 = new Pair&lt;short, messageid&gt; ((short) 0, NULL); Partitions.add (Partition1); Pair&lt;short, messageid&gt; partition2 = new Pair&lt;short, messageid&gt; (short) 1, null);p Artitions.add (Partition2) ; Pair&lt;short, messageid&gt; partition3 = new Pair&lt;short, messageid&gt; (short) 2, null);p Artitions.add (Partition3) ;//Add subscribed topics set&lt;string&gt; topics = new hashset&lt;string&gt; (); Topics.add ("topic_1"); Topics.add ("topic_2"); Topics.add ("Topic_3");//subscriber name String subscribername = "subscriber_1"; Subscription Subscription = new Subscription (subscribername, topics); subscription.addpartitions (partitions); Add callback function List&lt;messagelistener&gt; listeners = new linkedlist&lt;messagelistener&gt; (); MessagelisTener blackholelistener = new Blackholemessagelistener (subscribername); Listeners.add (blackholelistener);//Create Subscriber Subscriber = Queue.createsubscriber (subscription, listeners); Subscriber.start (); Thread.Sleep (600000L); Subscriber.stop ("Time out, request to stop Subscriber:" + subscribername);} catch (Exception ex) {log.error ("Received unexpected Exception when testing subscription.", ex);} finally {if (Queue!= null {try {queue.close (); Queue=null} catch (IOException ex) {//Ignore the exception}}}

HQueue queue = null;

Hqueuesubscriber subscriber = null;

try {

String queuename = "Subscription_queue";

Queue = new HQueue (queuename);

Set < Pair < short, messageid >> partitions = new HashSet < Pair < short, MessageID >> ();

Add subscribed partitions

Pair < short, MessageID > partition1 = new Pair < short, MessageID > (short) 0, NULL);

Partitions. Add (Partition1);

Pair < short, MessageID > partition2 = new Pair < short, MessageID > (short) 1, null);

Partitions. Add (Partition2);

Pair < short, MessageID > partition3 = new Pair < short, MessageID > (short) 2, null);

Partitions. Add (Partition3);

Add subscribed Topics

Set < string > topics = new HashSet < string > ();

Topics. Add ("Topic_1");

Topics. Add ("topic_2");

Topics. Add ("Topic_3");

Subscriber name

String subscribername = "subscriber_1";

Subscription Subscription = new Subscription (subscribername, topics);

Subscription. Addpartitions (partitions);

Add callback function

List < MessageListener > listeners = new LinkedList < MessageListener > ();

MessageListener Blackholelistener = new Blackholemessagelistener (subscribername);

Listeners. Add (Blackholelistener);

Create Subscribers

Subscriber = queue. Createsubscriber (subscription, listeners);

Subscriber. Start ();

Thread. Sleep (600000L);

Subscriber. Stop ("Time out, request to stop Subscriber:" + subscribername);

catch (Exception ex) {

LOG. Error ("Received unexpected exception when testing subscription.", ex);

finally {

if (queue!= null) {

try {

Queue. Close ();

queue = null;

catch (IOException ex) {

Ignore the exception

}

}

}

4.5. Thriftserver API

HBase's Thriftserver enables htable API support, HQueue extends HBase support in Thriftserver HQueue, making C + +, Languages such as Python and PHP also have easy access to hqueue.

The thrift APIs currently provided by HQueue are as follows:

1ScannerID Messagescanneropen (1:text queuename,2:i16 partitionid,3:tmessagescan messagescan) according to scan, Open Scanner2tmessage Messagescannerget (1:scannerid ID) on one of the partition in the queue to get message3list&lt;tmessage&gt; Messagescannergetlist (1:scannerid id,2:i32 nbmessages) bulk fetch messages4void messagescannerclose (1:ScannerID ID) Close Scannerid5void putmessage (1:text queuename,2:tmessage tmessage) writes a message to the queue using a random partition id6void Putmessages (1:text queuename,2:list&lt;tmessage&gt; tmessages) writes a batch of messages to the queue, using a random partition id7void Putmessagewithpid (1:text queuename,2:i16 partitionid,3:tmessage tmessage) writes a message to the queue, using the specified partition id8void Putmessageswithpid (1:text queuename,2:i16 partitionid,3:list&lt;tmessage&gt; tmessages) writes a batch of messages to the queue, Gets the address 5 of all queuename hosts in the queue using the specified partition id9list&lt;text&gt; getqueuelocations (1:text partition). Summary

The above is the HQueue concept, the characteristic, the system design, the processing flow as well as the application and so on aspect simple elaboration, hoped that has the help to everybody.