RABBITMQ Client Golang Combat

This is a creation in Article, where the information may have evolved or changed.

RABBITMQ message Mode

The formation of message control in RABBITMQ can be divided into the following parts:

1. The routing part in the EXCHANGE:RABBITMQ, which controls the forwarding path of the message;
2. QUEUE:RABBITMQ Message Queuing, which can have multiple consumers reading messages from the queue;
3. Consumer: The consumer of the message;

RABBITMQ can use queue for message delivery alone (for example, celery can use a single queue for many-to-many messaging), and it also uses Exchange and queue to build multiple message patterns, including Fanout, Direct and topic way, mode of use in this place a picture, no longer do this detailed explanation.

I am using the RABBITMQ process, mainly for the broadcast of messages and topic subscription:

[producer] -> [exchange] ->fanout-> [queue of consumer] -> [consumer]       |                             /|\       ------->[exchange] ->topic------

Different devices connect to the RABBITMQ to create their own queue, and the queue binds two different exchange to receive broadcast messages and subject messages, respectively. By configuring the persistence of the queue and the message expiration time, the message can be cached in the queue and then read from the queue when the device is briefly offline.

RABBITMQ Client

The RABBITMQ client is essentially a communication process that implements the AMQP protocol, which is used by the Golang base package github.com/streadway/amqp .

Here is a description of some of the issues in client building, detailed client building code see: Rabbitmq_client.go

Create a queue

Both Exchange and queue are actually created through the AMQP protocol, and if an Exchange or queue with the same name is already in RABBITMQ during the creation process, the attribute is not created to fail. In general, the properties of exchange do not change, but the queue may modify properties such as Expiration Time, message TTL, and so on, if the queue creation is unsuccessful then it is deleted and then created (in my scenario, the queue is bound to the consumer, Therefore there is no question of mistakenly deleting the queue in use):

func (clt *Client) queInit(server *broker, ifFresh bool) (err error) {var num intch := clt.chif ifFresh {num, err = ch.QueueDelete(server.quePrefix+"."+clt.device,false,false,false,)if err != nil {return}log.Println("[RABBITMQ_CLIENT]", clt.device, "queue deleted with", num, "message purged")}args := make(amqp.Table)args["x-message-ttl"] = messageTTLargs["x-expires"] = queueExpireq, err := ch.QueueDeclare(server.quePrefix+"."+clt.device, // nametrue,  // durablefalse, // delete when ususedfalse, // exclusivefalse, // no-waitargs,  // arguments)    // 注意在此配置的两个参数，详细用意请参见 http://next.rabbitmq.com/ttl.htmlif err != nil {return}for _, topic := range server.topics {err = ch.QueueBind(q.Name,topic.keyPrefix+"."+clt.device,topic.chanName,false,nil,)if err != nil {return}}clt.que = qreturn}

Message reception

The receiving process for consumer messages is as follows:

msgs, err := clt.ch.Consume(clt.que.Name, // queueclt.device,   // consumerfalse,        // auto ackfalse,        // exclusivefalse,        // no localfalse,        // no waitnil,          // args)if err != nil {clt.Close()log.Println("[RABBITMQ_CLIENT]", "Start consume ERROR:", err)return nil}clt.msgs = msgsclt.pubChan = make(chan *publishMsg, 4)go func() {cc := make(chan *amqp.Error)e := <-clt.ch.NotifyClose(cc)log.Println("[RABBITMQ_CLIENT]", "channel close error:", e.Error())clt.cancel()}()go func() {for d := range msgs {msg := d.BodymsgProcess(d.Exchange, msg)d.Ack(false)}}()

By ch.Consume invoking a msgs channel that receives a message, the auto ACK is not configured here, but after the end of the message processing, the feedback ack is called to d.Ack(false) ensure that the message is processed and then confirmed. During the consumption process, the ch.NotifyClose(cc) shut-down is also called to amqp.Channel listen.

Note: The simultaneous reading of the msgs and Notifyclose two channel in one gorontinue may result in a deadlock. Because the msgs is closed, the corresponding gorontinue is terminated, and the notifyclose is deadlocked because there is no receiver at the time of the amqp.channel shutdown.

Send Message

During the message sending process of AMQP, its acknowledgment mechanism for messages is slightly sore. Because the message ID sent is not configurable at the time of delivery, but when the acknowledgement is received, the message ID is incremented by the natural number, that is, the sender needs to maintain the sent message ID in the order in which the natural number is incremented. The relevant code looks like this:

Func (CLT *client) Publishproc () {Ticker: = time. Newticker (ticktime) Deliverymap: = Make (map[uint64]*publishmsg) defer func () {Atomic. AddInt32 (&clt.onpublish,-1) ticker.  Stop () for _, msg: = Range Deliverymap {msg.ackerr = Errcancelmsg.cancel ()}} () var deliverytag uint64 = 1var Acktag UInt64 = 1var pMsg *publishmsgfor {select {case <-clt.ctx.done (): returncase PMSG = <-clt.pubchan:pmsg.starttime = time. Now () Err: = Clt.sendpublish (Pmsg.topicid, Pmsg.keysuffix, pmsg.msg, pmsg.expire) if err! = Nil {Pmsg.ackerr = Errpmsg.canc El ()}deliverymap[deliverytag] = Pmsgdeliverytag++case C, OK: = <-clt.confirm:if!ok {log. Println ("[Rabbitmq_client]", "CLIENT Publish notify Channel error") Return}pmsg = deliverymap[c.deliverytag]//FMT. Println ("Deliverytag:", C.deliverytag) Delete (Deliverymap, C.deliverytag) if c.ack {pmsg.ackerr = Nilpmsg.cancel ()} else {pmsg.ackerr = Errnackpmsg.cancel ()}case <-ticker. C:now: = time. Now () for {if Len (deliverymap) = = 0 {break}pmsg = deliverymap[acktag]if PMSG!= Nil {if now. Sub (PMSG.STARTTIME.ADD (pubtime)) > 0 {pmsg.ackerr = errtimeoutpmsg.cancel () Delete (Deliverymap, Acktag)} else {break }}acktag++}}}}

The construction point of the sending process:

1. Using a map[uint64]*publishMsg stored message that has been sent, the key for the map is the ID of the message;
2. After receiving the confirmation message, the feedback mechanism of the message is fed back to confirm the information and the message is deleted from the map.
3. At each tick, check the map for timeout messages according to the incremented ID, and feedback the timeout information through the feedback mechanism of the message;
4. Sends feedback to each message when the process exits and deletes the message.

Note that message feedback does not use the channel because the recipient of the message may no longer listen to the channel because of a timeout, causing the sending process to block. It is possible to use a feedback channel with a length of not 0 to make the sending process non-blocking, but it is necessary to wait for the GC to release the memory of the feedback channel. So instead of using the channel to receive feedback, context the event is passed to tell the sender that the message was sent and the feedback is written in advance publishMsg ackErr .

Summarize

As a Golang entry-level player, in the implementation of the RABBITMQ client process or stepping on some pits, the final implementation can be considered efficient and reliable. The RABBITMQ library itself has a heartbeat mechanism to maintain the connection to the server, but based on the experience of the MQTT client, it realizes the heartbeat to ensure the reliability of the client's upper connection. Therefore, in the receiving and sending two aspects, the client implementation or has withstood the test, welcome everyone reference.