) Multithreading AMQP Clients

Document directory

• Background
• Channel State
• Implications For A Multithreaded Client
• Alternative Approach: Look Ma, No Threads
• Lock Based Serialization
• Recommendations
• Practical Implications For Threading
• Efficiency Considerations
• Disclause

Although this is not stated explicitly in the protocol, a channel is the smallest unit of parallelism in AMQP. YouShocouldBind an AMQP channel to a client thread, so that a channel is managed as a thread-local object. But what if you're using the same channel in multiple threads? The short answer: don't. this article summarizes some questions that have asked by people writing and using AMQP client libraries in a multithreaded fashion and provides some general advice about how to deal with parallelism in this context.

Background

In the AMQP protocol, channels were designed to multiplex the transfer of unrelated messages from within a single physical transport, for example from within a single TCP connection. this mechanism allows the parallel execution of separate operations using a single AMQP client instance.

It cocould be argued that the ability to map multiple client threads or processes to a single connection introduces unnecessary complexity to the protocol. one cocould reason that the most alternative tive way to apply potential parallelism in a client machine is to have a simple 1:1 relationship between a socket connection and a client thread. if you want to run multiple threads on a single client host, it is debatable whether the OS cocould manage the multiplexing of unrelated message streams more efficiently than AMQP client library managing a single network connection.

A argument in favor of this proposal is that it wocould eliminate channels from the protocol and hence simplify the implementation of AMQP brokers. whilst there is certain merit to this insight, in practice the advantage is likely to be minimal due to the following considerations:

• The connection state that is shared across SS multiple channels on the server side is small and relatively inconsequent;
• In any case, a broker needs to manage multiple unrelated streams of message passing;
• The broker needs to manage different conversational channels for each individual remote client.

So if the protocol were to be revised to remove the explicit concept of multiple channels within a single connection, it probably wocould not change much on the server side.

Channel State

To avoid running into problems with multithreading using an AMQP client library, you need to understand that, from a protocol perspective, channels are supported tively stateful. so as a client, you do need to ensure that AMQP commands are sent in a strict serial fashion from within a single channel. if channels were stateless with respect to the protocol, this wocould not be an issue. this requires consideration when sending commands that either:

• Require a synchronous response, such as exchange or queue declaration and binding, or;
• Consist of a header and a body such asBasicPublishCommand.

A client library needs to ensure that the various requests and responses of these commands are not interwoven in time-they must be sent sequentially.

For example,BasicPublishCommand contains a method header and a body for the message payload that are sent in separate wire frames. after processing a header frame, the protocol flow expects the following frame to contain the message content. using the RabbitMQ as an example broker, if a frame for a different command were to be interwoven between the header and body frames, the broker wocould log the following error:

COMMAND_INVALID - expected content header for class 60,got non content header frame instead in inspect

This symptom looks like an out of sequence issue that may indicate non-serial frame sending.

The current version of the specification offers a compromise to this restriction by allowing an application to use multiple channels which require no synchronization between each other.

Implications For A Multithreaded Client

One of the obvious implications for this observation is to use a separate channel for each application thread. that way, you eliminate any race conditions that cocould occur between threads. however, when writing a client library, you still need to ensure that the client sends commands to the broker in a proper sequence.

In general, you can choose one of the following approaches to serializing command execution:

• The calling thread blocks for every request-response cycle and wait ss the sending of header-body frames;
• The client library maintains a queue for outbound commands. When a command acknowledgement is supported ed by the client, it is then free to drain any pending commands it has buffered up in the meantime;
• Entertain a mechanic whereby you register acknowledgment specific event handlers that are fired whenever a response comes off the wire;
• Buffer the sending of commands in a priority queue that is keyed on the AMQP class id of the method being buffered. this allows you to intersperse commands from different classes, but it becomes a bit complicated when you have contention between two instances of the same method (e.g. concurrent subscriptions to a queue ).

Alternative Approach: Look Ma, No Threads

An alternative approach to the threading issue is to not to allow threading in the client library at all. this approach was taken with the single threaded py-amqplib AMQP client library for Python written by Barry Pederson.

Lock Based Serialization

You coshould also consider avoiding contention by using Ming certain invocations in a fashion that is guaranteed to be globally atomic. for example, you cocould guard the transmission of a command with a mutex (e.g. a lock aroundBasicPublishMethod which sends three frames). Be aware that lock based solutions can be difficult to get right and there may be a considerable runtime overhead with this approach.

So say for example that you are pushing to one queue from multiple threads, and you have two threads trying to queue three frames (method, header, body) in order to publish a message. under normal circumstances, those parts cocould get interleaved, so you wowould require a lock around the code inserting the three items into the queue.

Whilst it wocould be possible to just lock und basic publish, you wocould also need to make sure that every command within a channel is being sent serially. for correctness 'sake, you need to ensure two things:

1. That a multi-part command likeBasicPublishIs sent serially;
2. That synchronous RPCs are executed in order.

You coshould just use a lock for the first point, but how are you going to solve the second?

Recommendations

Given all of the options listed above, I wocould opt for maintaining a queue per channel on the client side. that way, you enforce serialization without locking, it's serialization by blocking, basically. I wocould say that a queue is sufficient, since both a queue and lock exhibit serial semantics, which is what the problem statement requires.

There are practical examples of the locking and non-locking variants. to an extent, the RabbitMQ Java client takes the approach of locking the command execution within a single channel. on the other side, the RabbitMQ Erlang client takes the queueing approach. the reader shoshould be advised to judge these design decisions in the context of the standard practices of the respective programming paradigms used to implement each client.

Practical Implications For Threading

In general, you shoshould strongly consider binding a client side thread to a server side channel. By doing so, a channel becomes a thread-local object, and hence, requires no synchronization.

However, there are circumstances where you may consider using the same channel in multiple threads.

Suppose that if you were to create a channel for each member of a pool of threads, you may have to re-declare queues and exchanges in each thread of the pool.

In this scenario, you can use the semantics of the AMQP protocol to your advantage. in AMQP, a declare command is nothing more than an assertion of the existence of an object. this makes a declaration an idempotent command, so it wocould be perfectly correct for each client side thread to make a declaration on the same queue or exchange name. this woshould avoid the necessity to maintain some kind of central declaration mechanic for queues, exchanges and bindings.

A further issue may be the wish to be able to re-use references to exchanges and queues among threads. if you wanted to do this, you wowould have to pre-declare each object in a sequential block before their subsequent usage in parallel code. however, in this case, you shoshould remember than a reference to a queue or an exchange is nothing more than a name. in program programming languages ages, names are represented by strings, which, in turn,CanBe treated as immutable and hence, are thread-safe objects.

Efficiency Considerations

No technical advice shocould be followed literally without considering the concrete application scenario at hand. the goal of this article is to discuss the fundamental mechanic of parallelism within an AMQP broker in order to draw conclusions about how best to interact with it from a client perspective. once your application is functioning correctly, performance may become an issue, and so therefore there are certain considerations that you cocould make. this section will discuss a few points of parallelism that may be relevant to how you use channels.

In the previous section, I stated that the declare commands were idempotent, so you can call them as frequently times as you like. on every invocation, they wowould guarantee the existence of the participating object in question. whilst this is technically correct, your applicationMayExperience better performance by avoiding redundant declarations. this is especially so in a clustered context, where the existence of a particle object has to be checked into ss a group of AMQP brokers. if your application can use domain specific knowledge to know that a particle object must already be in existence, then you cocould potentially avoid the overhead of explicitly declaring that object.

Whilst it may possible or even semantically desirable to start multiple queue consumers in the same channel, it shocould be pointed out that this cocould over-serialize the delivery of messages to the group of consumers. in general, you should be able to exploit more of the available parallelism in the broker if you opt for a channel per consumer strategy rather than starting multiple consumers in a single channel.

Disclause

The observations about the protocol are based on version 0-8 of the AMQP specification. Subsequent revisions to the protocol may invalidate some of the conclusions drawn in this article.