RABBITMQ Tutorial-Remote Procedure Call (RPC)

Remote Procedure Call (RPC)

(using Pika 0.9.8 python client)

In the second tutorial, we learned how to use a task Force to distribute time-consuming tasks between multiple workers.

But what if we need to execute a function on a remote computer and wait for the result? That would be a different thing. This pattern is often referred to as a remote procedure call or RPC.

in this tutorial, we will use RABBITMQ to build an RPC system: a client and a scalable RPC server. Since we do not have any time-consuming tasks worth distributing, we will create a virtual RPC service to return the number of Fibonacci.

client interface

to describe how to use an RPC service, we will create a simple client class. It exposes a method called Call , and the method sends an RPC request and blocks until the answer is received.

Fibonacci_rpc = fibonaccirpcclient () result = Fibonacci_rpc.call (4) print "FIB (4) is%r"% (result,)

a little note about RPC

Although RPC is a fairly common pattern in computing, it is often criticized. The problem comes from the programmer not realizing whether a function call is local or if it is a slow RPC. problems such as causing an unpredictable system and adding unnecessary debugging complexity. Not only did it not simplify the software, misuse of RPC could also lead to non-maintainable spaghetti-style code.

keep in mind, consider the following recommendations:

• Make sure that a call is local or remote and looks obvious.
• write a document for your system. Makes the components clear from the beginning.
• handle error conditions. What should clients do when the RPC server hangs or executes for a long time?

avoid using RPC when in doubt. If you can, you should use an asynchronous pipeline-instead of RPC-such as blocking, the result is pushed asynchronously to the next calculation step.

Callback Queue

It is generally easy to perform RPC based on RABBITMQ. A client sends a request message, and a server answers with a response message. In order to receive a response, the client needs to send a ' callback ' queue address in the request. Let's try it:

result = Channel.queue_declare (exclusive=true) callback_queue = result.method.queuechannel.basic_publish (exchange= ' ',                      routing_key= ' Rpc_queue ',                      Properties=pika. Basicproperties (                            reply_to = Callback_queue,                            ),                      body=request) # ... and some code to read a response message fro M The callback_queue ...

Message Properties

AMQP protocol pre-defines 14 properties that are sent along with a message. Most properties are seldom used, except for the following:

• Delivery_mode : Mark a message as persistent (with a value of 2) or transient (any other value). You may also remember This property in the second tutorial.
• Content_Type : The mime-type used to describe the encoding. For example, for commonly used JSON encoding, setting the attribute to Application/json is a good practice.
• reply_to : It is often used to name a callback queue.
• correlation_id : It is useful to correlate RPC responses and requests.

Association ID

In the method that appears above, we have created a callback queue for each RPC request. that's pretty inefficient, fortunately there's a better way-let's create a separate callback queue for each client.

that creates a new problem, and it's not clear which of the responses received in that queue belong to exactly which request. That's where the correlation_id attribute is applied. We will set a unique value for each request. Later, when we receive a message from the callback queue, we will look at this property, based on which we will be able to match a response to a request. If we see an unknown correlation_id value, we can safely discard the message-it does not belong to our request.

you might ask, why should we ignore unknown messages in the callback queue, instead of failing with an error? That is because a race condition may be generated on the server side. Although unlikely, the RPC server may have sent the answer to us, but died before sending a confirmation message to the request. If that happens, the restarted RPC server will process the request again. That's why on the client side we have to gracefully handle repetitive responses, and RPC should be an ideal idempotent.

Summary

Our RPC will work like this:

• when the client is up, it creates an anonymous exclusive callback queue.
• for an RPC request, the client sends a message with two properties: Reply_to, set to callback queue, and correlation_id, which is set to a unique value created for each request.
• the request is sent to a rpc_queue queue.
• The RPC Worker (Aka:server) waits on that queue. When a request appears, it executes the work and sends a message with a result to the client, using the queue from the Reply_to field.
• the client waits for data on the callback queue. When a message appears, it checks the Correlation_id property. If it matches the requested one, it will return the response to the app.

Full Code

rpc_server.py Code:

the server-side code is fairly straightforward:

• (4) As usual, we establish a connection and declare the queue at the outset.
• (11) We have declared our Fibonacci function. It assumes that the input is a valid positive number. (Don't expect this function to work when you enter a large number, which is probably the slowest recursive implementation).
• (19) We have declared the core of a CALLBACK,RPC server for Basic_consume. Executes the message when it is received. It completes the work and returns the response.
• (32) We may want to run multiple server processes. In order to balance the load across multiple servers, we need to set Prefetch_count setting.

rpc_client.py's code:

#!/usr/bin/env pythonimport pikaimport uuidclass fibonaccirpcclient (object): Def __init__ (self): self.connection = Pika. Blockingconnection (Pika. Connectionparameters (host= ' localhost ')) Self.channel = Self.connection.channel () result = SE Lf.channel.queue_declare (exclusive=true) self.callback_queue = Result.method.queue Self.channel.basic_consum  E (Self.on_response, No_ack=true, Queue=self.callback_queue) def on_response (self, CH, method, props, body): if self.corr_id = = Props.correlation_id:self.response = Body def call (self, n                                   ): Self.response = None self.corr_id = str (UUID.UUID4 ()) Self.channel.basic_publish (exchange= ", Routing_key= ' Rpc_queue ', Properties=pika.                                         Basicproperties (reply_to = Self.callback_queue, Correlation_id = self.corr_id,), BODY=STR (n)) While Self.response is None:self.connection.process_data_events () return int (self.response) FIBONACC I_rpc = Fibonaccirpcclient () print "[x] requesting FIB (+)" response = Fibonacci_rpc.call (print "[.] Got%r "% (response,)

The client code is a little more complicated:

• (7) We establish a connection, channel and declare a exclusive ' callback ' queue for the reply.
• (16) We subscribe to the ' callback ' queue so that we can receive the RPC response.
• (18) Each response, the ' On_response ' callback will be executed to do a very simple job, for each response message, it checks whether correlation_id is the one we are looking for. If it is, save the response in self.responseandbreak the consuming loop.
• (23) Next, we define our main call method-it executes the actual RPC request.
• (24) In this method, we first create a unique correlation_id number and save it-the ' on_response ' callback function will use this value to capture the appropriate response.
• (25) Next, we publish a request message with two properties:reply_to and correlation_id.
• (32) At this point we can sit down and rest and wait for the appropriate response to arrive.
• (33) Finally we return the response to the user.

Our RPC service is now ready. We can start the server:

to request a Fibonacci number, execute the client:

the current design is not the only possible implementation of an RPC service, but it has some important advantages:

• If the RPC server is slow, you can extend it by running another one. Try to run a second rpc_server.py in a new terminal.
• on the client, the RPC request is sent and received just a message. You do not need to call asynchronously, such as Queue_declare. Thus, for a single RPC request, the RPC client only needs one network back and forth.

our code is still too simplistic to solve more complex (but important) problems, such as:

• If no server is running, then how should the customer react?
• should a client have some RPC timeout mechanism?
• if the server fails and throws an exception, should it be forwarded to the client?
• protection against incoming invalid messages (such as checking boundaries, etc.) before processing the message.

can RPC implemented in this way respond to the same client issuing multiple RPC requests at the same time?

(full code for rpc_client.py and rpc_server.py).

Done .

The original address.

RABBITMQ Tutorial-Remote Procedure Call (RPC)