This paper mainly discusses the implementation process of thrift RPC supported by message bus. Since RABBITMQ's official Java client provides RABBITMQ-based JSON-RPC, the message bus also provides the JSON-RPC API. It also tried to increase support for THRIFT-RPC for the message bus, hoping that the message bus would provide the infrastructure for SOA at the same time.
Thrift Introduction
Thrift is a cross-language service deployment framework that was originally developed by Facebook in 2007 and entered the Apache Open source project in 2008. Thrift defines RPC interfaces and data types through an intermediate language (IDL, Interface Definition Language), and then generates code in different languages via a compiler (currently supports C++,java, python,php, Ruby, Erlang, Perl, Haskell, C #, Cocoa, Smalltalk, and OCaml), and the generated code is responsible for the implementation of the RPC protocol layer and the Transport layer.
Original intention
The idea of doing this is that RABBITMQ can be used to simulate a communication model such as Request/response, which is the communication model of the usual C/s and b/s architectures. And because of the popularity of RPC, its official Java client has provided an implementation of the JSON-based (text protocol) RPC. The thrift itself is an RPC framework that provides cross-language, multi-protocol, and multi-transmission communication mechanisms. If the two can be linked together, the message bus support for RPC is undoubtedly more perfect.
Ideas
The implementation of thrift is based on a multi-layer protocol stack model similar to TCP/IP. It is characterized by reciprocal communication, logical separation, and hierarchical decoupling. Such as:
At the protocol level, thrift currently supports numerous protocols, which are broadly divided into two categories:
- Binary protocol
- Text protocol (represented as XML, JSON)
Thrift also supports a number of communication transmission mechanisms: socket,iostream,httpclient,file,memory-input,nonblocking, etc., because of the implementation of too many classes, there is no class diagram.
Because its implementation is layered decoupled and interface-oriented, any communication protocol can be combined with any communication mechanism to form a specific RPC implementation! It is thanks to this design that we can implement a new communication transmission mechanism without having to make a fuss over the whole library.
Therefore, if we need to implement a communication mechanism based on RABBITMQ, does it mean that we simply have to implement the communication transport layer at both ends of C/s? To be precise, RPC's client implementation (service consumer) is like this, but RPC's service side (service provider) implementation is not just to implement a communication transport layer. Because the server acts as an responder (responser), it must itself be built into a process Loop (simply understood as a while (true)). Therefore, the implementation on the server side also needs to maintain a processing loop based on MQ message. Roughly as follows:
The current thinking is there, but the status quo is that we need to access not only RABBITMQ but the message bus (message bus is based on the RABBITMQ package). So we have to verify the feasibility of a two-step walk: The first use of RABBITMQ message mechanism, whether to achieve thrift C/s two-way communication; the second is good to fit the message bus proxy mechanism and provide the appropriate API.
Clients that implement RPC are relatively straightforward, so let's start with the client. The main task of the client side is to send the call information of the method according to the binary request data obtained after the protocol layer conversion to the target queue, and then block waiting to get the response, the binary response data is again based on the protocol used by the client to restore, this is the two opposite process, one is from the top down, one is bottom up, If you understand the TCP/IP protocol stack, this should be easy to understand.
Thanks to RABBITMQ's official Java Client, it provides a rpclient class that can initiate a request and fetch data directly in a blocking manner. This is our primary interface for sending and receiving data through the Java client of RABBITMQ. Now we need to adapt the thrift, just re-implement the thrift communication layer of the Client interface: Ttransport.
Thrift provides many implementations of client communication technologies by default, and these implementation classes must implement the Ttransport interface, which has the following class diagram:
This interface defines the interface methods such as input, output, open, and close. The key method of the input and output is flush, and its approximate implementation is as follows:
public void Flush () throws Ttransportexception { byte[] data = This.reqMsgStream.toByteArray (); This.reqMsgStream.reset (); byte[] ResponseData = new Byte[0]; try { responsedata = This.client.primitiveRequest (Secret, target, data, token, timeout), } catch (Exception e) { C6/>exceptionhelper.logexception (Logger, E, "[Flush]"); } This.respmsgstream = new Bytearrayinputstream (responsedata); }
You can see that it takes the request data from the output stream first, then calls the bus message to send the request, blocks the wait response data, and finally builds the input stream based on the response data. As far as RPC's client is concerned, we only need to implement the communication transmission mechanism based on message bus. Let's take a look at the usage code after the client uses the message bus as the communication mechanism:
public void Testthriftrpc () throws Exception { Ttransport transport = new Tamqpclienttransport (this.client, " KLIWHIDUHAIUCVARKJAJKSDBFKJABW ", " Emapdemorpcresponse ", " KLASEHNFKLJASHDNFLHKJAHWLEKDJF ", 10000); Transport.open (); Tprotocol protocol = new Tjsonprotocol (transport); Calcservice.client Client = new calcservice.client (protocol); int result = Client.calcsum (); Logger.info (result); Transport.close (); }
In general, the hierarchy is relatively clear, the construction process is from the lower layer to the upper layer of the way:
First, we build a client-side transport object, where we inject the client of the message bus to handle the client-to-server communication internally.
Then open the client's transport object, if the client does need to do something before or after the communication, you can override Open/close this method, because there are some communication mechanisms that need to open or recycle some resources, but if you do not need to open/ To do extra work in the Close method, write these two sentences on the encoding, as if it were a pattern code.
Finally, a protocol processor is built and the protocol processor is passed to the client code generated by IDL. You can close the transport object after you have finished using it. Here is an extra step: if you no longer need to use the message bus client, return the object.
Server-side implementation through the implementation mechanism before, we already know that for the implementation of the service side, we need to give a message bus as a communication transmission mechanism, but also need a message bus-based processing loop. Like the client, the RabbitMQ official Java client has provided a basic implementation of the rpcserver, and we simply use this as a prototype for the RPC server (no need to reinvent the wheel), only the thrift protocol to handle the related components. That means we need to build a consumer that listen a queue and hlod it, and then embed the thrift processor and the code that the Protocol handles.
This time we'll take a look at the server's usage code, and then see how it's implemented:
Messagebussinglepool Singlepool = new Messagebussinglepool (host, Port);; Messagebus client = Singlepool.getresource (); Server code Wrappedrpcserver rpcserver = null; try { Tprocessor processor = new Calcservice.processor (new Calcserviceimpl ()); Tprotocolfactory inprotocolfactory = new Tjsonprotocol.factory (); Tprotocolfactory outprotocolfactory = new Tjsonprotocol.factory (); Rpcserver = Client.buildrpcserver ("Mshdfjbqwejhfgasdfbjqkygaksdfa", new Thriftmessagehandler (processor, Inprotocolfactory, Outprotocolfactory)); Rpcserver.mainloop (); } finally { rpcserver.close (); Singlepool.returnresource (client); Singlepool.destroy (); }
It directly replaces the server implementation provided by thrift (as the client communicates thrift also provides multiple implementations of the server), instead takes a rpcserver based on the RABBITMQ build and then launches an event loop to block it. Waits for the client to request and to process.
The message bus provides a api:buildrpcserver that builds a encapsulated wrappedrpcserver. The Wrappedrpcserver encapsulates the rpcserver that the RABBITMQ Java client comes from above. Why encapsulation? The main reason is information hiding. In fact, if only access to RABBITMQ, not encapsulation is no problem. But now the purpose is to access the message bus, and the message bus on the Rabbitmqjava client and a layer of encapsulation, shielding some unnecessary settings, and these settings are exactly the necessary parameters to build this instance of the Rpcserver class (such as queue name, Channel, etc.).
So here we inject rpcserver from the constructor of the Wrappedrpcserver class based on a combinatorial approach, making Wrappedrpcserver a proxy for rpcserver, The constructor access identifier for Wrappedrpcserver is set to private because we have built an instance of rpcserver inside the message bus and then constructed an instance of Wrappedrpcserver through a reflection mechanism. Look at the code:
Public Wrappedrpcserver Buildrpcserver (String secret, final Irpcmessageprocessor rpcmsgprocessor) {Node Source = t His.getcontext (). Getconfigmanager (). Getnodeview (Secret). Getcurrentqueue (); try {rpcserver aserver = new Rpcserver (This.getcontext (). Getchannel (), Source.getvalue ()) {@Ov Erride public byte[] Handlecall (queueingconsumer.delivery request, AMQP. Basicproperties replyproperties) {return rpcmsgprocessor.onrpcmessage (Request.getbody ()); } }; constructor<wrappedrpcserver> Rpcserverconstructor = WrappedRpcServer.class.getDeclaredConstructor ( Rpcserver.class); Rpcserverconstructor.setaccessible (TRUE); Wrappedrpcserver wrappedrpcserver = rpcserverconstructor.newinstance (aserver); Rpcserverconstructor.setaccessible (FALSE); return wrappedrpcserver;
Buildrpcserverapi Required parameters In addition to a secret for its own identity, there is an RPC Message Processor interface: Irpcmessageprocessor. It is defined as follows:
Public interface Irpcmessageprocessor {public byte[] Onrpcmessage (byte[] in);}
Its primary role is to provide a gateway to thrift RPC (or other RPC frameworks that may subsequently be accessed) to connect to the message bus. It has an input as an argument, and each RPC server processes the request internally and returns the structure as an output parameter. As you can see, this interface does not have any dependencies on third parties, and the input parameters and output parameters are byte[], so it can be adapted to any RPC framework similar to thrift (assuming that these RPC frameworks are as good as thrift).
With the input and output, it is similar to having a "pipeline" in which the intermediate interception process can embed thrift RPC's third-party processing code. For thrift, we have implemented a thriftmessagehandler that shows how thrift seamlessly connects the message bus:
Public byte[] Onrpcmessage (byte[] inmsg) { InputStream in = new Bytearrayinputstream (inmsg); OutputStream out = new Bytearrayoutputstream (); Ttransport transport = new Tiostreamtransport (in, out); Tprotocol Inprotocol = Inprotocolfactory.getprotocol (transport); Tprotocol Outprotocol = Outprotocolfactory.getprotocol (transport); try { processor.process (Inprotocol, outprotocol); Return ((Bytearrayoutputstream) out). Tobytearray (); } catch (Texception e) { exceptionhelper.logexception (logger, E, "onrpcmessage"); throw new RuntimeException (e); } finally { transport.close (); } }
First it takes the method parameter as input, builds an input stream based on the input, and constructs an empty output stream. The Ttransport object (which is the base of the thrift communication transmission and the bottom of the Thrift protocol stack) can be built out of the input and output streams. Next, constructs the protocol processing object for the input decoding and the output encoding. The core processing object of the thrift is then passed in: Processor is processed, and the resulting raw output is returned as a method.
It explains thrift's server-side processing logic and thrift seamless access to the message bus through Irpcmessageprocessor, So we haven't mentioned when and how the code was triggered (that is, how does irpcmessageprocessor connect message events on the message bus)? This is done by overwriting the rpcserver message-handling code. See Code:
Rpcserver aserver = new Rpcserver (This.getcontext (). Getchannel (), Source.getvalue ()) { @Override public byte[ ] Handlecall (queueingconsumer.delivery request, AMQP. Basicproperties replyproperties) { return Rpcmsgprocessor.onrpcmessage (Request.getbody ()); } };
When we build the Rpcserver instance, we overwrite the processing of the message, which triggers a call to the Onrpcmessage method of the Irpcmessageprocessor. The invocation of this method joins the thrift code logic, and if the RPC framework here is not thrift, it can also be used to interface with other RPC frameworks.
Written in the end as the thrift itself provides a variety of communication transmission mode, the message bus as a thrift communication mechanism makes the thrift-based RPC communication mode more possible.
Queueing & Load Balancer the benefit of MQ-based RPC transport is that because of the buffering nature of the queue itself, it is possible to ensure that requests are not lost when dealing with high concurrency (in fact, for Web requests, a mechanism commonly used in the face of high concurrency is queued) In addition, the distributed cluster provided by MQ is preceded by a haproxy to provide good load balancing.
The lower throughput is certainly the lighter the protocol (the smaller the amount of data), the simpler the processing of the communication, the faster the Rpcserver processing requests, the more requests are processed per unit of time, and the higher the throughput. The final test results show that, in a single-threaded case, the THRIFT-RPC based on the message bus is an order of magnitude slower than the socket-based THRIFT-RPC (a non-dedicated PC, with a QPS of around 300). By comparing the JSON-RPC of the thrift based on the message bus with the JSON-RPC test based on the native of the message bus, it is found that the test results are similar, so we can guess the main bottleneck or the RABBITMQ-based transmission. Because the first AMQP protocol-oriented scenario is mainly high-availability scenarios, the implementation of its service side is slightly more complex, and the protocol data is much larger than the bare data of the direct socket, in addition to both the service side and the client has a pair of unpacking and encapsulation process, These all pull down the performance of the message bus-based RPC.
Is the balance point performance the only consideration? This is a good question, or whether performance is the most critical factor. In fact, a lot of things are trade-offs, and all you have to do is see if you can find a balance on all sides of your attention. All solutions for general purpose are not the most streamlined and the best in performance. This, like those of the ESB, has a much worse performance than a Web server that provides restful APIs.
Integration of message bus extensions THRIFT-RPC
