Golang High Performance Rpc:apache Thrift installation use full raiders

The use of RPC in enterprise applications can be said to be very extensive, the use of this technology can easily interact with a variety of programs without regard to their written language.

If you are not sure about the RPC concept, you can click here.

There are already many widely used RPC frameworks on the market, such as GRPC, and today we are going to introduce the same widely used Apache Thrift. This article will take you safely over all pit spots, please feel free to eat.

Thrift Introduction

Thrift is an open-source project on Facebook that later entered Apache for incubation. Thrift also supports cross-language, so it has its own set of IDL. It currently supports almost all major programming languages: C + +, Java, Python, PHP, Ruby, Erlang, Perl, Haskell, C #, Cocoa, JavaScript, node. js, Smalltalk, OCaml and Delphi and other languages. Thrift can support a variety of information formats, in addition to the thrift proprietary binary encoding Rules and a LVQ (similar to the TLV message format), as well as the regular JSON format. The Thrift network protocol is based on the TCP protocol, and supports the blocking IO model and the multiplexed IO multiplexing model. We will explain the use of Apache Thrift in detail later in this article. Thrift is also one of the most popular RPC frameworks, and the performance of thrift is leading from a variety of performance testing on the network. Thrift's official address is: http://thrift.apache.org/

Installation

The first is to install the Golang library:

Go get git.apache.org/thrift.git/lib/go/thrift/...

In order to compile thrift IDL, we have to compile Thrift-compiler, the main pit is all here.

First, if you're using Ubuntu, don't install it with APT, because the official source and the compiler version in all the PPA are heavily outdated;

Next to the online compilation compiler tutorial do not look, the basic is outdated, and now thrift version is 1.0.0-dev, if the tools compiled by those tutorials can not be used with Go get installed libraries, and a large probability you can not even normal compilation, If you do not use go get, then you can follow the method 2 here to install. (The steps to compile compiler in Method 2, please skip, use the method I want to say below)

Okay, here we go. Install Thrift-compiler, first of all install dependencies:

sudo Install pkg-config g++ Libssl-dev

Next we clone the newest thrift repository and compile:

git clone https://github.com/apache/thriftCD thrift. /bootstrap. SH . /configure--without-qt4--wihout-qt5makesudomakeinstall

To explain,--without-qt* is to speed up compiling and prevent errors, because QT always relies on the MOC, is prone to problems and we generally do not use, so let go.

If you do not need other languages to generate features, you can use--without-[name] to remove, specifically see./configure--help.

Make can also be changed to "Make-j n", N is the number of cores available for your CPU, and parallel compilation speeds up.

After installation we run the Thrift command to verify the installation:

Thrift-version# thrift version 1.0.0-dev

This thrift even if the installation is complete.

In addition, do not use go get to install with the source code, Please ensure that the Thrift-compiler and you install the same version of the library, otherwise there will be countless problems . Of course, there are no such problems with the go get+ self-compiling installation method.

Use of Thrift

First we will write the IDL file, define the RPC interface and data. If you are not familiar with the syntax of thrift IDL, you can refer to this, which is very similar to the syntax of C + + and easier to get started.

Let's take a look at an example where we define a "compute" module in which you define the DIVMOD (compute quotient and modulo) and mulrange (compute factorial) Two services and define a data type that contains the results of the DIVMOD calculation:

namespace Go Compute struct Result {    1: i64 div;     2 : i64 MoD;} Service Divmod {    Result dodivmod (12: i64 arg2);} Service Mulrange {    string bigrange (1: i64 max)}

Thrift IDL

Then we use Thrift-compiler to compile it into the Golang code:

Thrift-r--gen Go compute.thrift

It will be in the current directory to generate a gen-go directory, in which there is a compute directory, which is the module we generated, copy out to put in $gopath.

Next we implement the service side, now we have only defined the interface, and did not implement it, the establishment of the server requires the following several steps:

1. Implement the service processing interface, in version 1.0, the first parameter of the interface needs to be context.context to support cancellation or set timeouts on RPC calls .
2. Create Processor, which is to register the implemented interface into the RPC service to create transport,transport for managing network IO. It is a interface and can be an object returned by Tsocket,tserversocket as well as newthttpclient* or Ttransportfactory.gettransport.
3. Create Protocol, which is the transport protocol, typically uses a binary protocol to transfer data, and of course you can choose JSON or other formats.
4. Create the server, using the previous Processor,transport,protocol and the IP address to create the service side.
5. Run server.

Here we implement the interface first:

//Computethrift Implementing the methods defined in the serviceType Divmodthriftstruct {}//Each method returns an error in addition to the defined return value, including the method defined as void. Custom type adds an underscore after the name
Temporarily does not use the context, so ignore
Func (d *divmodthrift) Dodivmod (_ Context. Context, Arg1, arg2 Int64) (*Compute. Result_, error) {divres:= Int64 (Arg1/arg2) Modres:= Int64 (arg1%arg2)//generated functions for building custom Data ObjectsRes: =Compute. Newresult_ () Res. Div=divres Res. Mod=Modresreturnres, nil}//as much as possible a struct corresponds to a serviceType Mulrangethriftstruct{}func (M*mulrangethrift) Bigrange (_ Context. Context, Max Int64) (string, error) {Result:=New(big.) INT) result. SetString ("1",Ten) Result= result. Mulrange (2, Max)returnresult. String ()}

After the implementation of the interface we will set up and start the server, first we first set up transport:

// Create server servertransport, err: = Thrift. Newtserversocket (NET. Joinhostport ("127.0.0.1""9999"))if Err ! = nil {    FMT. Println ("error! " , Err)    Os. Exit (1)}

Then create the transport protocol:

// Create binary protocol protocolfactory: = thrift. Newtbinaryprotocolfactorydefault () transportfactory:= thrift. Newtframedtransportfactory (Thrift. Newttransportfactory ())

We then register the implemented interface as processor:

 //  Create processor and process multiple services with one port  Divmodprocessor: = Compute. Newdivmodprocessor (new   (Divmodthrift)) Mulrangeprocessor:  = compute.    Newmulrangeprocessor (new   (Mulrangethrift)) Multiprocessor:  = thrift. Newtmultiplexedprocessor ()  //  Give each service a name  multiprocessor.registerprocessor ( " divmod   "   " mulrange   ", Mulrangeprocessor) 

Here we use Tmultiplexedprocessor to implement a port to monitor multiple services.

The end is to start the server:

// Start server : = thrift. NewTSimpleServer4 (Multiprocessor, Servertransport, Transportfactory, protocolfactory) server. Serve ()
Stop server when exiting
Defer server. Stop ()

Boot server There's also a pit point, there are articles on the Internet that use NewTSimpleServer2 can directly enable the data transfer in binary format, which is wrong , want to use binary or other format to transfer, You must explicitly generate the corresponding protocolfactory and use NEWTSIMPLESERVER4 to create the server . This was done in the official case, and I confirmed the problem several times after being repeatedly dug in.

Client:

To establish a client, follow these steps:

1. Create Transport
2. Create Protocol
3. Create a Client based on Potocol
4. Open Transport (not necessarily open after client creation, but must be opened before interface call after protocol creation)
5. Calling interface

Let's look at the code:

Func Main () {//set up a socket connected to the server, and then create a transport through the socketSocket, err: = Thrift. Newtsocket (NET. Joinhostport ("127.0.0.1","9999"))    ifErr! =Nil {fmt. Println ("Error Opening socket:", err) os. Exit (1)} Transport:=Thrift. Newtframedtransport (socket)//Create a binary protocolProtocol: =Thrift. Newtbinaryprotocoltransport (transport)//Open Transport to connect to the server    ifERR: = transport. Open (); Err! =Nil {fmt. Fprintln (OS. Stderr,"Error opening socket to"+"localhost"+":"+"9999", err) os. Exit (1)} defer transport. Close ()//the interface requires a context so that the user can cancel the RPC call during a long operationCTX: =context. Background ()//using the Divmod serviceDivmodprotocol: = Thrift. Newtmultiplexedprotocol (Protocol,"Divmod")    //Create a proxy client to access the corresponding service using Tmultiplexedprotocolc: =Thrift. Newtstandardclient (Divmodprotocol, divmodprotocol) client:=Compute. Newdivmodclient (c) Res, err:= client. Dodivmod (CTX, -,3)    ifErr! =Nil {fmt. PRINTLN (ERR) OS. Exit (1)} FMT. Println (RES)//using the Mulrange service//the steps are the same as aboveMulprotocol: = Thrift. Newtmultiplexedprotocol (Protocol,"Mulrange") C=Thrift. Newtstandardclient (Mulprotocol, Mulprotocol) Client2:=Compute. Newmulrangeclient (c) num, err:= Client2. Bigrange (CTX, -)    ifErr! =Nil {fmt. PRINTLN (ERR) OS. Exit (1)} FMT. PRINTLN (num)}

Of course, most of the cases we may be a single service, for a single service, the 0.11 version of the interface and 1.0 has not changed, this look for examples on the Internet.

We ran client and server separately after go build.

Here is the output, calculate the results of 100÷3 and 100%3, and a 100 factorial, the client calls the interface after checking err, no error occurs on the output:

This example of a multi-service-enabled RPC is complete.

If you have questions or suggestions, please note in the comments.

Have a good time!