Introduction to the RPC of the Go language (including PROTOBUF-RPC)

This is a creation in Article, where the information may have evolved or changed.

Address of this article in Golang China Blog: http://blog.go-china.org/09-protorpc

RPC for Standard libraries

RPC is the short name of a remote call, which is simply called a function that calls the server as if it were a local function.

The standard library of the Go language already provides the RPC framework and different RPC implementations.

Here is an example of a server:

type Echo intfunc (t *Echo) Hi(args string, reply *string) error {    *reply = "echo:" + args    return nil}func main() {    rpc.Register(new(Echo))    rpc.HandleHTTP()    l, e := net.Listen("tcp", ":1234")    if e != nil {        log.Fatal("listen error:", e)    }    http.Serve(l, nil)}

Which is rpc.Register used to register the RPC service, the default name is the type name of the object (here is Echo ). If you need to specify a special name, you can use it rpc.RegisterName to register.

The type of the object being registered all methods that meet the following rules are exported to the RPC service interface:

func (t *T) MethodName(argType T1, replyType *T2) error

The registration must have at least one method to satisfy this feature, or it may fail to register.

It rpc.HandleHTTP is then used to specify the transport protocol for RPC, where the HTTP protocol is used as the carrier for RPC calls. Users can also use the rpc.ServeConn interface to customize their own transport protocols.

The client can call the interface this way Echo.Hi :

func main() {    client, err := rpc.DialHTTP("tcp", "127.0.0.1:1234")    if err != nil {        log.Fatal("dialing:", err)    }    var args = "hello rpc"    var reply string    err = client.Call("Echo.Hi", args, &reply)    if err != nil {        log.Fatal("arith error:", err)    }    fmt.Printf("Arith: %d*%d=%d\n", args.A, args.B, reply)}

The client first uses rpc.DialHTTP a link with the RPC server (the protocol must match).

The client remote function call is then made through the returned object. The name of the function is specified by the client.Call first parameter (is a string).

HTTP-based RPC calls are typically used during debugging, and you can view RPC statistics by default on the Browse "127.0.0.1:1234/debug/rpc" page.

JSON-based RPC calls

In the RPC example above, we used the default HTTP protocol as the transport carrier for RPC calls.

Because of the flaws in the design of the built-in net/rpc package interface, we cannot use jsonrpc such custom encodings as rpc.DialHTTP the underlying protocol. If you need to let the jsonrpc support rpc.DialHTTP function, you need to adjust the RPC interface.

There used to be a Issue2738 for this problem. I have submitted a CL10704046 patch to fix this problem. However, because of the interface to increase RPC, the official is not accepted (because DialHTTP it is easier to rewrite one).

In addition to the transport protocol, you can specify an RPC encoding protocol that is used to encode/program RPC call function parameters and return values. When RPC calls do not specify an encoding protocol, the default is to use the Go language-specific gob encoding protocol.

Because other languages generally do not support the Go Language gob protocol, if a cross-language RPC call is required
Use a common encoding protocol.

The standard library of Go also provides a "net/rpc/jsonrpc" package that provides JSON-encoded-based RPC support.

The server section only needs to use rpc.ServeCodec the specified JSON encoding protocol:

func main() {    lis, err := net.Listen("tcp", ":1234")    if err != nil {        return err    }    defer lis.Close()    srv := rpc.NewServer()    if err := srv.RegisterName("Echo", new(Echo)); err != nil {        return err    }    for {        conn, err := lis.Accept()        if err != nil {            log.Fatalf("lis.Accept(): %v\n", err)        }        go srv.ServeCodec(jsonrpc.NewServerCodec(conn))    }}

The client part value needs jsonrpc.Dial to rpc.Dial be replaced:

func main() {    client, err := jsonrpc.DialHTTP("tcp", "127.0.0.1:1234")    if err != nil {        log.Fatal("dialing:", err)    }    ...}

If you need to communicate with the jsonrpc go language in other languages, you need to encapsulate a jsonrpc
A matching library.

About jsonrpc The implementation details here will not unfold, interested in the words can refer to this article: Json-rpc:a Tale of interfaces.

PROTOBUF-based RPC calls

PROTOBUF is a coding protocol developed by Google. Its advantage is that the encoded data volume is small (not compression algorithm), more suitable for the command of the transmission encoding.

The PROTOBUF official team offers support in several languages Java/c++/python, and the Go language version is supported by the Go team, and other languages are supported by third parties.

The RPC interface can be defined in the PROTOBUF language specification. However, there is no implementation of RPC generation in the Go language and C + + versions of Protobuf.

However, the author developed the RPC implementation PROTORPC on the protobuf basis of the Go language version, and the protoc-gen-go commands provided can generate the corresponding RPC code. Project Address: https://code.google.com/p/protorpc/

The implementation supports the go language and the C + + language and is described in the third-party RPC implementation list on the Protobuf official wiki: https://code.google.com/p/protobuf/wiki/ThirdPartyAddOns#RPC_ Implementations

To use PROTORPC, you need to define the interface () in the proto file first arith.pb/arith.proto :

package arith;// go use cc_generic_services optionoption cc_generic_services = true;message ArithRequest {    optional int32 a = 1;    optional int32 b = 2;}message ArithResponse {    optional int32 val = 1;    optional int32 quo = 2;    optional int32 rem = 3;}service ArithService {    rpc multiply (ArithRequest) returns (ArithResponse);    rpc divide (ArithRequest) returns (ArithResponse);}

PROTORPC uses cc_generic_services selection to control whether the RPC code is output. Therefore, you need to set cc_generic_services the true .

Then download the Protoc-2.5.0-win32.zip and unzip it to get a protoc.exe compile command.

Then use the following command to get the PROTORPC and corresponding protoc-gen-go plug-ins.

go get code.google.com/p/protorpcgo get code.google.com/p/protorpc/protoc-gen-go

Need to be sure protoc.exe and protoc-gen-go.exe both in $PATH . Then run the following command to convert the previous interface file to the go code:

cd arith.pb && protoc --go_out=. arith.proto

The newly generated file is arith.pb/arith.pb.go .

The following are PROTOBUF-RPC-based servers:

package mainimport (    "errors"    "code.google.com/p/goprotobuf/proto"    "./arith.pb")type Arith intfunc (t *Arith) Multiply(args *arith.ArithRequest, reply *arith.ArithResponse) error {    reply.Val = proto.Int32(args.GetA() * args.GetB())    return nil}func (t *Arith) Divide(args *arith.ArithRequest, reply *arith.ArithResponse) error {    if args.GetB() == 0 {        return errors.New("divide by zero")    }    reply.Quo = proto.Int32(args.GetA() / args.GetB())    reply.Rem = proto.Int32(args.GetA() % args.GetB())    return nil}func main() {    arith.ListenAndServeArithService("tcp", ":1984", new(Arith))}

Where the name of the import "./arith.pb" is arith , arith.pb/arith.proto defined in the file (these 2 may be different names, be careful when importing).

arith.ArithRequestAnd arith.ArithResponse is the input and output parameters of the RPC interface, which are also arith.pb/arith.proto defined in the file.

There is also a function generated to arith.ListenAndServeArithService start the RPC service. The third parameter of the function is the RPC service object, which must satisfy arith.EchoService the definition of the interface.

The use of the client is also very simple, as long as one arith.DialArithService can be linked:

stub, client, err := arith.DialArithService("tcp", "127.0.0.1:1984")if err != nil {    log.Fatal(`arith.DialArithService("tcp", "127.0.0.1:1984"):`, err)}defer client.Close()

arith.DialArithServiceReturns an stub object that has been bound to various methods of RPC and can be called directly (no string is required to specify the method name):

var args ArithRequestvar reply ArithResponseargs.A = proto.Int32(7)args.B = proto.Int32(8)if err = stub.Multiply(&args, &reply); err != nil {    log.Fatal("arith error:", err)}fmt.Printf("Arith: %d*%d=%d", args.GetA(), args.GetB(), reply.GetVal())

Compared to the standard RPC library, the PROTORPC consists of the following advantages:

1. Use standard PROTOBUF protocol for easy interaction with other languages
2. Your own protoc-gen-go plug-in can generate RPC code, simplifying the use of
3. Server registration and calling clients are specific types rather than strings and interface{} can be secured by the compiler
4. The lower layer uses snappy compressed data to improve efficiency.

The disadvantage is that the process is cumbersome to use than standard RPC (the need to convert proto to go code).

C + + calls the PROTOBUF-RPC service provided by Go

PROTORPC also provides the implementation of the C + + language.

The C + + version is installed as follows:

1. hg clone https://code.google.com/p/protorpc.cxx/
2. cd protorpc.cxx
3. Build with CMake

The C + + version of PROTORPC has protoc.exe extended a
--cxx_outoption to generate the code for the RPC:

${protorpc_root}/protobuf/bin/protoc --cxx_out=. arith.proto

Note: --cxx_out the code generated by the option has support for serialization and deserialization of XML in addition to RPC support.

The following is a C + + client link to the Go language version of the server:

#include "arith.pb.h"#include <google/protobuf/rpc/rpc_server.h>#include <google/protobuf/rpc/rpc_client.h>int main() {  ::google::protobuf::rpc::Client client("127.0.0.1", 1234);  service::ArithService::Stub arithStub(&client);  ::service::ArithRequest arithArgs;  ::service::ArithResponse arithReply;  ::google::protobuf::rpc::Error err;  // EchoService.mul  arithArgs.set_a(3);  arithArgs.set_b(4);  err = arithStub.multiply(&arithArgs, &arithReply);  if(!err.IsNil()) {    fprintf(stderr, "arithStub.multiply: %s\n", err.String().c_str());    return -1;  }  if(arithReply.c() != 12) {    fprintf(stderr, "arithStub.multiply: expected = %d, got = %d\n", 12, arithReply.c());    return -1;  }  printf("Done.\n");  return 0;}

For detailed instructions please refer to: readme.md.
For more examples, please refer to: rpcserver.cc
and rpcclient.cc

Summarize

The Go language RPC client is a simple and powerful RPC library. Standard-based RPC libraries we can easily customize our RPC implementations (both the transport Protocol and the serialization Protocol can be customized).

However, in the process of developing PROTORPC also found net/rpc some shortcomings of the package:

• Built-in HTTP protocol for RPC serialization protocol and transport protocol coupling too tightly, the user-extended protocol cannot support the built-in HTTP transport protocol (because rpc.Server rpc.Client of problems caused by and interface defects)
• rpc.ServerYou can register only rpc.ServerCodec , but you cannot register a factory function. Instead of jsonrpc.NewServerCodec having to rely on establishing a link ( conn parameter), this results HTTP in a protocol that only supports built gob -in protocols
• rpc.ClientThe problems and rpc.Server similar

Because GO1 needs to ensure API compatibility, the above-mentioned problems can only be expected to improve in the future Go2.