Go RPC Source code analysis

This is a creation in Article, where the information may have evolved or changed.

1. Overview

Go source with the RPC framework, in a relatively streamlined way to implement the RPC function, the current source of RPC has announced the official no longer add new features, and recommend the use of GRPC.
As the RPC framework in the Go standard library, there are still a lot of places worth learning from, this will analyze the Go native RPC framework from the source point of view, and share some of the pits encountered during use.

2.server Terminal

The server side is divided into two steps, first method registration, the method is removed by reflection processing, and coexist into the map. Then there is the network call, which is mainly the listening port, reading the packet, decoding the request
Invokes the reflection-processed method, which returns a value encoded and returned to the client.

2.1 Method Registration

2.1.1 Register

// Register publishes the receiver's methods in the DefaultServer.func Register(rcvr interface{}) error { return DefaultServer.Register(rcvr) }// RegisterName is like Register but uses the provided name for the type// instead of the receiver's concrete type.func RegisterName(name string, rcvr interface{}) error {    return DefaultServer.RegisterName(name, rcvr)}

As above, there are two entry functions for the method registration, register and Registername, where interface{} is usually the object with the method. If you want to customize the receive object for a method, you can use Registername.

2.1.2 Reflection Processing Process

type methodType struct {    sync.Mutex // protects counters    method     reflect.Method    //反射后的函数    ArgType    reflect.Type      //请求参数的反射值    ReplyType  reflect.Type      //返回参数的反射值    numCalls   uint              //调用次数}type service struct {    name   string                 // 服务名,这里通常为register时的对象名或自定义对象名    rcvr   reflect.Value          // 服务的接收者的反射值    typ    reflect.Type           // 接收者的类型    method map[string]*methodType // 对象的所有方法的反射结果.}

The reflection processing process, in fact, is the object and the method of the object, through reflection to generate the above structure, such as registering arith.multiply (XX,XX) error Such an object, the resulting structure is map["Arith"]service, service Medium Ethod is map["Multiply"]methodtype.

Several key codes are as follows:

Build Service Object

func (server *Server) register(rcvr interface{}, name string, useName bool) error {    //生成service    s := new(service)    s.typ = reflect.TypeOf(rcvr)    s.rcvr = reflect.ValueOf(rcvr)    sname := reflect.Indirect(s.rcvr).Type().Name()     ....    s.name = sname    // 通过suitableMethods将对象的方法转换成map[string]*methodType结构    s.method = suitableMethods(s.typ, true)        ....    //service存储为键值对    if _, dup := server.serviceMap.LoadOrStore(sname, s); dup {        return errors.New("rpc: service already defined: " + sname)    }    return nil}

Build Map[string] *methodtype

Func suitablemethods (Typ reflect.  Type, reporterr bool) Map[string]*methodtype {methods: = Make (Map[string]*methodtype)//through reflection, traverse all methods for M: = 0; M < Typ. Nummethod (); m++ {method: = Typ. Method (m) Mtype: = method. Type Mname: = method.        Name//Method must be exported. If method.        Pkgpath! = "" {Continue}//Method needs three ins:receiver, *args, *reply. If Mtype. Numin ()! = 3 {if Reporterr {log. Println ("Method", Mname, "has wrong number of ins:", Mtype. Numin ())} continue}//Remove request parameter type Argtype: = Mtype. In (1) ...//Take out the response parameter type, the response parameter must be pointer replytype: = Mtype. In (2) if Replytype.kind ()! = reflect. PTR {if Reporterr {log.        Println ("Method", Mname, "reply type not a pointer:", Replytype)} Continue} ...        To remove the return value of a function, the return value of the function must be error. If returntype: = Mtype. Out (0); ReturnType! = Typeoferror {if Reporterr {log.                Println ("Method", Mname, "returns", Returntype.string (), "Not Error")} Continue}    Store method as Key-value methods[mname] = &methodtype{method:method, Argtype:argtype, Replytype:replytype} } Return methods}

2.2 Network calls

// Request 每次rpc调用的请求的头部分type Request struct {    ServiceMethod string   // 格式为: "Service.Method"    Seq           uint64   // 客户端生成的序列号    next          *Request // server端保持的链表}// Response 每次rpc调用的响应的头部分type Response struct {    ServiceMethod string    // 对应请求部分的 ServiceMethod    Seq           uint64    // 对应请求部分的 Seq    Error         string    // 错误    next          *Response // server端保持的链表}

As above, the network call mainly uses the above two structure body, is the request parameter as well as the return parameter, through the codec (Gob/json) realizes the binary to the structure the mutual transformation. Mainly involves the following steps:

The key code is as follows:
Take out the request and get the call parameters of the corresponding function

  func (server *server) Readrequestheader (codec Servercodec) (Svc *service, Mtype *methodtype, req *request, Kee    preading bool, err Error) {//Grab the request header. req = Server.getrequest ()//encoder read Generate Request Err = codec. Readrequestheader (req) If err! = Nil {//error handling ... return} Keepreading = TRUE//Remove service name and method Name dot: = Strings. LastIndex (req.    Servicemethod, ".") If dot < 0 {err = errors. New ("Rpc:service/method request ill-formed:" + req. Servicemethod) return} serviceName: = req. servicemethod[:d ot] methodName: = req. Servicemethod[dot+1:]//The structure of the corresponding method is queried from the map generated at the time of registration Svci, OK: = Server.serviceMap.Load (serviceName) if!ok {ER R = errors. New ("Rpc:can ' t find service" + req. Servicemethod) return} svc = Svci. (*service)//Gets the type of the method mtype = Svc.method[methodname] if Mtype = = Nil {err = errors. New ("Rpc:can ' t Find method" + req. Servicemethod)}  

Loop processing, constantly reading the link on the stream of bytes, decrypting the request, calling the method, encoding the response, write back to the client.

func (server *Server) ServeCodec(codec ServerCodec) {    sending := new(sync.Mutex)    for {        //读取请求        service, mtype, req, argv, replyv, keepReading, err := server.readRequest(codec)        if err != nil {            ...        }        //调用        go service.call(server, sending, mtype, req, argv, replyv, codec)    }    codec.Close()}

function calls via parameters

func (s *service) call(server *Server, sending *sync.Mutex, mtype *methodType, req *Request, argv, replyv reflect.Value, codec ServerCodec) {    mtype.Lock()    mtype.numCalls++    mtype.Unlock()    function := mtype.method.Func    // 通过反射进行函数调用    returnValues := function.Call([]reflect.Value{s.rcvr, argv, replyv})    // 返回值是不为空时,则取出错误的string    errInter := returnValues[0].Interface()    errmsg := ""    if errInter != nil {        errmsg = errInter.(error).Error()    }        //发送相应,并释放请求结构    server.sendResponse(sending, req, replyv.Interface(), codec, errmsg)    server.freeRequest(req)}

3.client Terminal

// 异步调用func (client *Client) Go(serviceMethod string, args interface{}, reply interface{}, done chan *Call) *Call {}// 同步调用func (client *Client) Call(serviceMethod string, args interface{}, reply interface{}) error {}

// Call represents an active RPC.type Call struct {    ServiceMethod string      // 服务名及方法名 格式:服务.方法    Args          interface{} // 函数的请求参数 (*struct).    Reply         interface{} // 函数的响应参数 (*struct).    Error         error       // 方法完成后 error的状态.    Done          chan *Call  // 方法调用结束后的channel.}

The client side part is relatively much simpler, mainly provides call and go two methods, respectively, to represent synchronous calls and asynchronous calls, but in fact, synchronous invocation of the underlying implementation is actually asynchronous call, called when the main use of the call structure, related interpretation as above.

3.1 Main processes

3.2 Key Codes

Send the request part of the code, each send a request, generate a Call object, and use the SEQ as key to save in the map, the server returned from the map call, the corresponding processing.

func (client *Client) send(call *Call) {    //请求级别的锁    client.reqMutex.Lock()    defer client.reqMutex.Unlock()    // Register this call.    client.mutex.Lock()    if client.shutdown || client.closing {        call.Error = ErrShutdown        client.mutex.Unlock()        call.done()        return    }    //生成seq,每次调用均生成唯一的seq,在服务端相应后会通过该值进行匹配    seq := client.seq    client.seq++    client.pending[seq] = call    client.mutex.Unlock()    // 请求并发送请求    client.request.Seq = seq    client.request.ServiceMethod = call.ServiceMethod    err := client.codec.WriteRequest(&client.request, call.Args)    if err != nil {        //发送请求错误时,将map中call对象删除.        client.mutex.Lock()        call = client.pending[seq]        delete(client.pending, seq)        client.mutex.Unlock()        if call != nil {            call.Error = err            call.done()        }    }}

The code that receives the response section, here is a for loop, constantly reading the stream on TCP and decoding it into the response object and the reply object of the method.

Func (client *client) input () {var err error var response response for err = = Nil {response = response{} Err = Client.codec.ReadResponseHeader (&response) if err! = nil {break}//through R Seq in esponse Gets the call object seq: = Response. Seq Client.mutex.Lock () Call: = Client.pending[seq] Delete (client.pending, seq) client.mutex.u Nlock () switch {case call = = Nil:err = Client.codec.ReadResponseBody (nil) if err! = Nil {err = errors. New ("Reading error body:" + err.) Error ())} case response. Error! = ""://The server returned a fault and returned the error directly to call. Error = Servererror (response. Error) Err = client.codec.ReadResponseBody (nil) if err! = Nil {err = errors. New ("Reading error body:" + err.)            Error ())} call.done () Default://Through the encoder, the body portion of the resonse is decoded into the reply object. Err = Client.codec.ReadResponseBody (call. Reply) If err! = Nil {call. Error = errors. New ("reading body" + err.)    Error ())} Call.done ()}}//Client exit Processing Client.reqMutex.Lock () Client.mutex.Lock () Client.shutdown = true Closing: = client.closing if Err = = Io. EOF {if closing {err = Errshutdown} else {err = io. Errunexpectedeof}} for _, Call: = Range Client.pending {call. Error = Err Call.done ()} client.mutex.Unlock () Client.reqMutex.Unlock () if Debuglog && err! = Io. EOF &&!closing {log. Println ("Rpc:client protocol error:", err)}}

4. Some pits

• Synchronous call cannot time out

Because the native RPC only provides two methods, synchronous call and asynchronous go, the synchronous call server does not return will always block, here if there is a large number of non-return, it will cause the co-process has been unable to release.

• Memory leak after asynchronous call times out

The time-out function based on the asynchronous call plus channel also has a leak problem because the client request exists in the map structure, and the go function exits without cleaning up the contents of the map, so if the server side does not return, the request in map will be stored, causing a memory leak.

5. Summary

In general, go native RPC is a basic version of RPC, code simplification, scalability, but only to achieve the most basic RPC network communication, such as time-out fuse, link management (keepalive and re-connected), service registration found, or lack of, so still can not reach the production environment out of the box, But Git has a feature-enhanced RPC-based version called RPCX, which supports most of the major RPC features.

6. Reference

RPC https://golang.org/pkg/net/rpc/