GRPC service discovery and load balancing

This is a creation in Article, where the information may have evolved or changed.

1) Introduction

The main implementation mechanism for GRPC load balancing is external load balancing, which is to provide the client with an updated list of servers through an external load balancer.

The GRPC client also has built-in support for a few load balancing policy APIs, including the GRPCLB policy, which enables external load balancing, but does not encourage users to add more policies in GRPC. The new load balancing strategy should be implemented in an external load balancer.

2) Workflow

The load balancing policy is compatible with the GRPC client in terms of name resolution and server connectivity, and the following is how it works:

1. First, the GRPC client issues a request for server name resolution, which is resolved to one or more IP addresses, each of which indicates whether it is a server address or a load balancer address, and the service configuration indicates the load balancing policy selected by the client (for example, Round_ Robin or GRPCLB).

2. The client instantiates the load balancing policy.

Note: If any address that is returned by the address resolver is a balancer address, the client will use the GRPCLB policy to directly ignore the load balancing policy noted in the service configuration. If there is no Balancer address, the client will use the load-balancing policy noted in the service configuration. If the load policy is not indicated in the service configuration, the client defaults to the policy that selects the first available server address.

3. The load balancing policy creates a sub-channel to each server address.

L for policies other than GRPCLB, this means that each parsed address has a sub-channel. Note: These policies ignore any resolved balancer addresses.

For GRPCLB policies, the workflow is as follows:

A) This policy creates a stream that leads to the balancer address. It asks the balancer for the server address where the client originally requested the server name.

Note: The current GRPCLB policy ignores all non-balancer resolved addresses. In the future, this mechanism may change, and these non-balancer parsing addresses will be an alternative to preventing any balancers from being associated.

b) The GRPC server cluster that is directed to the client by the load balancer informs the load balancer of the load.

c) The Load Balancer returns a list of servers to the GRPC client's GRPCLB policy. The GRPCLB policy then creates a sub-channel to the server in the list.

4. For each RPC send, the load balancing policy determines which sub-channel the RPC should be routed to.

For the GRPCLB policy, the client sends a request to the server in the order in which the load Balancer returns the list of servers, and if the server list is empty, the request is blocked until the list of non-empty servers is returned.

3) Introduction to other load balancing methods

1 , centralized LB ( Proxy Model )

There is a separate lb between service consumers and service providers, typically specialized hardware devices such as F5, or software-based implementations such as Lvs,haproxy. There is an address mapping table for all services on LB, which is usually registered by the operations configuration, and when a service consumer invokes a target service, it initiates a request to LB and forwards the request to the target service with a load balancer of a certain policy, such as polling (Round-robin). LB generally has the ability to check health, and can automatically remove unhealthy service instances. The main issues of the programme:

1. Single point of issue, all service call traffic through LB, when the number of services and the amount of calls, LB easily become a bottleneck, and once the LB failure affects the entire system;

2. The service consumer, the provider has increased the level, has a certain performance cost.

2 , in-process LB ( balancing-awareclient)

For the lack of the first scenario, this scenario integrates LB functionality into the service consumer process, also known as a soft-load or client-side load scenario. When the service provider starts, it first registers the service address with the service registry, and periodically reports the heartbeat to the service registry to indicate that the service is alive, equivalent to a health check, and when a service consumer accesses a service, it queries the service registry through the built-in lb component, caches and periodically refreshes the target service address list. Then select a target service address with a load balancing policy, and finally initiate a request to the target service. LB and service discovery capabilities are dispersed within the process of each service consumer, while the service consumer and the service provider are directly called, with no additional overhead and better performance. The main issues of the programme:

1. Development costs, the program integrates service callers into the client's process, if there are many different language stacks, it is necessary to cooperate with the development of a variety of different clients, there is a certain amount of research and development and maintenance costs;

2. In addition, in the production environment, if you want to upgrade the customer library, the service callers will be required to modify the code and re-release, the upgrade is more complex.

4) Grpc combined with ETCD load Balancing method (for reference only)

ETCD introduction in the text ETCD implementation of the technical summary has been described in detail, grpc combined with ETCD to achieve external load balancing the following four steps:

1. Service Registration

With the service name as key, one or more server IP as value, and with the lease put into the ETCD cluster;

2. Service Request

The GRPC client sends the service name to the ETCD server;

3. Service Response

ETCD obtains the server cluster IP of the service through the Get interface, sorts the server cluster IP through the server load report information, and returns the sorted result list to the client;

4.grpc Service Request and Response

The GRPC client connects to the GRPC server via the IP list returned by the ETCD server, enabling the GRPC service.

5) Implementation based on in-process lb scheme

According to the design ideas provided by GRPC, based on the in-process LB program (2nd case, Ali Open Source Service Framework Dubbo also adopt a similar mechanism), combined with distributed and consistent components (such as zookeeper, Consul, ETCD), Find actionable solutions for GRPC service discovery and load balancing. Next, take the go language as an example and briefly introduce the key code implementation based on ETCD3:

1) Named resolution implementation: RESOLVER.GO

Package Etcdv3

Import (

"Errors"

"FMT"

"Strings"

ETCD3 "GITHUB.COM/COREOS/ETCD/CLIENTV3"

"Google.golang.org/grpc/naming"

)

Resolver is the implementaion of Grpc.naming.Resolver

Type resolver struct {

ServiceName string//service name to resolve

}

Newresolver return resolver with service name

Func newresolver (ServiceName string) *resolver {

Return &resolver{servicename:servicename}

}

Resolve to Resolve the service from ETCD, Target is the dial address of ETCD

Target example: "http://127.0.0.1:2379,http://127.0.0.1:12379,http://127.0.0.1:22379"

Func (re *resolver) Resolve (target string) (naming. Watcher, error) {

if re.servicename = = "" {

return nil, errors. New ("Grpclb:no service name provided")

}

Generate ETCD Client

Client, Err: = Etcd3. New (ETCD3. config{

Endpoints:strings. Split (Target, ","),

})

If err! = Nil {

return nil, FMT. Errorf ("Grpclb:creat ETCD3 client failed:%s", err. Error ())

}

Return Watcher

Return &watcher{re:re, client: *client}, Nil

}

2) Service Discovery implementation: Watcher.go

Package Etcdv3

Import (

"Errors"

"FMT"

"Strings"

ETCD3 "GITHUB.COM/COREOS/ETCD/CLIENTV3"

"Google.golang.org/grpc/naming"

)

Resolver is the implementaion of Grpc.naming.Resolver

Type resolver struct {

ServiceName string//service name to resolve

}

Newresolver return resolver with service name

Func newresolver (ServiceName string) *resolver {

Return &resolver{servicename:servicename}

}

Resolve to Resolve the service from ETCD, Target is the dial address of ETCD

Target example: "http://127.0.0.1:2379,http://127.0.0.1:12379,http://127.0.0.1:22379"

Func (re *resolver) Resolve (target string) (naming. Watcher, error) {

if re.servicename = = "" {

return nil, errors. New ("Grpclb:no service name provided")

}

Generate ETCD Client

Client, Err: = Etcd3. New (ETCD3. config{

Endpoints:strings. Split (Target, ","),

})

If err! = Nil {

return nil, FMT. Errorf ("Grpclb:creat ETCD3 client failed:%s", err. Error ())

}

Return Watcher

Return &watcher{re:re, client: *client}, Nil

}

3) Service Registration implementation: REGISTER.GO

Package Etcdv3

Import (

"FMT"

"Log"

"Strings"

"Time"

ETCD3 "GITHUB.COM/COREOS/ETCD/CLIENTV3"

"Golang.org/x/net/context"

"Github.com/coreos/etcd/etcdserver/api/v3rpc/rpctypes"

)

Prefix should start and end with no slash

var Prefix = "Etcd3_naming"

var client etcd3. Client

var Servicekey string

var stopsignal = make (chan bool, 1)

Register

Func Register (name string, host string, port int, target string, interval time. Duration, TTL int) error {

Servicevalue: = Fmt. Sprintf ("%s:%d", host, Port)

Servicekey = FMT. Sprintf ("/%s/%s/%s", Prefix, name, Servicevalue)

Get endpoints for Register dial address

var err error

Client, Err: = Etcd3. New (ETCD3. config{

Endpoints:strings. Split (Target, ","),

})

If err! = Nil {

Return to FMT. Errorf ("Grpclb:create ETCD3 client failed:%v", err)

}

Go func () {

Invoke Self-register with Ticker

Ticker: = time. Newticker (interval)

for {

Minimum lease TTL is Ttl-second

Resp, _: = client. Grant (context. TODO (), Int64 (TTL))

Should get first, if not exist, set it

_, Err: = client. Get (context. Background (), Servicekey)

If err! = Nil {

If Err = = Rpctypes. Errkeynotfound {

If _, Err: = client. Put (context. TODO (), Servicekey, Servicevalue, ETCD3. Withlease (resp.id)); Err! = Nil {

Log. Printf ("Grpclb:set service '%s ' with TTL to ETCD3 failed:%s", name, err. Error ())

}

} else {

Log. Printf ("Grpclb:service '%s ' Connect to ETCD3 failed:%s", name, err. Error ())

}

} else {

Refresh set to True for notifying the watcher

If _, Err: = client. Put (context. Background (), Servicekey, Servicevalue, ETCD3. Withlease (resp.id)); Err! = Nil {

Log. Printf ("Grpclb:refresh service '%s ' with TTL to ETCD3 failed:%s", name, err. Error ())

}

}

Select {

Case <-stopsignal:

Return

Case <-ticker. C:

}

}

}()

return Nil

}

UnRegister Delete registered service from ETCD

Func UnRegister () error {

Stopsignal <-True

Stopsignal = Make (chan bool, 1)//Just a hack to avoid multi UnRegister deadlock

var err error;

If _, Err: = client. Delete (context. Background (), Servicekey); Err! = Nil {

Log. Printf ("Grpclb:deregister '%s ' failed:%s", Servicekey, err. Error ())

} else {

Log. Printf ("Grpclb:deregister '%s ' OK.", Servicekey)

}

return err

}

4) Interface description file: Helloworld.proto

Syntax = "Proto3";

Option Java_multiple_files = true;

Option Java_package = "COM.MIDEA.JR.TEST.GRPC";

Option Java_outer_classname = "Helloworldproto";

Option Objc_class_prefix = "HLW";

Package HelloWorld;

The Greeting service definition.

Service Greeter {

Sends a greeting

RPC SayHello (hellorequest) returns (Helloreply) {

}

}

The request message containing the user ' s name.

Message Hellorequest {

string name = 1;

}

The response message containing the greetings

Message helloreply {

String message = 1;

}

5) Implement the service-side interface: Helloworldserver.go

Package Main

Import (

"Flag"

"FMT"

"Log"

"NET"

"OS"

"Os/signal"

"Syscall"

"Time"

"Golang.org/x/net/context"

"Google.golang.org/grpc"

GRPCLB "Com.midea/jr/grpclb/naming/etcd/v3"

"COM.MIDEA/JR/GRPCLB/EXAMPLE/PB"

)

VAR (

Serv = flag. String ("service", "Hello_service", "Service Name")

Port = flag. Int ("Port", 50001, "Listening port")

REG = Flag. String ("Reg", "http://127.0.0.1:2379", "Register ETCD Address")

)

Func Main () {

Flag. Parse ()

Lis, Err: = Net. Listen ("TCP", FMT. Sprintf ("0.0.0.0:%d", *port))

If err! = Nil {

Panic (ERR)

}

Err = GRPCLB. Register (*serv, "127.0.0.1", *port, *reg, time. SECOND*10, 15)

If err! = Nil {

Panic (ERR)

}

CH: = Make (chan os. Signal, 1)

Signal. Notify (CH, syscall. SIGTERM, Syscall. SIGINT, Syscall. SIGKILL, Syscall. SIGHUP, Syscall. Sigquit)

Go func () {

S: = <-ch

Log. PRINTF ("Receive Signal '%v '", s)

GRPCLB. UnRegister ()

Os. Exit (1)

}()

Log. Printf ("Starting Hello Service at%d", *port)

S: = Grpc. NewServer ()

Pb. Registergreeterserver (S, &server{})

S.serve (LIS)

}

Server is used to implement HelloWorld. Greeterserver.

Type Server struct{}

SayHello implements HelloWorld. Greeterserver

Func (S *server) SayHello (CTX context. Context, in *PB. Hellorequest) (*PB. Helloreply, error) {

Fmt. Printf ("%v:receive is%s\n", time. Now (), in. Name)

Return &PB. Helloreply{message: "Hello" + in. Name}, Nil

}

6) Implement Client interface: Helloworldclient.go

Package Main

Import (

"Flag"

"FMT"

"Time"

GRPCLB "Com.midea/jr/grpclb/naming/etcd/v3"

"COM.MIDEA/JR/GRPCLB/EXAMPLE/PB"

"Golang.org/x/net/context"

"Google.golang.org/grpc"

"StrConv"

)

VAR (

Serv = flag. String ("service", "Hello_service", "Service Name")

REG = Flag. String ("Reg", "http://127.0.0.1:2379", "Register ETCD Address")

)

Func Main () {

Flag. Parse ()

r: = grpclb. Newresolver (*serv)

B: = Grpc. Roundrobin (R)

CTX, _: = Context. Withtimeout (context. Background (), 10*time. Second)

Conn, err: = Grpc. Dialcontext (CTX, *reg, Grpc. Withinsecure (), Grpc. Withbalancer (b))

If err! = Nil {

Panic (ERR)

}

Ticker: = time. Newticker (1 * time. Second)

For T: = Range Ticker. E =

Client: = PB. Newgreeterclient (conn)

RESP, err: = client. SayHello (context. Background (), &PB. Hellorequest{name: "World" + StrConv. Itoa (T.second ())})

If Err = = Nil {

Fmt. Printf ("%v:reply is%s\n", T, resp. Message)

}

}

}

7) Run the test

1. Run 3 server S1, S2, s3,1 client C to see if the number of requests received by each server is equal?

2. Close 1 server S1 to see if the request will be transferred to another 2 server?

3. Restart the S1 server to see if the other 2 server requests will be evenly distributed to S1?

4. Shut down the ETCD3 server to see if the client communicates properly with the service side.

Shutdown communication is still normal, but the new server will not be registered in, the service side dropped off the line can not be removed.

5. Restart the ETCD3 server, the service end of the offline can automatically return to normal.

6. Close all server side, the client request will be blocked.

Reference:

http://www.grpc.io/docs/

Https://github.com/grpc/grpc/blob/master/doc/load-balancing.md