Go Micro Service Practice

This is a creation in Article, where the information may have evolved or changed.

Brief introduction

In the last one or two years, the microservices architecture has become a hot topic (microservices.io), compared with the traditional integrated application architecture, the micro-service architecture has many attractive points in the development, testing and deployment, and more and more new projects with no historical burden have been developed by the mode of micro-service architecture.

Our team after deep thinking, decided to together the beauty of this app in the back-end development, choose go as the development language, the use of micro-service model to achieve, after nearly half a year of practice, the formation of some experience, a simple summary after sharing, I hope to give you some help.

Frame selection

Different teams have different elements to consider when choosing the base framework (library), and our team prefers a small, beautiful framework that allows the framework to break into the business, easy to upgrade, maintain, and replace, so we prefer to choose the library rather than the framework.

On the web side, we chose Negroni as the middleware library, replacing the mux of the Go standard library with a well-performing httprouter, without any web-related frameworks.

In terms of RPC calls between microservices, for future extensibility, cross-language calls, and so on, we did not directly use the Go standard library RPC module, but adopted the latest Google Grpc. But Grpc itself belongs to the relatively heavy RPC framework, the business code has a certain degree of intrusion, we do a series of libraries (including WORPC, WORC, wonaming, etc. https://github.com/wothing) to block these unnecessary business code intrusion, Keep the business code itself tidy.

Micro Service Division

In the micro-service system, how to divide the micro-service is also an important topic, in our practice, we have followed the following principles:

1. Logic Independent, clear-boundary module as a standalone micro-service

2. Each table is operated by only one microservices (including insert, read, change, delete, etc.)

3. FOREIGN KEY constraints are not introduced between table, and ID fields all use UUID

4. Keep data consistency in one microservices and avoid data consistency challenges across services

5. Communication between microservices, using Message Queuing to achieve loose coupling whenever possible, and then using RPC when synchronous calls are required

6. Micro-services are deployed independently to enable registration and discovery of services through ETCD

Overall architecture

Gateway

Gateway is a barrier to the external service of microservices, and its core points are:

1. Shielded microservices provide a RESTful API interface based on HTTP protocol for Web pages and mobile apps through communication such as Message Queuing, RPC, etc.

2. Perform the necessary authentication and data integrity and legality checks on every HTTP business request to reduce the burden of microservices and make the MicroServices code more pure

3. In a microservices deployment system, each microservices may deploy multiple instances, and the gateway also assumes the ability to load-balance these instances

4. Perform the necessary log output, monitor and control functions, generate a unique trace ID for each HTTP request from the app and page, and transmit the trace ID to each subsequent microservices for subsequent troubleshooting and performance tuning

5. Each of the gateway HTTP requests is stateless and uses the JWT (Json Web Token) mechanism to implement a client's request state information delivery

Registration and discovery of services wonaming

In MicroServices, the registration and discovery of services is important for the overall architecture, especially for synchronous RPC calls, how many instances each service has, and the address of each instance, all of which require a unified management. We use ETCD to store service information while encapsulating Wonaming as a middleware for micro-service registration and discovery, and its main features include:

1. When the service starts, it calls wonaming to Etcd to register the Service "index" that contains the TTL,

2. After registration, the service and ETCD maintain a timed heartbeat, when the microservices active exit or timeout, the service is registered and "offline"

3. In the gateway, the service discovery through the resolver, with the GRPC provided by the balancer to achieve load balancing, resolver the ETCD in the/wonaming directory monitoring, when there is a service registration or registration, dynamic maintenance of the list of available services.

r: = wonaming. Newresolver (name) b: = Grpc. Roundrobin (R) Conn, err: = Grpc. Dial (ETCD, Grpc. Withinsecure (), Grpc. Withbalancer (b))

RPC invocation of the service Worc

GRPC is a relatively heavy RPC framework, when the client through the GRPC call server, need a lot of repetitive code to establish connections, calls, processing error return, etc., affecting the cleanliness of business code, and the business code is very intrusive, in order to circumvent this problem, we encapsulated the WORC, To enable convenient GRPC calls:

RESP, err: = Worc. Callrpc (CTX, "Hello", "Hello", req)

Grpc's middleware chain Worpc

GRPC provides the interceptor mechanism, but does not provide the chain to implement the order of different middleware, in order to encapsulate different middleware functions (such as authentication, logging, recover) in different functions, WORPC provides a combination GPRC Interceptor as a chain ability, according to the needs of their own business, writing different GRPC middleware combinations, such as the implementation of GRPC recovery and log middleware:

 
 Func Recovery (CTX context. Context, req interface{}, Info *grpc. Unaryserverinfo, Handler Grpc. Unaryhandler) (Resp interface{}, err error) {defer func () {if r: = Recover (); r! = nil {//log Stackstack: = Make ([]byte, maxstacksize) stack = stack[:runtime. Stack (stack, false)]log. Ctxerrorf (CTX, "Panic Grpc Invoke:%s, err=%v, stack:\n%s", info.) Fullmethod, R, String (stack))//If panic, set custom error to ' err ', in order the client and sense It.err = Grpc. Errorf (codes. Internal, "Panic Error:%v", R)}} () return handler (CTX, req)}func Logging (CTX context. Context, req interface{}, Info *grpc. Unaryserverinfo, Handler Grpc. Unaryhandler) (Resp interface{}, err Error) {start: = time. Now () log. Ctxinfof (CTX, "calling%s, req=%s", info.) Fullmethod, Marshal (req)) resp, err = Handler (CTX, req) log. Ctxinfof (CTX, "finished%s, Took=%v, Resp=%v, err=%v", info.) Fullmethod, time. Since (start), Marshal (resp), err) return resp, err} 
 

S: = Grpc. NewServer (GRPC. Unaryinterceptor (WORPC. Unaryinterceptorchain (WORPC. Recovery, Worpc. Logging)))

Through the above combination, can provide panic recovery ability for micro-service, ensure the service is stable and available, and also take out the trace ID in the injection context mentioned above, so that the gateway and the Micro Service log through the trace ID to link up, easy to check the wrong, tune.

Other experience

1. Use GRPC. Logic errors in the Errorf encapsulation business are returned with the GRPC service call, separating the business response from the error.

2. The data can be assembled in the gateway, but there is no need to deliberately avoid micro-service intermodulation and clarify dependencies, especially when protobuf upgrades, depending on the business to determine whether the referral microservices need to be re-deployed synchronously.

3. Micro-services, although good, but to a certain extent, will increase the difficulty of implementation, according to the volume of business reasonable into the pit.

Summarize

The above is a micro-service architecture in our team's practical solution, perfectly formed. Through the flexible combination of various middleware, to ensure that the business order and high availability of services, not to grasp the practice? In the following article, we will also introduce the current micro-service testing, operations and deployment scenarios.

The author of this article: Together Beauty-Elvizlai

Github:https://github.com/elvizlai