Go microservices framework go-micro deep Learning (a) Introduction to the overall architecture

Product mouth of a small project, from project to development on-line, with time and demand of the continuous explosion, will become more and more complex, become a large project, if the previous project structure is not well designed, code will become more and more bloated, difficult to maintain, late every product iteration on the line will be reaching. Project MicroServices, loosely coupled with the relationship between the modules, is a good choice, with the increase in maintenance costs, but it is still worthwhile.

Micro-service projects in addition to stability I personally also more concerned about several issues:

One: The efficiency and security of data transfer between services.

Second: The dynamic expansion of services, that is, the registration and discovery of services, service clustering.

Third: the micro-service function can be customized, because not all the functions will be in line with your needs, inevitably need to according to their own needs two times to develop some features.

Go-micro is a good RPC microservices framework in the go language, and the functions are perfect, and several of my concerns are well addressed:

One: the inter-service transfer format is PROTOBUF, the efficiency is not said, very fast, also very safe.

Second: Go-micro's service registration and discovery are varied. I personally prefer Etcdv3 service discovery and registration.

Three: The main functions have the corresponding interface, as long as the corresponding interface, you can customize the plug-in according to their own needs.

Spare time to go-micro the source of the system to read it again, the more read the feeling that the framework is well written, and learned a lot of things from it. Just want to tidy up a series of posts, learn Go-micro experience and share with you.

Communication process

The Go-micro communication process is as large as

The server listens for client calls, and Brocker pushes over the information to be processed. And the server side needs to register its existence or extinction to register so that the client can know its state.

Register discovery of the Register service.

The client side obtains the server information from the register, and then each invocation is based on the algorithm to select one of the server to communicate, of course, the communication is to be encoded/decoded, select the transport protocol and a series of processes.

If there is a need to notify all server side can use Brocker for information push.

Brocker information queue to receive and publish information.

Go-micro can be highly customized and his frame structure is inseparable, Go-micro is composed of 8 key interface, each interface can be re-realized according to their own needs, These 8 main inteface also constitute the GO-MICRO frame structure.

These interface Go-micir have his own default implementation, and a go-plugins is a replaceable item implemented on those interfaces. You can also implement your own plugin based on your needs.

This post is mainly to introduce the main structure of GO-MICRO and the functions of these interfaces, the specific details of the later article we will say slowly:

Transort

Interfaces for communication between services. That is, the final implementation of the service delivery and reception, which is customized by these interfaces.

Source:

Type SocketInterface{Recv (*Message) Error Send (*Message) Error Close () Error}type ClientInterface{socket}type ListenerInterface{Addr ()stringClose () Error Accept (func (Socket)) Error}type TransportInterface{Dial (addrstring, opts ... Dialoption) (Client, error) Listen (addrstring, opts ... Listenoption) (Listener, error) String ()string}

The Listen method of Transport is usually called by the server, and he listens on a port waiting for the client to call.

Transport's dial is the way the client connects to the service. He returns a client interface, which returns a client interface that embeds the socket interface, which is the information that sends and receives the communication specifically.

The HTTP transport is the GO-MICRO default synchronous communication mechanism. Of course there are a lot of other plugins: Grpc,nats,tcp,udp,rabbitmq,nats, which is the way it has been implemented now. You can find them in the go-plugins.

Codec

With the transmission, the following is to solve the transmission encoding and decoding problems, Go-micro has a number of encoding and decoding methods, the default implementation is PROTOBUF, of course, there are other ways of implementation, JSON, PROTOBUF, JSONRPC, Mercury and so on.

Source

Type CodecInterface{Readheader (*Message, MessageType) error Readbody (Interface{}) error Write (*message,Interface{}) Error Close () error String ()string}type Messagestruct{Id UInt64 Type MessageType TargetstringMethodstringErrorstringHeader map[string]string}

The Write method of the codec interface is the encoding process, and two read is the decoding process.

Registry

Service registration and discovery, currently implemented by Consul,mdns, etcd,etcdv3,zookeeper,kubernetes. et cetera,

Interface {    Register (*Service, ... registeroption) Error    deregister (*Service) error    GetService (string) ([]*  Service, error)    listservices () ([]*service, error)    Watch (...) Watchoption) (Watcher, error)    string    options () options}

Simply put, the service registers, the client uses the Watch method to monitor, and when a service is added or deleted, this method is triggered to alert the client to update service information.

The default is service registration and discovery is consul, but personal is not recommended because you cannot use consul cluster directly

I personally prefer etcdv3 clusters. You can choose according to your preferences.

Selector

Based on registry, Selector is a client-level load balancer, and when a client sends a request to the service, Selector according to the different algorithms from the Registery host list, gets the available service nodes to communicate. At present, there are cyclic algorithms and stochastic algorithms, the default is random algorithm.

Source:

Type SelectorInterface{Init (opts ... Option) the error options () options//Select returns a function which should return the next nodeSelect (Servicestring, opts ... Selectoption) (Next, error)//Mark sets the Success/error against a nodeMark (Servicestring, node *Registry. Node, err Error)//Reset returns state back to zero for a serviceReset (Servicestring)    //Close renders the selector unusableClose () error//Name of the selectorString ()string}

The default is the implementation is the local cache, the current implementation of the blacklist,label,named, and so on.

Broker

Broker is the interface for message publishing and subscriptions. A very simple example, because the node of the service is not fixed, if you need to modify all the service behavior needs, you can make the service subscribe to a topic, when there is information published, all the listening service will receive information, according to your needs to do the corresponding behavior.

Source

Interface {    options () options    string    Connect () error    Disconnect () error    Init (... Option) Error    Publish (string, *Message, ...) publishoption) Error    Subscribe (string, Handler, ... Subscribeoption) (subscriber, error)    string}

Broker is implemented by default in HTTP mode, but this is not used in production environments. There are many mature message queue implementations in Go-plugins, such as Kafka, NSQ, RABBITMQ, Redis, and so on.

Client

The client is the interface for requesting the service, and he encapsulates transport and codec for RPC calls, and also encapsulates brocker for information publishing.

Source

 type Client interface   {Init (... Option) Error options () Options newmessage (topic  string , msg interface  {}, opts ... messageoption) Message newrequest (service, method  string , req interface  {}, Reqopts ... requestoption) Request Call (CTX context. Context, req Request, RSP  interface  {}, OPTs ... calloption) Error Stream (CTX context. Context, req Request, opts ... Calloption) (Stream, error) Publish (CTX context. Context, msg Message, opts ... publishoption) Error String ()  string  } 

Of course, he also supports the duplex communication Stream these specific implementation methods and use, will be explained in detail later.

The default is RPC implementation, he also has GRPC and HTTP way, in Go-plugins can find

Server

The server looks at the name and everyone knows what it is. Listens for RPC requests. Listen to the broker's subscription information, wait for the message queue to push, and so on.

Source

Interface {    options () options    Init (... Option) Error    Handle (Handler) Error    Newhandler (interface{}, ... handleroption) Handler    newsubscriber (stringinterface{}, ... subscriberoption) Subscriber    Subscribe (subscriber) Error    Register () error    deregister () error    Start () Error    Stop () Error    string}

The default is RPC implementation, he also has GRPC and HTTP way, in Go-plugins can find

Service

Service is the client and server package, he contains a series of methods using the initial value to initialize the service and client, so that we can easily create an RPC service.

Source:

Interface {    Init (... Option) Options    ()    client (). Client    Server () server. Server    Run () Error    string}

Specific details, my future post will be for everyone to expand, I hope this post, can help you to Go-micro's overall framework has a preliminary understanding