Go microservices-part Seventh-service discovery and load balancing

This is a creation in Article, where the information may have evolved or changed.

Part VII: Go microservices-service discovery and load balancing

This section deals with two basic parts of a robust microservices architecture-service discovery and load balancing-as well as how they facilitate the horizontal scaling of important non-functional requirements in 2017.

Brief introduction

Load Balancing is a well-known concept, but I think service discovery needs a deeper understanding, and I'll start with a question.

If service A wants to talk to service B, but does not know where to look for service B how to handle it?

In other words, if our service B runs 10 instances on any number of cluster nodes, then someone needs to track these 10 instances.

Therefore, when service a needs to communicate with service B, it must provide at least one appropriate IP address or hostname (client load balancer) for service A, or service a must be able to delegate address resolution and route to a third party the logical name of a known service B (server-side load balancer). In the changing context of microservices, both of these approaches require service discovery. In its simplest form, service discovery simply registers a running instance for one or more services.

If that sounds like a DNS service to you, it does. The difference is that service discovery is used within the cluster to allow MicroServices to find each other, and DNS is generally more static and suitable for external routing, so the external party can request routing to your service. In addition, DNS services and DNS protocols are often not suitable for handling topologies with changing microservices environments, containers and nodes coming and going, and clients often do not follow TTL values, failure monitoring, and so on.

Most microservices frameworks provide one or more options for service discovery. By default, Spring Cloud/netflix OSS uses Netflix Eureka (supports Consul, ETCD, and Zookeeper), and the service uses known Eureka instances to register itself, The heartbeat is then intermittently sent to ensure that the Eureka instances know they are still active. Consul provides an option that includes a rich set of features for DNS integration that has become increasingly popular. Other popular options are the use of distributed and replicable Key-value storage, such as ETCD in which services can register themselves. Apache Zookeeper will also be aware of the need for such a group of people.

In this article, we mainly deal with some of the mechanisms provided by Docker swarm (Docker in swarm mode) and demonstrate the service abstraction we explored in part fifth, and how it actually provides service discovery and server-side load balancing for us. In addition, we'll look at the simulations in our unit tests using Gock to simulate HTTP request output, as we do the inter-service communication.

Note: When we refer to Docker swarm, I mean running Docker version 1.12 or above in swarm mode. Docker Swarm no longer exists as a standalone concept after Docker 1.12.

Two types of load balancing

In MicroServices, it is common to differentiate between the two types of load balancing mentioned above:

• Client load Balancing.
• Server-side load balancing.

Client Load Balancing
The client queries the discovery service to get the actual address information (IP, hostname, port number) that they want to invoke the service, and when found, they can use a load-balancing policy (such as polling or random) to select a service. In addition, in order to make it unnecessary for each incoming call to query the discovery service, each client typically maintains a local cache of endpoints that must be kept reasonably synchronized with the master information from the discovery service. One example of client load balancing in Spring cloud is the Netflix Ribbon. Similar things also exist in the Go-kit ecology supported by ETCD. Some of the benefits of client load Balancing are elasticity, dispersion, and no central bottleneck, because each service consumer maintains its own production-side registration. The downside is a high level of internal service complexity and the risk that local registrations may contain outdated entries.

Server-Side Load balancing
In this model, the client relies on load balancing to provide the service logical name to query the appropriate instance of the service to invoke. This mode of operation is often referred to as a proxy because it acts as both a load balancer and a reverse proxy. I think the main advantage of it is simplicity. Load balancers and service discovery mechanisms are typically built into your container orchestration, and you don't have to worry about installing and managing these components. In addition, the client (e.g. our service) does not need to know about service registration-the Load balancer is responsible for this for us. Relying on a load balancer to route all calls can reduce resiliency, and the load balancer can theoretically be a bottleneck for performance.

The graphs for client load balancing and server-side load balancing are very similar and differ in the position of lb.

Note: When we use the Swarm mode of Docker's service abstraction, for example the above service-side production service registration is actually completely transparent to you as a developer. In other words, our production services are not even aware of the context in which they operate the server load balancer (or even in the context of container orchestration). Swarm Mode Docker is responsible for all of our registration, heartbeat, and cancellation of registration.

In the 2nd part of the blog series, we've been using the example field, and we might want to request Accountservice to get the current random quote from Quotes-service. In this article, we will focus on using the Docker Swarm service discovery and load balancing mechanism. If you are interested in how to integrate with Go language-based microservices and Eureka, you can refer to my 2016 blog post. I have also written a simple self-contained integrated go application and Eureka Client class library that contains basic lifecycle management.

Consumer Service Discovery Information

Suppose you want to build a custom monitoring application, and you need to query the/health endpoint (route) for each instance of each deployment service. How does your monitoring program know which IP and port to request? You need to master the actual service discovery details. If you are using Docker swarm as a provider of service discovery and load balancing, and need these IPs, how can you master the IP address of each instance that Docker swarm holds for us? For client-side resolution, such as Eureka, you only need to use its API to consume it. However, this may not be as simple in the case of a service discovery mechanism that relies on the orchestration. I think there is a need to pursue a major choice, as well as some minor options to consider more specific use cases.

Docker Remote API

First, I recommend using Docker's remote API-for example, using the Docker API from within the service to query Swarm manager for service and instance information. After all, you are using the container composer's built-in service discovery mechanism, which is the source of your query. For portability, this is a problem and you can always select an adapter for the orchestration you choose. However, it should be noted that the API for using the orchestration has some caveats-it ties your solution closely to the specific container API, and you have to make sure that your application can talk to the Docker manager, for example, that they are aware of some of the contexts in which they are running, The use of the Docker remote API does somewhat increase the complexity of the service.

Alternative Solutions (alternatives)

• Using a separate service discovery mechanism-that is, running Netflix Eureka, Consul, or something like that-and ensuring that, in addition to the Docker swarm mode mechanism, the services can be registered/unregistered microservices found in these service discovery mechanisms. Then we just need to use the Discovery Service's registration/query/Heartbeat API. I don't like this option because it introduces more complex stuff into the service, when the swarm mode of Docker can be more or less transparent for us to deal with most of these things inside. I almost think it's a rice pattern, but if you have to do it, don't do it.
• Apply a specific discovery token-in this way, the service wants to broadcast their presence and can periodically post a discovery token with IP, service name, and so on on a message topic. Consumers need to know the instances and their IP, subscribe to this topic (Topic), and keep their own service instances registered for immediate updating. We will use this mechanism to provide information to a custom Turbine discovery plug-in when we do not use Eureka's Netflix Turbine in a later article. This is a bit different because they don't need to take full advantage of the full service registry-after all, in this particular use case, we only care about a particular set of services.

Source

Please be assured to cut out this part of the code: Https://github.com/callistaen ....

Scaling and load Balancing

Let's move on to this section to see how we can extend our accountservice, let them run into multiple instances, and see if we have the Docker swarm automatically load-balance the requests for us.

In order to know what the actual instance really is for us, we need to add a field to the account that we can populate with the IP address of the production service instance. Open the/accountservice/model/account.go file.

type Account struct {
    Id string `json:" id "`
    Name string `json:" name "`
    
    // NEW
    ServedBy string `json:" servedBy "`
}
Then add the ServedBy attribute to the account in the GetAccount method that provides the account service.

func GetAccount (w http.ResponseWriter, r * http.Request) {
    // Read the 'accountId' path parameter from the mux map
    var accountId = mux.Vars (r) ["accountId"]

    // Read the account struct BoltDB
    account, err: = DBClient.QueryAccount (accountId)

    account.ServedBy = getIP () // NEW, add this line
    ...
}

// ADD THIS FUNC
func getIP () string {
    addrs, err: = net.InterfaceAddrs ()
    if err! = nil {
        return "error"
    }
    for _, address: = range addrs {
        // check the address type and if it is not a loopback the display it
        if ipnet, ok: = address. (* net.IPNet); ok &&! ipnet.IP.IsLoopback () {
            if ipnet.IP.To4 ()! = nil {
                return ipnet.IP.String ()
            }
        }
    }
    panic ("Unable to determine local IP address (non loopback). Exiting.")
}
We use getIP () to get the machine IP and fill it to ServedBy. In a real project, the getIP function should be placed in a specific toolkit, so that each microservice can use it when it needs to obtain a non-loopback ip address.

Then use copyall.sh to rebuild and deploy the accountservice service.

./copyall.sh
Wait a few seconds, then enter the following command:

> docker service ls
ID NAME REPLICAS IMAGE
yim6dgzaimpg accountservice 1/1 someprefix / accountservice
Access using curl is as follows:

> curl $ ManagerIP: 6767 / accounts / 10000
{"id": "10000", "name": "Person_0", "servedBy": "10.255.0.5"}
Great, we've seen that the container's IP address is included in the response. Let's extend this service.

> docker service scale accountservice = 3
accountservice scaled to 3
Wait a few seconds, and then run the docker service ls to check, you get the following:

> docker service ls
ID NAME REPLICAS IMAGE
yim6dgzaimpg accountservice 3/3 someprefix / accountservice
The above shows that the accountservice was copied 3 times. Then make a curl request for account multiple times to see if we get a different IP address each time.

curl $ ManagerIP: 6767 / accounts / 10000
{"id": "10000", "name": "Person_0", "servedBy": "10.0.0.22"}

curl $ ManagerIP: 6767 / accounts / 10000
{"id": "10000", "name": "Person_0", "servedBy": "10.255.0.5"}

curl $ ManagerIP: 6767 / accounts / 10000
{"id": "10000", "name": "Person_0", "servedBy": "10.0.0.18"}

curl $ ManagerIP: 6767 / accounts / 10000
{"id": "10000", "name": "Person_0", "servedBy": "10.0.0.22"}
Before processing the current request at 10.0.0.22, we can see that the 4 calls are looped within three instances. The load balancing provided by this container orchestration using the Docker Swarm service abstraction is very attractive because it removes the complexity of clients based on load balancing (such as Netflix Ribbon), and we can load balance without relying on service discovery Mechanism to provide us with a list of available IP addresses. In addition, no traffic will be routed from Docker Swarm 1.3 to nodes that do not report that they are healthy, provided that health checks are implemented. This is very important. When you need to make the scale larger or smaller, especially when your service is very complicated, it may take more than several hundred milliseconds to start the accountservice we currently need.

FOOTPRINT AND PERFORMANCE WHEN SCALING
Interestingly, if we scale the accountservice instance from one to four, how will it affect latency and CPU / memory usage. Is there a substantial overhead when the load balancer in Swarm mode polls our request?

docker service scale accountservice = 4
Wait a few seconds to get everything ready.

CPU and memory usage during load testing
Gatling tests are run with 1000 requests per second.

CONTAINER CPU% MEM USAGE / LIMIT
accountservice.3.y8j1imkor57nficq6a2xf5gkc 12.69% 9.336 MiB / 1.955 GiB
accountservice.2.3p8adb2i87918ax3age8ah1qp 11.18% 9.414 MiB / 1.955 GiB
accountservice.4.gzglenb06bmb0wew9hdme4z7t 13.32% 9.488 MiB / 1.955 GiB
accountservice.1.y3yojmtxcvva3wa1q9nrh9asb 11.17% 31.26 MiB / 1.955 GiB
Very good, our four instances enjoy almost the same workload, and we see that the other 3 new instances keep memory below 10M. Given this situation, each instance should not need to serve more than 250 requests / s.

performance
First, Gatling refers to an instance:


Gatling then references 4 instances:


The difference is not huge-but it shouldn't be-all four service instances run on the same virtual host Docker Swarm node after all, and share the same underlying hardware (such as my laptop). If we add more visualization instances to Swarm, they can utilize the resources of the unused host OS, then we will see a greater reduction in latency, as it will be separated into different logical CPUs etc to handle the load. However, we saw a slight increase in performance, with an average of about 95/99 percent. We can fully conclude that in this particular scenario, Swarm mode load balancing has no negative impact on performance.

Bring out Quote service
Remember the quote service we implemented in Java in Part 5? Let's extend it too, and then call it from accountservice, using the quotes-service name. The purpose of adding this is to show how transparent service discovery and load balancing are. The only thing we need to do is to know the logical service name of the service we want to call.

We will edit the /goblog/accountservice/model/account.go file, so our response will include a quote.

type Account struct {
    Id string `json:" id "`
    Name string `json:" name "`
    ServedBy string `json:" servedBy "`
    Quote Quote `json:" quote "` // NEW
}

// NEW struct
type Quote struct {
    Text string `json:" quote "`
    ServedBy string `json:" ipAddress "`
    Language string `json:" language "`
}
Note that we used the json tag above to map the fields from the quotes-service output to the quote field of our byte structure, which contains the quote, ipAddress and servedBy fields.

Continue editing /goblog/accountservice/service/handler.go. We will field a simple getQuote function, execute an HTTP call, request http: // quotes-service: 8080 / api / quote, this request will return a quote value, and then we use it to generate a new structure Quote. We call it in the GetAccount () method.

First, we deal with the connection: Keep-Alive problem, which will cause load balancing problems, unless we explicitly configure the Go language client properly. In handlers.go, add the following code above the GetAccount function:

var client = & http.Client {}

func init () {
    var transport http.RoundTripper = & http.Transport {
        DisableKeepAlives: true,
    }
    client.Transport = transport
}
The init function will ensure that any HTTP request from the client instance has the appropriate headers to ensure that Docker Swarm based on load balancing can work properly. Next, in the GetAccount function, add a package-level getQuote function.

func getQuote () (model.Quote, error) {
    req, _: = http.NewRequest ("GET", "http: // quotes-service: 8080 / api / quote? strength = 4", nil)
    resp, err: = client.Do (req)

    if err == nil && resp.StatusCode == 200 {
        quote: = model.Quote {}
        bytes, _: = ioutil.ReadAll (resp.Body)
        json.Unmarshal (bytes, & quote)
        return quote,nil
    } else {
        return model.Quote {}, fmt.Errorf ("Some error")
    }
}
nothing special. The parameter strength = 4 is unique to the quotes-service API and can be used to make it consume more or less CPU. There are still some issues with this request, we returned a generalized error.

We will call the new getQuote function in the GetAccount function, and if no error occurs, assign the Quote property of its return value to the Account instance.

// Read the account struct BoltDB
account, err: = DBClient.QueryAccount (accountId)
account.ServedBy = getIP ()

// NEW call the quotes-service
quote, err: = getQuote ()
if err == nil {
    account.Quote = quote
}
All error checking is one of my least favorite things in Go. Although it can produce very safe code, it can also express the intent of the code more clearly.
Unit tests that do not generate HTTP requests
If we run the unit test for /accountservice/service/handlers_test.go, it will fail. The GetAccount function under test will now try to make an HTTP request to get the famous quote, but since no quote-service operates at a specific URL (I guess it can't solve anything), the test fails.

We can use two alternative strategies for this, given a unit test a context.

Extract the getQuote function as an interface, providing a real implementation and a mock implementation, just like we did for the Bolt client in Part 4.
Use HTTP-specific mocking frameworks to intercept requests we are about to make and return a pre-determined response.
The built-in httptest package can open an embedded HTTP server for us, which can be used for unit testing, but I prefer the third-party gock framework, which is more concise and easy to use.

func init () {
    gock.InterceptClient (client)
}
Above we added an init function. This will ensure that our http client instance will be stolen by gock.

The gock DSL provides fine-grained control over the expected HTTP requests and responses. In the following example, we use New (), Get () and MatchParam () to tell Gock to expect http: // quotes-service: 8080 / api / quote? Strength = 4 GET request and respond to HTTP 200, and hardcoded Response body.

Add the following code to the TestGetAccount function:

func TestGetAccount (t * testing.T) {
        defer gock.Off ()
        gock.New ("http: // quotes-service: 8080").
                Get ("/ api / quote").
                MatchParam ("strength", "4").
                Reply (200).
                BodyString (`{" quote ":" May the source be with you. Always. "," IpAddress ":" 10.0.0.5:8080 "," language ":" en "}`)
defer gock.Off () ensures that HTTP capture is turned off after the current test is completed. Since gock.New () will return http capture, this may make subsequent tests fail.

Let's assert the quote we expect to return. Add a new assertion to the Convey block inside the TestGetAccount test:

Convey ("Then the response should be a 200", func () {
    So (resp.Code, ShouldEqual, 200)

    account: = model.Account ()
    json.Unmarshal (resp.Body.Bytes (), & account)
    So (account.Id, ShouldEqual, "123")
    So (account.Name, ShouldEqual, "Person_123")
    
    // NEW!
    So (account.Quote.Text, ShouldEqual, "May the source be with you. Always.")
})
Run test
> go test. / ...
? github.com/callistaenterprise/goblog/accountservice [no test files]
? github.com/callistaenterprise/goblog/accountservice/dbclient [no test files]
? github.com/callistaenterprise/goblog/accountservice/model [no test files]
ok github.com/callistaenterprise/goblog/accountservice/service 0.011s
Deploy and run on Swarm
We also use the copyall.sh script to rebuild and deploy. Then call the account route via curl:

> curl $ ManagerIP: 6767 / accounts / 10000
  {"id": "10000", "name": "Person_0", "servedBy": "10.255.0.8", "quote":
      {"quote": "You, too, Brutus?", "ipAddress": "461caa3cef02 / 10.0.0.5: 8080", "language": "en"}
  }
Then extend quotes-service to two instances.

docker service scale quotes-service = 2
Wait for a period of time, about 15-30 seconds, because Spring Boot services are not as fast as Go services. Then call it a few more times with curl, and the result might look like this:

{"id": "10000", "name": "Person_0", "servedBy": "10.255.0.15", "quote": {"quote": "To be or not to be", "ipAddress": " 768e4b0794f6 / 10.0.0.8: 8080 "," language ":" en "}}
{"id": "10000", "name": "Person_0", "servedBy": "10.255.0.16", "quote": {"quote": "Bring out the gimp.", "ipAddress": "461caa3cef02 /10.0.0.5:8080","language":"en "}}
{"id": "10000", "name": "Person_0", "servedBy": "10.0.0.9", "quote": {"quote": "You, too, Brutus?", "ipAddress": " 768e4b0794f6 / 10.0.0.8: 8080 "," language ":" en "}}
We can see that servedBy loops well in the accountservice instance. We can also see that the quote's ipAddress field also has two different IPs. If we have disabled the keep-alive behavior, we may see that the same accountservice service maintains the same quotes-service to provide services.

to sum up
In this section, we have touched on the concepts of service discovery and load balancing in the context of microservices, as well as the implementation of calling other services, and only need to provide the service logical service name.

In Section 8, we turn to the most important concept in another freely scalable microservice, centralized configuration.

Reference connection
English part 7
Eureka
Consul
Etcd
ZooKeeper
Go microservices
eeureka
Turbine AMQP Plugin
gock
Featured Home
Next section