Spring Cloud (i): Service Governance Technology Overview "version Finchley"

Spring Cloud (i): Service Governance Technology Overview "version Finchley" Posted in 2018-04-14 | updated on 2018-05-07 |

Spring Cloud Netflix is a framework in spring cloud that was developed by Netflix and later incorporated into the spring Cloud family, and its main modules include service discovery, circuit breakers and monitoring, intelligent routing, client load balancing, and more.

Starting with spring cloud Netflix, the core project in spring Cloud, this article explains the advantages of spring cloud Netflix and describes the technical principles of service governance for spring cloud Netflix.

The benefits of Spring Cloud Netflix

For the governance of microservices, the core is the registration and discovery of services. So the choice of which component depends to a large extent on its solution for service registration and discovery. In this area, there are many open source architectures, the most common being Zookeeper, but this is not the best choice.

There is a well-known CAP theorem in the field of distributed systems:c--Data consistency, a--service availability, p--service fault tolerance for network partitioning . These three features cannot be met in any distributed system at the same time, up to a maximum of two.

Zookeeper is a sub-project of the famous Hadoop, and in many scenarios Zookeeper also serves as Service discovery Services solution. the Zookeeper guarantees that the CP, which is a consistent data result for Zookeeper access requests at any time, and that the system is fault tolerant of the network segmentation, does not guarantee the availability of each service request. From a factual analysis, when using Zookeeper to get a list of services, if Zookeeper is choosing the master, or if more than half of the machines in the Zookeeper cluster are unavailable, then data will not be available. Therefore, Zookeeper cannot guarantee the availability of services.

Admittedly, for most distributed environments, especially those involving data storage, data consistency should be guaranteed first, which is why zookeeper is designed to be CP. But for service discovery scenarios, the situation is not the same: for the same service, even if the different nodes of the Registry store service provider information is not the same, it does not cause catastrophic consequences. Because for service consumers, spending is the most important thing-try to spend it when you get an incorrect service instance information, and it's better than not spending because you can't get the instance information. Therefore, for service discovery, availability is more important than data consistency--ap isbetter than CP. and Spring Cloud Netflix adheres to the AP principle when designing Eureka.

Eureka itself is a product of Netflix's open source offering service registration and discovery, and provides the appropriate Java package. In its implementation, the nodes are equal to each other, some of the registry nodes are not affected by the cluster, even if the cluster is only one node surviving, can also provide a normal discovery service. Even if all service registration nodes are hung up, the service invocation information is cached on the Eureka clients. This ensures that the mutual invocation between our microservices is robust enough.

Beyond that, the powerful open source power behind spring cloud Netflix has prompted us to choose Spring Cloud Netflix:

• As mentioned earlier, the community of Spring Cloud is very active, its application in the industry is very wide (especially abroad), and the entire framework has withstood the test of Netflix's harsh production environment.
• In addition to service registration and discovery, the other features of Spring Cloud Netflix are also powerful, including components such as Ribbon,hystrix,feign,zuul, which combine to make service invocation and routing extremely easy.
• With spring Cloud Netflix as the spring's heavyweight integration framework, it also means that we can get great convenience from spring. Spring Cloud's other sub-projects, such as Spring Cloud Stream, Spring Cloud Config, and more, provide a one-stop solution to the various needs of microservices.

What is the relationship between Netflix and Spring Cloud?
The first time I had the word Netflix, it was in the title of the American drama "The House of Cards". Yes, this is an Internet streaming media player, so you can say that the U.S. opera on the site should be the most fire. As the US video giant, the traffic is very large, so that the rapid development of its technology behind the support, and it is so, Netflix began to the overall system to the micro-service migration.
His family's microservices do not do the earliest, but are the largest in the production-level microservices attempt. A few years ago, Netflix contributed its almost entire micro-service framework stack to the community, called Netflix OSS.
Spring Cloud's Open source product, launched by Pivotal in 2015, is a further encapsulation of Netflix's open source components, allowing spring developers to build a microservices infrastructure. (Although Spring Cloud has so far not only the solutions that Netflix offers can be integrated, there are many options, but Netflix is the most mature.) )

Spring Cloud Netflix Main components

The core of spring cloud Netflix is the Eureka for service registration and discovery, and next we'll take Eureka as a clue to the Eureka, Ribbon, Hystrix, feign, which are the main components of the spring cloud Netflix.

Service Registration and Discovery--eureka

The word Eureka is derived from ancient Greek, meaning "I found it!" I found it! ", it is rumored, Archimedes in the bath found buoyancy principle, happy too late to put on trousers, ran to the street shouting:" Eureka (I found)! ”。

Spring Cloud Eureka is part of the spring cloud Netflix microservices Suite, with two packages based on Netflix Eureka, primarily responsible for service governance capabilities in the MicroServices architecture, Service governance can be said to be the most core and basic module in MicroServices architecture, which is mainly used to realize automatic registration and discovery of each micro-service instance.

• Service Registration : In the service governance framework, a registry is usually built, each service unit registers its own services with the registry, informs the Registry of additional information such as host and port number, version number, communication protocol, etc., and the registry organizes the list of services by service name. The service registry also needs to monitor the availability of services in the list with a heartbeat, and if it is not available, it needs to be removed from the list of services to achieve the effect of troubleshooting services.
• Service Discovery : Because it runs under the service governance framework, calls between services are no longer implemented by specifying a specific instance address, but rather by initiating request invocation to the service name.

Spring Cloud Eureka uses Netflix Eureka for service registration and discovery, including server-side components and client-side components, and both the server and client are written in Java, so Eureka is primarily for distributed systems implemented with Java. Or a system built with a JVM-compatible language, however, because the service governance mechanism of the Eureka server provides a complete RESTful API, he also supports the inclusion of non-Java language-built microservices into the Eureka service governance system.

Eureka consists of multiple instance (service instances), which can be divided into two types: Eureka Server and Eureka Client. For the sake of understanding, we divide the Eureka client into service Provider and service Consumer. As shown in the following:

• Eureka Server: A Registry of services that maintains a list of registered services, like other service registries, that support highly available configurations.
• Service Provider: As a Eureka Client, services to Eureka Server for service registration, renewal and offline operations, registration of the main data including service name, machine IP, port number, domain name and so on.
• Service Consumer: Consumer, as a Eureka Client, obtains registration information for service Provider to Eureka Server and through remote calls to service Provider Letter.

Service Provider and service Consumer are not strict concepts, and service Consumer can be registered with Eureka Server at any time to make itself a service Provider.

Spring Cloud provides a layer of abstraction for service registration and discovery, and offers three implementations: Eureka, Consul, Zookeeper. At present, the best support is Eureka, followed by Consul, and finally Zookeeper. Under the effect of layer abstraction, we can seamlessly switch service governance implementations without affecting any other logic such as service registration, service discovery, service invocation, and so on.

Eureka Server

Eureka Server acts as a standalone deployment unit, providing registration, management, and querying for service instances in the form of REST APIs. At the same time, Eureka server provides us with a visual monitoring page that can visually see the current state of operation of each Eureka server and the status of all registered services.

Eureka Server's highly available cluster

Eureka server can run multiple instances to build a cluster to solve a single point of issue, but unlike ZooKeeper's election leader process, Eureka Server uses peer to peer communication. This is a centralized architecture, no master/slave distinction, and each peer is equal to each other. In this architecture, nodes increase availability by registering with each other, and each node needs to add one or more valid serviceurl to point to the other nodes. Each node can be treated as a copy of another node.

If a Eureka server goes down, the Eureka Client's request is automatically switched to the new Eureka server node, and once the outage server is restored, Eureka will again be included in the server cluster management. When a node begins to accept a client request, all operations are performed replicateToPeer (Inter-node replication), and the request is replicated to all the nodes that are currently known to other Eureka servers.

After a new Eureka Server node is started, it will first attempt to get all the instance registry information from the neighboring node and complete the initialization. Eureka Server getEurekaServiceUrls() obtains all the nodes through the method and is updated periodically through a heartbeat renewal. By default, if Eureka server does not receive a heartbeat for a service instance for a certain amount of time, Eureka server will unregister the instance (default is 90 seconds, through eureka.instance.lease-expiration-duration-in-seconds configuration). When the Eureka Server node loses too much heartbeat in a short period of time (such as a network partition failure), the node enters a self-protected mode. Architecture map for Eureka official website

Self-protection mode

If you see this hint on the first page of Eureka Server, the Eureka has entered protection mode.

emergency! EUREKA may incorrectly claiming INSTANCES is up when the they ' RE not. Renewals is LESSER THAN THRESHOLD and HENCE the INSTANCES is not BEING EXPIRED JUST to be SAFE.

By default, Eureka Server triggers self-protection if the number of heartbeat renewals received per minute is less than a threshold and lasts for 15 minutes.

Copy

1	Threshold = Number of Instance x (60/instance heartbeat interval seconds) x self-protection factor

In self-protected mode, Eureka Server protects the information in the service registry and no longer unregisters any service instances. When the number of heartbeats it receives is restored above the threshold, the Eureka Server node automatically exits self-protected mode. Its philosophy of design mentioned earlier that it would prefer to keep the wrong service registration information and not blindly unregister any potentially healthy service instances. Doing so makes it easy for the client to get a service instance that does not already exist, and the failure of the call can occur. Therefore, the client should have a fault-tolerant mechanism, such as request retry, circuit breaker. This mode can be eureka.server.enable-self-preservation = false disabled by, and eureka.instance.lease-renewal-interval-in-seconds can be used to change the heartbeat interval (default 30s), which eureka.server.renewal-percent-threshold can be used to modify the self-protection factor (default 0.85).

Special attention
The above threshold is theoretically calculated, but in practice it is not!!! Not!!! Not!!!
Because the actual code is hard-coded, each registered instance is directly + 2, look at eureka-core-1.8.7.jar!com.netflix.eureka.registry.AbstractInstanceRegistry#register the code below to understand

Copy

1 2 3 4 5 6 7 8 9 10 11

The lease does not exist and hence it is a new registration Synchronized (lock) { if (this.expectednumberofrenewspermin > 0) { //Since the client wants to cancel it, reduce the threshold //(1 //For the seconds, 2 for a minute) this.expectednumberofrenewspermin = this.expectednumberofrenewspermin + 2; This.numberofrenewsperminthreshold = (int) (this.expectednumberofrenewspermin * serverconfig.getrenewalpercentthreshold ()); } }

This is the time to study this self-protection pattern carefully.

Eureka Server's Region, Zone

The relationship between region, Zone, Eureka cluster, as shown in:

Region and zone (or availability zone) are the concept of Amazon Web Services (AWS). In a non-AWS environment, we can first simply understand region as Eureka cluster, zone understanding into a room. Can be understood as a Eureka cluster is deployed in the Zone1 room and the Zone2 room.

Why is there such a perfect support for AWS?
Because Netflix relies on the AWS cloud on server operations, it says spring Cloud Netflix is a spring package on Netflix's open source component, Netflix OSS.

Service Provider Services Registration

Service Provider is essentially a Eureka Client. When it starts, it invokes the service registration method and registers its own information with Eureka Server. Eureka Server maintains a list of registered services, which is a nested HASHMAP:

Copy

1 2	Final concurrenthashmap<string, map<string, lease<instanceinfo>>> Registry New concurrenthashmap<string, Map<string, lease<instanceinfo>>> ();

• The first layer, application name and the corresponding service instance.
• The second layer, the service instance and its corresponding registration information, including IP, port number and so on.

When the state of an instance changes, such as when it detects that it is down, it also updates its service state to Eureka server, while replicateToPeers() synchronizing the state with the other Eureka server nodes.

Renewal and rejection

As mentioned earlier, after the service instance is started, the heartbeat is periodically sent to the Eureka Server to renew its information and to prevent its registration information from being rejected. The way to renew the contract is basically the same as the service registration: First update its state, then sync to other Peer.

If Eureka server does not receive a heartbeat for a microservices node for a period of time, Eureka server will unregister the MicroServices node (except for self-protected mode).

Service Consumer

Service Consumer is essentially a Eureka Client (it also registers with Eureka Server, but this registration information does not matter). When it starts, it obtains registration information for all instances from the Eureka Server, including the IP address, port, and so on, and caches it locally. This information is updated every 30 seconds by default. As mentioned earlier, if communication with Eureka Server is interrupted, service Consumer can still communicate with the service Provider through the local cache.

In the process of actually developing the Eureka, there are times when the Service Consumer gets a delay in obtaining information about the Server Provider, and there is a paragraph in the Eureka Wiki:

All operations from Eureka client could take some time to reflect in the Eureka servers and subsequently in other Eureka CLI Ents. This was because of the caching of the payload on the Eureka server which are refreshed periodically to reflect new Informat Ion. Eureka clients also fetch deltas periodically. Hence, it may take up to 2 mins for changes to propagate to all Eureka clients.

In the last sentence, it may take up to 2 minutes for the server to propagate to all clients, for reasons not described. This is because Eureka has three caches and one delay.

• Eureka Server caches the registration list, which is a default time of 30s.
• The Eureka Client caches the obtained registration information, which is a default time of 30s.
• The Ribbon will fetch a list of services from the Eureka Client mentioned above, caching the results of the load-balanced 30s.
• If these components are not used in a Spring Cloud environment (Eureka, Ribbon), the service will not register with Eureka immediately after it is started, but will need to wait until the heartbeat request is first sent. The heartbeat request's send interval defaults to 30s. Spring Cloud has made changes to this and will register immediately after the service is started.

We can make service calls based on the service instance information that Consumer gets. Spring Cloud also provides service Consumer with a wealth of services invocation tools:

• Ribbon, which enables client-side load balancing.
• Hystrix, Circuit breaker.
• The feign,restful Web Service client integrates the Ribbon and Hystrix.

Let's introduce each of them.

Service call-side load Balancing--ribbon

The Ribbon is an open source project by Netflix, and the main feature is load balancing for REST clients. It consists mainly of six components:

• ServerList, the list of servers used for load balancing. This list is cached in the load balancer and updated periodically. When the Ribbon is used in conjunction with Eureka, the ServerList implementation class is Discoveryenabledniwsserverlist, which saves the service instance tables registered in Eureka Server.
• Serverlistfilter, server List filter. This is an interface that is primarily used to pre-filter the list of servers that the Service Consumer gets to, and the filtered results are also serverlist. The Ribbon provides a variety of filter implementations.
• IPing, a strategy for detecting if a service instance is alive.
• IRule, load Balancing strategy, the implementation of class formulation strategy includes: polling, random, based on response time weighting, and its class structure as shown.
  

  We can also define our own load balancing strategy, for example, we use our own implementation of the strategy to achieve the service version control and direct-attached configuration. Once implemented, the implementation classes are re-injected into the Ribbon.

• Iloadbalancer, Load balancer. This is also an interface where the Ribbon provides multiple implementations, such as Zoneawareloadbalancer. The upper-layer code selects the load Balancer for service invocation by invoking its API. A IRule is referenced in the general Iloadbalancer implementation class.
• Restclient, the service invoker. As the name implies, this is the tool for the Ribbon to initiate REST requests to Service Provider after load balancing.

There are four things to do when the Ribbon works:

1. Preference for Eureka Server with less load in the same Zone;
2. Periodically update and filter the list of service instances from Eureka;
3. Select the address of an instance in the list of service registrations from the Server, based on user-specified policies;
4. Service calls are made through restclient.

Service call end fusing--hystrix avalanche effect

In a microservices architecture, there are often multiple service layer calls, and the failure of the underlying service can lead to cascading failures that could result in an entire system being unavailable, a phenomenon known as service avalanche effect. The service avalanche effect is a process in which the service consumer is not available and will not be usable due to the unavailability of the service provider.
If: A serves as a service provider, B is a service consumer, and C and D are service consumers of B. A unavailability causes the unavailability of B and will not be available as a snowball to C and D, the avalanche effect is formed.

Circuit Breaker

Netflix created a library called Hystrix, which implements the mode of the circuit breaker. "Circuit Breaker" itself is a switching device, when a service unit fails, through the fault monitoring of the circuit breaker (similar to a fuse fuse), to the caller to return an expected, can handle the alternative response (FallBack), rather than a long wait or throw the caller cannot handle the exception, This ensures that the service caller's threads are not consumed for long periods of time and unnecessarily, thus avoiding the spread of failures in distributed systems, or even avalanches.

Of course, if the request fails at a low frequency, Hystrix will return the fault directly to the client. Only when the number of failures reaches the threshold, the circuit breaker opens and does not perform subsequent communication, but instead returns the alternative response directly. Of course, Hystrix's alternative response can also be customized by the developer.

Monitoring

In addition to isolating service-dependent calls, Hystrix provides quasi-real-time call monitoring (hystrix Dashboard), which hystrix continuously records execution information for all requests initiated through Hystrix. and presented to the user in the form of statistical reports and graphs, including how many requests per second, how many successes, how many failures, etc. Netflix monitors the above metrics through the Hystrix-metrics-event-stream project. Spring Cloud also provides the integration of Hystrix Dashboard, which transforms the monitoring content into a visual interface, detailing the meaning of each of the indicators on the Hystrix Dashboard Wiki.

Service call-side code abstraction and encapsulation--feign

Feign is a declarative Web service client that is designed to make Web service calls easier. It integrates the Ribbon and the hystrix so that we no longer need to explicitly use these two components. Feign also provides templates for HTTP requests, and by writing simple interfaces and inserting annotations, we can define the parameters, formats, addresses, and so on for HTTP requests. Next, feign will fully proxy the HTTP request, and we just need to invoke it as a method to complete the service request.

The feign has the following characteristics:

• pluggable annotation support, including feign annotations and jax-rs annotations
• Support pluggable HTTP encoders and decoders
• Support for Hystrix and its Fallback
• Support for Ribbon load balancing
• Support for HTTP request and response compression

Related reading

Spring Cloud (i): Overview of service governance Technologies
Spring Cloud (ii): Service Registration and Discovery Eureka
Spring Cloud (iii): service delivery and invocation Eureka
Spring Cloud (iv): Service-tolerant protection hystrix
Spring Cloud (v): Hystrix Monitor Panel
Spring Cloud (vi): Hystrix monitoring Data Aggregation Turbine
Spring Cloud (vii): Configuration Center (Git vs. dynamic refresh)
Spring Cloud (eight): Configuration Center (service and high availability)
Spring Cloud (ix): Configuration Center (message bus)
Spring Cloud (10): Service Gateway Zuul (routing)
Spring Cloud (11): Service Gateway Zuul (filter)
Spring Cloud (12): Distributed Link Tracking (Sleuth and Zipkin)

Sample code: GitHub

Reference

Spring-cloud
Spring Cloud Technical Analysis (1)--Service governance
Netflix/eureka Wiki
Spring Cloud, Eureka FAQ Summary
Dive into Eureka
Spring Cloud Netflix Eureka Source reading and principle analysis
Understanding the self-protection mechanism of Eureka
The surprise group effect of Eureka
Understanding Spring Cloud Eureka Server Self preservation and renew threshold
The Mystery of Eureka self-preservation
Hystrix Use and analysis

• this article Yibo
• This article link: https://windmt.com/2018/04/14/spring-cloud-1-services-governance/
• Copyright Notice: All articles in this blog are subject to the CC BY-NC-SA 4.0 license Agreement except for special statements. Reprint please specify the source!

Spring Cloud (i): Service Governance Technology Overview "version Finchley"