Service discovery: Zookeeper vs Etcd vs Consul

This is a creation in Article, where the information may have evolved or changed.
"Editor's note" This article compares three service discovery tools for zookeeper, ETCD, and Consul, and explores the best solution for service discovery, for informational purposes only.

If you use a predefined port, the more services you have, the greater the likelihood of a conflict, after all, it is impossible to have two services listening on the same port. Managing a crowded list of all the ports used for example by hundreds of services is a challenge in itself, and the number of databases and numbers that these services need will be increasing once added to the list. So we should deploy a service that doesn't have to specify a port, and let Docker assign us a random port. The only problem is that we need to find the port number and let people know.

When we start deploying services to one of the servers on a distributed system, things get more complicated, and we can choose to predetermine which server runs which service, but this can cause a lot of problems. We should try our best to make use of server resources, but it is almost impossible to make the best use of server resources if the deployment location of each service is predefined. Another problem is that automatic scaling of services can be difficult, not to mention automatic recovery, such as server failures. On the other hand, if we deploy the service to a server where only a minimum number of containers are running, we need to add the IP address to the data list, which needs to be discoverable and stored somewhere.

There are many other examples when we need to store and discover some information related to the services we are working on.

To be able to locate the service, we need at least the next two useful steps.

• Service Registration -The information stored by this step includes at least the host and port information for the running service
• Service Discovery -This step allows other users to discover information stored during the service registration phase.

In addition to the above steps, we also need to consider other aspects. If a service stops working and deploys/registers a new service instance, should the service be logged out? What if there are multiple copies of the same service? How do we do load balancing? What if a server goes down? All of these issues are closely related to the registration and discovery phases. Today, we limit the scope of service discovery only (common names, around the above steps) and tools for service discovery tasks, most of which use highly available distributed key/value stores.

Service Discovery Tool

The main goal of the Service Discovery tool is to serve to find and interact with each other, so the tool needs to know each service, which is not a new concept, and there are a lot of similar tools before Docker, however, the container gives these tools a whole new level of demand.

The basic idea behind service discovery is that each new instance (or application) of the service is able to identify the current environment and storage-related information. The stored registry information itself is usually in the form of a key/value pair, which is required to scale, support fault tolerant, and all nodes in a distributed cluster because service discovery is often used in distributed systems. The primary purpose of this storage is to provide information to all interested parties, such as service IP addresses and ports, for mutual communication between them, which is often extended to other types of information service discovery tools that tend to provide some form of API for the service's own registration and service information lookups.

Let's say we have two services, one is the provider, the other is the consumer of the first service, and once the service provider is deployed, it needs to store its information in the Service Discovery registry. Then, when the consumer tries to access the service provider, it first queries the service registry, using the acquired IP address and port to invoke the service provider. To decouple from the specific implementations of the service providers in the registry, we often use some kind of proxy service. In this way, consumers always request information from the proxy of the fixed IP address, and the agent then uses service discovery to find the service provider information and redirect the request, which we will do later through the reverse proxy. It is important to understand the service discovery process based on three roles (service consumers, providers, and agents).

The Service Discovery tool is looking for data, at least we should be able to find out where the service is? is the service healthy and available? What is the configuration? Now that we are building a distributed system on multiple servers, the tool needs to be robust enough to ensure that the downtime of one of the nodes does not compromise the data, and that each node should have exactly the same copy of the data, and further, we want to be able to start the service in any order, kill the service, or replace the new version of the service. We should also be able to reconfigure the service and see the corresponding changes to the data.

Let's take a look at some common options to accomplish the goals we set above.

Manual configuration

Most services still need to be managed manually, we pre-decide where to deploy the services, how to configure them, and for whatever reason, the service will continue to work until the end of the day. Such a goal is not easy to achieve. Deploying a second service instance means that we need to start the whole process of manual processing, we need to introduce a new server, or find out which server resource utilization is low, then create a new configuration set and start the service. The situation may become more complex, for example, when a hardware failure results in a slow response time under manual management. Visibility is another pain point, we know what is static configuration, after all we are prepared, however, most services have a lot of dynamically generated information, this information is not easily visible, there is no single place for us to refer to this data when needed.

Reaction times are inevitably slowed down, and recovery and monitoring can become very difficult to manage, given the many mobile components that need to be handled manually.

Although there is an excuse not to do this work in the past or when the number of services/servers is low, this excuse does not exist with the advent of service discovery tools.

Zookeeper

Zookeeper is one of the oldest in this type of project, originating from Hadoop and helping to maintain various components in a Hadoop cluster. It is very mature and reliable and is used by many large companies (YouTube, EBay, Yahoo, etc.). Its data storage format is similar to the file system, if run in a server cluster, Zookeper will share the configuration state across all nodes, each cluster elects a leader, the client can connect to any one server to obtain data.

The main advantage of zookeeper is its maturity, robustness, and richness, but it also has its drawbacks, with Java development and complexity being the culprit. Although Java is great in many ways, and then it is too heavy for this type of work, Zookeeper uses Java and a considerable amount of reliance to make it very hungry for resource competition. Because of these problems, zookeeper becomes very complex, and maintaining it requires more knowledge than we expect from the benefits of this type of application. This is partly due to the richness of the features instead of turning it from an advantage to a liability. The more features the application has, the greater the likelihood that these features will not be required, so we will eventually pay the cost of complexity for these unwanted features.

Zookeeper paved the way for significant improvements in other projects, and "Big data players" are using it because there is no better choice. Today, zookeeper is already senile, we have a better choice.

Etcd

ETCD is a health/value pair storage System with HTTP protocol, which is a distributed and functional hierarchy configuration system, which can be used to build service discovery system. It is easy to deploy, install, and use, providing reliable data persistence features. It is secure and the documentation is complete.

ETCD is better than zookeeper, because it is simple, however, it needs to be paired with some third-party tools to provide service discovery functionality.

Now that we have a place to store information about our services, we also need a tool to automatically send messages to ETCD. But after that, why do we need to manually send the data to ETCD? Even if we want to send the information to ETCD manually, we usually don't know what the information is. With this in mind, the service may be deployed to a server running a minimum number of containers and randomly assigned a port. Ideally, this tool should monitor the Docker container on all nodes and update etcd whenever a new container is running or one of the existing containers stops, and one of the tools that can help us achieve our goals is registrator.

Registrator

Registrator is currently supporting ETCD, Consul, and Skydns 2 by checking the container online or by stopping the status of automatic registration and registration services.

Registrator and Etcd are a simple but powerful combination that can run many advanced technologies. Whenever we open a container, all the data is stored in ETCD and propagated to all nodes in the cluster. We will decide what information is ours.

The puzzle game is still missing a piece, we need a way to create the configuration file, and the data are stored in ETCD, by running some commands to create these configuration files.

Confd

CONFD is a lightweight configuration management tool that can be used to reload applications when a configuration file changes by using the latest state of data retention profiles stored in Etcd, Consul, and some other data registries. In other words, we can reconfigure all services with information stored in ETCD (or other registries).

Final thoughts on the combination of ETCD, Registrator and CONFD

When Etcd, Registrator, and CONFD are combined, you get a simple and powerful way to automate the configuration of all our service discoveries and needs. This combination also shows the validity of the correct combination of the "small" tool, which can be done exactly as we wish, and if the scope is slightly smaller, we will not be able to accomplish the goal before us, and on the other hand, if they design with a larger scope, it is possible to do so. We will introduce unnecessary complexity and server resource overhead.

Before we make our final decision, let's look at another tool set that has the same goal, after all, we should not be content with alternatives that are not available.

Consul

Consul is a strongly consistent data store that uses gossip to form dynamic clusters. It provides a hierarchical key/value store that not only stores data, but can also be used to register devices for a variety of tasks, from sending data change notifications to running health checks and custom commands, depending on their output.

Unlike zookeeper and Etcd, Consul implements the service discovery system inline, so there is no need to build your own system or use a third-party system. In addition to the features mentioned above, this discovery system includes node health checks and services running on it.

Zookeeper and ETCD only provide raw key/value team storage, requiring application developers to build their own systems to provide service discovery capabilities. While Consul provides a built-in framework for service discovery. The customer only needs to register the service and perform service discovery through the DNS or HTTP interface. The other two tools require a hands-on solution or third-party tools.

Consul provides out-of-the-box native support for a variety of data centers, where the gossip system works not only within the same cluster, but also across data centers.

Consul there is another good feature that distinguishes it from other tools, which not only can be used to discover deployed services and the node information they reside in, but also provides easy-to-expand health check features through HTTP requests, TTLs (time-to-live), and custom commands.

Registrator

Registrator has two Consul protocols in which the CONSULKV protocol produces a result similar to the ETCD protocol.

In addition to the usual IP and port storage in the ETCD or CONSULKV protocol, the Registrator Consul protocol stores more information, and we can get information about the service running node, as well as the service ID and name. We can also use some additional environment variables to store additional information in a certain token.

Consul-template

CONFD can be used as a consul with Etce, but Consul has its own template service, which is more suitable for consul.

With the information obtained from Consul, Consul-template is a very handy way to create files, and one additional benefit is that you can run any command after the file is updated, as confd,consul-template can also use the Go template format.

Consul health Check, web interface and data center

Monitoring the health status of cluster nodes and services is as important as testing and deploying them. While we should strive for a stable environment that has never failed, we should also admit that unexpected failures occur at any time, and we are ready to take appropriate measures. For example, we can monitor memory usage and, if a certain threshold is reached, migrate some services to another node in the cluster, which is a precaution to be performed before a "disaster" occurs. On the other hand, not all potential failures can be detected and acted on in a timely manner. A single service may be toothed white, and a full node may stop working due to a hardware failure. In this case we should be ready to act as quickly as possible, such as replacing a node with a new one and migrating the failed service. Consul has a simple, elegant but powerful way to perform health checks that help the user define what should be done when the health threshold reaches a certain number.

If the user Google searches for "Etcd UI" or "Etec dashboard", the user may see only a few available solutions, may ask why we have not introduced to the user, the reason is very simple, ETCD is only key/value pair storage, that's all. There is not much use in rendering data through a UI, as we can easily get the data through ETCDCTL. This does not mean that the Etcd UI is useless, but given its limited scope of use, it does not have much impact.

Consu is not just a simple key/value pair store, as we have seen, in addition to storing simple key/value pairs, it also has a concept of the service and the data it belongs to. It can also perform health checks, so it becomes a good candidate for dashboard, where we can see the state of our nodes and the services that are running. Finally, it supports the concept of a multi-data center. The combination of all these features allows us to see the necessity of introducing dashboard from different angles.

Through the Consul Web interface, users can view all services and nodes, monitor health check status, and read set key/value pair data by switching data center.

Final thoughts on Consul, Registrator, Template, Health Check, and Web UI

Consul and the tools we discussed together in many cases provide a better solution than ETCD. This is a simple and powerful solution designed to serve architecture and discovery from the bottom of your heart. It provides a complete, yet concise solution, which in many cases is the best service discovery and a tool to meet the needs of a health check.

Conclusion

All of these tools are based on similar principles and architectures that run on nodes, require quorum to run, and are strongly consistent, providing some form of key/value pair storage.

Zookeeperis one of the oldest of its kind, and the lifespan shows its complexity, resource utilization and efforts to achieve it, designed to work with the different eras of other tools we evaluate (even if it's not too old).

Etcd、 RegistratorAnd CONFDis a very simple but very powerful combination that can solve most of the problems if not all of them meet the needs of service discovery. It also shows that we can get powerful service discovery by combining very simple and specific tools, each of which performs a very specific task, communicates through a well-designed API, has the ability to work relatively autonomously, and from both the architectural and functional pathways Micro-ServiceWay.

ConsulThe difference is that there is no need for third-party tools to natively support multiple data centers and health checks, which does not mean the use of third-party tools is not good. In fact, in this blog we try to combine different tools by choosing tools that perform better and do not introduce unnecessary functionality. Use the right tools to get the best results. If the tool introduces features that are not needed for work, productivity will decrease, and on the other hand, if the tool does not provide the required features for the job, it is useless. Consul a good balance of weights, with as few things as possible to achieve a good goal.

Consul uses gossip to propagate cluster information in a way that makes it easier to build than ETCD, especially for large data centers. The ability to store data as a service makes it more complete and useful than ETCD only the health/value store features (even if Consul has this option). While we can achieve the same goal by inserting multiple keys in the ETCD, the consul service achieves a more compact result, typically requiring only one query to get all the data related to the service. In addition, registrator a good implementation of the consul of the two Protocols, make it into one, especially added consul-template to the puzzle. Consul's Web UI provides a great way to visualize services and health checks.

I can't say Consul is a definite winner, but it has a slight advantage over ETCD. Service discovery as a concept, and as tools are very new, we can expect a lot of changes in this area. With an open mind, you can take reservations about the recommendations of this article, try different tools and then make your own conclusions.

original link: Service discovery:zookeeper vs Etcd vs Consul (Translate: Hu Zhen)

=================================================================

Translator: Hu Zhen, former chief architect of internet finance startups &cto, is now responsible for technical management and architecture design at Ping An financial science and Technology Center architecture group.