How to build MicroServices with API gateways

When you choose to build your application as a microservices, you need to determine how the application client interacts with the microservices. In a single application, there is only one set of endpoints. In the microservices architecture, each microservices exposes a set of endpoints that are usually fine-grained. In this article, we'll discuss how this affects the communication between the client and the application, and propose a way to use the API gateway.

When you choose to build your application as a set of microservices, you need to determine how the application client interacts with the microservices. In a single application, there is only one set of endpoints (usually repetitive, load-balanced). However, in a microservices architecture, each microservices exposes a set of endpoints that are usually fine-grained. In this article, we'll discuss how this affects the communication between the client and the application, and propose a way to use the API gateway.

Let's imagine that you are developing a native mobile client for a shopping application. You will probably need to implement a product detail page that shows information about any given product.

For example, it shows what you see when scrolling through product details in an Amazon Android mobile app.

Although this is a smartphone app, the product detail page also shows a lot of information. For example, the page contains not only basic product information (such as name, description, price), but also the following:

The number of pieces in the shopping cart;

Order history;

Customer reviews;

Low inventory warning;

Delivery options;

Various recommendations, including other products that are often purchased with the product, other products purchased by the customer who purchased the product, and other products that the customer of the product has seen;

Optional purchase options.

When a monolithic application schema is used, the mobile client obtains the data by initiating a rest call to the application (get api.company.com/productdetails/<productid>). The load balancer routes the request to one of n identical application instances. The application then queries the various database tables and returns the responses to the client.

By contrast, when using the MicroServices architecture, the data displayed on the product detail page is owned by multiple microservices. Here are some of the possible microservices that have data to display in the sample Product Details page:

Shopping Cart Service--The number of items in the shopping cart;

Order service-order history;

Directory Services-Basic product information, such as name, picture, and price;

Comment Service-customer's comments;

Inventory Services-Low inventory early warning;

Delivery service-delivery options, deadlines and fees, which are obtained separately from the delivery party API;

Recommended services-Recommended products.

We need to decide how mobile clients can access these services. Let's see what options are available.

Client-to-micro service direct communication

In theory, a client can send a request directly to each micro-service. Each micro-service has a public endpoint (https://<servicename>.api.company.name). The URL maps to the microservices load balancer, which is responsible for distributing requests between the available instances. To obtain product details, the mobile client sends requests one at a to the N services listed above.

Unfortunately, there are challenges and limitations to this approach. One problem is that the client requirements and the fine-grained APIs exposed by each microservices do not match. In this example, the client needs to send 7 separate requests. In more complex applications, you may want to send more requests. For example, as Amazon says, they call hundreds of services when they display their product pages. However, the client sends many requests over the LAN, which can be inefficient on the public web and not feasible on the mobile network at all. This approach also makes client-side code very complex.

Another problem with clients calling MicroServices directly is that the protocol used by some services is not a web-friendly protocol. One service may use thrift binary RPC, while another service may use the AMQP messaging protocol. Regardless of which protocol is not browser-friendly or firewall-friendly, it is best to use it internally. Outside the firewall, applications should use protocols such as HTTP and WebSocket.

Another drawback of this approach is that it makes microservices difficult to refactor. Over time, we may want to change how the system is partitioned into services. For example, we might merge two services, or split a service into two or more services. However, if the client communicates directly with the microservices, it is very difficult to perform such refactoring.

Because of these problems, it is seldom reasonable for clients to communicate directly with MicroServices.

Using the API Gateway

In general, a better approach is to use the so-called API gateways. An API gateway is a server that is the only portal to the system. From an object-oriented design perspective, it is similar to the appearance pattern. The API gateway encapsulates the system's internal architecture and provides a custom API for each client. It may also have other responsibilities, such as authentication, monitoring, load balancing, caching, request shaping, and management, and static response processing.

Shows how API gateways typically fit into the architecture:

The API Gateway is responsible for service request routing, composition, and protocol conversions. All requests to the client go through the API Gateway first, and then it routes the request to the appropriate microservices. API network management often processes a request by invoking multiple microservices and merging the results. It can be converted between Web protocols, such as HTTP and WebSocket, and non-web-friendly protocols that are used internally.

The API Gateway can also provide a custom API for each client. Typically, it exposes a coarse-grained API to mobile clients. For example, consider a scenario where product details are considered. The API gateway can provide an endpoint (/PRODUCTDETAILS?PRODUCTID=XXX) that allows the mobile client to obtain all the product details through a single request. API gateways Process requests by invoking individual services (product information, recommendations, comments, and so on) and merging results.

The Netflix API Gateway is a good example of an API gateway. Netflix streaming services are available to hundreds of different types of devices, including TVs, set-top boxes, smartphones, gaming systems, tablets, and more. Initially, Netflix tried to provide a common API for their streaming services. However, they found that this approach did not work well because of the unique needs of various devices. Today, they use an API gateway to provide a custom API for each device by running device-specific adapter code. Typically, an adapter processes each request by calling an average of 6 to 7 back-end services. The Netflix API gateway processes billions of requests per day.

Advantages and disadvantages of API gateways

As you might have expected, there are advantages and disadvantages to using API gateways. The biggest advantage of using an API gateway is that it encapsulates the internal structure of the application. The client only needs to interact with the gateway without having to invoke a specific service. API gateways provide specific APIs for each type of client. This reduces the number of interactions between the client and the application and simplifies the client code.

API Gateways also have some shortcomings. It adds a highly available component that we have to develop, deploy, and maintain. There is also a risk that the API gateway becomes a development bottleneck. To expose the endpoints of each microservices, the developer must update the API gateway. It is important that the API Gateway update process be as simple as possible. Otherwise, in order to update the gateway, developers will have to wait in line.

However, despite these shortcomings, it is reasonable to use an API gateway for most real-world applications.

Implementing an API Gateway

So far, we've explored the motivations for using API gateways and their pros and cons. Let's look at the various design issues that need to be considered.

Performance and Scalability

Only a handful of companies have the size of Netflix and need to process billions of of requests every day. However, for most applications, the performance and scalability of API gateways is often important. Therefore, it is reasonable to build an API gateway on a platform that supports asynchronous, I/O non-blocking. There are a number of different technologies that can be used to implement an extensible API gateway. On the JVM, you can use a NIO-based framework, such as one of the Netty, VERTX, Spring reactor, or JBoss undertow. A very popular non-JVM option is node. js, which is a platform built on the chrome JavaScript engine. Another option is to use Nginx Plus. NGINX Plus provides a sophisticated, scalable, high-performance Web server and an easy-to-deploy, configurable, and programmable reverse proxy. NGINX Plus can manage authentication, access control, load balancing requests, cache responses, and provide application-aware health checks and monitoring.

Using the Responsive programming model

The API gateway processes a partial request by simply routing the request to the appropriate backend service, while processing other requests by invoking multiple backend services and merging the results. For some requests, such as multiple requests related to product details, their requests to the backend service are independent of the other requests. To minimize response time, the API gateway should concurrently execute separate requests. Sometimes, however, there is a dependency between requests. Before the request is routed to the backend service, the API gateway may first need to invoke the legitimacy of the Authentication Service authentication request. Similarly, in order to obtain product information from the customer's wish list, the API gateway must first obtain the customer data that contains the information and then obtain information about each product. Another interesting example of the API mix is the Netflix Video Grid.

Writing API mix code with a traditional asynchronous callback method will let you quickly fall into a callback hell. Code can become confusing, incomprehensible, and error-prone. A better approach is to use a responsive approach to write API gateway code in a declarative style. Examples of responsive abstractions are the future in Scala, the completablefuture in Java 8, and promise in JavaScript, as well as the original Microsoft. NET platform developed by reactive Extensions (RX). Netflix created the Rxjava for JVM, dedicated to their API gateways. In addition, there is RXJS for JavaScript, which can run either in the browser or in node. js. Using a responsive approach will allow you to write simple but efficient API gateway code.

Service invocation

A microservices-based application is a distributed system that must use an inter-process communication mechanism. There are two types of interprocess communication mechanisms to choose from. One is the use of asynchronous, message-based mechanisms. Some implementations use message proxies such as JMS or AMQP, while other implementations (such as ZEROMQ) have no proxies and direct communication between services. Another type of interprocess communication is a synchronization mechanism such as HTTP or thrift. Typically, one system uses both asynchronous and synchronous types. It may even use multiple implementations of the same type. In summary, API gateways need to support multiple communication mechanisms.

Service discovery

The API gateway needs to know the location (IP address and port) of each micro-service it communicates with. In traditional applications, this location may be hardwired, but in modern, cloud-based microservices applications, this is not an easy problem to solve. Infrastructure services, such as message brokers, typically have a static location that can be specified through an OS environment variable. However, determining the location of an application service is not so simple. The location of the application service is dynamically allocated. Also, a set of instances of a single service will change dynamically as auto-scaling or upgrades occur. In summary, like other service clients in the system, API gateways need to use the system's service discovery mechanism, either server-side discovery or client discovery. The next article will describe service discovery in more detail. Now, note that if the system uses client-side discovery, the API gateway must be able to query the service registry, which is a database that contains all the microservices instances and their locations.

Handling Local Failures

When implementing an API gateway, there is another problem that needs to be addressed, which is the problem of partial failure. This problem occurs in all distributed systems, whenever a service invokes another service that responds slowly or is not available. API gateways can never be blocked by waiting indefinitely for downstream services. However, how to handle the failure depends on the particular scenario and which service failed. For example, in the product detail scenario, if the referral service is unresponsive, the API gateway should return the other content of the product details to the client because they are still useful to the user. The recommended content can be empty or, for example, replaced with a fixed top 10 list. However, if the product information service is unresponsive, the API gateway should return an error message to the client.

If the cached data is available, the API gateway can also return cached data. For example, because product prices do not change frequently, the API gateway can return cached price data if the price service is unavailable. The data can be cached by the API gateway itself, or it can be stored in an external cache like Redis or memcached. By returning default data or caching data, the API gateway ensures that a system failure does not affect the user's experience.

Netflix Hystrix is an exceptionally useful library for writing code to invoke remote services. The Hystrix will time out the call exceeding the set threshold. It implements a "circuit breaker (Circuit breaker)" mode that prevents clients from waiting unnecessarily for unresponsive services. If the error rate of the service exceeds the set threshold, then Hystrix will cut off the circuit breaker, and all requests will fail immediately in a specified time frame. Hystrix allows a user to define a backup operation after a failed request, such as reading data from the cache, or returning a default value. If you are using the JVM, then you should definitely consider using Hystrix. And if you are using a non-JVM environment, then you should use an equivalent library.

Summary

For most microservices-based applications, implementing an API Gateway makes sense, and it can be the only entry for the system. The API Gateway is responsible for service request routing, composition, and protocol conversions. It provides a custom API for each application client. The API gateway can also fail by returning cached data or the default data masking back-end service. In the next article in this series, we'll explore inter-service communication.

How to build MicroServices with API gateways