The API gateway Ocelot uses Polly to handle some failures.

The API gateway Ocelot uses Polly to handle some failures.

In the process of implementing the API Gateway, another issue that needs to be considered is part of failure. This problem occurs in distributed systems when one service calls another service and times out or is unavailable. API Gateway should not be blocked and wait for downstream services indefinitely. However, how to handle such failures depends on specific scenarios and specific services. If the product information service does not respond, the API Gateway should return an error to the client.

Ocelot is an API Gateway on the. NET Core platform. Recently, I was involved in the development of this project. The first one after development was completed was to use Polly to handle some failures. You may not be familiar with the Polly project. First, I would like to briefly introduce Polly, an open-source project under the. NET Foundation. Polly records calls that exceed the predefined limit. It implements the circuit break model, allowing the client to stop from the endless waiting of the unresponsive service. If the error rate of a service exceeds the preset value, Polly will interrupt the service and all requests expire immediately within a period of time. Polly can define a fallback operation for request failure, for example, read the cache or return the default value, sometimes we need to call other APIs when there is a temporary connection failure timeout, then you can also perform Retry through Polly, specific information refer to the http://www.thepollyproject.org/2016/10/25/polly-5-0-a-wider-resilience-framework.

In terms of implementation, Ocelot is a series of middleware combinations. When an HTTP request arrives at Ocelot, it is processed by a series of middleware and forwarded to downstream services. The middleware responsible for invoking downstream services is HttpRequestBuilderMiddleware, by calling HttpClient request downstream HTTP service, we here is to call the HttpClient with the fuse function, Code refer to the https://github.com/TomPallister/Ocelot/pull/27/files, the main part of the code is as follows:

using Ocelot.Logging;using Polly;using Polly.CircuitBreaker;using Polly.Timeout;using System;using System.Net;using System.Net.Http;using System.Threading;using System.Threading.Tasks;namespace Ocelot.Requester{    public class CircuitBreakingDelegatingHandler : DelegatingHandler    {        private readonly IOcelotLogger _logger;        private readonly int _exceptionsAllowedBeforeBreaking;        private readonly TimeSpan _durationOfBreak;        private readonly Policy _circuitBreakerPolicy;        private readonly TimeoutPolicy _timeoutPolicy;        public CircuitBreakingDelegatingHandler(int exceptionsAllowedBeforeBreaking, TimeSpan durationOfBreak,TimeSpan timeoutValue            ,TimeoutStrategy timeoutStrategy, IOcelotLogger logger, HttpMessageHandler innerHandler)            : base(innerHandler)        {            this._exceptionsAllowedBeforeBreaking = exceptionsAllowedBeforeBreaking;            this._durationOfBreak = durationOfBreak;            _circuitBreakerPolicy = Policy                .Handle<HttpRequestException>()                .Or<TimeoutRejectedException>()                .Or<TimeoutException>()                .CircuitBreakerAsync(                    exceptionsAllowedBeforeBreaking: exceptionsAllowedBeforeBreaking,                    durationOfBreak: durationOfBreak,                    onBreak: (ex, breakDelay) =>                    {                        _logger.LogError(".Breaker logging: Breaking the circuit for " + breakDelay.TotalMilliseconds + "ms!", ex);                    },                    onReset: () => _logger.LogDebug(".Breaker logging: Call ok! Closed the circuit again."),                    onHalfOpen: () => _logger.LogDebug(".Breaker logging: Half-open; next call is a trial.")                    );            _timeoutPolicy = Policy.TimeoutAsync(timeoutValue, timeoutStrategy);            _logger = logger;        }        protected override Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)        {            Task<HttpResponseMessage> responseTask = null;            try            {                responseTask = Policy.WrapAsync(_circuitBreakerPolicy, _timeoutPolicy).ExecuteAsync<HttpResponseMessage>(() =>                {                    return  base.SendAsync(request,cancellationToken);                });                return responseTask;            }            catch (BrokenCircuitException ex)            {                _logger.LogError($"Reached to allowed number of exceptions. Circuit is open. AllowedExceptionCount: {_exceptionsAllowedBeforeBreaking}, DurationOfBreak: {_durationOfBreak}",ex);                throw;            }            catch (HttpRequestException)            {                return responseTask;            }        }        private static bool IsTransientFailure(HttpResponseMessage result)        {            return result.StatusCode >= HttpStatusCode.InternalServerError;        }    }}

The above Code uses Policy. wrapAsync combines the fuse and retry policies to solve some of the failures. The idea is very simple. It defines the exceptions to be handled, such as the Policy. handle <HttpRequestException> (). or <TimeoutRejectedException> (). or <TimeoutException> (). When an exception occurs, use the fuse Or try again. The above code is also suitable for calling third-party services.

Welcome to join the construction of the. NET Core microservice development framework. Starting from the Ocelot thumb up and fork code for the project, I have contributed two feature codes, service discovery, and fuse described in this article to the project during the Spring Festival.