API Gateway Ocelot using Polly to handle partial failure issues

Another problem that needs to be considered in implementing the API gateway is partial failure. This problem occurs in distributed systems where a service invokes another service timeout or is not available. API gateway should not be blocked and wait indefinitely for downstream service status. However, how to handle this failure depends on the specific scenario and the specific service. If the product information service is unresponsive, then API Gateway should return an error to the client.

Ocelot is an API Gateway using the. NET core platform, and I've recently been involved in the development of this project, and the first thing to do is to use Polly to deal with some of the failures. The students may be unfamiliar with the project Polly, the first simple introduction, Polly is an open source project under the. NET Foundation, Polly Records those calls that exceed the pre-set limit values. It implements the circuit break mode, which allows the client to stop in an endless wait for unresponsive services. If the error rate of a service exceeds the preset value, Polly interrupts the service and all requests are immediately invalidated for a period of time, Polly can define a fallback operation for request failure, such as reading the cache or returning the default value. Sometimes we need to call other APIs when there is a temporary connection timeout, which can also be done through Polly Retry, specific information reference http://www.thepollyproject.org/2016/10/25/ polly-5-0-a-wider-resilience-framework/.

Ocelot from the implementation is a series of middleware combination, in the HTTP request to reach Ocelot, after a series of middleware processing forwarding to downstream services, which is responsible for invoking downstream services middleware is httprequestbuildermiddleware, By calling HttpClient to request the downstream HTTP service, we are here to add a fuse function to the httpclient call, code see https://github.com/TomPallister/Ocelot/pull/27/ Files, the main piece of code is as follows:

using Ocelot.Logging;
using Polly;
using Polly.CircuitBreaker;
using Polly.Timeout;
using System;
using System.Net;
using System.Net.Http;
using System.Threading;
using System.Threading.Tasks;

namespace Ocelot.Requester
{
    public class CircuitBreakingDelegatingHandler : DelegatingHandler
    {
        private readonly IOcelotLogger _logger;
        private readonly int _exceptionsAllowedBeforeBreaking;
        private readonly TimeSpan _durationOfBreak;
        private readonly Policy _circuitBreakerPolicy;
        private readonly TimeoutPolicy _timeoutPolicy;

        public CircuitBreakingDelegatingHandler(int exceptionsAllowedBeforeBreaking, TimeSpan durationOfBreak,TimeSpan timeoutValue
            ,TimeoutStrategy timeoutStrategy, IOcelotLogger logger, HttpMessageHandler innerHandler)
            : base(innerHandler)
        {
            this._exceptionsAllowedBeforeBreaking = exceptionsAllowedBeforeBreaking;
            this._durationOfBreak = durationOfBreak;

            _circuitBreakerPolicy = Policy
                .Handle<HttpRequestException>()
                .Or<TimeoutRejectedException>()
                .Or<TimeoutException>()
                .CircuitBreakerAsync(
                    exceptionsAllowedBeforeBreaking: exceptionsAllowedBeforeBreaking,
                    durationOfBreak: durationOfBreak,
                    onBreak: (ex, breakDelay) =>
                    {
                        _logger.LogError(".Breaker logging: Breaking the circuit for " + breakDelay.TotalMilliseconds + "ms!", ex);
                    },
                    onReset: () => _logger.LogDebug(".Breaker logging: Call ok! Closed the circuit again."),
                    onHalfOpen: () => _logger.LogDebug(".Breaker logging: Half-open; next call is a trial.")
                    );
            _timeoutPolicy = Policy.TimeoutAsync(timeoutValue, timeoutStrategy);
            _logger = logger;
        }

        protected override Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
        {
            Task<HttpResponseMessage> responseTask = null;

            try
            {
                responseTask = Policy.WrapAsync(_circuitBreakerPolicy, _timeoutPolicy).ExecuteAsync<HttpResponseMessage>(() =>
                {
                    return  base.SendAsync(request,cancellationToken);
                });
                return responseTask;
            }
            catch (BrokenCircuitException ex)
            {
                _logger.LogError($"Reached to allowed number of exceptions. Circuit is open. AllowedExceptionCount: {_exceptionsAllowedBeforeBreaking}, DurationOfBreak: {_durationOfBreak}",ex);
                throw;
            }
            catch (HttpRequestException)
            {
                return responseTask;
            }
        }

        private static bool IsTransientFailure(HttpResponseMessage result)
        {
            return result.StatusCode >= HttpStatusCode.InternalServerError;
        }
    }
}

In the above code, we use Policy.WrapAsync to combine the two strategies of fuse and retry to solve some failures. The idea is simple and define what exceptions need to be handled, such as Policy.Handle <HttpRequestException> () .Or <TimeoutRejectedException> () .Or <TimeoutException> (), what to do when an exception occurs, use a fuse or try again. Of course, the above code is also suitable for calling third-party services.

Everyone is welcome to join in building a microservices development framework for .NET Core. I started to like the project Ocelot and fork the code and build it together. During the Spring Festival, I have contributed 2 feature codes to the project, service discovery and the fuses described in this article.