Ocelot integrates Butterfly for Distributed tracking and ocelotbutterfly

Ocelot integrates Butterfly for Distributed tracking and ocelotbutterfly

Microservices are usually implemented using complex and large-scale distributed clusters. Microservices are built on different software modules, it may be developed by different teams and may be implemented using different programming languages. It may be deployed on thousands of servers, spanning multiple different data centers. Therefore, some tools are needed to help you understand system behavior and analyze performance problems.

As an important component of microservices, API gateway Ocelot appears on the system boundary as an API-oriented, serially centralized, and highly managed service. The boundary here is the boundary of enterprise IT systems, it mainly serves to isolate external access from internal systems. OpenAPI is usually released through API gateway, which is followed by many distributed applications, such as microservices, message sending and receiving, distributed databases, distributed cache, distributed object storage service, and cross-origin calls, these components constitute a complex distributed network.

When application A sends A request, dozens or more services may be called ". If we compare a distributed system to a highway network, each front-end request is equivalent to a high-speed vehicle, and the request processing application is a high-speed toll station, the vehicle traffic information is recorded as a log on the toll station, including the time, license plate, site, road, price, etc. If all the logs on the toll station are integrated, the complete traffic record of the vehicle can be determined by the unique license plate number. The distributed call system tracking and monitoring is similar to this idea, then, the application and time consumption of each request are clarified.

Butterfly is designed as a distributed Tracing and APM Server. It will contain Collector, Storage, and an independent Web UI, and uses Open Tracing specifications to design Tracing data. Currently, only Open Tracing APIs are implemented according to specifications, and google's opencensus will be compatible in the future. Here, by the way, we don't need zipkin or Jaeger. They only do tracing. Butterfly is a little more than they do at the same time to implement metrics and warning, that is, to implement a three-dimensional monitoring system. Currently, Butterfly is in its infancy, and many features need to be developed. Currently, there are two tasks with a higher priority: metrics at the application process level, one is the performance optimization of the backend collector and es. You are welcome to join in the development. We believe that through continuous construction, we will. the NET community can also reach the height of Java. Looking back at the development history of Ocelot, we have been developing Ocelot for two years in 2016 and completed the development of version 3.0. Now it is an increasingly mature API gateway, connect to the following service through API gateway, like today I share with you the support of recently developed distributed tracing in my spare time, this task comes out on a one-year premise, https://github.com/TomPallister/Ocelot/issues/19 here for our discussion, now we integrate Butterfly to implement this function, so that our microservices can be maintained.

Butterfly. Client. AspNetCore provides us with the integration of Butterfly components in the ASP. NET Core project.ConfigureServicesRegister Butterfly services

public void ConfigureServices(IServiceCollection services){   //your other code   services.AddButterfly(option =>  {      option.CollectorUrl = "http://localhost:9618";      option.Service = "my service";  });}

Http: // localhost: 9618 is the Butterfly server that provides the UI. We can access it in the browser through http: // localhost: 9618.

What is the difference between the integration of Butterfly in API gateway Ocelot? After we add the above Code to the Ocelot project, we can see our tracing data on the Butterfly UI, but the data is not linked together. Therefore, the main tasks of integration are as follows:

I. concatenate tracing data so that we can intuitively view the data of each node on the Butterfly UI.

Ii. Definition of data that Ocelot itself needs to be added to system tracking

Ocelot integrated Butterfly to implement distributed tracking code has not been added to the trunk, you can see in my code library branch https://github.com/geffzhang/Ocelot/tree/Monitoring, we first add a configuration item in the Ocelot routing configuration, whether to enable distributed tracing:

{
"ReRoutes ":[
{
"DownstreamPathTemplate": "/api/values ",
"DownstreamScheme": "http ",
"UpstreamPathTemplate": "/api/values ",
"UpstreamHttpMethod": ["Get"],
"DownstreamHostAndPorts ":[
{
"Host": "localhost ",
Port: 5002
}
],
"HttpHandlerOptions ":{
"AllowAutoRedirect": true,
"UseCookieContainer": true,
"UseTracing": true
}
},

UseTracing indicates whether to enable distributed tracing. The default value is false, that is, disable it. Then add an interface method in Ocelot. DependencyInjection. IOcelotBuilder:

The implementation of the method is also very simple:

The main is to add the Ocelot itself needs to be added to the system tracking data definition, implementation mainly use DiagnosticSource, the official documentation: https://github.com/dotnet/corefx/blob/master/src/System.Diagnostics.DiagnosticSource/src/DiagnosticSourceUsersGuide.md. Similar to asp.net core, asp.net Core has a Diagnostics middleware https://github.com/aspnet/diagnostics. The main function is to report and handle exceptions and errors in asp.net core, and diagnose Entity Framework Core migration errors. There are several other functions, including the welcome page, error code page, and page 404. As well as a fairly good Log Viewing function, this function is also required by many people to directly view logs online.

Implemented the Butterfly interface ITracingDiagnosticListener. After DI injection, Butterfly will help us register it.

Next we will concatenate our distributed tracing data, OpenTracing (link: opentracing. io) allows developers to conveniently add (or replace) tracing systems by providing platform-independent and vendor-independent APIs. OpenTracing is providing unified concepts and data standards for global distributed tracing. The standard Chinese version is translated by our MVP Wu Sheng and is also a major Member of OpenTracing: https://wu-sheng.gitbooks.io/opentracing-io/content /.

In a broad sense, a trace represents the execution process of a transaction or process in a (distributed) system. In the OpenTracing standard, trace is a directed acyclic graph (DAG) composed of multiple spans. Each span represents a continuous execution segment named and timing in the trace.

Each component in Distributed tracing contains one or more of its own spans. For example, in a conventional RPC call process, OpenTracing recommends that the RPC client and server have at least one span, which is used to record the client and server information of the RPC call.

A parent span displays multiple child spans that start in parallel or in serial mode. In the OpenTracing standard, a child span can even have multiple parent spans (for example, the cache for parallel writing may be written through a refresh operation ).

The key point of integration is the Id processing of the association between tracerId and spanId.

Tracerid indicates a global id. Similar to Ocelot's RequestId upload header, its key is ot-traceid. Therefore, you can set the global RequestId to ot-traceid in Ocelot.

At the same time, you also need to process the spanid so that the spanid of the downstream component is the spanid of its upper level and also stored in the http header. Its key is ot-spanId, we need to process the spanid in OcelotRequestTracer and OcelotHttpTracingHandler.

As we have finished code integration, let's take a look at the results. I built a Demo environment, service front-end-> Ocelot-> service backend. Butterfly generates a globally unique ID (Traceld) for each request. It associates the "isolated" call information of different systems to restore more valuable data.

Is the call chain of an API call request. In the Span column, you can see a series of application components that pass through the request intermediate process. You can see the HttpClient component that first passes through the request, subsequent calls to Ocelot, HttpClient, and backend form a call tree (the indentions in the tree indicate nested relationships). From the call tree, you can easily see the complete process of front-end requests. The page shown in clearly shows the specific time consumed by each application to process requests, which is very intuitive to locate. In addition, click a specific component to view the log records in this component.

For the distributed call tracing system, it not only provides the call chain function, because it implements tracking for all middleware calls, therefore, all the situations on the middleware can be monitored. Therefore, a detailed call monitoring report is also formed when a call chain is formed. Unlike other monitoring reports, this report provides the drill-up and drill-down functions. Because the call chain is a detailed bottom-layer statistics, the report dimensions that can be formed on the preceding pages are very rich. In the call reports shown, you can not only view the service conditions, you can drill down the services it calls. In addition, you can drill down the call chain from the monitoring report to view the clear call chain information. Currently, Butterfly is a feature that requires further development. You are welcome to join the development process.

There is also link analysis. Different from the call chain, the link is a statistical concept, and the call chain is a single call process. Topology Analysis of analysis links: analyze the source and destination, and identify unreasonable sources;

Is a global call topology. You can clearly see the complex call relationships between different services, the call relationships between a service and other services, and the call frequency; through this topology, the architect can clearly observe the call situation on the system.