Micro-service Distributed Tracking System (Springboot+zipkin)

Micro-service Distributed Tracking System (Springboot+zipkin)

First, what is Zipkin?

Zipkin is an open source distributed tracking system, open-source by Twitter, dedicated to collecting timing data for services to address latency issues in microservices architectures, including data collection, storage, discovery, and presentation. Its theoretical model comes from the Google Dapper paper.

Each service reports timing data to Zipkin, and Zipkin generates a dependency graph through the Zipkin UI based on the calling relationship, showing how many trace requests pass through each service, which allows developers to easily collect and analyze data through a Web front end, such as the processing time of a user requesting a service, etc. The bottleneck in the system can be conveniently monitored.

Ii. what needs a distributed tracking system (Zipkin)

Modern Internet services are often implemented with complex, large-scale distributed clusters. Especially with the rise of microservices architectures and container technologies (accelerating enterprise agility, adapting quickly to business changes, and meeting the high availability and scalability of architectures), Internet applications are often built on different services that may be developed by different teams, possibly using different programming languages, It is possible to fabric on thousands of servers spanning multiple different data centers. As a result, you need tools that can help you understand system behavior and quickly analyze performance issues. First Google developed its distributed tracking system and published a dapper paper, and then by the Twitter reference dapper thesis design ideas to develop zipkin distributed tracking system, while open source.

Zipkin collects tracking data to help developers gain insight into how a particular request is executed in a distributed system. If we now have a user request timeout, we can display this timeout request chain in the UI. We can quickly pinpoint what is causing a slow response to a service. If the details of this service are also very clear, then we can also locate which problem in the service caused the timeout. At the same time, the service call link can quickly locate the performance bottleneck of the system.

Third, Zipkin download and launch

In this section, we will describe downloading and launching Zipkin instances to check for Zipkin locally. There are three ways to install: Use the website's own packaged jar to run, Docker mode or download the source code to package the jar itself (because Zipkin uses springboot, built-in server, so you can run directly with the jar). Zipkin recommend using Docker mode, I will write about the operation of Docker, and the source of operation is the advantage of the opportunity to experience the latest features, but may also bring some more strange pits, so do not explain, Below I am directly using the official website packaged jar Run process:

(1) Download jar file

         Wget-o Zipkin.jar  ' https://search.maven.org/remote_content?g=io.zipkin.java&a=zipkin-server&v= Latest&c=exec '

However, I was in the process of running and found unable to download. I then downloaded the latest JAR file (Zipkin-server-1.17.1-exec.jar) through the wall software, and I also provided it here.

(2) Launch instance

Java-jar Zipkin-server-1.17.1-exec.jar or Java-jar Zipkin.jar (note the need for an JDK8 or above) to start successfully as shown in:

(3) View running effect

Through, we found that Zipkin uses springboot, and the boot port is 9411, and then we access it through the browser, the effect is as follows:

Iv. Architecture and core concepts of Zipkin

Components that send data to Zipkin's instrumented applications are called reporter. It sends trace data to the Zipkin collector through one of several modes of transmission, and the Zipkin collector saves the trace data to storage. Later, the store is queried by the API to provide data to the UI. To guarantee the invocation of the service link trace, Zipkin uses the transport ID, for example, when a trace operation is in progress and it needs to emit an outgoing HTTP request, some headers information is added to propagate the ID, but it cannot be used to send details (operation name, data, etc.). The schema diagram is as follows:

A, Span

Basic unit of work, a link call creates a span, identifies it through a 64-bit ID, and spans through other data, such as descriptive information, timestamps, key-value pairs of (Annotation) tag information, Parent-id, etc. Where Parent-id can represent a span invocation link source, a popular understanding of span is a request for information.

B, Trace

A span collection similar to a tree structure that represents a call link that has a unique identity.

C, Annotation

Annotations, which are used to record information about a request-specific event, such as time, and typically contain four annotation information:

(1) Cs-clientstart, which indicates that the client initiated the request

(2) Sr-server receive, indicating that the server received the request

(3) Ss-server send, which means the server finishes processing and sends the results to the client

(4) Cr-client Received, indicating that the client obtains the return information to the server

D, Transport

Collect the spans of the trace's services, and transmit to Zipkin's collector, there are three main transports: Http,kafka and scribe.

E, Collector

Zipkincollector will validate, store, and index an incoming trace data (span).

F, Storage

Storage, Zipkin The default storage mode is In-memory, that is, no persistence is performed. If you want to do data persistence, you can store data in Cassandra, because Cassandra is extensible, has a flexible pattern, and is heavily used in Twitter, and we make this component pluggable. In addition to Cassandra, we natively support Elasticsearch and MySQL. Other backend may be provided as a third-party extension.

G, QueryService

Once the data is stored and indexed, we need a way to extract it. The query daemon provides a simple JSON API for locating and retrieving traces, and the primary consumer of this API is WebUI.

H, WebUI

The Display page provides a nice interface to see the traces. The Web UI provides a way to view the trace based on service, time, and comments (via query service). Note: There is no built-in authentication in the UI.

Five, distributed Tracking System practice (Springboot+zipkin)

5.1 scene setting and analysis

There is now a service a called Service B, and service B calls services C and D, respectively, and the diagram for the entire link process is as follows:

It invokes the workflow call link Detail graph:

Represents a request link, a link through the Traceid unique identity, span identifies the requested information that is initiated, and each span is associated with the parent ID, Parentid==null, indicating that the span is the root span, as shown in:

Zipkin provides clients in a variety of languages (Java, Go, Scala, Ruby, JavaScript) and uses them to Zipkin report data.

5.2 Code Writing

Below I use the Java client brave as an example to complete the above four service call code writing, source code: Https://github.com/dreamerkr/mircoservice.git folder Springboot+zipkin below , specifically as follows:

(1) Serivce1

A, Springboot startup class

@SpringBootApplication @enableautoconfigurationpublic class Application {public    static void Main (string[] args) { C7/>springapplication.run (Application.class, args);}    }

B, Zipkin collection and configuration classes

/** *  * TODO zipkin configuration  * * @author Wangzhao (mailto:[email protected])  */@ Configurationpublic class Zipkinconfig {       //span (one request message or one link call) Information collector     @ bean    Public Spancollector spancollector () {        Config config = httpspancollector.conf Ig.builder ()                . compressionenabled (FALSE)//default False, Whether span will be gzipped                ConnectTimeout (transport)     and nbsp           Flushinterval (1)                 ReadTimeout (6                . Build ();        return httpspancollector. Create ("http://localhost:9411", config, New Emptyspancollectormetricshandler ());   }        //As each call link, it is only the responsibility to send the data in the specified format to zipkin    @Bean &nbsp   Public Brave Brave (Spancollector Spancollector) {        Builder builder = new Builder ("Service1 ");//Specify servicename        Builder.spancollector (spancollector);        Builder.tracesampler (Sampler.create (1));//Acquisition rate         return Builder.build ();   }   //Set up server (the service side receives the request and the server finishes processing and sends the results to the client) filters     @Bean     public braveservletfilter Braveservletfilter (Brave Brave) {        braveservletfilter filter = new Braveservletfilter ( Brave.serverrequestinterceptor (),                Brave.serverresponseinterceptor (), New Defaultspannameprovider ());        return filter;   }       //Set up the client (initiate request and get to server return information) Interceptor     @Bean     Public Okhttpclient okhttpclient (Brave Brave) {        okhttpclient httpClient = new OkhttpclieNt. Builder ()                . Addinterceptor (New  Braveokhttprequestresponseinterceptor                         Brave.clientrequestinterceptor (),                        Brav E.clientresponseinterceptor (),                         NE W Defaultspannameprovider ())). Build ();        return httpclient;   }}

C, service 1 business code

@Api ("Service Api Interface") @RestController @requestmapping ("/service1") public class Zipkinbravecontroller {@Autowired pri        vate okhttpclient Client; @ApiOperation ("Trace First step") @RequestMapping ("/test") public String Service1 () throws Exception {Thread.Sleep (1        00);        Request Request = new Request.builder (). URL ("Http://localhost:8082/service2/test"). Build ();        Response Response = client.newcall (Request). Execute ();    Return Response.body (). String (); }    }

D, Pom file

  <dependencies> <dependency> <groupId>org.springframework.boot</groupId>        <artifactId>spring-boot-starter</artifactId> <version>1.3.5.RELEASE</version>            </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-web</artifactId> <version>1.3.5.release</version&gt        ; </dependency> <dependency> <groupId>io.zipkin.brave</groupId> <a Rtifactid>brave-core</artifactid> <version>3.9.0</version> </dependency>& lt;! --<dependency> <groupId>io.zipkin.reporter</groupId> <artifactid>zip Kin-reporter-urlconnection</artifactid> <version>0.2.0</version> </dependency&gt          ;-<dependency> <groupId>io.zipkin.brave</groupId> <artifactid >brave-spancollector-http</artifactId> <version>3.9.0</version> &LT;/DEPENDENCY&G         T <dependency> <groupId>io.zipkin.brave</groupId> <artifactid>brave-web-ser vlet-filter</artifactid> <version>3.9.0</version> </dependency> <d Ependency> <groupId>io.zipkin.brave</groupId> <artifactid>brave-okhttp</ar            tifactid> <version>3.9.0</version> </dependency> <dependency>            <groupId>io.springfox</groupId> <artifactId>springfox-swagger2</artifactId> <version>2.6.1</version> </dependency> <dependency> <groupid>i O.springfox&lT;/groupid> <artifactId>springfox-swagger-ui</artifactId> <version>2.6.1</ve Rsion> </dependency> </dependencies>

E, application.properties

application.name:service1server.port:8081

(2) Serivce2

Pom File and startup class is the same as Service1, configuration class Zipkinconfig Service1 changed to Service2,application.properties name Service2, change port to 8082, service 2 business code is as follows :

@Api ("Service Api Interface") @RestController @requestmapping ("/service2") public class Zipkinbravecontroller {    @ autowired    private okhttpclient client;        @ApiOperation ("Trace Second Step")    @RequestMapping ("/test") public    String Service1 () throws Exception {        Thread.Sleep ($);        Request request3 = new Request.builder (). URL ("Http://localhost:8083/service3/test"). Build ();        Response response3 = Client.newcall (REQUEST3). Execute ();                Request request4 = new Request.builder (). URL ("Http://localhost:8084/service4/test"). Build ();        Response Response4 = Client.newcall (REQUEST4). Execute ();        return response3.tostring () + ":" +response4.tostring ();    }    }

(3) Serivce3

The pom file and the startup class are the same as the Service1, the configuration class Zipkinconfig change Service1 to Service3,application.properties name Service3, port to 8083, Service 3 business code is as follows:

@Api ("Service Api Interface") @RestController @requestmapping ("/service3") public class Zipkinbravecontroller {@ApiOperation        ("Trace Third Step") @RequestMapping ("/test") public String Service1 () throws Exception {Thread.Sleep (300);    return "Service3"; }    }}

(4) Serivce4

The pom file and the startup class are the same as the Service1, the configuration class Zipkinconfig change Service1 to Service4,application.properties name Service4, port to 8084, Service 4 business code is as follows:

@Api ("Service Api Interface") @RestController @requestmapping ("/service4") public class Zipkinbravecontroller {@ApiOperation        ("Trace Fourth Step") @RequestMapping ("/test") public String Service1 () throws Exception {Thread.Sleep (300);    return "Service4"; }    }

5.3 Operating Effect

(1) Start each service separately and then Access Service 1, browser access (http://localhost:8081/service1/test)

(2) Enter the Zipkin address, the list of each trace

Click on the trace to see the tree structure of the trace, including the time consumed by each service:

Click each span to get the delay information:

You can also view dependencies between services:

This article mainly shares with you zipkin for a simple service distributed tracking, but the data collected in memory, restart data loss, but Zipkin also provides Cassandra, MySQL, Elasticsearch and other storage methods, The following chapters will be explained in succession.

Micro-service Distributed Tracking System (Springboot+zipkin)