Springboot monitoring and management, actuator

         a brief preface

in the service architecture, by splitting a large monolithic system into multiple applications that provide different services. While the internal logic of each application is simplified by decomposition, the complexity of the system is greatly increased by the number of deployed applications.

With the increasing application, the number of failures in the system cluster is more and more, although under the protection of the high availability mechanism, individual failures will not affect the external services of the system, but these frequent applause need to be discovered and processed in time to ensure the system is in a healthy and usable state. To achieve this, we need an automated monitoring and maintenance mechanism, which is capable of continuously collecting indicators for each micro-service application and developing monitoring and early warning rules based on these basic indicator information.

Initial Actuator

This module is introduced to automatically provide a series of endpoints for monitoring for applications built by spring boot. Spring Cloud has made a number of extensions to the module as it implements the various service components. For example, more metrics and metrics are added to the native endpoint.

Introduction of Actuator

1. Spring-boot-starter-actuator dependencies are introduced in the Pom.xml file of the existing spring boot application.

   <dependency>     <groupId>org.springframework.boot</groupId>     

2. Restart the app after you increase the dependency. You can see the following in the console at this point:

Displays a batch of endpoint definitions, which are not created by ourselves in the program, but are automatically created by the Spring-boot-starter-actuator module to monitor and manage endpoints based on application dependencies and configurations. Through these endpoints, we can get the monitoring metrics of the application in real time, such as: access to the/health endpoint, we can get the application health information returned as follows:

{    "status": "UP",    "diskSpace": {        "status": "UP",        "total": 491270434816,        "free": 383870214144,        "threshold": 10485760    }}

Native Endpoint

By adding the Spring-boot-starter-actuator module to the QuickStart example, we have a preliminary understanding of it. Next, let's look at some of the native endpoints already implemented in the Spring-boot-starter-actuator module. If, depending on the role of the endpoint, we can divide the native endpoint into three main categories:

• Application Configuration class: Gets the configuration class information that is closely related to the spring boot application, such as application configuration, environment variables, and automation configuration reports loaded in the application.
• Metric class: Gets the metrics that are used for monitoring during application run, such as: memory information, thread pool information, HTTP request statistics, and so on.
• Operation Control class: Provides operation class functions such as closing the application.

Let's take a look at each of these three types of endpoints to provide us with useful information and powerful features, and how we can extend and configure them.

The following are all the monitoring descriptions:

HTTP method	Path	Describe	Authentication
GET	/autoconfig	View the usage of automatic configuration, which shows all auto-configuration candidates and why they are applied or not applied	True
GET	/configprops	Displays a list of all @configurationproperties	True
GET	/beans	Displays a complete list of all spring beans in an app	True
GET	/dump	Print thread stacks	True
GET	/env	View all environment variables	True
GET	/env/{name}	View specific variable values	True
GET	/health	View App Health Metrics	False
GET	/info	View app Information	False
GET	/mappings	View all URL mappings	True
GET	/metrics	View App Basic Metrics	True
GET	/metrics/{name}	View specific metrics	True
POST	/shutdown	Allow apps to close gracefully (not enabled by default)	True
GET	/trace	View basic tracking information	True

Apply Configuration Class

Because spring boot is designed to improve the complex configuration of traditional spring applications, the package scanning and automation configuration mechanisms are used to load the content that was originally concentrated in the XML file. While this makes our code very concise, information such as instance creation and dependencies for the entire application is dispersed over the annotations of each configuration class, making it very difficult to analyze the relationships of resources and instances throughout the application. This type of endpoint can help us easily get a series of detailed reports about the configuration of the spring application, such as reports of automated configuration, reports of bean creation, reports of environmental properties, and so on.

• /autoconfig: This endpoint is used to obtain an automated configuration report for an app that includes candidates for all automation configurations. It also lists the individual prerequisites for each candidate automation configuration that are met. So, this endpoint can help us find some specific reasons why the automation configuration is not effective. The content of the report divides the automation configuration into two parts:

  • Positivematches returns a successful condition matching automation configuration
  • Negativematches returned in an automated configuration with an unsuccessful condition match

   {"positivematches": {//condition matching successful "endpointwebmvcautoconfiguration": [ {"Condition": "Onclasscondition", "message": "@ConditionalOnClass classes Found:javax.s Ervlet. Servlet,org.springframework.web.servlet.dispatcherservlet "}, {" condition ":" ONWEBAP        Plicationcondition "," message ":" Found Web Application Standardservletenvironment "}], ...}, "Negativematches": {//condition mismatch succeeded "Healthindicatorautoconfiguration.datasourceshealthindicatorco Nfiguration ": [{" Condition ":" Onclasscondition "," message ":" Required @Conditio Nalonclass classes not Found:org.springframework.jdbc.core.JdbcTemplate "}], ...}}  

As we can see from the example above, there are a number of conditions in each automation configuration candidate, For example, there is no successful match on the healthindicatorautoconfiguration.datasourceshealthindicatorconfiguration configuration, its prerequisite is to include in the project Org.springframework The. Jdbc.core.JdbcTemplate class, because we do not introduce dependencies, it does not implement the automation configuration content. So, when we find that some of the desired configurations are not in effect, you can see the specific reason for not taking effect through that endpoint.

• /beans: This endpoint is used to get all the beans created in the app context.

  [{"Context": "hello:dev:8881", "parent": null, "beans": [{"Bean": "O                Rg.springframework.boot.autoconfigure.web.dispatcherservletautoconfiguration$dispatcherservletconfiguration ", "Scope": "Singleton", "type": "Org.springframework.boot.autoconfigure.web.DispatcherServletAutoConfigura tion$dispatcherservletconfiguration$ $EnhancerBySpringCGLIB $$3440282b "," resource ":" Null ","                    Dependencies ": [" Serverproperties "," Spring.mvc.CONFIGURATION_PROPERTIES ", "Multipartconfigelement"]}, {"Bean": "Dispatcherservlet"                , "Scope": "Singleton", "type": "Org.springframework.web.servlet.DispatcherServlet", "Resource": "Class path Resource [Org/springframework/boot/autoconfigure/web/dispatcherservletautoconfiguration$di SpatcherservletconfiguraTion.class] "," dependencies ": []}]}] 

  As in the example above, we can see that the following information is included in each bean:

  • Name of the Bean:bean
  • Scope of Scope:bean
  • Java type of Type:bean
  • Specific path to the Reource:class file
  • Dependencies: Dependent Bean name

• /configprops: This endpoint is used to obtain a report of the property information configured in the app. From the fragment that returns the example below, we see that the configuration information about the SMS is returned, the Prefix property represents the configuration prefix for the property, and properties represents the name and value of each property. So, we can use this report to see the configuration path of each property, for example, if we want to close the endpoint, we can complete the setup by using Endpoints.configprops.enabled=false.

  {    "configurationPropertiesReportEndpoint": {        "prefix": "endpoints.configprops",        "properties": {            "id": "configprops",            "sensitive": true,            "enabled": true        }    },    ...}

• /env: This endpoint differs from/configprops, which is used to get all the available Environment property reports for the app. Includes: Environment variables, JVM properties, application configuration configuration, parameters on the command line. From the sample fragment returned by the endpoint below, we can see that it returns not only the configuration properties of the app, but also rich configuration information such as system properties, environment variables, and so on, including the configuration that the app has not yet used. So it can help us to easily see the configuration information that the current application can load and use it with @configurationproperties annotations to bring them into our application. In addition, in order to configure the security of the property, for some sensitive information like password, the endpoint will be privacy protection, but we need to have the property name contains: Password, secret, key these keywords, so that the end of the endpoint when returning to use * to replace the actual property values.

   {"Profiles": ["Dev"], "Server.ports": {"Local.server.port": 88 Bayi}, "Servletcontextinitparams": {}, "Systemproperties": {"idea.version": "2016.1.3", "JAVA.R Untime.name ":" Java (TM) SE Runtime Environment "," Sun.boot.library.path ":" C:\\Program Files\\java\\jdk1.8.0_91\\jr E\\bin "," java.vm.version ":" 25.91-b15 "," Java.vm.vendor ":" Oracle Corporation ", ...}," System        Environment ": {" Configsetroot ":" C:\\windows\\configsetroot "," Rabbitmq_base ":" E:\\TOOLS\\RABBITMQ ", ...}, "Applicationconfig: [Classpath:/application-dev.properties]": {"Server.port": "8881"}, "Appl Icationconfig: [Classpath:/application.properties] ": {" Server.port ":" 8885 "," spring.profiles.active ":" Dev "," Info.app.name ":" Spring-boot-hello "," info.app.version ":" v1.0.0 "," Spring.application.name ":" He Llo "}}  

• /mappings: This endpoint is used to return the controller mapping relationship report for all spring MVC. From the sample fragment below, we can see that the information in the report is similar to the log information we output when we enable Spring MVC's Web app, where the bean attribute identifies the request processor for the mapping relationship, and the method property identifies the specific processing class and handler function for the mapping relationship.

   {"/webjars/**": {"Bean": "Resourcehandlermapping"}, "/**": { "Bean": "Resourcehandlermapping"}, "/**/favicon.ico": {"Bean": "Faviconhandlermapping"}, "{[/hello ]} ": {" Bean ":" Requestmappinghandlermapping "," Method ":" Public java.lang.String Com.didispace.web.HelloCon Troller.index () "}," {[/mappings | | /mappings.json],methods=[get],produces=[application/json]} ": {" Bean ":" Endpointhandlermapping "," Method ":" Public Java.lang.Object Org.springframework.boot.actuate.endpoint.mvc.EndpointMvcAdapter.invoke () "}, ...}  

• /info: This endpoint is used to return some application-customized information. By default, the endpoint returns only an empty JSON content. We can set some properties through the info prefix in the application.properties configuration file, such as the following:

  info.app.name=spring-boot-helloinfo.app.version=v1.0.0

  Again to access the/info endpoint, we can get the following return report, which contains the above two parameters that we used to customize.

  {    "app": {        "name": "spring-boot-hello",        "version": "v1.0.0"    }}

Metric class

The information report provided by the application configuration class endpoint described above has basically determined its return when the app is launched, which can be said to be a static report. The report content provided by the metric class endpoint is dynamically variable, which provides some snapshot information of the application during operation, such as memory usage, HTTP request statistics, external resource metrics, and so on. These endpoints are very helpful to us in building a monitoring system in the microservices architecture, and because the spring boot application itself implements these endpoints, we can easily use them to gather the information we want to develop a variety of automation strategies. Let's take a look at these powerful endpoint features separately.

• /metrics: This endpoint is used to return the current application of a variety of important metrics, such as: memory information, thread information, garbage collection information and so on.

  {"Mem": 541305, "Mem.free": 317864, "processors": 8, "Instance.uptime": 33376471, "uptime": 33385352, "systemload.a Verage ":-1," heap.committed ": 476672," Heap.init ": 262144," heap.used ": 158807," heap ": 3701248," nonheap.committed " : 65856, "Nonheap.init": 2496, "nonheap.used": 64633, "Nonheap": 0, "Threads.peak": $, "Threads.daemon": "Threa" Ds.totalstarted ":", "Threads": "Classes": 7669, "classes.loaded": 7669, "classes.unloaded": 0, "Gc.ps_scavenge." Count ": 7," Gc.ps_scavenge.time ": 118," Gc.ps_marksweep.count ": 2," Gc.ps_marksweep.time ": 234," Httpsessions.max ":-1 , "httpsessions.active": 0, "Gauge.response.beans":, "gauge.response.env": Ten, "Gauge.response.hello": 5, "GAUGE.R  Esponse.metrics ": 4," Gauge.response.configprops ": 153," Gauge.response.star-star ": 5," Counter.status.200.beans ": 1, "Counter.status.200.metrics": 3, "Counter.status.200.configprops": 1, "Counter.status.404.star-star": 2, " Counter.status.200.hello ": One," CounTer.status.200.env ": 1} 

  From the above example, we see that there are these important measures:

  • System Information: Including number of processors processors, run time uptime and instance.uptime, system average load systemload.average.
  • mem.*: Memory profile information, including the total amount of memory allocated to the app and the amount of memory currently idle. This information is from Java.lang.Runtime.
  • heap.*: Heap memory usage. This information is derived from the Java.lang.management.MemoryUsage obtained by the Getheapmemoryusage method in the Java.lang.management.MemoryMXBean interface.
  • nonheap.*: Non-heap memory usage. This information is derived from the Java.lang.management.MemoryUsage obtained by the Getnonheapmemoryusage method in the Java.lang.management.MemoryMXBean interface.
  • threads.*: Thread usage, including the number of threads, number of daemons (daemon), peak of thread, and so on, all from Java.lang.management.ThreadMXBean.
  • Classes.*: Applies the load and unload class statistics. The data are from Java.lang.management.ClassLoadingMXBean.
  • Gc.*: Details of the garbage collector, including garbage collection times Gc.ps_scavenge.count, garbage collection consumption time gc.ps_scavenge.time, number of token-purge algorithms gc.ps_marksweep.count, Mark-Clears the algorithm's consumption time gc.ps_marksweep.time. The data are from Java.lang.management.GarbageCollectorMXBean.
  • Httpsessions.*:tomcat the session usage of the container. Includes the maximum number of sessions Httpsessions.max and the number of active sessions httpsessions.active. This metric information is only available when embedded Tomcat is introduced as an application container.
  • One of the performance indicators of the GAUGE.*:HTTP request, which is mainly used to reflect an absolute value. As in the example above, gauge.response.hello:5 indicates a delay of 5 milliseconds for the last hello request.
  • One of the performance metrics of the counter.*:http request, which is used primarily as a counter to record the amount of increase and decrease. As counter.status.200.hello:11 in the example above, it represents the number of times that the Hello request returned a 200 status of 11.

  For gauge.* and counter.* statistics, there is a special content request Star-star, which represents access to a static resource. These two kinds of metrics are very useful, and we can not only use the default statistical indicators, but also easily add custom statistics to the program. You only need to inject org.springframework.boot.actuate.metrics.CounterService and org.springframework.boot.actuate.metrics.GaugeService to achieve a custom Statistical indicator information. For example, we can customize the number of accesses to the Hello interface as follows.

  @RestControllerpublic class HelloController {    @Autowired    private CounterService counterService;    @RequestMapping("/hello")    public String greet() {        counterService.increment("didispace.hello.count");        return "";    }}

  The/metrics endpoint can provide a complete measurement report of the application run status, which is very practical, but for monitoring the system's monitoring functions, their monitoring content, data collection frequency is different, if we each time through the full amount of reports collected, slightly rude. So, we can also use the/metrics/{name} interface to get more granular metrics, such as we can access the/metrics/mem.free to get the current amount of available memory.

• /health: This endpoint has been used in the first example to get information about the various health metrics for your app. In the Spring-boot-starter-actuator module, we implement some common resources health indicator detectors. These detectors are implemented through the Healthindicator interface , and are automatically assembled according to the introduction of dependencies, such as diskspacehealthindicator for detecting disks, Detects if the DataSource connection is available datasourcehealthindicator, and so on. Sometimes, we may also use some spring boot starter poms products that are not encapsulated in the development, such as: When using ROCKETMQ as the message agent, because there is no automated configuration of the detector, So we need to implement a detector to collect health information. For example, we can implement a ROCKETMQ detector class for the Org.springframework.boot.actuate.health.HealthIndicator interface in the Spring boot application:

  @Component Public classRocketmqhealthindicatorImplementsHealthindicator {@Override PublicHealth Health () {intErrorCode =check (); if(ErrorCode! = 0) {          returnHealth.down (). Withdetail ("Error Code", ErrorCode). Build (); }        returnhealth.up (). build (); }      Private intCheck () {//detecting operations on monitored objects      }}

  By overriding the health () function to achieve a healthy check, a total of two items are returned from the Heath object, one of which is state information, in addition to the up and down in the example, and unknown and Out_of_service, which can be implemented as needed to return There is also a detailed information that is stored in map mode, where an error code message is injected through the Withdetail function, and we can also fill in other information, such as the IP address, port, etc. of the object being detected. To restart the application and access the/health interface, we will include the following information in the returned JSON string:

  "rocketMQ": {  "status": "UP"}

• /dump: This endpoint is used to expose thread information in program runs. It uses Java.lang.management.ThreadMXBean's Dumpallthreads method to return all active thread details that contain synchronization information.

• /trace: This endpoint is used to return basic HTTP trace information. By default, the storage of tracking information takes the form of memory implemented by Org.springframework.boot.actuate.trace.InMemoryTraceRepository and always retains the last 100 request records. It records content in the following format:

  [{"Timestamp": 1482570022463, "info": {"method": "GET", "path": "/metrics/mem", "Headers": {"request": {"host": "localhost:8881", "Conne Ction ":" Keep-alive "," Cache-control ":" No-cache "," user-agent ":" Mozilla/5.0 (Windo WS NT 10.0; WOW64) applewebkit/537.36 (khtml, like Gecko) chrome/53.0.2785.143 safari/537.36 "," Postman-token ":" 98 17ea4d-ad9d-b2fc-7685-9dff1a1bc193 "," Accept ":" */* "," accept-encoding ":" Gzip, DEFL                    Ate, Sdch "," Accept-language ":" zh-cn,zh;q=0.8 "}," response ": {                     "X-application-context": "hello:dev:8881", "Content-type": "Application/json;charset=utf-8",                    "Transfer-encoding": "chunked", "Date": "Sat, 09:00:22 GMT", "StATUs ":" 200 "}}}}, ...] 

Operation Control class

A careful reader may find that all of the monitoring endpoints that we run in the console of the sample when we "first know actuator" are already covered in the application configuration class endpoint and metric class endpoint. So what are the operations control class endpoints? In fact, because all of the endpoints described earlier are used to reflect the properties of the application itself or the running state, they are enabled by default relative to the operation control class endpoint, which is not so sensitive. The operations control class endpoints have more control, and if you want to use them, you need to configure them with properties to open.

In the native endpoint, only one endpoint to close the app is provided:/shutdown. We can open it with the following configuration:

endpoints.shutdown.enabled=true

Once you have configured the above attributes, you can only access the/shutdown endpoint of the app to enable remote operations to close the app. Since the operation of the open shutdown application is inherently dangerous, we need to add some protection to it when it is really online, such as customizing the endpoint path of the actuator, integrating the spring security, and so on.

Springboot monitoring and management, actuator