Use Grafana to display monitoring charts for Docker containers and set up email alert rules

First, Docker container monitoring alarm mode

　　

Then the previous article record, see Grafana version has been updated to 4.2, and in 4.0 later version, added the alert notifications function, so in the container monitoring, you can add alarm rules. According to the official website, there are many kinds of alarm methods, common email, slack instant messenger, Webhook and so on.

This article records the alarm settings for the message. The environment and the above are basically consistent, are in the Docker Platform test environment, in addition this article uses the Grafana container version is the latest version of the Dockerhub, this version is grafana/grafana:latest; Since the container is generated using Docker file and the service startup script for the container process Grafana-server is run.sh, we will modify the run.sh, then reboot the container and then configure it on the interface.

Overall approximate steps: Create a container--Modify the run.sh file within the container, add SMTP-related configuration--restart container--Login Grafana interface, add alarm notifications--> Add influxdb data Source-- > Edit graph, define alarm rules--Test alarm message verification

Ii. Deploy Grafana containers and configure mail alerts

1. Run Create a Grafana container

+- e influxdb_host=localhost-e influxdb_port=8086 -E influxdb_name=cadvisor-e Influxdb_user=root-e influxdb_pass=root--link influxsrv:influxsrv--name grafana-alerting grafana/  GRAFANAF9495C492B9E4C735F0002D47FC4AF668691F83E60B70D7B09DEFBFA2C806F6D

2. Modify the script that Grafana service runs within the container

Divided into 2 ways,

The first type:

Copy the run.sh from the Grafana-alerting container to the host, modify it, and then copy the past overwrite replace.

CP Grafana-alerting:/run. sh /tmp/run. SH

Edit the/tmp/run.sh file (the full contents of the file will be given later) after the modification is complete, replace the script in the container with the script

CP /tmp/run. sh grafana-alerting:/run. SH

The second type:

Enter into the container, first apt-get update system, then install VIM (the default official image, the Vim, VI, nano-like editor is not found.) I did not find ha, perhaps there may be, this way will not elaborate on it.

The complete contents of the run.sh file are as follows:

#!/bin/bash-e:"${gf_paths_data:=/var/lib/grafana}": "${gf_paths_logs:=/var/log/grafana}": "${gf_paths_plugins:=/var/lib/grafana/plugins}": "${gf_smtp_enabled:=true}": "${gf_smtp_host:=smtp.163.com:25}": "${gf_smtp_user:[email protected]}": "${gf_smtp_password:=fuck123}": "${gf_smtp_from_address:[email protected]}"Chown-R Grafana:grafana"$GF _paths_data" "$GF _paths_logs"Chown-R grafana:grafana/etc/Grafanaif[!-Z ${gf_aws_profiles+x}]; Then    mkdir-P ~grafana/.aws/Touch~grafana/.aws/Credentials forProfileinch${gf_aws_profiles}; DoAccess_key_varname="gf_aws_${profile}_access_key_id"Secret_key_varname="Gf_aws_${profile}_secret_access_key"Region_varname="gf_aws_${profile}_region"        if[!-Z"${!access_key_varname}"-A! -Z"${!secret_key_varname}"]; Then            Echo "[${profile}]">> ~grafana/.aws/CredentialsEcho "aws_access_key_id = ${!access_key_varname}">> ~grafana/.aws/CredentialsEcho "Aws_secret_access_key = ${!secret_key_varname}">> ~grafana/.aws/Credentialsif[!-Z"${!region_varname}"]; Then                Echo "Region = ${!region_varname}">> ~grafana/.aws/Credentialsfi        fi     Done    ChownGrafana:grafana-r ~grafana/. AWSchmod  -~grafana/.aws/Credentialsfiif[!-Z"${gf_install_plugins}"]; Thenoldifs=$IFS IFS=','   forPlugininch${gf_install_plugins}; DoGrafana-CLI--pluginsdir"${gf_paths_plugins}"PluginsInstall${plugin} DoneIFS=$OLDIFSfiexec Gosu Grafana/usr/sbin/grafana-Server--homepath=/usr/share/Grafana--config=/etc/grafana/Grafana.ini Cfg:default.paths.data="$GF _paths_data"Cfg:default.paths.logs="$GF _paths_logs"Cfg:default.paths.plugins="$GF _paths_plugins"cfg:default.smtp.enabled="$GF _smtp_enabled"Cfg:default.smtp.host="$GF _smtp_host"Cfg:default.smtp.user="$GF _smtp_user"Cfg:default.smtp.password="$GF _smtp_password"cfg:default.smtp.from_address="$GF _smtp_from_address"

The file only adds 10 perspiration content to the file in the container. Content added to the first half:

: "${gf_smtp_enabled:=true}" #启用smtp配置: "${gf_smtp_host:=smtp.163.com:25}" #smtp服务器的地址: Port : "${gf_smtp_user:　　　　[Email protected]} " #认证的邮箱账户: "${gf_smtp_password:=fuck123}" #邮箱密码 (Note: 163 uses the client authorization code as the user password): "${gf_smtp_from_address:[email Protected]} "#邮箱发件人地址, pay attention! This must be consistent with the above email authentication account. Otherwise it will report 553 errors.!!

Content added to the latter part:

  cfg:default.smtp.enabled= "$GF _smtp_enabled"     cfg:default.smtp.host= "$GF _smtp_host"     cfg: default.smtp.user= "$GF _smtp_user"     cfg:default.smtp.password= "$GF _smtp_password"     cfg: default.smtp.from_address= "$GF _smtp_from_address"   

Small notes:

The Grafana server has configuration options that can be specified in the. ini configuration file or using environment variables. The default profile path is/etc/grafana/grafana.ini, or you can specify a profile path when you start Grafana-server.

environment variable format:gf_<sectionname>_<keyname>; where sectionname refers to the section name in brackets in the configuration file, KeyName refers to the parameter name. To capitalize.

　　Cfg:default.smtp.host: This is the parameter specified when the Grafana script starts.

This article belongs to original article, if reproduced, please indicate the source and link. From flying away cannot: http://www.cnblogs.com/hanyifeng/p/6293620.html

3. After modifying the script, restart the container, equivalent to restart the service

[[email protected] ~]# Docker stop grafana-alertinggrafana-~]# Docker start grafana-  Alertinggrafana-alerting

4. Open the browser, visit HTTP://IP, the homepage is as follows:

Add a notification group (can add multiple, convenient grouping to send alarm settings)

Name: Defines a notification name that specifies the send to when the alert alarm rule is added to the post graph

Type: the email defined here

Email addresses: Define recipient mailing List

Last save, click Sendtest, if your mailbox receives the following mail, the configuration is successful.

5. Add the data source, just as you would add information in the previous article. This step is skipped.

6. Add a Dashboard

Then create a new graph, edit, and an alert below, creating an alert. Such as:

Alert Config fills in the following:

Notifications fill in the following:

Primary selection sent to a previously defined notification group

7. Enter the monitored container and use the following command to simulate an increased memory

This article belongs to original article, if reproduced, please indicate the source and link. From flying away cannot: http://www.cnblogs.com/hanyifeng/p/6293620.html

Enter the container:

[email protected] ~]# Docker exec-it Cadvisor/bin/ash

Use the DD command, as follows:

/# Swapoff-addif=/dev/zero of=/dev/shm/test bs=1k count=80000kdd'  /dev/shm/test': No space left on device65537+0 in 65536+0 Records out

Then observe the Grafana graph interface, you can find that the alarm rules have been effective, you can log in to set the mailbox, see if there is no mail received.

The contents of the message are as follows:

To free up memory, just delete the/dev/shm/test file.

RM -f/dev/shm/test

After waiting for 5 minutes (this 5 minutes is defined by the alarm rule 5m), the warning will be lifted. The color of ♥ will also change from red to green.

OK email has also been received, as follows:)

In addition, you can view the list of warnings in Alerting--alert llist.

Third, PostScript

Currently Grafana only support graph add alarm rules, the official said follow-up will support singlestat and table panel, detailed introduction, you can refer to here. Grafana monitoring and alarming aspects of the temporary study record here, the latter has what to expand Learning, continue to share. There are shortcomings in the text, but also hope that we have a lot of advice.

This article belongs to original article, if reproduced, please indicate the source and link. From flying away cannot: http://www.cnblogs.com/hanyifeng/p/6293620.html

_

Use Grafana to display monitoring charts for Docker containers and set up email alert rules