GRAFANA+TELEGRAF+INFLUXDB Monitoring Tomcat Cluster scenario

Objective

A few years ago since the house of several cats often problems, because there is no effective monitoring and warning means, so that the problem arises or a long time will be notified. One o'clock in the morning this pot can no one want to back, to this end based on the current situation to build the following a set of monitoring and warning system.

Related software

• Nginx: Proxy Access Grafana
• Grafana: Visualization Panel (Dashboard) with very nice charting and layout display
• Influxdb: Open Source time series database for recording metrics, events, and execution analysis
• Telegraf: Collecting statistics on systems and services
• Docker: Open source application container engine that allows developers to package their applications and dependencies into a portable container

Monitoring architecture

GTI Monitoring early Warning system, architecture process Description:

• First step: Data acquisition, TELEGRAF acquisition of TOMCAT-related parameters
• The second step: data storage, INFLUXDB storage Telegraf collected data
• Step three: Data visualization, Grafana configuration Tomcat monitoring Panel
• Fourth step: Alert notification, configure nails, mail and other early warning

Installation configuration

Here only Grafana, Telegraf, Influxdb, Tomcat to do the corresponding installation instructions, Nginx and Docker, please check the information yourself.

Grafana

Grafana is just a visual panel to access the data source, for convenience, we choose Docker installation.

Mkdir grafana
ID=$(id -u)
Docker run -d --user $ID --name=grafana --volume "$PWD/grafana:/var/lib/grafana" -p 3000:3000 grafana/grafana
# If the production environment is configured, it is best to configure the domain name in advance.
Docker run -d --user $ID --name=grafana --volume "$PWD/data:/var/lib/grafana" -p 3000:3000 -e "GF_SERVER_ROOT_URL=http://monitor.52itstyle.com" Grafana/grafana

After successful execution, execute the following command:

docker ps

If the Grafana run container appears, the installation succeeds.

To view container-related parameters:

docker inspect docker.io/grafana/grafana

Enter:

docker exec -it grafana /bin/sh

The default configuration file for Grafana Grafana.ini is located in the/etc/grafana in the container, which is not mapped. However, you can create and run a container and copy it to recreate the running container.

Parameter description (some key parameters are intercepted here):

##################### Grafana Several important parameters (refer to the following) ################### ##
[paths]
# Storing temporary files, sessions, and directories for the sqlite3 database
;data = /var/lib/grafana

# Where to store the log
;logs = /var/log/grafana

# Place the relevant plugin
;plugins = /var/lib/grafana/plugins

################################################################################################# #######################
[server]
#default protocol support (http, https, socket)
;protocol = http

# default port
;http_port = 3000

# Configure the access address here. If a reverse proxy is used, configure the domain name. When sending an alarm notification, use it as the access address.
Root_url = http://grafana.52itstyle.com

############################################################################################# #######################
[database]

# The default database sqlite3 is located in the /var/lib/grafana directory.
;path = grafana.db

############################################################################################# #######################
[session]
#session Storage mode, the default is file Either "memory", "file", "redis", "mysql", "postgres", default is "file"
;provider = file

######################################### SMTP / Emailing ########### ###############
[smtp]
# Mail server configuration, modify the configuration by yourself
Enabled = true
Host = smtp.mxhichina.com: 465
User = admin@52itstyle.com
# If the password contains # or ; you have to wrap it with trippel quotes. Ex """#password;"""
Password = 123456
;cert_file =
;key_file =
;skip_verify = false
From_address = admin@52itstyle.com
# Do not set Chinese here, otherwise it will fail to send
From_name = Grafana

Influxdb

Create and run the container

docker run -d -p 8083:8083 -p 8086:8086 -e ADMIN_USER="root" -e INFLUXDB_INIT_PWD="root" -e PRE_CREATE_DB="telegraf" --name influxdb tutum/influxdb:latest

The meaning of each parameter:

-d: the container is running in the background

--name: container name

-e: Specify an environment variable that can be used in the container

-p: maps the port inside the container to the host port in the format of the host port: the port inside the container; 8083 is the web management tool port of influxdb, and 8086 is the HTTP API port of influxdb

After successful execution, execute the following command:

docker ps

If the Influxdb run container appears, the installation succeeds.

Access Address: http://ip:8083/

Telegraf

docker pull telegraf

Copy the Telegraf related configuration to the host machine

docker cp telegraf:/etc/telegraf/telegraf.conf ./telegraf

Collect Tomcat data:

If you want to monitor multiple tomcat, configure multiple [[Inputs.tomcat]] here, but be sure to configure a different tags identity.

[[inputs.tomcat]]
Url = "http://192.168.1.190:8080/manager/status/all?XML=true"
# Tomcat access account password must be configured
Username = "tomcat"
Password = "tomcat"
Timeout = "5s"
# Identifies the Tomcat name, depending on the actual project deployment
[inputs.tomcat.tags]
Host = "blog"

[[inputs.tomcat]]
Url = "http://192.168.1.190:8081/manager/status/all?XML=true"
# Tomcat access account password must be configured
Username = "tomcat"
Password = "tomcat"
Timeout = "5s"
# Identifies the Tomcat name, depending on the actual project deployment
[inputs.tomcat.tags]
Host = "bbs"

Collect data to Influxdb:

[[outputs.influxdb]]
       # urls = ["udp://localhost:8089"] # UDP endpoint example
       Urls = ["http://localhost:8086"] # required, this url is changed to your own host
       ## The target database for metrics (telegraf will create it if not exists).
       Database = "telegraf" # This will create a library in the influx library 

Copy the configuration file to the container:

docker cp telegraf.conf telegraf:/etc/telegraf/telegraf.conf

Restart the Telegraf service:

docker restart docker

Tomcat

Since Telegraf collects tomcat-related data to configure access, here we choose TOMCAT7 to do configuration instructions.

Modify the Tomcat-users.xml file located under Conf:

<tomcat-users><user username="tomcat" password="tomcat" roles="manager-gui,manager-script,manager-jmx,manager-status"/></tomcat-users>

Restart the Tomcat container to access the following address:

http://ip:8080/manager/status/all?XML=true

If the above interface appears, the configuration is successful.

Monitoring Configuration

After launching Tomcat, Influxdb, Telegraf, Grafana, we entered Grafana background management for the relevant configuration.

To configure the INFLUXDB data source:

Select Datasources/add DataSource

Enter the correct HTTP address and database account password, click Save, if a green prompt appears, the configuration is successful.

To configure the Tomcat dashboard:

Select Dashboard/import

There are three ways to import panels:

• Select Enter official panel ID or URL
• Direct copy paste JSON format code
• Import third-side board JSON format file

Here we import our own custom-saved Tomcat monitor panel, and finally click Import to save.

If not unexpected, will be the appearance.

Alarm Configuration

So much has been done in the early stages, and our ultimate goal is to make early warning announcements before the system is ready for disaster. Here we use Tomcat's thread count threshold as an alert notification.

Click on the Threads panel-select Edit:

To configure the relevant parameters:

1, alert name, can be customized.
2, the frequency of implementation, here I choose to detect every 60s.
3, judge the standard, the default is AVG, here is the drop-down box, their own according to the needs of choice.
4, query (A,5m,now), the letter A for the selected metrics set in the SQL, you can also choose other settings in metrics, but here is a radio. The 5m represents five minutes from now onwards, that is, the point before 5m is the starting point of the time, now is the end point of the time, and here you can manually enter the time.
5, set the warning point, here Manual input, and 6 is the same function, 6 can be manually moved, the two operations are equivalent.

Configure alert information and how to notify:

Here we choose is the Mail alert notification, but to be configured in advance, see the beginning Grafana.ini smtp/emailing related parameter configuration.

Click Send Test, the prompt will send a copy of the alert demo to the designated mailbox:

Summarize

To be reasonable, this set of things is still quite powerful. Especially for small and medium-sized companies, a variety of mature open-source group between the integration of a perfect set of monitoring system, time costs, labor costs, technical costs can be minimized.

Reference documents

During the installation process, the version may not be the same, the related page exhibition is inconsistent, but will not affect the final feature rendering.

http://docs.grafana.org/

https://docs.influxdata.com/influxdb/

https://docs.influxdata.com/telegraf/

https://blog.52itstyle.com/archives/2014/

https://blog.52itstyle.com/archives/2029/

https://github.com/influxdata/telegraf/pull/3277

GRAFANA+TELEGRAF+INFLUXDB Monitoring Tomcat Cluster scenario