Collectd+influxdb+grafana Building Performance Monitoring Platform

Collectd+influxdb+grafana Building Performance monitoring PlatformPreface

• InfluxDB is an open-source distributed time-series database for Go language development, which is well suited for storing metrics, events, analytics and other data; key-value times database performance is good.

• COLLECTD is a system performance acquisition tool written in the C language.

• Grafana is a front-end tool for pure Javascript development for accessing InfluxDB, customizing reports, displaying charts, and more. V3.0 the above version supports Zabbix database, it is very convenient to collect data directly from Zabbix_agent.

1 Environmental Information

Test environment 174,175 Twin Unit, press 172

COLLECTD Collect data, influxdb store data, Grafana display data

The three relationships are:

Data collection (COLLECTD), storage data (INFLUXDB), display (Grafana)

According to the test needs, installed on three machines COLLECTD collect data, Influxdb and Grafana installed on 172

The author three machines for the same network segment, the operating system are CentOS Linux release 7.4.1708 (Core)

2 Installing COLLECTD

Installing dependent packages and COLLECTD

Yum-y Install Epel-release
Yum-y Install COLLECTD

COLLECTD the installed version is now collectd-5.8.0-4.el7.x86_64

COLLECTD Dependent epel-release Installation source

Installing the RRDtool Plugin

In order to communicate with Influxdb, COLLECTD as the client, need to connect the INFLUXDB port 25826, so to open the network plugin and configure the server properties;

In order for Influxdb to be able to identify COLLECTD data, install the RRDtool plugin, otherwise the/VAR/LIB/COLLECTD/RRD directory will not generate

RRDtool plug-in with dependent package installation:

yum install collectd-rrdtool rrdtool rrdtool-devel

Parameter configuration

The configuration file path for COLLECTD under the default installation path is/etc/collectd.conf

In configuration file collectd.conf:

• Two # #开头表示该插件还没有构建, you can't use it

• A # start indicates that the plugin has been built, but cannot use

• No # indicates that the plugin has been built and can be used

Vi/etc/collectd.conf
Hostname "client174"
Fqdnlookup true
BaseDir "/VAR/LIB/COLLECTD"
Pidfile "/var/run/collectd.pid"
Plugindir "/USR/LIB64/COLLECTD"
Typesdb "/usr/share/collectd/types.db"
Loadplugin syslog
Loadplugin CPU
Loadplugin disk
Loadplugin interface
Loadplugin Memory
 loadplugin rrdtool 
  loadplugin swap 
  ? 
  <plugin cpu> 
  reportbycpu true 
  reportbystate true 
  valuespercentage true 
 Span></plugin> 
  ? 
  <plugin interface> 
        Interface "eth0" 
        ignoreselected false 
  </plugin> 
  ? 
  <plugin load> 
        reportrelative true 
  </plugin> 
  ? 
  <plugin network> 
        Server "*.*.*.*" "25826" 
  </plugin> 
  ? 
  <plugin rrdtool> 
        DataDir "/var/lib/collectd/rrd" 
  </Plugin>  

The most important part of the configuration item is the network plug-in, where IP is configured to refer to the IP that is installed INFLUXDB

COLLECTD at least input/output network, RRDtool plug-in

Kai-Stop COLLECTD

Systemctl Stop Collectd.service
Systemctl Start Collectd.service
Systemctl Enable Collectd.service Boot
Systemctl status Collectd.service Check plug-in loading status

Log

Open the log and configure properties such as log level and path

Loadplugin logfile
<plugin logfile>
  LogLevel Info
  File "/var/log/collectd.log"
</Plugin>

Restart COLLECTD, you can see the log in/var/log/collectd.log

RRD Directory

After starting COLLECTD, you can see the RRD directory in the/VAR/LIB/COLLECTD directory

[Email protected] ~]# cd/var/lib/collectd/rrd/node172
[email protected] node172]# LL
Total 0
Drwxr-xr-x 2 root root 209 June 09:14 cpu-0
Drwxr-xr-x 2 root root 209 June 09:14 Cpu-1
Drwxr-xr-x 2 root root 209 June 09:14 Cpu-2
Drwxr-xr-x 2 root root 209 June 09:14 cpu-3
Drwxr-xr-x 2 root root 209 June 09:14 Cpu-4
Drwxr-xr-x 2 root root 209 June 09:14 cpu-5
Drwxr-xr-x 2 root root 209 June 09:14 cpu-6
Drwxr-xr-x 2 root root 209 June 09:14 Cpu-7
Drwxr-xr-x 2 root root 124 June 09:48 disk-dm-0
Drwxr-xr-x 2 root root 94 June 09:14 Disk-dm-1
Drwxr-xr-x 2 root root 94 June 09:14 Disk-dm-2
Drwxr-xr-x 2 root root 94 June 09:14 Disk-sr0
Drwxr-xr-x 2 root root 147 June 09:48 Disk-xvda
Drwxr-xr-x 2 root root 94 June 09:14 disk-xvda1
Drwxr-xr-x 2 root root 147 June 09:48 Disk-xvda2
Drwxr-xr-x 2 root root the June 09:14 Interface-eth0
Drwxr-xr-x 2 root root for June 11:05 load
Drwxr-xr-x 2 root root 162 June 09:14 memory
Drwxr-xr-x 2 root root 195 June 09:14 processes
Drwxr-xr-x 2 root root for June 09:14 swap
drwxr-xr-x 2 root root 09:14 users             

3 Installing Influxdb

The high version of INFLUXDB does not have a Web page, the author installs the influxdb-0.9.4.2-1.x86_64 version

INFLUXDB Installation

Yum-y Install http://influxdb.s3.amazonaws.com/influxdb-0.9.4.2-1.x86_64.rpm

Configuration parameters

The configuration file path for INFLUXDB under the default installation path is/etc/opt/influxdb/influxdb.conf

Configuration file indicates Influxdb uses COLLECTD collector

Edit under [COLLECTD]:

[COLLECTD]
  enabled = True
  bind-address = "127.0.0.1:25826"
  database = "COLLECTD"
  typesdb = "/usr/share/collectd/types.db"

Kai-Stop Influxdb

Systemctl Start Influxdb.service
Systemctl Enable Influxdb.service
Systemctl Stop Influxdb.service

Test Influxdb

Check if the parameters collected by COLLECTD are received by influxdb

  [[email protected] ~]#/opt/influxdb/influx 
  Connected to http://localhost:8086 version 0.9.4.2 
  InfluxDB shell 0.9.4.2 
  > use COLLECTD 
  Using database collectd 
  > show Measurements 
  name:measurements 
 ---- --------------
  name 
  cpu_value 
  disk_io_time 
  disk_read 
  disk_value 
  disk_weighted_io_time 
  disk_write 
  interface_rx 
  interface_tx 
  load_ Longterm 
  load_midterm 
  load_shortterm 
  memory_value 
  processes_value 
  swap_value 
  users_value 
  ? 
  >                

Ports Used by Influxdb

• 8083:web Admin Management Service port, http://yourIP:8083

• Ports for the 8086:http API

• netstat -tlnpu |grep influxdView INFLUXDB Port UDP protocol monitoring

View Port 8083 in a browser to access the Influxdb front end, and use SQL-like commands to access the data

4 Installing Grafana

Grafana Installation

Grafana installation is not recommended to install the low version, the version is too low, the subsequent cannot monitor MySQL and so on

It is recommended to install the latest version of Grafana

Website reference

Grafana Start and stop

Systemctl Stop Grafana-server.service
Systemctl Start Grafana-server.service
Systemctl Enable Grafana-server.service

Grafana page

Web Access page:

http://yourIP:3000

Pre-fabricated user admin, password admin, user configurable

For the Grafana panel that has been set

Grafana Connection Database

Click Data Sources-add New, fill in the following parameters and save

Setting up the Grafana panel

Load Settings

Network I/O, you can only switch to switch editor mode and modify the left y-axis unit to BYTES/S, as shown below:

Incoming SQL script:

Select derivative ("value") as "value" from "Interface_rx" WHERE "host" = ' client174 ' and ' type ' = ' if_octets ' and "Instanc E "= ' eth0 '

When flowing, change the INTERFACE_RX to Interface_tx

Similar configurations, configurable Cpu,memory,swap, etc.

Collectd+influxdb+grafana Building Performance Monitoring Platform