Build a modern monitoring system using Grafana, collectd, and InfluxDB

Build a modern monitoring system using Grafana, collectd, and InfluxDB

To create a beautiful real-time monitoring system like New Relic, we only need the three InfluxDB, collectd, and Grafana tools. The relationship between these three tools is as follows:

Collect data-> store data (InfluxDB)-> display data (Grafana ).

• InfluxDB is an open-source Distributed Time Series database developed by the Go language. It is ideal for storing metrics, events, analysis, and other data. You can see that this project is still very young when you look at the version number (v0.8.8;
• Collectd is a system performance collection tool written in C language;
• Grafana is a front-end tool developed by pure Javascript. It is used to access InfluxDB, customize reports, and display charts.

The following installation and configuration steps are completed on Ubuntu 14.04 Server 64bit. Restart the entire system after upgrading:

1. $ sudo apt-get update
2. $ sudo apt-get upgrade
3. $ sudo reboot

 

Install InfluxDB

InfluxDB is written in Go and is clean without any other packages or libraries. Easy to install:

1. $ wget https://s3.amazonaws.com/influxdb/influxdb_latest_amd64.deb
2. $ sudo dpkg -i influxdb_latest_amd64.deb

Start InfluxDB:

1. $ sudo /etc/init.d/influxdb start
2. Setting ulimit -n 65536
3. Starting the process influxdb [ OK ]
4. influxdb process was started [ OK ]

Open the web Management Interface http: // 192.168.2.183: 8083/the default username and password are root and root. the Web management interface port of InfluxDB is 8083, And the http api listening port is 8086. to change these default settings, modify the InfluxDB configuration file/opt/influxdb/current/config. after toml, restart InfluxDB.

 

Create a database named collectd on the newly installed InfluxDB, which can be created using the command line or the Web management interface:

1. $ curl "http://192.168.2.183:8086/db?u=root&p=root"-d "{\"name\": \"collectd\"}"

 

Install collectd

Install collectd:

1. $ sudo apt-get install collectd

Configure collectd as the client and send the collected data directly to InfluxDB:

1. $ sudo vi /etc/collectd/collectd.conf
2. ...
3. LoadPlugin network
4. ...
5. <Plugin network>
6. Server"192.168.2.183""25826"
7. </Plugin>
8. ...

Restart collectd:

1. $ sudo /etc/init.d/collectd restart

InfluxDB now comes with a collectd plug-in to obtain the data sent from the collectd client, which is not so convenient in the past, before Version 0.8.4, you can only use a third-party program such as influxdb-collectd-proxy to connect collectd and InfluxDB. if you check the port opened on the server, you will find that the influxdb plug-in has started port 25826. If the InfluxDB database does not have (collected) data, check whether port 25826 is enabled properly:

1. $ sudo netstat -tupln
2. ActiveInternet connections (only servers)
3. ProtoRecv-Q Send-Q LocalAddressForeignAddressState PID/Program name
4. tcp 000.0.0.0:220.0.0.0:* LISTEN 622/sshd
5. tcp6 00:::8086:::* LISTEN 668/influxdb
6. tcp6 00:::22:::* LISTEN 622/sshd
7. tcp6 00:::8090:::* LISTEN 668/influxdb
8. tcp6 00:::8099:::* LISTEN 668/influxdb
9. tcp6 00:::8083:::* LISTEN 668/influxdb
10. udp6 00:::25826:::*668/influxdb

The built-in collectd plug-in of InfluxDB is disabled by default. You need to manually enabled = true and fill in the row "database =" collectd, the "collectd" here is the database we created above. After changing the configuration, remember to restart InfluxDB:

1. $ sudo vi /opt/influxdb/current/config.toml
2. $ sudo vi /opt/influxdb/shared/config.toml
3. ...
4. # Configure the collectd api
5. [input_plugins.collectd]
6. enabled =true
7. # address = "0.0.0.0" # If not set, is actually set to bind-address.
8. # port = 25826
9. database ="collectd"
10. # types.db can be found in a collectd installation or on github:
11. # https://github.com/collectd/collectd/blob/master/src/types.db
12. # typesdb = "/usr/share/collectd/types.db" # The path to the collectd types.db file
13. ...
14. $ sudo /etc/init.d/influxdb restart
15. Setting ulimit -n 65536
16. Setting ulimit -n 65536
17. influxdb process was stopped [ OK ]
18. Setting ulimit -n 65536
19. Starting the process influxdb [ OK ]
20. influxdb process was started [ OK ]

Now, InfluxDB is ready to accept and process data from collectd. Use the command line or Web management interface to verify whether data exists in the database:

1. $ curl -G 'http://192.168.2.183:8086/db/collectd/series?u=root&p=root&q=list+series&pretty=true'
2. [
3. {
4. "name":"list_series_result",
5. "columns":[
6. "time",
7. "name"
8. ],
9. "points":[
10. [
11. 0,
12. "192.168.2.183/cpu-0/cpu-idle"
13. ],
14. ...
15. ]
16. }
17. ]

 

Install Grafana

Download grafana and decompress it on the web server. This saves the trouble of configuring Nginx/Apache, and directly uses the simplest Web Server python-m SimpleHTTPServer DRIVER:

1. $ wget http://grafanarel.s3.amazonaws.com/grafana-1.9.1.tar.gz
2. $ tar xzvf grafana-1.9.1.tar.gz
3. $ cd grafana-1.9.1.tar.gz
4. $ cp config.sample.js config.js
5. $ vi config.js
6. ...
7. // InfluxDB example setup (the InfluxDB databases specified need to exist)
8. datasources:{
9. influxdb:{
10. type:'influxdb',
11. url:"http://192.168.2.183:8086/db/collectd",
12. username:'root',
13. password:'root',
14. },
15. ...
16. },
17. ...
18. $ sudo python -m SimpleHTTPServer

Access Grafana through a browser. The default port here is 8000:

This article permanently updates the link address: