Icinga is not sending any metricts to InfluxDB

Icinga Web Modules Visualization, Metrics & Reporting
1

Hi all.

I have a running installation of Icinga2. It monitors different parameters in different servers without any issue.

My next step was to store all that readings into an InfluxDB database for further analysis and dashboard setup for our Support team.

I have installed InfluxDB in the same server as Icinga2 and IcingaWeb2.

InfluxDB is reachable via influx shell command.

In InfluxDB I have created a database named icinga2.

I’m able to inject fake measurement into influx using

curl -i -XPOST "http://localhost:8086/write?db=icinga2" --data-binary 'weather,location=us-midwest temperature=82 1465839830100400200'

On Icinga I have enabled the influxdb feature and restarted the service.

This is the contents of /etc/icinga2/features-enables/influxdb.conf

object InfluxdbWriter "influxdb" {
  host = "127.0.0.1"
  port = 8086
  database = "icinga2"
  host_template = {
    measurement = "$host.check_command$"
    tags = {
      hostname = "$host.name$"
    }
  }
  service_template = {
    measurement = "$service.check_command$"
    tags = {
      hostname = "$host.name$"
      service = "$service.name$"
    }
  }
}

Everything looks OK for me, but Icinga is not writing any data into InfluxDB

What I have forgotten?

Thanks in advance!!

Javier Vilarroig

2

I don’t see username and password in your influxdb.conf, not sure if this is working. You can check icinga.log for any errors.

3

Why would one need a username and password for Icinga, when the curl command
works fine without?

Antony.

4

Good point. Have overlooked this.

5

Hi,

which version of Icinga 2 and InfluxDB you are using? Are there any entries in the Icinga 2 log file about errors regarding the InfluxDB connection.

Best regards
Michael

6

Just one thought: InfluxDB Version 2 was released and I am sure, this is not implemented in Icinga2 yet. You have to use the latest Influx 1 Version.

1 Like
7

Hi.

Icinga2 is version 2.12.1
InfluxDB is version 1.0.2

In /var/log/icinga2/icinga2.log I’m getting every minute a line similar to

information/WorkQueue: #7 (InfluxdbWriter, influxdb) items: 0, rate: 1/s (60/min 298/5min 897/15min);

No errors. But nothing gets written in InfluxDB.

Using influx CLI I have checked icinga2 database and also _internal database but nothing is there.

error.log is empty.

I have activated debug in icinga2. The only mention to influxdb is in inserts into IDO where there ara attributes called influxdb.

Thanks!
Javier

8

Do you see any perfdata in Icingaweb2?

BTW: Your InfluxDB is pretty old, we ware using 1.8.

9

Yes, I have perfdata shown in Icingawed.

About the InfluxDB version. We are still in Debian 9 (old old stable), planning to migrate to Debian 10 in January. If this can be the issue I can take a look at backports to see if there is a more modern version.

In general my management is very cautious about upgrading :wink:

Thanks!

10

Why not getting it directly from https://repos.influxdata.com/?

Here a hint points to SELinux as possible root cause.

11

Interesting reading. I will go through it and share what I find.

Thanks!

12

Do you know if this is still the case, that influx 2.0 is not yet supported? I am running Icinga r2.12.3-1 and influx2.0 in a new build and am not seeing data being written.

13

I think so. I haven’t heard any different yet. But maybe @theFeu can give more detailed information here. I also don’t know, if grafana is already working with InfluxDB2

14

Hey there,

I went and asked one of our developers and it seems like we didn’t build a special implementation for InfluxDB 2.0
So if it doesn’t work out of the box with our InfluxDB writer, then I am afraid we don’t support it at the time of writing :slight_smile:

1 Like
15

As Icinga 2 uses the standard protocol of InfluxDB it should be no problem to use it with 2.0. Only if one of the other, now not directly supported protocols would be used, then a telegraf instance would be needed for translation. So if something does not work, it would still be worth investigating as it could be a bug in the protocol implementation or similar.

16

Unfortunately in German but a colleague wrote a quite note on how to connect Icinga 2 to InfluxDB 2: https://www.netways.de/blog/2021/01/28/icinga2-und-influx2-so-bringen-wir-beide-zum-reden/

Commands are included to create a version 1 compatible bucket and this should be enough. If an English version is required, please tell me and I will try my best to get one published.

17

Has taken some time. Busy issues somewhere else. Sorry for the delay.

I have moved to InfluxDB OSS 2.0, changed Icinga configuration Influxdb2Writer, and seems that I’m getting exactly the same result.

When querying InfluxDB I can’s see any data point and in incinga2.log I’m getting that lines related to InfluxDB

/var/log/icinga2/icinga2.log:[2021-09-03 10:27:58 +0000] information/WorkQueue: #5 (Influxdb2Writer, influxdb2) items: 0, rate: 1.43333/s (86/min 417/5min 849/15min);
/var/log/icinga2/icinga2.log:[2021-09-03 10:32:58 +0000] information/WorkQueue: #5 (Influxdb2Writer, influxdb2) items: 0, rate: 1.36667/s (82/min 416/5min 1254/15min);
/var/log/icinga2/icinga2.log:[2021-09-03 10:37:58 +0000] information/WorkQueue: #5 (Influxdb2Writer, influxdb2) items: 0, rate: 1.4/s (84/min 416/5min 1249/15min);
/var/log/icinga2/icinga2.log:[2021-09-03 10:42:58 +0000] information/WorkQueue: #5 (Influxdb2Writer, influxdb2) items: 0, rate: 1.38333/s (83/min 415/5min 1247/15min);
/var/log/icinga2/icinga2.log:[2021-09-03 10:47:58 +0000] information/WorkQueue: #5 (Influxdb2Writer, influxdb2) items: 0, rate: 1.36667/s (82/min 414/5min 1245/15min);

I’m having that line every 5 minutes. As far as I can understand them, the communication with InfluxDB is OK, but no items are written. :frowning:

18

It looks like you don’t have any checks defined that return perfdata.

1 Like
19

I will check that.
Thanks

20

I have checked in Icingaweb and there I can see Performance Data there.

So it seems that the checks are returning it.