Influxdb2Writer - dynamic service parsing issue

Icinga 2
1

Hi All!

We started using the Influxdb2Writer function in Icinga2 v2.13 and found that due to a parsing error, influxdbWriter cannot process dynamic service names. The icinga2.log shows the following entries only:

[2022-01-10 13:45:30 +0100] warning/Influxdb2Writer: Unexpected Content-Type: application/json; charset=utf-8
...
[2022-01-10 13:45:30 +0100] warning/Influxdb2Writer: Unexpected response code: Bad Request

Analyzing with Wireshark, we found that in case of dynamic service, parsing errors occure. I.e. in case of inner service name ‘[DMZ] Apache vhost: { host = “my-vhost-name.local” }’ the Wireshark shows this:

check_http,hostname=my-host.local,service=[DMZ]\ Apache\ vhost:\ {
                host\ \=\ \"my-vhost-name.local\"
},metric=time unit="seconds",value=0.137194 1642064342
check_http,hostname=my-host.local,service=[DMZ]\ Apache\ vhost:\ {
                host\ \=\ \"my-vhost-name.local\"
},metric=size unit="bytes",value=19807 1642064342
...

Content-Type: application/json; charset=utf-8
X-Platform-Error-Code: invalid
Date: Thu, 13 Jan 2022 08:57:05 GMT
Content-Length: 419

{"code":"invalid","message":"unable to parse 'check_http,hostname=my-host.local,service=[DMZ]\\ Apache\\ vhost:\\ {': missing fields\nunable to parse 'host\\ \\=\\ \\\"my-vhost-name.local\\\"': missing fields\nunable to parse 'check_http,hostname=my-host.local,service=[DMZ]\\ Apache\\ vhost:\\ {': missing fields\nunable to parse 'host\\ \\=\\ \\\"my-vhost-name.local\\\"': missing fields"}

The very last line of the above output shows the parsing error message itself:

{"code":"invalid","message":"unable to parse 'check_http,hostname=my-host.local,service=[DMZ]\\ Apache\\ vhost:\\ {': missing fields\nunable to parse 'host\\ \\=\\ \\\"my-vhost-name.local\\\"': missing fields\nunable to parse 'check_http,hostname=my-host.local,service=[DMZ]\\ Apache\\ vhost:\\ {': missing fields\nunable to parse 'host\\ \\=\\ \\\"my-vhost-name.local\\\"': missing fields"}

The related host and service definitions:

object Host "my-host.local" {
  ...

  vars.apache.vhosts = [
    ...
    { 
      host = "my-vhost-name.local"
    },
    ...
  ]
}

apply Service "[DMZ] Apache vhost: " for (vhost in host.vars.apache.vhosts) {
  ...
  check_command = "check_http"
  display_name = "[DMZ] Apache vhost: " + vhost.host
  ...
  assign where ...
}

Has anyone experienced this sort of issue?
Please help.

Regards,
Endre

2

No one has experience with Influxdb2Writer?

3

Hi @bking!

As I see you prefer display_name to be [DMZ] Apache vhost: my-vhost-name.local. Have you tried to make your service name identical?

object Host "my-host.local" {
  ...

  vars.apache.vhosts = [
    ...
    "my-vhost-name.local",
    ...
  ]
}

apply Service "[DMZ] Apache vhost: " for (vhost in host.vars.apache.vhosts) {
  ...
  check_command = "check_http"
  display_name = "[DMZ] Apache vhost: " + vhost // actually obsolete
  ...
  assign where ...
}
4

Hi Klimov,

Thanks for your reply.

One question: What do you mean exactly that the display_name definition is obsolete? The service name and the display name shouldn’t be different?

Regards,
Endre

5

I just mean: the line doesn’t matter in this case.

6

Finally we solved it :slight_smile:

It was simple: we used $service.display_name" in the Influxdb2Writer service template definition instead of $service.name$:

  service_template = {
    measurement = "$service.check_command$"
    tags = {
      hostname = "$host.name$"
      service = "$service.display_name$"  
    }
  }