Icinga2 not connecting to InfluxDB in openshift

I am working on deploying Icinga2 and InfluxDB on ARO (Azure Openshift). I am currently encountering issues where Icinga2 is not initializing a connection to InfluxDB.

About my setup:
Icinga2: v2.13.2
InfluxDB: V2.1.1

The Icinga2 logs show that it has an InfluxDB2Writer object, but the performace data is no getting written. (I do see the performance data on IcingaWeb). In the icinga logs I see the following:

[2022-04-18 20:29:32 +0000] information/WorkQueue: #6 (Influxdb2Writer, influxdb2) items: 0, rate: 0.2/s (12/min 63/5min 65/15min);

Things that work.

  • I can create a connection to InfluxDB using a cURL command from within the Icinga2 container.
  • I can post to InfluxDB using a deployment of Icinga2 running on Docker.

Things I have tried:

  • I’ve verified my influxdb2.conf is identical to that working in the docker deployment.
  • In the Influxdb2.conf file, I specifically set an invalid token. When I check the InfluxDB logs I do not see an associated “Unauthorized access” line, like I do when I perform the same test with cURL or with the docker Icinga2 deployment.
  • Verified selinux is disabled in the container. (I found mention of the icinga2_can_connect_all boolean for selinux and this seems like the exact issue, but selinux is disabled)
  • I’ve tried writing to a docker version of InfluxDB that I have been using for months, but it didn’t work.
  • Set a bogus host in influxdb.conf, and no errors are generated. I see errors if I do this in my Docker version of Icinga2.

Based on the facts above it seems like the issue is somewhere with Icinga2 itself within the openshift environment, or my configuration it. I’m not sure if somehow Openshift is running selinux underneath the container level and is causing issues or what. Any import is greatly appreciated.

Thanks in advance!

The issue was resolved by deleting and recreating both the icinga and icingaweb persistent volumes. I believe the issue was caused by a corrupted Icinga Director deployment which was causing the checks to not actually execute, but IcingaWeb was still showing that checks were running fine.