I’ve been experiencing occasional, random occurrences where icinga2 greatly reduces the number of of checks until the UI complains that icinga2 is not running. It is running, it’s just not doing anything.
It happens often enough that I need to implement some kind of watchdog to check on this state, and of course it can’t run from icinga2’s scheduler. My first thought is to use whatever mechanism the UI uses, but I am open to ideas.
So my two questions:
- What exactly is the UI looking at (something in the mariadb database, apparently), so I could look at it, too?
- Any other ideas for a watchdog trigger?
The solution I implement would probably run often from systemd, and have the authority to restart icinga2.