Problem with non-executing checks

Icinga 2
1

  • Version used 2.9

  • Operating System Centos 7 x64

  • Enabled features
    Enabled features: api checker command compatlog debuglog ido-mysql livestatus mainlog notification perfdata statusdata

  • Icinga Web 2 version 2.6.1

Hello
We have a serious problem with non-executing checks on an Icinga 2.9 instance at a customer. It happened that the system time was inadvertently set to a future year (!) for a short time, which means that almost all Icinga services got a “last check” time stamp in the future. This has now led to a permanent state of non-execution of all affected services. How can we enforce checking to resume? (Setting “next check” to now, anywhere, or restoring the “icinga” database didn’t help!)

grafik

Thx

2

I’d try:

systemctl stop icinga2
rm /var/lib/icinga2/icinga2.state
systemctl start icinga2
3 Likes
3

If this doesn’t work, I would imagine it’s reading for the next check from the state file or DB – you might need a way to clear out the last executed check timestamp from one or both of these locations (as well as the history for Icingaweb2 – those probably show future dates too I would imagine).

Off hand, I’m not sure if that information exists in the state file, database, or both, but iirc when Icinga2 restarts, it drops a lot of the db tables, and then populates them from the state file on start, so I would check there first. Also not sure if the db writes back to the state file, so maybe the DB is the place to start :sweat_smile:

Worst case scenario, if nothing works, set up a new master, move the object to the new master, reconnect any agents/zones to the new master and see if that helps – but again I would only treat that as a last resort.

4

Hello Roland,

Supper.
Has solved problem

Thx