I have a situation where we have a long-period check_interval (1 hour) for an expensive (compute-wise) check. Once it goes Hard, checks are lightweight in nature until it is resolved.
Unfortunately, with the 1h check interval, the situation might be automatically or manually resolved in (for example) 5 minutes, but it will be another 55m until Icinga notices. (Unless someone hits Check now).
Ideally I would like to change the check_interval from 60m to 5m until the state goes to OK again. Then, go back to 60m.
Hello, this isn’t a simple task, but there are few ideas :
_ update the check_interval of the service in your check script by sending the appropriate request to the rest api, i’m not sure however icinga can get the modification without needing a restart, this is to be tested. https://icinga.com/docs/icinga2/latest/doc/12-icinga2-api/#modifying-objects
_ use a custom function to set the check interval when stats goes hard down. I have done several tests for this one, and so far i didn’t found a way to have a function executed at runtime.
Regarding your needs, i think it would be easier for you to use a passive check if possible.