Since Icinga 2.11, notification suppressed during downtime are sent when the downtime ends. I think it’s that feature that resulted in:
- It’s 2:00 am.
- plan downtime for host and services till 04:00 am.
- Do work
- See that related services are also going into error. Planned downtime. This may or may not be before/after a notification is sent out.
- Work is done.
- Dashboard is green
- It’s 03:00 am.
- Woken up with a phone text message at 04:00 saying ‘recovered’.
Looking at FireSuppressedNotifications(), I can’t quite match this with the documentation “Once the downtime ends and the problem state is still intact , Icinga checks whether a re-notification should be sent immediately”. The code seems to collect recovery messages for later sending, but that’s not a ‘problem state’, right?
(also, I don’t quite see how the compiler won’t complain about uninitialized booleans there)
I wanted to mention it here instead of Github right away, because it’s likely I misunderstood something, and the feature already had a somewhat ‘rich’ history. Fact is however, that we did get those recovery messages after downtime ended, waking me up for now reason.