Do not send notifications on new hosts in first hour

Icinga 2
1

Hi,
is there a way how to avoid sending notifications on hosts and their services in the first hour these objects are created?

We use custom script which calls Icinga API to create host objects. The services are assign based on host vars.

We create and object and then we create a downtime on the host and all services. Unfortunately users sometimes receive notification about the problem, even that the downtime is set.

We have two Icinga2 masters in a cluster and few satellites. Only Icinga2 master instances are allowed to send notifications.

The weird thing we can’t see all retries in the histroy. And the notification was sent few seconds after first occurency of the error. See attached picture.

What we would prefer, would be to automatically disable notifications for the host and all its services íf the host exists less than 1 hour. Is it possible?

Screenshot 2022-09-26 at 10.20.20
Screenshot 2022-09-26 at 10.20.201298×1086 123 KB

The check on the picture is active check. We use also passive checks.

icinga2 - The Icinga 2 network monitoring daemon (version: 2.13.2-1)

System information:
Platform: CentOS Stream
Platform version: 8
Kernel: Linux
Kernel version: 5.18.6-2.gdc.el8.x86_64
Architecture: x86_64

Build information:
Compiler: GNU 8.4.1
Build host: runner-hh8q3bz2-project-322-concurrent-0
OpenSSL version: OpenSSL 1.1.1k FIPS 25 Mar 2021

Disabled features: compatlog debuglog elasticsearch gelf graphite icingadb influxdb influxdb2 livestatus opentsdb perfdata statusdata syslog
Enabled features: api checker command ido-mysql mainlog notification

2

you could change you notification apply rule and add some datetime-arithmetic:

https://icinga.com/docs/icinga-2/latest/doc/18-library-reference/#datetime-arithmetic

maybe you have to save the timestamp in a custom var if you cant access it via dsl but:
something like that should do the trick: (DateTime(created-unixTimestamp) - DateTime() ) >= (60 * 60)

1 Like
3

This is interesting idea, unfortunately the apply rule is used only when the object is created or icinga restarted. This will not change the notification when 1 hour will pass. But, we can use very similar approach directly in notification command.
Thank you for the idea.

1 Like
4

you could change your notification script and append the host creation date as parameter => this way your notification script can decide to send the notification or not