Notifications are disabled for service

Icinga 2
, ,
1

Hello everyone,

I’ve been troubleshooting an issue which I can’t seem to find a straight answer for. We received a report that some alerts failed to trigger a notification to support team. Upon closer inspection, I’ve found messages in /var/log/icinga2/icinga2.log with the legend: “Notifications are disabled for service ”

We run four zones, two are triggering notifications fine while the other two show “Notifications are disabled for service ”. In my troubleshooting I’ve found that satellites failing to send out notifications have file /var/lib/icinga2/modified-attributes.conf with the following content in it:

var obj = get_object("IcingaApplication", "app")
if (obj) {
	obj.modify_attribute("enable_notifications", false)
	obj.version = 1712328115.352699
}

We rarely work Icinga2 API so I can’t say how this modification landed on these zones/satellite. How can I re-enable these alerts for the whole cluster?

I’ve tried clearing up the file content but upon restart and connecting the Icinga2 master, the issue comes back. Any clues on this issue would help a lot.

For reasons, we run Icinga2 r2.6.3.3-1 and can’t upgrade to a newer release.

Icinga2 --version
icinga2 - The Icinga 2 network monitoring daemon (version: r2.6.3.3-1)

Copyright (c) 2012-2017 Icinga Development Team (https://www.icinga.com/)
License GPLv2+: GNU GPL version 2 or later http://gnu.org/licenses/gpl2.html
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Application information:
Installation root: /usr
Sysconf directory: /etc
Run directory: /run
Local state directory: /var
Package data directory: /usr/share/icinga2
State path: /var/lib/icinga2/icinga2.state
Modified attributes path: /var/lib/icinga2/modified-attributes.conf
Objects path: /var/cache/icinga2/icinga2.debug
Vars path: /var/cache/icinga2/icinga2.vars
PID path: /run/icinga2/icinga2.pid

System information:
Platform: CentOS Linux
Platform version: 7 (Core)
Kernel: Linux
Kernel version: 3.10.0-1062.el7.x86_64
Architecture: x86_64

Build information:
Compiler: GNU 4.8.5

2

Found this post How to stop icinga2 notifications via API - #7 by Tqnsls which mentions how to disable notifications. Did the opposite and now see

var obj = get_object(“IcingaApplication”, “app”)
if (obj) {
obj.modify_attribute(“enable_notifications”, true)
obj.version = 1712328115.352699
}

3

What kind of update should I send to the API to clear out this file and have an even environment? As mentioned, two zones have this file modified-attributes.conf empty while these other have

var obj = get_object(“IcingaApplication”, “app”)
if (obj) {
    obj.modify_attribute(“enable_notifications”, true)
    obj.version = 1712328115.352699
}