Notifications to multiple users inlcuding opsgenie

Icinga 2
1

Hey,

I’m using Icinga r2.13.7-1 with Director and I’m struggling with notifications configuration to be sent to multiple users.
First user is called Desks, and second one is the opsgenie user for the opsgenie integration.
If there are both assigned, notifications are usually sent to Desks. If there is only opsgenie user, it works as well and opsgenie is receiving notification.

Let’s say I have a disk_usage check and host chd-dev-test001

This is what I see in the logs:

[2024-07-26 09:12:03 -0500] information/Checkable: Checkable 'chd-dev-test001!disk_usage' has 3 notification(s). Checking filters for type 'Problem', sends will be logged.
[2024-07-26 09:12:03 -0500] information/Notification: Sending 'Problem' notification 'chd-dev-test001!disk_usage!everyone-service-apply-warning-to-critical' for user 'Desks'
[2024-07-26 09:12:03 -0500] information/Notification: Completed sending 'Problem' notification 'chd-dev-test001!disk_usage!everyone-service-apply-warning-to-critical' for checkable 'chd-dev-test001!disk_usage' and user 'Desks' using command 'mail-service-notification'.

Output from icinga2 object list --type "Notification" | grep chd-dev-test001

Object 'chd-dev-test001!disk_usage!everyone-service-apply' of type 'Notification':
  * __name = "chd-dev-test001!disk_usage!everyone-service-apply"
  * host_name = "chd-dev-test001"

Object 'chd-dev-test001!disk_usage!notify-opsgenie-service' of type 'Notification':
  * __name = "chd-dev-test001!disk_usage!notify-opsgenie-service"
  * host_name = "chd-dev-test001"

Users definitions:

object User "opsgenie" {
    import "generic-user"
    display_name = "OpsGenie Contact"
    enable_notifications = true
}

object User "Desks" {
    import "user-template"
    enable_notifications = true
    states = [ Critical, Down ]
}

Notifications definitions:

apply Notification "everyone-service-apply" to Service {
    command = "mail-service-notification"
    assign where host.vars.notification_useremail && (service.name == "cpu_throttling" || match("disk_usage*", service.name) || match("memory*", service.name) || service.name == "ssh" || match("mysql_slave*", service.name))
    states = [ Critical, OK ]
    types = [ Custom, Problem, Recovery ]
    users = [ "Desks" ]
    vars.notification_from = "support@example.com"
}

apply Notification "notify-opsgenie-service" to Service {
    command = "opsgenie-service-notification"
    assign where service
    states = [ Critical, OK, Warning ]
    types = [ Problem, Recovery ]
    users = [ "opsgenie" ]
}

If anyone has an idea what is wrong with this approach, please let me know, I run out of ideas.

This is what I see in the GUI for this specific check, not sure where I can find this in the config
image

2

As you have defined specific state for the “Desks” user and not for the “opsgenie” user, I’m guessing this user is missing those, as they are also not set in the user template. Thus it is not being notified because Icinga does not know for what state it should send notifications.

Either define states (and transition types) in the user object or the imported user template generic-user.

For clarification:
You need to define the states & transition types that the user should get notifications for and you need to define for what states & transition types a notification should be triggered.
This way you could have one apply rule with

states = [ Critical, OK, Warning ]
types = [ Custom, Problem, Recovery ]

where both users are mentioned and only send the desired notifications based on the states & transition types based on the users settings.

Hint: With this user config and the notification apply rules you share the “Desks” user would never receive the OK recovery messages, because you limited the states to Critical (for services) and Down (for hosts)

3

Thank you for your answer.

Users definitions were fine, as everything else as well. The issue was in the HA configuration. I’m using two masters. Somehow on the second one notifications feature was enabled, but not on the first one. No idea why, as it was working until some point. Email notifications were sent via second master, opsgenie notifications via first master.

Anyway, after enabling notifications feature on the first master, notifications are going to opsgenie as well.

The opsgenie user doesn’t need to have defined states or types. It just takes everything.

Thanks!

1 Like
4

Just as a follow-up, for a different host this could have been the other way round, as the object authority is object-dependent.
https://icinga.com/docs/icinga-2/latest/doc/19-technical-concepts/#high-availability-object-authority