Configure notifications in service sets for each service/host individually

kiko · June 25, 2024, 8:51am

Give as much information as you can, e.g.

Director version (System - About): 1.9.1
Icinga Web 2 version and modules (System - About): 2.9.5
Icinga 2 version (icinga2 --version): 2.13.2-1
Operating System and version: Oracle Linux 8
Webserver, PHP versions . PHP 7.4.33 and apache/2.4.37

Dear all,

I´m currently migrating from icinga1 to icinga2 a infrastructure of +200 hosts, many of them have the same services, at least the common performance checks for each OS.

I´m have a stopped now and I hope you can help me.

We decided to use serviceset´s for service grouping and we finished this part, all services had been imported on this way.

Now we are configuring notifications and I need to know the properly way for assigning different contacts for each host that which uses this service set.

I use host/service.groups to apply each notification apply rule and I have a notification apply rule for each “department” that I want to notify. The problem is that I need to use different contacts for each host or service, If I configure a servicegroup to notify 24x7shifts and I apply this to the service in the serviceset, It´s automatically applied for each host that uses this serviceset, but I need to remove it from some hosts that should not notify to 24x7, In case of these hosts I would like to configure another servicegroup, but as I´m using service sets is not possible for me or I don´t know how to do it

I hope that you understand my case

If you need more information about this or I should explain it in another way, please just ask for it

log1c · June 25, 2024, 12:00pm

hello
I think with some examples your setup would be easier to understand.

What I understood from your description what you want to achieve:

Host A → Contact 1
– Service X → Contact 3
– Service Y → Contact 5
Host B → Contact 2
– Service X → Contact 4
– Service Y → Contact 6

kiko · June 25, 2024, 12:56pm

Hi, thank you for the celerity

Of course, for example:

serviceset.windows.servers → when host.group = group.windows.servers

Notification apply rule1 → when service.group = servicegroup.windows.department (it will notify to windows department)

Notification apply rule2 → when service.group = servicegroup.24x7 (it will notify to 24x7)

What do I need?:

server1 → group.windows.servers (using serviceset.windows.servers)
services:
disk_C → servicegroup.notification.windows.department (It will notify to windows department)
disk_D → servicegroup.notification.windows.department (It will notify to windows department)

server2 → group.windows.servers (using serviceset.windows.servers)
services:
disk_C → servicegroup.notification.24x7 (It will notify to 24x7)
disk_D → servicegroup.notification.24x7 (It will notify to 24x7)

And this is what I have right now(due to in icinga director I´m able to only modify the service in the serviceset and not the single service in each service of different hosts):

server1 → group.windows.servers (using serviceset.windows.servers)
services:
disk_C → servicegroup.notification.windows.department (It will notify to windows
department)
servicegroup.notification.24x7 (It will notify to 24x7) (Added in serviceset, I want
to remove it, but icinga director does not let me modify a single service, only this
service in the serviceset)
disk_D → servicegroup.notification.windows.department (It will notify to windows
department)
servicegroup.notification.24x7 (It will notify to windows department) (Added in
serviceset, I want to remove it, but icinga director does not let me modify a
single service, only this service in the serviceset)

server2 → group.windows.servers (using serviceset.windows.servers)
services:
disk_C → servicegroup.notification.windows.department (It will notify to windows
department) (Added in serviceset, I want to remove it, but icinga director does
not let me modify a single service, only this service in the serviceset)
servicegroup.notification.24x7 (It will notify to 24x7)

disk_D → servicegroup.notification.windows.department (It will notify to windows
department) (Added in serviceset, I want to remove it, but icinga director does
not let me modify a single service, only this service in the serviceset)
servicegroup.notification.24x7 (It will notify to windows department)

This scenary means that there are many hosts that have added both servicegroups directly in the serviceset, and all windows servers will notify to 24x7 and my windows department, when I only want to choose one in each service of different hosts

If I try to edit a single service of a host:

I hope it helps, thank you in advance

rivad · June 25, 2024, 3:55pm

I use the setup the Linuxfabrik recommended to me.
Every server and optionally the services can have 2 variables (vars.teams and vars.criticality). If the variable isn’t set on the service, the one from the host is used.

Teams maps to the contact groups and criticality to the notification timeperiods.

Both of this variables can be overwritten on services that come from a service set via the regular director overwrite mechanism.

github.com

Linuxfabrik/monitoring-plugins/blob/main/ICINGA.rst#criticality

Icinga Director
===============

Installing the Plugins
----------------------

Single / specific Plugins
~~~~~~~~~~~~~~~~~~~~~~~~~

By importing Baskets
    For each check, we provide an Icinga Director Basket that contains at least the Command definition and a matching Service Template (for example, ``check-plugins/cpu-usage/icingaweb2-module-director/cpu-usage.json``). Import this:

    * either via the WebGUI using Icinga Director > Configuration Baskets > Upload, select the latest entry in the Snapshots tab and restore it
    * or via ``icingacli director basket restore < cpu-usage.json -v``

    Now use the new commands within Service Templates, Service Sets and/or a Single Services.

By defining them manually
    If you want to define plugins manually, this is how to do it (example).

This file has been truncated. show original

/etc/icinga2/zones.d/master/notifications.conf:

apply Notification "notification-host-mail" to Host {
    import "116-tpl-notification-host-mail"

    assign where host.vars.teams
    user_groups = host.vars.teams
    vars.notify_mail_icingaweb2_url = "https://icinga.example.com/"
    vars.notify_mail_recipient = "$user.email$"
    vars.notify_mail_notes_url = "$service.notes_url$"
}

apply Notification "notification-service-mail" to Service {
    import "116-tpl-notification-service-mail"

    assign where service.vars.teams
    user_groups = service.vars.teams
    vars.notify_mail_icingaweb2_url = "https://icinga.example.com/"
    vars.notify_mail_port = "25"
    vars.notify_mail_recipient = "$user.email$"
    vars.notify_mail_server = "localhost"
    vars.notify_mail_notes_url = "$service.notes_url$"
}

apply Notification "notification-service-mail" to Service {
    import "116-tpl-notification-service-mail"
    assign where host.vars.teams && ! service.vars.teams
    user_groups = host.vars.teams
    vars.notify_mail_icingaweb2_url = "https://icinga.example.com/"
    vars.notify_mail_port = "25"
    vars.notify_mail_recipient = "$user.email$"
    vars.notify_mail_server = "localhost"
    vars.notify_mail_notes_url = "$service.notes_url$"

log1c · June 26, 2024, 7:00am

Exactly what I would have suggested!

Yes, and below that you should see all the fields of that service where you can overwrite them with new information and save that only for this one services belonging the this one host.
If you don’t see any fields there (like in my screenshot below) you need to add them first to your service template (or the check command).

Example from our setup for the cpu check for one specific server:

kiko · July 2, 2024, 8:29am

Morning,

It´s just what I need.
Thank you so much!

kiko · July 2, 2024, 8:49am

Hi,

Thanks for the information, I will think about it!