So I’m not sure if I’m following you here. Do I need to make a new notification that has “Service” in the “Apply to” field? And then use the ‘service-by-mail’ script as a command in this one? Basically, it’s not enough to know that a host is ‘CRITICAL’ I also need to have a notification for the specific service that caused this state?
Icinga separates host and service notifications. Some users prefer only host notifications, others want them for all services, others want to just create service notifications for a specific type of services, others again apply notifications only to production host groups.
So, generally speaking you’ll always have two or more apply rules for notifications, one for each type: host and service. More on this here.
Cheers,
Michael
Could you point me towards an example configuration or something? I went through the documentation that you linked but I’m still unsure how this should work… all I want is the notification to fire if a host cannot be pinged. Based on what you said my apply rule should say 'assign where host.address is set (that I can do) AND the ping4 service is set?
Hi,
specify what if a host cannot be pinged. means. Would you use the host check, typically hostalive being a ping with more gentle thresholds, or are you planning to add the ping service and others for notifications in the future?
Likewise, you will configure services for viewing them in the web interface, like disk, http or anything else and want notifications for them in case they break, right?
Cheers,
Michael
Host cannot be pinged means that the ping4 service says “CRITICAL host unreachable”. If that happens I want a notification.
Yes in the future I plan to monitor more stuff, like disk space for example but I think I can figure the rest out if I can get this to work at least.
I guess that means I want a service type notification not a host type?
Yes, service type notification is what you need.
I would suggest to configure the service notification that sends(or better creates) the notifications for the types/states you want to be notified for. PLUS the notification rule should contain the type/state OK/Recovery, because this is needed to reset the state of a warning/critical service, that the it will be considered for notifications again.
Then you can “filter” what notifications are sent out on the user level.
Examples
template Notification "generic-service-notification" {
command = "mail-service-notification"
interval = 0s
period = "24x7"
states = [ Critical, OK, Warning ]
types = [ Custom, Problem, Recovery ]
}
object User "Autoticket" {
email = "AutoTicket-Intern"
enable_notifications = true
period = "24x7"
states = [ Critical, Down, Warning ]
types = [ Problem ]
}
Thanks for answering!
Now I changed my approach and made a service type notification that looks like this:
apply Notification "service-notification" to Service {
command = "service-by-mail"
period = "always"
zone = "myserver.local"
assign where service.enable_notifications
states = [ Critical, OK, Unknown, Warning ]
types = [ Acknowledgement, Problem, Recovery ]
users = [ "myuser" ]
}
So I have a notification that should fire on every service that has notifications enabled (they all do) when the states are as what is listed? OR, should I have the apply rule state specifically that I want the notification if any of these services are in CRITICAL for example?
That’s an important detail. If you would stick with hosts only, the above config works just fine. Moving along to services, this requires additional notification rules, as noted above.
Inside the Director, you can create such notification rules already. The key thing is to
- Create user templates and objects beforehand
- Create a notification command for the host and one for the service if not existing
- Create two notification templates which hide all the basic information already, or have users assigned
Then you can go and add an apply rule similar to the host, just change the apply to dropdown.
Full example looks like this:
Save, store, deploy.
This is my first time doing this inside the Director (normally I write DSL a lot), and it feels very easy.
Cheers,
Michael
1 Like
I have the user template, the user with notifications enabled.
I have a the notification commands for both service and host. They both work if I send a custom notification from the webui.
I have the two notification templates.
At this point for simplicitys sake I deleted my host notificaition and try to get the service notification to work at least.
This is my service notification right now. It doesn’t work:
zones.d/myserver.local/notification_apply.conf
apply Notification "service-notification" to Service {
command = "service-by-mail"
period = "always"
zone = "myserver.local"
assign where service.enable_notifications && service.state == "CRITICAL"
states = [ Critical, OK, Unknown, Warning ]
types = [ Acknowledgement, Problem, Recovery ]
users = [ "myuser" ]
}
I tried it without the service.state statement as well but no cigar. I have two services applied to my test host, ping4 and hostalive. They both go critical when I do a test so they work fine. All I need is the thing to fire when this happens. The log says that the service has no notifications… why not?
not needed
also not needed
Try applying the service with assign where service.name == "my-service-name" to have the assign rule specifically for that service.
I would also suggest having multiple smaller assign rules for specific services/hosts instead of two huge ones, that then can get quite messy when excluding host/services.
Also be sure to have the services in HARD state, because notifications are only sent when a service or host is in a hard state.
1 Like
I modified my notification like you said:
apply Notification “service-notification” to Service {
command = “service-by-mail”
period = “always”
assign where service.name == “hostalive”
states = [ Critical, OK, Unknown, Warning ]
types = [ Acknowledgement, Problem, Recovery ]
users = [ "myuser ]
}
I do have a service called hostalive, it has notifications enabled and right now is in CRITICAL state.
Webui says its: "Check attempts 1/3 (hard state) " so it should be hard state.
In log this is all I get:
[2019-05-09 10:34:03 +0200] information/Checkable: Checking for configured notifications for object 'myserver.local!hostalive'
And then nothing, not even the usual “does not have any notifications” text.
Ok I figured it out. It wasn’t a problem with my apply rule or anything like that. It was a the timeperiod.
Can you share the complete working example for others who seek for the same question, please?
Also, please don’t try to set attributes other than suggested next time. Timeperiods are an advanced topic here and complicate the initial understanding.
Cheers,
Michael
Ok so here is my service notification template:
zones.d/myserver.local/notification_templates.conf
template Notification "service_notify_template" {
command = "service-by-mail"
users = [ "myuser" ]
}
Here is the service notification:
zones.d/myserver.local/notification_apply.conf
apply Notification "service-notification" to Service {
command = "service-by-mail"
assign where service.enable_notifications
states = [ Critical, OK, Unknown, Warning ]
types = [ Acknowledgement, Problem, Recovery ]
users = [ "myuser" ]
}
Here is the command itself (the script is the one made by Spiller taken from her github):
zones.d/myserver.local/commands.conf
object NotificationCommand "service-by-mail" {
import "plugin-notification-command"
command = [ "/etc/icinga2/scripts/service-by-mail.sh" ]
arguments += {
"-4" = "$address$"
"-6" = "$address6$"
"-d" = "$icinga.long_date_time$"
"-e" = "$service.name$"
"-l" = "$host.name$"
"-n" = "$host.display_name$"
"-o" = "$service.output$"
"-r" = "$user.email$"
"-s" = "$service.state$"
"-t" = "$notification.type$"
"-u" = "$service.display_name$"
}
}
This works for me. Thanks for the help @dnsmichi and @log1c!
2 Likes