Warning Notifications during specific period

mani.vajrala · June 22, 2020, 8:52pm

Hi Team,

I would like to understand & example would be really helpful.

We are notify warning alerts during business hours only where as critical & recovery alerts all hours…
In timeperiods.conf file we have 9*5… i am trying use that for only warning alerts…

Can anyone provide an example. to send notification for warning alerts during business hours 9 to 5PM.

thanks,
mani

log1c · June 23, 2020, 6:43am

For this you will need two different notification rules.

One rule should have only the warning state and problem type (as well as ok state and recovery type, to “reset the notification status”) and the 9-to-5 time period.
The second rule has the critical state and problem type (+ok/recovery) and the 24*7 time period.

Then you configure your users so that they get notifications 24*7 and states warning & critical.
If you want your users to get the ok/recovery notifications as well, add them. If not, just leave them out.

mani.vajrala · June 23, 2020, 4:57pm

Thanks for your response. You are correct… my case is i want to send warning alerts for 9*5 timeperiod… & all other alerts

For warning… I have made a separate templates.

Here, Notification template

template Notification "warning-host-notification" {

  command = "mail-host-notification"

  states = [ Up, Down ]

  types = [ Problem, Acknowledgement, Recovery, Custom,

            FlappingStart, FlappingEnd,

            DowntimeStart, DowntimeEnd, DowntimeRemoved ]

  vars += {

    // notification_icingaweb2url = "https://www.example.com/icingaweb2"

    // notification_from = "Icinga 2 Host Monitoring <icinga@example.com>"

    notification_logtosyslog = false

  }

  period = "8to5"

}

/**

 * Provides default settings for service notifications.

 * By convention all service notifications should import

 * this template.

 */

template Notification "warning-service-notification" {

  command = "mail-service-notification"

  states = [ OK, Warning, Critical, Unknown ]

  types = [ Problem, Acknowledgement, Recovery, Custom,

            FlappingStart, FlappingEnd,

            DowntimeStart, DowntimeEnd, DowntimeRemoved ]

  vars += {

    // notification_icingaweb2url = "https://www.example.com/icingaweb2"

    // notification_from = "Icinga 2 Service Monitoring <icinga@example.com>"

    notification_logtosyslog = false

  }

  period = "8to5"

}

Here, Apply notification block.

apply Notification "warning-admin" to Host {

  import "warning-host-notification"

  user_groups = host.vars.notification.mail.groups

  users = host.vars.notification.mail.users

  #nterval = 0 // disable re-notification

  interval = 4h

  //vars.notification_logtosyslog = true

  assign where host.vars.notification.mail

}

apply Notification "warning-admin" to Service {

  import "warning-service-notification"

  user_groups = host.vars.notification.mail.groups

  users = host.vars.notification.mail.users

  

   interval = 4h

    #interval = 6h // disable re-notification

  # assign where host.vars.notification.mail

  assign where host.vars.notification.mail

}

In users.conf… I have defined as shown below.

object User "icingaadminwarn" {

  import "generic-user"

  display_name = "Notify icinga admin for warning"

  enable_notifications = true

  states = [ Warning ]

  types = [ Problem ]

  groups = [ "icingausers" ]

  email = "Icinga_Admin@nicholasandco.com"

}

Does this make sense? on my all hostfiles…

we defined as shown below…

vars.notification["mail"] = {

        groups = [ "icingausers" ]

        users = [ "icingaadmin" ]

}

Does this make sense?

log1c · June 24, 2020, 6:20am

Having looked over this quickly: Yes, that should work to send warnings during the 8to5 period.
Keep in mind though, that you won’t receive any other notifications (up/ok/critical/…) for the “icingaadminwarn”, because you only configured the warning state.

mani.vajrala · June 29, 2020, 4:44pm

Thanks for you response… Actually i have been struggling to understand how timeperiod notifications getting applied…

Suppose we have a usergroup for critical & recovery… same group should only receive warning alerts during business hours 8to9 for every 4 hours…

users.conf:

object User "icingaadmin" {
  import "generic-user"
  display_name = "Icinga 2 Admin"
  enable_notifications = true
  states = [ OK, Critical ]
  types = [ Problem, Recovery ]
  groups = [  "icingausers" ]

  email = "Icinga_Admin@nicholasandco.com"
}

object User "icingaadminwarn" {
  import "generic-user"

  display_name = "Notify icinga admin for warning"
  enable_notifications = true
  states = [ Warning ]
  types = [ Problem ]
  groups = [ "icingausers" ]

  email = "Icinga_Admin@nicholasandco.com"
}

templates.conf:

template Notification "warning-host-notification" {
 command = "mail-host-notification"
  states = [ Up, Down ]
  types = [ Problem, Acknowledgement, Recovery, Custom,
            FlappingStart, FlappingEnd,
            DowntimeStart, DowntimeEnd, DowntimeRemoved ]
  vars += {
    // notification_icingaweb2url = "https://www.example.com/icingaweb2"
    // notification_from = "Icinga 2 Host Monitoring <icinga@example.com>"
    notification_logtosyslog = false
  }
  period = "8to5"
}

template Notification "warning-service-notification" {
  command = "mail-service-notification"
  states = [ OK, Warning, Critical, Unknown ]
  types = [ Problem, Acknowledgement, Recovery, Custom,
            FlappingStart, FlappingEnd,
            DowntimeStart, DowntimeEnd, DowntimeRemoved ]
  vars += {
    notification_logtosyslog = false
  }
  period = "8to5"
}

Notification.conf:

apply Notification "warning-admin" to Host {
  import "warning-host-notification"
  # user_groups = host.vars.notification.mail.groups
  users = [ "icingaadminwarn" ]
  #nterval = 0 // disable re-notification
  interval = 4h
  //vars.notification_logtosyslog = true
  assign where host.vars.notification.mail
}

apply Notification "warning-admin" to Service {
  import "warning-service-notification"
  # user_groups = host.vars.notification.mail.groups
   users = [ "icingaadminwarn" ]
   interval = 4h
  assign where host.vars.notification.mail
}

We have defined timeperiod 8*5… somehow new templates & notification blocks are not getting applied to users… somehow people are getting warning alerts after businesshours… looks like user.conf still taking the default notifications & notification templates… can you advise how notification templates will be applied on users.conf for warnings to send only during business hours…

Many thanks in advance…

log1c · June 30, 2020, 6:14am

Configs look good to me. This should work to only send the warning notitifactions during the 8to5 period.

mani.vajrala:

apply Notification "warning-admin" to Host {
  import "warning-host-notification"
  # user_groups = host.vars.notification.mail.groups
  users = [ "icingaadminwarn" ]
  #nterval = 0 // disable re-notification
  interval = 4h
  //vars.notification_logtosyslog = true
  assign where host.vars.notification.mail
}

This can be deleted though, as there is no warning state for hosts (just up or down). So this would never be used.

Are there any other notifications apply rules, templates and users that you have not posted?

mani.vajrala · July 6, 2020, 5:01pm

Many thanks for your response…

Yes there are other notification apply rules… that are shown as below… & other notification apply that i posted earlier. Looks like the below apply notification is getting applied on 8to5? We are defintely getting warning alerts after business hours…& not able to understand how this apply rules are using appropriate timeperiod?..

apply Notification "mail-icingaadmin" to Host {
  import "mail-host-notification"
  user_groups = host.vars.notification.mail.groups
  users = host.vars.notification.mail.users
  interval = 2h
  assign where host.vars.notification.mail
}
apply Notification "mail-icingaadmin" to Service {
  import "mail-service-notification"
   if (service.vars.notification.mail.users) {
    users = service.vars.notification.mail.users
  } else if (host.vars.notification.mail.users) {
    users = host.vars.notification.mail.users
  } else {
    users = [ "icingaadmin" ]
  }
 interval = 2h
  if (service.vars.notification.mail.groups) {
    user_groups = service.vars.notification.mail.groups
  } else if (host.vars.notification.mail.groups) {
    user_groups = host.vars.notification.mail.groups
  }

 assign where ( host.vars.notification.mail && typeof(host.vars.notification.mail) == Dictionary ) || ( service.vars.notification.mail && typeof(service.vars.notification.mail) == Dictionary )
}

log1c · July 7, 2020, 6:17am

You are using mail-service-notifications and mail-host-notification as templates here.
If these are the default templates they use the 24x7 time period and all the states and types.

Also you are applying the rules to every host/service with users or user groups.
So this is the reason why you are getting warning notifications after business hours.

Also using the same mail address for both users does pretty much invalidate what you are trying to achieve, if I think about it.

mani.vajrala · July 8, 2020, 4:14pm

Thanks for your response… Yes. that make sense now… Actually we don’t have so many users & user groups… we have only one group icingausers.
First user icingaadmin that receive critical & recovery alerts during all business hours for every 2 hours… Another user icingaadminwarn that receive only warning alerts during business hours for every 4 hours

So at this time… I have the following options that would like to proceed…

Remove warning state type from default notification template. Even this template applied to icingaadminwarn user it just ignore the warning state type.

or another template/application notification something like below
apply Notification “warning-admin” to Service {

  import "warning-service-notification"
      users = [ "icingaadminwarn" ]
       interval = 4h 
      assign where host.vars.notification.mail
    }

Can you please advise the options that i have? Apologies for asking so many questions… Many thanks in advance…