Monitoring Warning Alert by Email and Critical Alert By Email and Call by Pager Duty

Service Monitoring
1

Good Afternoon The Community,

I explain to you my issue. I have configure the Prod Server with 2 Alerting Levels. The first is Warning and the secund Critical.

We have plugged Pager duty to call on duty but with this configuration the call are also done for Warning Alerting.

I want only call for Critical Alerting and Icinga send email for Warning.

Thanks for the help

2

Hi,

please share the two notification apply rule configurations as well as the user/group objects you’re using.

Cheers,
Michael

3

Hi Michael,

I’m novice on Icinga can you tell me where can I found it please ?

4

Hi,

you should have them configured by yourself. Depending on how you configured Icinga 2, either they are static config files in /etc/icinga2 or within the Director. If you didn’t touch anything, the example config lies underneath /etc/icinga2/conf.d.

Cheers,
Michael

6

You’ll want to adjust this in your notification object with the states value. Here’s a glimpe at one of mine:

apply Notification "pagerduty-service" to Service {
  command = "notify-service-by-pagerduty"
  states = [ Critical, Unknown ]
  types = [ Problem, Acknowledgement, Recovery ]
  users = [ "yourpagerdutyuserhere" ]

In this case, I don’t have it paging me for warnings, just critical and unknown. You can adjust that for whatever notification objects you have as needed.

7

Hello Blake,
Thanks for your feed Back but I want only send an Email when we have got warning Status not use pagerduty.

When the Alert is Critical, PagerDuty send mail and Call but when it’s warning Icinga send an email only

I don’t know if it’s clear ?

Thanks.

8

Change your notification script to send to pager when state is critical. That is what i have done. You only need to send all needed variables to your script.

9

Hello Carsten,

Thanks. Can you give me an example please ?

10

I, personally, would not want to modify PagerDuty’s scripts. They’re pretty beastly and the states variable just keep Icinga from triggering on those in the first place.

What I suggested above works fine; I never get paged for Warning. For email alerts, you would take a similar approach. You can check the default template to get an idea.

template Notification "mail-service-notification" {
  command = "mail-service-notification"

  states = [ OK, Warning, Critical, Unknown ]
  types = [ Problem, Acknowledgement, Recovery, Custom,
            FlappingStart, FlappingEnd,
            DowntimeStart, DowntimeEnd, DowntimeRemoved ]
11

Hi Blake,

Thanks for your help.
The configuration will be:

template Notification “mail-service-notification” {
command = “mail-service-notification”

states = [ Warning]
types = [ Problem, Acknowledgement, Recovery, Custom,
FlappingStart, FlappingEnd,
DowntimeStart, DowntimeEnd, DowntimeRemoved ]

12

Roughly, that’s an example. Basically, go to your current email notification object and define states and types to what you want to receive.