Creating dependencies to limit remote notifications

Icinga 2
1

Greetings. Relatively new Icinga2 user here having trouble with notifications that a dependency should squelch.

My Icinga2 setup monitors systems at two locations, BigTown and LittleTown. The Icinga2 system is in BigTown.

LittleTown network connectivity is a little flaky, and I’d like to be notified only when Icinga2 can’t see the LittleTown router, not all the systems behind it.

Problem is, I’m getting notifications for all the LittleTown systems, not just the remote router, when LittleTown connectivity goes out.

What’s wrong in this setup? Thanks in advance! (Note: All the Host names are really FQDNs; I’ve shortened them here to get around the 2-link rule for new-user posts.)

#BigTown router
object Host “BigTownRouter” {
import “generic-host”
address = “172.31.0.1”
notes = “BigTown”

 vars.os = "OpenBSD"

 vars.notification["mail"] = {
     groups = [ "icingaadmins" ]
 }

}

LittleTown router

object Host “LittleTownRouter” {
import “generic-host”
address = “172.31.1.1”
notes = “LittleTown”

 vars.os = "OpenBSD"

 vars.notification["mail"] = {
     groups = [ "icingaadmins" ]
 }

}

#LittleTown host
object Host “ap11” {
import “generic-host”
address = “172.31.1.11”
notes = “LittleTown”

 vars.notification["mail"] = {
     groups = [ "icingaadmins" ]
 }

}

make LittleTown hosts children of LittleTown router

apply Dependency “LittleTown” to Host {
parent_host_name = “LittleTownRouter”
disable_checks = true
disable_notifications = true

assign where host.notes == “LittleTown” && host.name !=
“LittleTownRouter”
}

There are lots more hosts in LittleTown, and they all send notifications whenever the LittleTown link goes down.

What’s wrong here, and how can I get notifications for only the LittleTown router?

Thanks again!

2

Change it to

assign where host.notes == “LittleTown” && !(host.name == “LittleTownRouter”)

I think it should work - hopefully

Also make sure LittleTown Router Check achieves hard state before your Little Town

3

Another assign option

assign where host.notes == “LittleTown”
ignore where host.name == “LittleTownRouter”
4

Thank you both for your speedy replies. I tried both sets of modifications, and rebooted LittleTownRouter after each change. Unfortunately, other devices from LittleTown still send ping-critical problem notifications.

Is there something else needed so that Icinga only sends notifications for LittleTownRouter, and not other devices at the remote location?

Thanks again.

5

Maybe your retry and check interval are to short. Can you share the “generic-host” template?

Or try using the dependency “ignore_soft_state” attribute.

apply Dependency “LittleTown” to Host {
parent_host_name = “LittleTownRouter”
disable_checks = true
disable_notifications = true
ignore_soft_states = false

assign where host.notes == “LittleTown” && host.name != “LittleTownRouter”
}

If that still does not work the “notes” attribute in the apply rule is the problem. I never seen the notes attribute used before. Can you set a variable for the LittleTown devices and use that for the apply rule?

Regards
Alex

6

Thanks, Alex.

((EDIT: The 15-minute power down did trigger notification for LittleTownRouter.))

Good news/bad news. The good news is that with the addition of the ignore_soft_state directive there are no longer any notifications for LittleTown devices when LittleTownRouter reboots. I rebooted three times and didn’t get notifications. I then powered off the router for 15 minutes; again, no notifications.

The bad news is that I received no notifications for LittleTownRouter either. ((EDIT: Icinga did send notifications; it just took awhile to receive them.))

Here’s the generic-host template. I think this is the stock version included with the FreeBSD package; AFAIK the stock timers are unchanged.

template Host “generic-host” {
max_check_attempts = 3
check_interval = 1m
retry_interval = 30s

check_command = “hostalive”
}

As for the “notes” attribute: This was the only way I could find in the docs to describe device location. I thought of it as similar to the “description” field on Cisco router and switch interface configurations, but I could be mistaken.

In case that’s the problem, I’d be glad to try variables, which are new to me. (Hey, I said I’m a clueless newbie with Icinga2 :slight_smile:

It this how it should look?

object Host “ap11.int.example.com” {
import “generic-host”
vars = {
location = “LittleTown”
}

address = "172.31.1.11"

vars.notification["mail"] = {
    groups = [ "icingaadmins" ]
}

}

Also, if this is correct, would you recommend trying it with or without the ignore_soft_state directive?

Thanks again!

7

Hello @nycvelo
I am glad the dependency is working now. The problem before probably was the devices at LittleTown were entering a hard state before the router was in a hard state, which caused the notification to get sent out for the devices. It all would depend on when the last host check were scheduled for. Sometimes it is all a matter to timing.

As far as you dependency apply rule. Do you have a standard naming convention for your devices? You could use that for your apply rule.

object Host “LittleTownRouter” {
import “generic-host”
address = “172.31.1.1”
...
}

object Host “LittleTownap11” {
import “generic-host”
address = “172.31.1.11”
...
}

apply Dependency “LittleTown_Router” to Host {
parent_host_name = “LittleTownRouter”
disable_checks = true
disable_notifications = true
ignore_soft_states = false

assign where match ( “LittleTown*”, host.name)  //  This will add the dependency to all host with LittleTown at the beginning of their name
ignore where host.name == “LittleTownRouter”  // This will exclude the LittleTownRouter from the dependency
}

Or you can do the same with a variable.

object Host “LittleTownRouter” {
import “generic-host”
address = “172.31.1.1”
vars.site = "LT"
...
}

object Host “ap11” {
import “generic-host”
address = “172.31.1.11”
vars.site = "LT"
...
}

apply Dependency “LittleTown_Router” to Host {
parent_host_name = “LittleTownRouter”
disable_checks = true
disable_notifications = true
ignore_soft_states = false

assign where host.vars.site == "LT"  //  This will add the dependency to all host with variable site = LT
ignore where host.name == “LittleTownRouter”  // This will exclude the LittleTownRouter from the dependency
}

BTW - you don’t need to take your router down to test each time. You could just change the IP address for the router in Icinga to test, then change back when finished testing.

Regards
Alex

8

Great suggestions, thanks. This is all working well now

Really appreciate the help. Thanks again!

9

Hi @nycvelo, would you please be so kind and pick a solution to the topic?
This way the topic will be shown as resolved and that no more help is needed. It also is a nice “reward” for the helpers :wink:

Another tip/appeal: Please use markdown formatting for code snippets in future posts, so they become a bit easier to read :slight_smile:
You can find a guide on that here

Thanks and have fun with your setup :+1: