Newbie: Simple Email notification for like FTP down?

Hi,
coming from Nagios, i just installed Icinga to fresh VM server, i got services monitored at webpage, but cannot get any email notifications.

And server mail function seems to work because i get every night email:


[DOWNTIMESTART] load on xxx.com is OK!

***** Service Monitoring on icinga *****
load on icinga.xxx is OK!

Info: OK - load average: 0.00, 0.00, 0.00

When: 2019-05-27 02:00:03 +0300
Service: load

Comment by icingaadmin:
Scheduled downtime for backup

But i don’t get any email that some service is down, when they show red in Icinga web interface.

I have changed email @ users.conf , not touched notifications.conf

Can someone give just super simple example what file i have to modify to get simple email notification for services i have added to hosts.conf ?

Installed from Ubuntu-Server 18.04 / icinga2 2.10.5-1 packages.

Your system added " Comment by icingaadmin" , was that because i try to seperate log lines with dash marks ?? :slight_smile: :slight_smile:

The docs you must read, young Padawan
https://icinga.com/docs/icinga2/latest/doc/04-configuring-icinga-2/

Hi and welcome to the community!

Please avoid such swear words, and try to explain what exactly happened. When you look up the docs URL @MarcusCaepio just posted, you’ll see that there is some example configuration in place.

All parts together with hosts, services and notifications already generate such. One specific thing here is that a recurring downtime (“ScheduledDowntime”) generates a runtime downtime for the load service already. During that timeperiod, load may be higher and as such, a downtime start notification is sent.

Since you’re coming from the old world, please also learn about the fact that you need notification, user and notification command objects to build alerts. The old habit was to just assign contacts to host/service objects, this is not the case anymore.

In order to start over, remove the inclusion for conf.d in icinga2.conf and create your own object storage.

mkdir -p /etc/icinga2/objects.d
vim /etc/icinga2/icinga2.conf

//include_recursive "conf.d"

include_recursive "objects.d" 
cat >>/etc/icinga2/objects.d/`hostname`.conf <<EOF
object Host "`hostname`" {
 address = "127.0.0.1"

 check_command = "hostalive"
}
EOF

The snippet above create the local host, and allows you did further with your first service. Once that’s working, continue with users and notifications.

Cheers,
Michael

Hi,
i have like 10 nodes succesfully monitored, ping and services, all green @ icinga web view.

BUT i cannot get any email alert if i shutdown e.g. FTP service.

Monitoring is just for 1 admin (me), nothing special.

I added this to hosts.conf

object Service “ftp” {
host_name = “www4.XX.fi
check_command = “ftp”
vars.notification[“mail”] = {
groups = [ “icingaadmins” ]
}
}

And users.conf has:

object User “icingaadmin” {
import “generic-user”

display_name = “Icinga 2 Admin”
groups = [ “icingaadmins” ]

email = "info@XX.fi"
}

object UserGroup “icingaadmins” {
display_name = “Icinga 2 Admin Group”
}

What else i need to add just have plain simple setup ?

Your documents looks fine but for normal 20yrs as linux adminstrator they are just to complicated.

Server is Ubuntu 18.04 and installed from packages (icinga 2.10.5).

Hi,

did you define any notification rules which take the notification custom attribute into account from the host?

If so, there also is a thing with max_check_attempts - only if the retries hit a HARD state, a notification will be triggered. If you’re in a SOFT state, e.g. first out of three, then the service turns not-ok, but won’t fire a notification.

If that still doesn’t work, check the logs whether the notification is actually triggered. If yes, but not sent, enable the debug log and check whether filers prevent this from sending.

Cheers,
Michael

Your forum is not easy to use. it says, “Sorry new user can only put 2 links on the post.” There was not any links but i remove server addresses…

Thanks for your support,
log shows this, but i really cannot understand what should i add to notification.conf to make it send an email ?

[2019-05-31 18:41:00 +0300] information/Checkable: Checking for configured notifications for object ‘www4!ftp’
[2019-05-31 18:41:00 +0300] information/Checkable: Checkable ‘www4!ftp’ does not have any notifications.


hosts.conf

object Host “www4” {
address = “www4”
check_command = “hostalive”
}

object Service “http” {
host_name = “www4”
check_command = “http”
vars.notification[“mail”] = {
groups = [ “icingaadmins” ]
}
}

object Service “ssh” {
host_name = “www4”
check_command = “ssh”
vars.notification[“mail”] = {
groups = [ “icingaadmins” ]
}
}

object Service “ftp” {
host_name = “www4”
check_command = “ftp”
vars.notification[“mail”] = {
groups = [ “icingaadmins” ]
}
}


notifications.conf

apply Notification “mail-icingaadmin” to Host {
import “mail-host-notification”
user_groups = host.vars.notification.mail.groups
users = host.vars.notification.mail.users

//interval = 2h

//vars.notification_logtosyslog = true

assign where host.vars.notification.mail
}

apply Notification “mail-icingaadmin” to Service {
import “mail-service-notification”
user_groups = host.vars.notification.mail.groups
users = host.vars.notification.mail.users

//interval = 2h

//vars.notification_logtosyslog = true

assign where host.vars.notification.mail
}


users.conf

object User “icingaadmin” {
import “generic-user”

display_name = “Icinga 2 Admin”
groups = [ “icingaadmins” ]

email = “infofi”
}

object UserGroup “icingaadmins” {
display_name = “Icinga 2 Admin Group”
}


groups.conf

object HostGroup “linux-servers” {
display_name = “Linux Servers”

assign where host.vars.os == “Linux”
}

object HostGroup “windows-servers” {
display_name = “Windows Servers”

assign where host.vars.os == “Windows”
}

object ServiceGroup “ping” {
display_name = “Ping Checks”

assign where match(“ping*”, service.name)
}

object ServiceGroup “http” {
display_name = “HTTP Checks”

assign where match(“http*”, service.check_command)
}

object ServiceGroup “disk” {
display_name = “Disk Checks”

assign where match(“disk*”, service.check_command)
}

Hi,

Better format your content, you can use back ticks and code blocks to encapsulate configuration, logs, etc. - check the howto here: Create topics and master Markdown formatting (linked in the FAQ as well).

Discourse needs a little learning, I can highly recommend to do the discobot tutorial, you should find a message in your inbox :slight_smile:

That’s a good starting point. Now you’ll know that Icinga doesn’t have any notification objects configured.

Not Notification objects are there

So why is that? You’ll know that you’re using apply rules to generate notification objects for hosts and services.

This generation depends on a pattern match, a boolean expression, which must evaluate to true. Let’s inspect the notification apply rule first.

apply Notification “mail-icingaadmin” to Service {

//....

  assign where host.vars.notification.mail
}

The expression will be true, if the host has a custom attribute in vars which is named notification. This is a dictionary, as it also contains a sub key called mail.

Now let’s inspect the host object, does it provide such a custom attribute …

object Host “www4” {
  address = “www4”
  check_command = “hostalive”
}

Obviously no, it doesn’t.

This is the explanation why no notification objects for services are generated.

Making it work for services

You’re defining the notification custom attribute on the service level instead. This is totally fine, but for now this is just additional information never used anywhere.

Modify the service notification apply rule to actually use the provided details.

  • Fetch the values from service.vars.notification.mail into usersanduser_groups`
  • Ensure that the assign where expression matches on service.vars.notification.mail
apply Notification “mail-icingaadmin” to Service {
  import “mail-service-notification”

  user_groups = service.vars.notification.mail.groups

  //TODO: The same applies for users, try that

  // Important: Evaluate this apply rule for all services which provide the notification details.
  assign where service.vars.notification.mail
}

Can you improve this for host defaults and service exceptions?

Yes, and you definitely should once the above works.

So, how can we make this more simple to only manage the defaults on the host level, while creating exceptions on the service objects?

Coming back to the notification apply rule. With using conditions, you can check whether the service provides the details and assign the values.

Additionally, the assign where expression needs to be modified to match both, services and hosts with a logical OR condition. Otherwise Icinga won’t recognize this.

apply Notification “mail-icingaadmin” to Service {
  import “mail-service-notification”

   // Prefer the service object details for notifications
  if (service.vars.notification.mail) {
    user_groups = service.vars.notification.mail.groups
  } else { // Use the host defaults if none specified in the service
    user_groups = host.vars.notification.mail.groups
  }

  //TODO: The same applies for users, try that

  // Important: Evaluate this apply rule for all services AND hosts which provide the notification details.
  assign where service.vars.notification.mail || host.vars.notification.mail
}

That’s too complicated

Yeah, the DSL allows for many advanced techniques. If you prefer to not use apply rules and attribute fetching from hosts/services, you can also create simple notification objects.

From your example, this looks like this:

object Notification "mail-icingaadmin" {
  host_name = "www4"
  service_name = "http"

  groups = [ "icingaadmins" ]

  //...
}

Typing and copying this everything can become hard, that is why apply rules are here to make this more reliable and generic. Decide whatever fits best. You can always refactor the configuration later on after making it work.

Cheers,
Michael

1 Like