Notification in HA scenario

Icinga 2
1

Hey Everyone

I had a look through existing topics and there were a few regarding the subject but can’t find a clear answer (or just don’t understand it).

I have a problem that notifications in my HA scenario where notifications seem not to be sent no matter what I do.

Current setup:

  • 2 x Master nodes
  • 1 x Sattelite node
  • Multiple clients connecting to the Sattelite node

Currently, I have a notification feature that enables only Master nodes and after enabling debug logs noticed something interesting:

Master1

[2025-01-16 10:52:47 +0000] notice/NotificationComponent: Reminder notification '*********': HA cluster active, this endpoint does not have the authority (paused=true). Skipping.
[2025-01-16 10:52:51 +0000] information/NotificationComponent: 'notification' stopped.

Master2

[2025-01-16 10:52:30 +0000] notice/NotificationComponent: Reminder notification '******': HA cluster active, this endpoint does not have the authority (paused=true). Skipping.
[2025-01-16 10:52:31 +0000] information/NotificationComponent: 'notification' stopped.

Am I reading it correctly that both masters think they are not active and that’s why they not sending?

Please note we are using IcingaDB Health seems OK in cluster
MasterA

Icinga DB OK: Uptime: 14 days, 4 hours, 35 minutes and 1 second. Version: v1.2.0.
Icinga 2:
* Last full dump: 37 minutes and 55 seconds ago, took 338 milliseconds
Icinga DB:
* Last seen: 586 milliseconds ago
* Responsible
* Last full sync: 37 minutes and 54 seconds ago, took 805 milliseconds

MasterB

Icinga DB OK: Uptime: 59 minutes and 37 seconds. Version: v1.2.0.
Icinga 2:
* Last full dump: 39 minutes and 38 seconds ago, took 274 milliseconds
Icinga DB:
* Last seen: 579 milliseconds ago
* Not responsible, but another instance is

Also just in case it’s matters this is my zones from masters
MasterA

object Endpoint "ic2-master-a" {
    host = "ic2-master-a"
}
object Endpoint "ic2-master-b" {
        host = "ic2-master-b"
}
object Zone "master" {
        endpoints = [ "ic2-master-a", "ic2-master-b" ]
}
object Zone "director-global" {
        global = true
}
object Endpoint "ic2-satellite-a" { }

object Zone "ic2-satellite-a" {
    endpoints = [ "ic2-satellite-a ]
    parent = "master"
}

MasterB

object Endpoint "ic2-master-b" {
}

object Endpoint "ic2-master-a" {
}

object Zone "master" {
        endpoints = [ "ic2-master-b", "ic2-master-a" ]
}

object Zone "director-global" {
        global = true
}

object Endpoint "ic2-satellite-a" { }

object Zone "ic2-satellite-a" {
    endpoints = [ "ic2-satellite-a" ]
    parent = "master"
}

Also, this is an example Notification object that is a check Executed on the Endpoint connected to the Sattelite zone

Object '**********' of type 'Notification':
  % declared in '/etc/icinga2/zones.d/global-templates/notifications/email.conf', lines 2:1-2:131
  * __name = "**************
  * command = "mail-service-notification"
    % = modified in '/etc/icinga2/zones.d/global-templates/notifications/templates.conf', lines 43:5-43:41
  * command_endpoint = ""
  * host_name = "ENDPOINT_HOST"
    % = modified in '/etc/icinga2/zones.d/global-templates/notifications/email.conf', lines 2:1-2:131
  * interval = 3600
    % = modified in '/etc/icinga2/zones.d/global-templates/notifications/email.conf', lines 9:64-9:77
  * name = "standard-email-service-notificationStandardBS"
  * package = "_etc"
    % = modified in '/etc/icinga2/zones.d/global-templates/notifications/email.conf', lines 2:1-2:131
  * period = "dynamic-bsupport"
    % = modified in '/etc/icinga2/zones.d/global-templates/notifications/email.conf', lines 7:24-7:43
  * service_name = "linux_systemd-invantage-scheduler-db@pflive"
    % = modified in '/etc/icinga2/zones.d/global-templates/notifications/email.conf', lines 2:1-2:131
  * source_location
    * first_column = 1
    * first_line = 2
    * last_column = 131
    * last_line = 2
    * path = "/etc/icinga2/zones.d/global-templates/notifications/email.conf"
  * states = null
  * templates = [ "standard-email-service-notificationStandardBS", "notification-email-service-standard" ]
    % = modified in '/etc/icinga2/zones.d/global-templates/notifications/email.conf', lines 2:1-2:131
    % = modified in '/etc/icinga2/zones.d/global-templates/notifications/templates.conf', lines 42:1-42:59
  * times = null
  * type = "Notification"
  * types = null
  * user_groups = null
    % = modified in '/etc/icinga2/zones.d/global-templates/notifications/email.conf', lines 13:5-13:34
  * users = [ "Business-Support-uk" ]
    % = modified in '/etc/icinga2/zones.d/global-templates/notifications/email.conf', lines 12:5-12:22
  * vars
    % = modified in '/etc/icinga2/zones.d/global-templates/notifications/email.conf', lines 5:5-5:18
    * period = "dynamic-bsupport"
    * targets = "ALL"
      % = modified in '/etc/icinga2/zones.d/global-templates/notifications/email.conf', lines 8:25-8:53
    * users = [ "Business-Support-uk" ]
  * zone = "ic2-satellite-a"
    % = modified in '/etc/icinga2/zones.d/global-templates/notifications/email.conf', lines 2:1-2:131

Also, I tried to enable notifications on sattelite and they are no debug logs about it and also notifications are not sent.

Anyone can point what I did wrong?

Thanks
Dariusz

2

Hey Guys

I think I fixed it by adding

enable_ha = true

For notification.conf on both masters. So is that right solution and this is now HA ready?

Thanks
Dariusz

1 Like
3

In my case it works without this line as enable_ha = true is the default according to Object Types - Icinga 2.

1 Like
4

For me it worked when I did enable_ha = true. Weird but fixed now!

Thanks for assistance
D

1 Like