Notifications is not working on HA configuration

Hello,

I’ve discovered that notifications is only working with mail and not working my other application. Do you know how I can check it the HA configuration is provoking the issue?

as I can see on the notification:

→ HA cluster active, this endpoint does not have the authority (paused=true). Skipping.

Hi @alperez,
hard to say with that information, is the notification feature active on both master nodes?

Hello,

Yes, we have active notifications in both master nodes. As I could see, only works notifications related with “HOSTS” all the notifications with services are not working when the nodeB is active. To check it that could be an issue with the HA configuration I’ve stopped the nodeB and I was testing all the notifications creating some alerts and forcing these alerts and works. So this is why I think that the issue is related with the nodeB HA configuration.

In the other hand, I’ve raised another ticket here related to “check now” funcionatily and it’s the same issue, if nodeB is working “check now” option and some services applies rules are hang.

I didn’t do the configuration of HA was another person and sincerelly i don’t know how he did it. as I can see on the document must exists a CA certicated that for me not exists in none master nodes and besides we have two satellites configured. So my problem also is that I cannot check it if the HA is correctly configured or not.

This is why I’ve created the thread.

Please share from both masters

Also if you can only send mails from one of both masters check your MTA (postfix most likley) if it is configured correctly on both masters.

Hi,

Information added. Right now Master2 is down because I’ve stopped the icinga2 daemon.

In the other hand the problem is not with the sendmail both nodes are sending mails is not working or script to use the oncall.
I’ve tested manually the scripts and works so the problem is with MASTER2, besides, services apply rules are hang with MASTER 2 configuration check now is not working and satellites are hang also.

MASTER 1 icinga2 daemon -C

[2024-07-23 08:12:19 +0200] information/cli: Icinga application loader (version: 2.13.2-1)
[2024-07-23 08:12:19 +0200] information/cli: Loading configuration file(s).
[2024-07-23 08:12:19 +0200] information/ConfigItem: Committing config item(s).
[2024-07-23 08:12:19 +0200] information/ApiListener: My API identity: eurvlii75429.xmp.net.intra
[2024-07-23 08:12:19 +0200] information/ConfigItem: Instantiated 1 SyslogLogger.
[2024-07-23 08:12:19 +0200] information/ConfigItem: Instantiated 1 NotificationComponent.
[2024-07-23 08:12:19 +0200] information/ConfigItem: Instantiated 1 IdoMysqlConnection.
[2024-07-23 08:12:19 +0200] information/ConfigItem: Instantiated 1 ExternalCommandListener.
[2024-07-23 08:12:19 +0200] information/ConfigItem: Instantiated 1 CheckerComponent.
[2024-07-23 08:12:19 +0200] information/ConfigItem: Instantiated 3 Users.
[2024-07-23 08:12:19 +0200] information/ConfigItem: Instantiated 7 TimePeriods.
[2024-07-23 08:12:19 +0200] information/ConfigItem: Instantiated 1700 Services.
[2024-07-23 08:12:19 +0200] information/ConfigItem: Instantiated 4 Zones.
[2024-07-23 08:12:19 +0200] information/ConfigItem: Instantiated 1 NotificationCommand.
[2024-07-23 08:12:19 +0200] information/ConfigItem: Instantiated 96 HostGroups.
[2024-07-23 08:12:19 +0200] information/ConfigItem: Instantiated 2856 Notifications.
[2024-07-23 08:12:19 +0200] information/ConfigItem: Instantiated 4 Downtimes.
[2024-07-23 08:12:19 +0200] information/ConfigItem: Instantiated 1 IcingaApplication.
[2024-07-23 08:12:19 +0200] information/ConfigItem: Instantiated 438 Hosts.
[2024-07-23 08:12:19 +0200] information/ConfigItem: Instantiated 4 Endpoints.
[2024-07-23 08:12:19 +0200] information/ConfigItem: Instantiated 12 Comments.
[2024-07-23 08:12:19 +0200] information/ConfigItem: Instantiated 1 FileLogger.
[2024-07-23 08:12:19 +0200] information/ConfigItem: Instantiated 2 ApiUsers.
[2024-07-23 08:12:19 +0200] information/ConfigItem: Instantiated 256 CheckCommands.
[2024-07-23 08:12:19 +0200] information/ConfigItem: Instantiated 1 ApiListener.
[2024-07-23 08:12:19 +0200] information/ScriptGlobal: Dumping variables to file ‘/var/cache/icinga2/icinga2.vars’
[2024-07-23 08:12:19 +0200] information/cli: Finished validating the configuration file(s).

MASTER 1 zones.conf

/*

  • Generated by Icinga 2 node setup commands
  • on 2023-02-17 17:36:43 +0100
    */

/Master endpoint/
object Endpoint “MASTER 1” {
host = “MASTER 1”
}

object Endpoint “MASTER 2” {
host = “MASTER 2”
}

/Satellite-1/
object Endpoint “SATELLITE-1” {
host = “SATELLITE-1”
}

/Satellite-2/
object Endpoint “SATELLITE-2” {
host = “SATELLITE-2”
}

object Zone “master” {
endpoints = [ “MASTER 1” , “MASTER 2”]
}

object Zone “zone-emea” {
endpoints = [ “SATELLITE-1” , “SATELLITE-2”]
parent = “master”
}

object Zone “global-templates” {
global = true
}

object Zone “director-global” {
global = true
}

icinga2 feature list
Disabled features: compatlog debuglog elasticsearch gelf graphite icingadb ido-pgsql influxdb influxdb2 livestatus opentsdb perfdata statusdata
Enabled features: api checker command ido-mysql mainlog notification syslog

Master2 icinga2 daemon -C

[2024-07-23 11:03:39 +0200] information/cli: Icinga application loader (version: 2.13.2-1)
[2024-07-23 11:03:39 +0200] information/cli: Loading configuration file(s).
[2024-07-23 11:03:39 +0200] warning/ConfigCompiler: Cannot compile file ‘/etc/icinga2/features-enabled/notification.conf’: Error: Function call ‘std::ifstream::open’ for file ‘/etc/icinga2/features-enabled/notification.conf’ failed with error code 13, ‘Permission denied’

Context:
(0) Compiling configuration file ‘/etc/icinga2/features-enabled/notification.conf’
[2024-07-23 11:03:39 +0200] information/ConfigItem: Committing config item(s).
[2024-07-23 11:03:39 +0200] information/ApiListener: My API identity: eurvlii75430.xmp.net.intra
[2024-07-23 11:03:40 +0200] information/ConfigItem: Instantiated 1 SyslogLogger.
[2024-07-23 11:03:40 +0200] information/ConfigItem: Instantiated 1 IdoMysqlConnection.
[2024-07-23 11:03:40 +0200] information/ConfigItem: Instantiated 1 ExternalCommandListener.
[2024-07-23 11:03:40 +0200] information/ConfigItem: Instantiated 1 CheckerComponent.
[2024-07-23 11:03:40 +0200] information/ConfigItem: Instantiated 3 Users.
[2024-07-23 11:03:40 +0200] information/ConfigItem: Instantiated 6 TimePeriods.
[2024-07-23 11:03:40 +0200] information/ConfigItem: Instantiated 1480 Services.
[2024-07-23 11:03:40 +0200] information/ConfigItem: Instantiated 4 Zones.
[2024-07-23 11:03:40 +0200] information/ConfigItem: Instantiated 1 NotificationCommand.
[2024-07-23 11:03:40 +0200] information/ConfigItem: Instantiated 94 HostGroups.
[2024-07-23 11:03:40 +0200] information/ConfigItem: Instantiated 1923 Notifications.
[2024-07-23 11:03:40 +0200] information/ConfigItem: Instantiated 4 Downtimes.
[2024-07-23 11:03:40 +0200] information/ConfigItem: Instantiated 1 IcingaApplication.
[2024-07-23 11:03:40 +0200] information/ConfigItem: Instantiated 449 Hosts.
[2024-07-23 11:03:40 +0200] information/ConfigItem: Instantiated 4 Endpoints.
[2024-07-23 11:03:40 +0200] information/ConfigItem: Instantiated 12 Comments.
[2024-07-23 11:03:40 +0200] information/ConfigItem: Instantiated 1 FileLogger.
[2024-07-23 11:03:40 +0200] information/ConfigItem: Instantiated 256 CheckCommands.
[2024-07-23 11:03:40 +0200] information/ConfigItem: Instantiated 1 ApiListener.
[2024-07-23 11:03:40 +0200] information/ScriptGlobal: Dumping variables to file ‘/var/cache/icinga2/icinga2.vars’
[2024-07-23 11:03:40 +0200] information/cli: Finished validating the configuration file(s).

MASTER 2 zones.conf

/*

  • Generated by Icinga 2 node setup commands
  • on 2024-04-11 20:22:29 +0200
    */

object Endpoint “MASTER 1” {
host = “MASTER 1 IP”
port = “5665”
}

object Endpoint “MASTER 2” {
host = “MASTER 2”
}

object Zone “master” {
endpoints = [ “MASTER 1” , “MASTER 2” ]
}

/Satellite-1/
object Endpoint “SATELLITE-1” {
host = “SATELLITE-1”
}

/Satellite-2/
object Endpoint “SATELLITE-2” {
host = “SATELLITE-2”
}

object Zone “zone-emea” {
endpoints = [ “SATELLITE-1” , “SATELLITE-2”]
parent = “master”
}

object Zone “global-templates” {
global = true
}

object Zone “director-global” {
global = true
}

icinga2 feature list
Disabled features: compatlog debuglog elasticsearch gelf graphite icingadb ido-pgsql influxdb influxdb2 livestatus opentsdb perfdata statusdata
Enabled features: api checker command ido-mysql mainlog notification syslog

the icinga user ( nagios on debianish OSes) does not have read permissions on that file :slight_smile:

1 Like

so… what’s the permission that must exists on /etc/icinga2 folder? because I can see that zones.conf is root:root and other inside of features also the owner and the group is root:root

That folder and it’s contents should belong to the user running the icinga2 service.
icinga on rhel/rpm-based systems
nagios on debian-based systems

How come that it is changed on your setup? Did someone copy all the files from one node to the other?
The installation process will place the files with the correct ownership.

Hello,

Sincerelly I don’t know how it was intalled the second node. Because as I don’t have access to the repo. I think that all packages were download and installed manually but I don’t know if there was a copy or not.

This is also the thing that I’m triying to understand.

Hi again,

Do you have another idea? I was thinking that could be an issue with HA because we have both masters “UP”.

What’s the normal behaviour of HA configuration for icinga2?

Because the “checks” for service apply rules are not working with both masters “UP” I think that something internall, is not ok.

Regards