we have an HA master → satellite-zones set up which has been fine for a long time. But recently one satellite zone reports that the satellites are not connected to each other - based on the default “load” and “icinga” service checks going into an UNKNOWN state every minute. they were fine for several months until adding some additional host objects config then doing a “reload” on the primary master.
output from load check:
Remote Icinga instance 'icinga-s6.mon.domain' is not connected to 'icinga-s7.mon.domain'
i’ve stopped/restarted each of them as well as both masters which did not resolve it. i then reran the node wizard on both satellites but still did not resolve the issue. the zones.conf file is the same as in all the other satellite zones which do not exhibit this behavior.
on satellites zones.conf
object Endpoint “icinga-master1.mon.domain” {
// host = “icinga-master1.mon.domain”
//do not actively connect to the satellite by leaving out the ‘host’ attribute
}object Endpoint “icinga-master2.mon.domain” {
// host = “icinga-master2.mon.domain”
//do not actively connect to the satellite by leaving out the ‘host’ attribute
}//Jinja2 syntax to add the corresponding satellite zone’s corresponding endpoint
object Endpoint “icinga-s6.mon.domain” {
host = “icinga-s6.mon.domain”
}
object Endpoint “icinga-s7.mon.domain” {
host = “icinga-s7.mon.domain”
}object Zone “master” {
endpoints = [ “icinga-master1.mon.domain”, “icinga-master2.mon.domain” ]
}//Jinja2 syntax to add the corresponding satellite zone and then the zone’s corresponding endpoint
object Zone “satellite-zone5” {
endpoints = [ “icinga-s6.mon.domain”, “icinga-s7.mon.domain” ]parent = “master”
}object Zone “global-templates” {
global = true
}object Zone “director-global” {
global = true
}
yes i know the satellites can have one-way connection by commenting the other satellite but all the other satellite zones have the same set-up and have been fine for years.
the satellite host objects are defined in the zones.d/master/satellite-zone5.conf so get synched to second master without having to update the zones.conf on both whenever a new satellite zone is added. again this has been working completely fine for years.
how do i get these two satellites to stay connected to each other?
masters:
icinga2 r2.11.2-1
icingaweb 2.7.3
satellites:
icinga2 r2.10.5-1
yes i know we should run the same version but this is a production environment and can’t afford downtime for upgrade testing - and it’s been operating just fine for a couple of years so please don’t recommend updating.
