Satellite sees one agent as zone and failed, because its redefined

Icinga 2
1

My setup is standard master-satellite(s)-agents. It works well since years. Now I’ve added some new hosts to one of the satellites and (only) one agent makes trouble and I don’t know, where to look for the reason.

On master there is a zones.conf with

object Endpoint "universe" {
  host = "<ip>"
  port = "5665" 
}

object Zone "universe" {
  endpoints = [ "universe" ]
}

object Endpoint "sun" {
    host = "<ip>"
    port = "5665"
}

object Zone "sun" {
    parent = "universe"
    endpoints = [ "sun" ]
}
# several...other zones

Then there is a zones.d/sun folder with three files

object Zone "mercury.sol.home" {
    endpoints = [ "mercury.sol.home" ]
    parent = "sun"
}

object Endpoint "mercury.sol.home" {
    host = "<ip>"
    port = "5665"
}

object Zone "venus.sol.home" {
    endpoints = [ "venus.sol.home" ]
    parent = "sun"
}

object Endpoint "venus.sol.home" {
    host = "<ip>"
    port = "5665"
}

object Zone "saturn.sol.home" {
    endpoints = [ "saturn.sol.home" ]
    parent = "sun"
}

object Endpoint "saturn.sol.home" {
    host = "<ip>"
    port = "5665"
}
...

the hosts.conf contains the host objects, and the services.conf contains a lot of assign service rules. It looks like 1 day ago it worked. But today I see on “sun” in icinga log

Config validation failed for staged cluster config sync in '/var/lib/icinga2/api/zones-stage/'. Aborting. Logs: '/var/lib/icinga2/api//zones-stage-startup-last-failed.log'

and failed log

Error: Object 'saturn.sol.home' of type 'Host' re-defined: in /var/lib/icinga2/api/zones-stage//saturn.sol.home/_etc/hosts.conf: 3:1-3:29; previous definition: in /var/lib/icinga2/api/zones-stage//sun/_etc/hosts.conf: 131:1-131:29

And thats correct, the generated files contain host definition on sun and there is a zone containing the host “saturn”. But I dont understand, why this happens and why this is not happens on all other satellites and there is no zone venus or mercury.

Any idea where to look?

2

You define zones in zones which is no longer working since V2.11.

3

Aha, ok, did not know that. I’m not really sure, if I understood the documentation. It is a little bit confusing to me. There is a zone master/universe a satellite zone “sun” whcih contain agents. And yes, then there is a zone “saturn” and I don’t need to define a zone for that? And only because it is defined in the folder structure for satellite “sun” its known as an agent for zone “sun” (because in my understanding that was the main reason to define a zone, because you can define a “parent-zone”)

But why is the rest of the setup is still working then?

4

Zones needs to be defined outside the zones directories in zones.d. I’ve no idea why it is working partly.

5

Zones needs to be defined outside the zones directories in zones.d

I don’t read that in the documentation. Concrete example from the page:

mkdir /etc/icinga2/zones.d/master
vim /etc/icinga2/zones.d/master/satellite.conf

object Endpoint “icinga2-satellite1.localdomain” { … }
object Endpoint “icinga2-satellite2.localdomain” { … }

object Zone “satellite” {
endpoints = [ “icinga2-satellite1.localdomain”, “icinga2-satellite1.localdomain” ]
parent = “master”
}

So, there is a zone definition in zones.d.

I run icinga2 object list --name saturn.sol.home and get the shortened output:

Object 'saturn.sol.home' of type 'Zone':
  % declared in '/etc/icinga2/zones.d/sun/zones.conf', lines 23:1-23:29
  * __name = "saturn.sol.home"
  * endpoints = [ "saturn.sol.home" ]
  * global = false
  * name = "saturn.sol.home"
  * package = "_etc"
  * parent = "sun"
  * source_location
    * path = "/etc/icinga2/zones.d/sun/zones.conf"
  * templates = [ "saturn.sol.home" ]
  * type = "Zone"
  * zone = "sun"

Object 'saturn.sol.home' of type 'Endpoint':
  % declared in '/etc/icinga2/zones.d/sun/zones.conf', lines 28:1-28:33
  * __name = "saturn.sol.home"
  * host = "saturn.sol.home"
  * log_duration = 86400
  * name = "saturn.sol.home"
  * package = "_etc"
  * port = "5665"
  * source_location
    * path = "/etc/icinga2/zones.d/sun/zones.conf"
  * templates = [ "saturn.sol.home" ]
  * type = "Endpoint"
  * zone = "sun"

Object 'saturn.sol.home' of type 'Host':
  % declared in '/etc/icinga2/zones.d/sun/hosts.conf', lines 131:1-131:29
  * __name = "saturn.sol.home"
  * action_url = ""
  * address = "192.168.10.2"
  * address6 = ""
  * check_command = "hostalive"
  * check_interval = 60
  * check_period = "8to10"
  * check_timeout = null
  * command_endpoint = ""
  * display_name = "saturn.sol.home"
6

The problem with failed.log is solved. The config was generated, and during the first run, I didn’t defined saturn as a child of sun, so the generator creates a saturn-zone under universe, that was written to /var/lib/icinga2/api/zones on the master. After “moving” saturn in our CMDB under “sun” the configuration was recreated. But somehow, the original zone under /var/lib/icinga2/api/zones was not removed.

PS: I love that naming convention :wink: