Top down config sync in three level cluster. Sattelites won't sync configuration to agents

Hi,

I maybe misunderstood the documentation but it seems the satellites, in the three-level cluster setup, don’t support config sync to agents.

My aim is to have a three-level cluster similar to described in the documentation but I’d like to have some agents checked by themself, not by the satellite (Top Down Config Sync).


Environment:

  • Linux, Debian - Buster.
  • Icinga version: 2.12.0-1.buster
  • Configuration realized by hand, no director.

Bellow is example configuration similar to one I use.

root@master# cat /etc/icinga2/zones.conf

object Zone "global-templates" {
  global = true
}

object Endpoint "master_server" {
//  that's us
}

object Endpoint "satellite_server" {
  host = "satellite_server"
}

object Endpoint "agent" {
}

object Zone "master" {
  endpoints = [ "master_server" ]
}

object Zone "satellite" {
  parent = "master"
  endpoints = [ "satellite_server" ]
}


object Zone "agent" {
  parent = "satellite"
  endpoints = [ "agent" ]
}


root@master# ls /etc/icinga2/zones.d/

global-templates/ master/ satellite/

root@master# ls /etc/icinga2/zones.d/satellite/

satellite_server.conf agent.conf

root@master# cat /etc/icinga2/zones.d/satellite/agent.conf

object Host "agent" {
  import "generic-host"

  address = "agent"
  zone = "satellite"
  vars.os = "Linux"
  vars.distro = "Debian"
  vars.agent_type = "Icinga"


object Service "disk" {
  import "generic-service"

  host_name = "agent"
  check_command = "disk"

}

root@satellite# cat /etc/icinga2/zones.conf

object Endpoint "master" {
// Connect to us
}


object Endpoint "satellite" {
// That's us
}

object Endpoint "agent" {
    host = "agent"
}

object Zone master {
  endpoints = [ "master_server" ]

object Zone satellite {
  parent = "master"
  endpoints = [ "satellite_server" ]
}

object Zone agent {
  endpoints = [ "agent" ]
  parent = "satellite"

object Zone "global-templates" {
  global = true
}


root@agent# cat /etc/icinga2/zones.conf

object Endpoint "satellite" {
// Connect to us
}

object Endpoint "agent" {
// That's us
}

object Zone "satellite" {
    endpoints = [ "satellite_server" ]
}

object Zone "agent" {
  endpoints = [ "agent" ]
  parent = "satellite"
}

object Zone "global-templates" {
	global = true
}

The satellite zone will be synced to the satellite node with no problem but on the agent node only the global-templates are in sync.

So, do I misunderstood the documentation, and thus the configuration is valid but the setup is not in the scope of the Icinga, or is there anything wrong with my config?

I realize I can create a separate zone just for the agent, so it will be connected directly to the master but I’d like to agent in the satellite zone because of some other requirements (some agents are not accessible from the master for example.)

My guess is that the agent cannot receive the “satellite zone” configuration because it is the superior zone, in that case, I would need something like a zone for the agent inside the satellite zone. I’ve tried that approach but that doesn’t work It seems the satellite is not aware of the “child zone” which should be synced to the agent(child).

Thank you in advance for any help or clarification.

Your

object Zone “satellite”

does not match

parent = “satellite_zone”

typo. Fixed. Problem still persist. Can you confirm the setup I am trying to create is valid?

Instead of defining service the way you are doing I’d recommend to get familiar with apply and move those objects to a global zone. This will make life easier.

Thanks for idea but host configuration needs to be separated (agent.conf > object Host) so even If I will use apply logic it will not help with missing host configuration on agent node.

That’s right and as already mentioned this would need manually steps and some tricky work arounds.

Does it mean that by default Top Down Config Sync From master to satellite and from satellite to agent is not supported?

I just need to known if the idea is valid or not.

I can think of at least two solutions how to work around that’s not a problem.

Sync to satellites is supported as described here but not to agents.

Thank you. It is not clear from documentation or is it? Maybe I am just blind… :slight_smile:

Just as an idea (I’ve never tried it): create a directory for the agent under zones.d and put it’s host conf in there. Create a symbolic link in the corresponding satellite directory to that host conf.

Even if it’s working, as this is not intended use a upcoming upgrade might break it in future.

Smart idea, unfortunatelly. it will not work because of the symlink the configuration is redundant and that’s not valid from point of Icinga.

I’ve tried something similar. I created zone inside zone and hoped the satellite will sync the nested to zone to agent, but that didn’t work.

So, I will create separated zones for hosts which I’d like to sync config to and treat the rest of the infra as described in three-level cluster scenario seems like most suitable and the distribution still lies in Icinga mechanisms.

Thanks for help, I really appreciate it.

The fact that top-down config sync only works for one level (Master to Agents,
or Master to Satellite, but not then on to Agents) has always struck me as:

  1. not sufficiently clearly documented

  2. a flaw in the way Icinga operates.

After all, anyone who sets up a Master with a few Satellites and then several
Agents is very likely to want to manage the whole setup from the Master, so
not ending up with the apparently configured service checks actually being
performed on the Agents is firstly a surprise, and then a configuration
management inconvenience.

Is it planned to support full end-to-end configuration propagation across
multiple levels in a future version?

Antony.

1 Like

Are you sure about your facts? Host objects are never synced to any agent incl. first level. Second, (haven’t been trying, but I’d assume this is true) host objects are synced the whole chain in case you have more then one satellite level.

Well, I have two quite independent Icinga setups here:

One has a Master with around 25 Agents, all performing service checks locally,
and the config on the Master definitely gets propagated down to the Agents
(otherwise they wouldn’t know what service checks to perform).

The other is a Master with Four Satellites, and each Satellite has between 5
and 50 Agents under it. The Master syncs to the Satellites, but the Agents
don’t get that synced onwards to them (which also seems to me to match up with
your comment earlier in this thread, that “Sync to satellites is supported as
described [here] but not to agents.”).

Antony.

I’m talking about host objects only. Could you please check if you have them on any agent:

icinga2 object list -t host

Checks are scheduled by your master.

Right. No, I don’t.

Antony.