CheckCommands does not exist

Hi all,
I’ve just setup Icinga2 using your Puppet module and everything seems to be working great except for the fact that my check commands don’t seem to be ending up on agents.
Interestingly if I look in /var/lib/icinga2/zones-stage/_etc/global-commands I can find my file, but it doesn’t seem to be making it over to the zones proper folder:

root@docker01:~# ls /var/lib/icinga2/api/zones/global-commands/
root@docker01:~# ls /var/lib/icinga2/api/zones-stage/global-commands/_etc/
check_mem.conf

I was wondering if someone could point me in the direction of where I’m going wrong?

Here’s some of my configuration:
zones.conf

object Endpoint "docker01.testdomain.local" {
  host = "docker01.testdomain.local"
}
object Endpoint "monitoring01.testdomain.local" {
  host = "monitoring01.brownserve.co.uk"
}
object Zone "docker01.testdomain.local" {
  endpoints = [ "docker01.testdomain.local", ]
  parent = "master"
}
object Zone "global-commands" {
  global = true
}
object Zone "global-notifications" {
  global = true
}
object Zone "global-services" {
  global = true
}
object Zone "global-templates" {
  global = true
}
object Zone "global-users" {
  global = true
}
object Zone "master" {
  endpoints = [ "monitoring01.testdomain.local", ]
}

the check_mem command configuration:

# This file is managed by Puppet. DO NOT EDIT.

object CheckCommand "check_mem" {
  import "plugin-check-command"

  command = [ PluginDir + "/check_mem.pl", ]
  arguments = {
    "-f" = {
      set_if = "$check_free$"
      order = -1
    }
    "-u" = {
      set_if = "$check_used$"
      order = 1
    }
    "-w" = {
      value = "$mem_warn$"
      order = 2
    }
    "-c" = {
      value = "$mem_crit$"
      order = 3
    }
    "-C" = {
      set_if = "$mem_cache$"
      order = 4
    }
  }
}

Puppet copies the file over to the agent system:

root@docker01:~$ find /usr/lib/nagios/plugins/ -iname check_mem.pl
/usr/lib/nagios/plugins/check_mem.pl

Service definitition:

apply Service "memory-linux" to Host {
  import "ServiceTemplate"

  check_command = "check_mem"
  command_endpoint = host.vars.client_endpoint
  assign where host.vars.kernel == "Linux" && host.vars.memory

  if(host.vars.memory.max_check_attempts) {
    max_check_attempts = host.vars.memory.max_check_attempts
  }

  if(host.vars.memory.check_interval) {
    check_interval = host.vars.memory.check_interval
  }

  if(host.vars.memory.retry_interval) {
    retry_interval = host.vars.memory.retry_interval
  }

  if(host.vars.memory.check_free) {
    vars.check_free = host.vars.memory.check_free
  }

  if(host.vars.memory.check_used) {
    vars.check_used = host.vars.memory.check_used
  }

  if(host.vars.memory.mem_warn) {
    vars.mem_warn = host.vars.memory.mem_warn
  }
  
  if(host.vars.memory.mem_crit) {
    vars.mem_crit = host.vars.memory.mem_crit
  }

  if(host.vars.memory.mem_cache) {
    vars.mem_cache = host.vars.memory.mem_cache
  }
}

If there’s any other information you need then please let me know.
I’m pretty new to Icinga2.

Thanks in advance.

Typically after posting this I found the solution.

It seems one of my configuration files that should only be for the master zone had made it’s way into one of the global zones resulting in the sync failing.
I was able to find the offending configuration by checking:
/var/lib/icinga2/api/zones-stage//startup.log (note the double backslash)
From here I could see that I was getting an error about a timeperiod configuration that I had inadvertently put in global-users, once I moved it to the master zone everything is now working as it should.

Hopefully this helps someone in the future