How to configure fallback string/state for passive check

Hi all,

I have a passive check in Icinga defined. This check works fine, however after adding the check to a server, if no passive check results have been received yet I get an error.

This is the check as it is defined:

apply Service "borgmatic" {
  check_interval = 18h
  enable_passive_checks = true
  enable_active_checks = true

  max_check_attempts = 1

  check_command = "dummy"

  vars.dummy_state = {{
    var service = get_service(macro("$host.name$"), macro("$service.name$"))
    if (service.last_check + service.last_check_result.ttl < get_time()) {
      var state = 3
    } else {
      var state = service.last_state
    }
    return state
  }}

  vars.dummy_text = {{
    var service = get_service(macro("$host.name$"), macro("$service.name$"))
    var last_text = service.last_check_result.output
    if (service.last_check + service.last_check_result.ttl < get_time()) {
      if (last_text.contains("STALE:")) {
        var text = last_text
      } else {
        var text = "STALE: No passive check result received. Last output: " + last_text
      }
    } else {
      var text = last_text
    }
    return text
  }}

  vars.generic_notify = false
  vars.day_notify = false
  vars.night_notify = false
  assign where host.address && host.vars.borgmatic
}

With this, I get the following error when no passive check results have been received yet:

Exception occurred while checking 'myserver!borgmatic': Error: Argument is not a callable object.
Location: in /var/lib/icinga2/api/zones/global-templates/_etc/borgmatic.conf: 26:11-26:38
/var/lib/icinga2/api/zones/global-templates/_etc/borgmatic.conf(24):     var last_text = service.last_check_result.output
/var/lib/icinga2/api/zones/global-templates/_etc/borgmatic.conf(25):     if (service.last_check + service.last_check_result.ttl < get_time()) {
/var/lib/icinga2/api/zones/global-templates/_etc/borgmatic.conf(26):       if (last_text.contains("STALE:")) {
                                                                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
/var/lib/icinga2/api/zones/global-templates/_etc/borgmatic.conf(27):         var text = last_text
/var/lib/icinga2/api/zones/global-templates/_etc/borgmatic.conf(28):       } else {

	(0) Resolving macros for string '$dummy_text$'
	(1) Executing check for object 'myserver!borgmatic'

I tried to change the script so that when no previous check results can be retrieved it does a fallback to a predefined string. However with this I get infinite recursion errors so sadly it does not work:

apply Service "borgmatic" {
  check_interval = 18h
  enable_passive_checks = true
  enable_active_checks = true

  max_check_attempts = 1

  check_command = "dummy"

  vars.dummy_state = {{
    try {
      var service = get_service(macro("$host.name$"), macro("$service.name$"))
      if (service.last_check + service.last_check_result.ttl < get_time()) {
        var state = 3
      } else {
        var state = service.last_state
      }
      return state
    } except {
      var state = 3
      return state
    }
  }}

  vars.dummy_text = {{
    try {
      var service = get_service(macro("$host.name$"), macro("$service.name$"))
      var last_text = service.last_check_result.output
      if (service.last_check + service.last_check_result.ttl < get_time()) {
        if (last_text.contains("STALE:")) {
          var text = last_text
        } else {
          var text = "STALE: No passive check result received. Last output: " + last_text
        }
      } else {
        var text = last_text
      }
      return text
    } except {
      var text = "UNKNOWN: No previous check result found."
      return text
    }
  }}

  vars.generic_notify = false
  vars.day_notify = false
  vars.night_notify = false
  assign where host.address && host.vars.borgmatic
}

Does anyone have an idea of how I should configure this?

Cheers,
Niels

Hi,

better check whether service.last_check_result is not null and return early. try/catch may not work in this regard.

Cheers,
Michael

Hi Michael,

Cheers, I tried that as well but I received a similar issue. Please see below:

Exception occurred while checking 'myserver!borgmatic': Error: Error while evaluating expression: Infinite recursion detected while resolving macros
Location: in /var/lib/icinga2/api/zones/global-templates/_etc/borgmatic.conf: 25:31-25:50
/var/lib/icinga2/api/zones/global-templates/_etc/borgmatic.conf(23): 
/var/lib/icinga2/api/zones/global-templates/_etc/borgmatic.conf(24):   vars.dummy_text = {{
/var/lib/icinga2/api/zones/global-templates/_etc/borgmatic.conf(25):     var service = get_service(macro("$host.name$"), macro("$service.name$"))
                                                                                                   ^^^^^^^^^^^^^^^^^^^^
/var/lib/icinga2/api/zones/global-templates/_etc/borgmatic.conf(26):     if (!service || !service.last_check_result) {
/var/lib/icinga2/api/zones/global-templates/_etc/borgmatic.conf(27):       var text = "UNKNOWN: No previous backup results reported."

  (0) Resolving macros for string 'Exception occurred while checking 'myserver!borgmatic': Error: Error while evaluating expression: Infinite recursion detected while resolving macros
Location: in /var/lib/icinga2/api/zones/global-templates/_etc/borgmatic.conf: 25:31-25:50
/var/lib/icinga2/api/zones/global-templates/_etc/borgmatic.conf(23): 
/var/lib/icinga2/api/zones/global-templates/_etc/borgmatic.conf(24):   vars.dummy_text = {{
/var/lib/icinga2/api/zones/global-templates/_etc/borgmatic.conf(25):     var service = get_service(macro("$host.name$"), macro("$service.name$"))
                                                                                                   ^^^^^^^^^^^^^^^^^^^^
/var/lib/icinga2/api/zones/global-templates/_etc/borgmatic.conf(26):     if (!service.last_check_result) {
/var/lib/icinga2/api/zones/global-templates/_etc/borgmatic.conf(27):       var text = "UNKNOWN: No previous backup results reported."

  (0) Resolving macros for string 'Exception occurred while checking 'myserver!borgmatic': Error: Error while evaluating expression: Infinite recursion detected while resolving macros
Location: in /var/lib/icinga2/api/zones/global-templates/_etc/borgmatic.conf: 25:31-25:50
/var/lib/icinga2/api/zones/global-templates/_etc/borgmatic.conf(23): 
/var/lib/icinga2/api/zones/global-templates/_etc/borgmatic.conf(24):   vars.dummy_text = {{
/var/lib/icinga2/api/zones/global-templates/_etc/borgmatic.conf(25):     var service = get_service(macro("$host.name$"), macro("$service.name$"))
                                                                                                   ^^^^^^^^^^^^^^^^^^^^
/var/lib/icinga2/api/zones/global-templates/_etc/borgmatic.conf(26):     var last_text = service.last_check_result.output
/var/lib/icinga2/api/zones/global-templates/_etc/borgmatic.conf(27):     if (!service.last_check_result) {

  (0) Resolving macros for string 'Exception occurred while checking 'myserver!borgmatic': Error: Argument is not a callable object.

And this is the check definition which I belive is as you recommend it ?

apply Service "borgmatic" {
  check_interval = 18h
  enable_passive_checks = true
  enable_active_checks = true

  max_check_attempts = 1

  check_command = "dummy"

  vars.dummy_state = {{
    var service = get_service(macro("$host.name$"), macro("$service.name$"))
    if (!service || !service.last_check_result) {
      var state = 3
    } else if (service.last_check + service.last_check_result.ttl < get_time()) {
      var state = 3
    } else {
      var state = service.last_state
    }
    return state
  }}

  vars.dummy_text = {{
    var service = get_service(macro("$host.name$"), macro("$service.name$"))
    if (!service || !service.last_check_result) {
      var text = "UNKNOWN: No previous backup results reported."
      return text
    } else {
      var last_text = service.last_check_result.output
      if (service.last_check + service.last_check_result.ttl < get_time()) {
        if (last_text.contains("STALE:")) {
          var text = last_text
        } else {
          var text = "STALE: No passive check result received. Last output: " + last_text
        }
      } else {
        var text = last_text
      }
      return text
    }
  }}

  vars.generic_notify = false
  vars.day_notify = false
  vars.night_notify = false
  assign where host.address && host.vars.borgmatic
}

Hi,

Does anyone have another idea to fix this?
Cheers

Mh I don’t know exactly where the recursion comes from, but I’d guess this line:

var service = get_service(macro("$host.name$"), macro("$service.name$"))

creates a reference to itself. So var service points to the object, but is declared inside the object. So called “DSL inception”.

I actually don’t know whether we may self-reference ourselves here. Since anonymous lambdas don’t allow to pass the locals or this context, one would likely need to use closures with 2 function objects. But I am not sure if you can follow up to this point, so I’ll wait.

Cheers,
Michael