How to debug a configuration issue?

Any hints are welcome for debugging the following problem:

I’ve got a custom checkcommand defined like this:

object CheckCommand "uptime1" {
        command = [ PluginDir + "/check_uptime" ]

        arguments = {
        "--warning" = {
            value = "$uptime1_warning$"
            required = true
        }
        "--critical" = {
            value = "$uptime1_critical$"
            required = true
        }
        "--perfparse" = {
            set_if = "$uptime1_perfdata$"
            description = "Add performance data"
        }
    }
    // some defaults
    vars.uptime1_perfdata = true
    vars.uptime1_warning = "15"
    vars.uptime1_critical = "5"
}

Now, when using this CheckCommand in an apply-rule like e.g.
then the check fails.

apply Service "uptime" {
  check_command = "uptime1"
  import "ssh-service"

  vars += host.vars.uptime1

  assign where (host.address || host.address6) && host.vars.os == "Linux"
}

Icinga complains:
Error: Non-optional macro ‘uptime1_critical’ used in argument ‘–critical’ is missing.

(The template “ssh-service” is coming from here: https://blog.netways.de/2016/03/21/check_by_ssh-mit-icinga-2/)

Apparently the variable with the default parameter is not being passed through.
What’s even stranger is that the same setup works for other checkcommands.
I have no clue what’s different here.

How do I go about debugging this problem?
Thanks for any hints!
Andy

Hello Andy,
You are receiving this error because the check command argument “required” field is set to “true” for the critical and warning argument. Set the required field to false to get past this error.

Read more details here.

You have a default value set in the check command object for the critical value, so not need to set the required field to true.

True, but - no matter whether required set to true or false - the command option is not being used.
Somehow Icinga doesn’t see the default value which I set.

If you change your code to this does it work?

object CheckCommand "uptime1" {
        command = [ PluginDir + "/check_uptime" ]

        arguments = {
        "--warning" = {
            value = "$uptime1_warning$"
            required = true
        }
        "--critical" = {
            value = "$uptime1_critical$"
            required = true
        }
        "--perfparse" = {
            set_if = "$uptime1_perfdata$"
            description = "Add performance data"
        }
    }
    // some defaults
    vars.uptime1_perfdata = true
}
apply Service "uptime" {
  check_command = "uptime1"
  import "ssh-service"

  vars.uptime1_warning = "15"
  vars.uptime1_critical = "5"

  assign where (host.address || host.address6) && host.vars.os == "Linux"
}

Regards
Alex

Yes, but the option --perfparse isn’t added to the command line. :sleepy:

If I remove this line:

import "ssh-service"

then the variables get passed through as they should. So it must have to do with the by_ssh-template.

I thought this line takes care of this in the by_ssh-template:
vars.by_ssh_arguments = {{ get_check_command(service.vars.original_check_command).arguments }}

Am I wrong?

I’m still struggling with this. :cold_face:

Anyone else has a hint for me on how to debug that?

Hello,

Maybe you post also the template. Makes it easier to help you.

Regards,
Carsten

Oh, sure.

template Service "ssh-service" {
  import "generic-service"

  // "save" original command name, and replace it
  vars.original_check_command = check_command
  check_command = "by_ssh"
  vars.by_ssh_port = "5522"
  vars.by_ssh_quiet = 1
  vars.by_ssh_timeout = 90

  // these get evaluated at runtime
  vars.by_ssh_command = {{ get_check_command(service.vars.original_check_command).command }}
  vars.by_ssh_arguments = {{ get_check_command(service.vars.original_check_command).arguments }}

  // for integration with graphite
  // https://github.com/Icinga/icingaweb2-module-graphite/blob/master/doc/03-Configuration.md
  // To make the respective graphs work as expected
  // you have to tell the monitored object's "actual" check command
  // by setting its custom variable "check_command"
  vars.check_command = vars.original_check_command
}

Hello,

i changed the check command arguments = to arguments += and added vars.uptime1_perfdata = true to your service. The trick only handles variables that are defined in the service and not if they are defined in the command.

Final command:

object CheckCommand "uptime1" {
        command = [ PluginDir + "/check_uptime" ]

        arguments += {
        "--warning" = {
            value = "$uptime1_warning$"
            required = true
        }
        "--critical" = {
            value = "$uptime1_critical$"
            required = true
        }
        "--perfparse" = {
            set_if = "$uptime1_perfdata$"
            description = "Add performance data"
        }
    }
    // some defaults
    vars.uptime1_perfdata = true
}

and the service:

apply Service "uptime" {
  check_command = "uptime1"
  import "ssh-service"

  vars.uptime1_warning = "15"
  vars.uptime1_critical = "5"
  vars.uptime1_perfdata = true

  assign where (host.address || host.address6) && host.vars.os == "Linux"
}

Output (i did not install ssh keys etc.)

To debuf such things in the future you can use the log function to write it to logfile or console
Regards,
Carsten

Interesting. What is the reason for the + ?

Yes, but I want to avoid having to set all default values again.
If you don’t add that line, does the service pick up the default value from the command definition and thus add the “–perfparse” parameter?

Oh, too bad. But in other commands it works.
I have the feeling it only works if I use it within a “for” loop.

Here’s an example:

object CheckCommand "lm-sensors" {
    command = [ PluginLocalDir + "/check_lm_sensors" ]

    arguments = {
        "--nodrives" = {
            set_if = "$lm_sensors_nodrives$"
        }
        "--sanitize" = {
            set_if = "$lm_sensors_sanitize$"
        }
        "--high" = {
            value = "$lm_sensors_high$"
            description = "specifies a check for a sensor value which is too high"
        }
        "--low" = {
            value = "$lm_sensors_low$"
            description = "specifies a check for a sensor value which is too low"
        }
        "--range" = {
            value = "$lm_sensors_range$"
            description = "specifies a check for a sensor value which should stay in a given range"
        }
    }

    vars.lm_sensors_nodrives = true
    vars.lm_sensors_sanitize = true
    vars.lm_sensors_high = "Core0=60,90"
}

And the service:

apply Service for (lmsensor => config in host.vars.lmsensors) {
  check_command = "lm-sensors"
  import "ssh-service"
        
  vars += config
}

Why does it pick up the default values in this case?

Thanks,
Andy

Nice! So I would add sth like
log(LogCritical, "Console", vars.uptime1_perfdata)
to my command (or service?) definition?

How exactly did you get this output of the plugin?
Thank you!
Andy

Hello @ciberandy,

the += adds/merges all variables under the variable arguments or vars and does not complete overwrite the arguments or vars. For example if you have a command that imports another command and adds only 1 argument. If you write arguments = ... all the arguments from the import are gone and only the one you defined in the new command is there. If you use arguments += the variables from arguments are merged.

Why it works in a foor loop you have to ask the developers, maybe @theFeu can get the information from one.

Regards,
Carsten

Do you have a hint for me where I can find usage examples of debugging with “log”?
How exactly did you get this explicit output of the plugin?

Thanks
Andy