Discrepancy between check_disk in web console and running check locally

Service Monitoring
1

I’m having an issue where I have a simple check_disk service defined on our hosts as follows:

apply Service "disk" {
  check_command = "disk"
  vars.disk_cfree = "8%"
  vars.disk_wfree = "10%"
  vars.disk_ereg_path = [ "/" ]
  vars.disk_ignore_eregi_path = [ "/snap.*", "/var/lib/docker.*" ]
  command_endpoint = host.vars.agent_endpoint
  assign where host.address
}

When I run the check_disk plugin directly on a host with these same options, it returns OK. However, in the Icinga web console, the check is permanently stuck in a Soft 1/3 CRITICAL state. However, if I use the “Check now” link in the web console, the check returns OK. Then the next time the check runs automatically, it goes back to the Soft 1/3 CRITICAL state.

Our main Icinga server is running version 2.12.0-1, and most if not all of our hosts are on 2.8.1-1. Icingaweb2 is on 2.8.2.

I’m at a loss for why this is happening, so any help is greatly appreciated.

2

Hi, realy can’t say why that is happening, as the soft state should not stuck on 1/3. But your master and your agent are so hard out of version, i’am pretty sure that this is not supported as 2.11- 2.12 changes so much on agent communication, i’am not sure that this can work.

I would test to update the agent on one of the systems that are affected, just to be sure.

3

I can test upgrading one of the agents, but I’m not convinced that is the issue, as we have hundreds of other service checks that work just fine. It’s only the check disk one that is having issues.