How to set the refresh interval for a passive check?

[j4n]

I have an icinga2 server as listed below; I’ve set it up with mostly passive checks, reporting through the REST API, which works fine. There is just one problem: The checks become UNKNOWN too soon, and I can’t find where to change it.

I use a python script to update the services, but this is what is sent to the server, printed out by the script:

service.report: https://vogon.comind.io:5665/v1/actions/process-check-result?service=knox!checkcpu
data: {"exit_status": 0, "plugin_output": "OK: CPU usage 24%", "check_command": ["vmstat 1 2|tail -1|awk '{print $15}'"]}

Comparing to some of the examples in the API docs, there seems to be more parameters than what is listed for process-check-result; which parameter controls the refresh interval?

root@vogon:~# icinga2 --version
icinga2 - The Icinga 2 network monitoring daemon (version: r2.14.0-1)

Copyright (c) 2012-2023 Icinga GmbH (https://icinga.com/)
License GPLv2+: GNU GPL version 2 or later <https://gnu.org/licenses/gpl2.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

System information:
  Platform: Debian GNU/Linux
  Platform version: 11 (bullseye)
  Kernel: Linux
  Kernel version: 5.10.0-25-amd64
  Architecture: x86_64

Build information:
  Compiler: GNU 10.2.1
  Build host: runner-hh8q3bz2-project-575-concurrent-0
  OpenSSL version: OpenSSL 1.1.1w  11 Sep 2023

Application information:

General paths:
  Config directory: /etc/icinga2
  Data directory: /var/lib/icinga2
  Log directory: /var/log/icinga2
  Cache directory: /var/cache/icinga2
  Spool directory: /var/spool/icinga2
  Run directory: /run/icinga2

Old paths (deprecated):
  Installation root: /usr
  Sysconf directory: /etc
  Run directory (base): /run
  Local state directory: /var

Internal paths:
  Package data directory: /usr/share/icinga2
  State path: /var/lib/icinga2/icinga2.state
  Modified attributes path: /var/lib/icinga2/modified-attributes.conf
  Objects path: /var/cache/icinga2/icinga2.debug
  Vars path: /var/cache/icinga2/icinga2.vars
  PID path: /run/icinga2/icinga2.pid

• Operating System and version

root@vogon:~# cat /etc/*release*
PRETTY_NAME="Debian GNU/Linux 11 (bullseye)"
NAME="Debian GNU/Linux"
VERSION_ID="11"
VERSION="11 (bullseye)"
VERSION_CODENAME=bullseye
ID=debian
HOME_URL="https://www.debian.org/"
SUPPORT_URL="https://www.debian.org/support"
BUG_REPORT_URL="https://bugs.debian.org/"

• Enabled features (icinga2 feature list)

root@vogon:~# icinga2 feature list
Disabled features: compatlog debuglog elasticsearch gelf graphite influxdb influxdb2 journald livestatus opentsdb perfdata statusdata syslog
Enabled features: api checker command icingadb mainlog notification

• Icinga Web 2 version and modules (System - About)

Icinga Web 2 Version 	2.12.0
Git commit 	7cd79a5c7ff66a6a67ecd55875c214ba4b2c7db4
PHP Version 	7.4.33
Git commit date 	2023-09-21
Get Icinga Support
Icinga Community
Report a bug
Icinga Documentation
Pending Migrations
Icinga Web 	1
Show all
Loaded Libraries
icinga/icinga-php-library 	0.13.0
icinga/icinga-php-thirdparty 	0.12.0
Loaded Modules
doc 		2.12.0 	Configure
icingadb 		1.1.0 	Configure
pdfexport 		0.10.2 	Configure
reporting 		0.10.0 	Configure

• Config validation (icinga2 daemon -C)

root@vogon:~# icinga2 daemon -C
[2023-10-16 11:41:00 +0000] information/cli: Icinga application loader (version: r2.14.0-1)
[2023-10-16 11:41:00 +0000] information/cli: Loading configuration file(s).
[2023-10-16 11:41:00 +0000] information/ConfigItem: Committing config item(s).
[2023-10-16 11:41:00 +0000] information/ApiListener: My API identity: vogon.comind.io
[2023-10-16 11:41:00 +0000] information/ConfigItem: Instantiated 2 NotificationCommands.
[2023-10-16 11:41:00 +0000] information/ConfigItem: Instantiated 16 Notifications.
[2023-10-16 11:41:00 +0000] information/ConfigItem: Instantiated 1 IcingaApplication.
[2023-10-16 11:41:00 +0000] information/ConfigItem: Instantiated 21 Hosts.
[2023-10-16 11:41:00 +0000] information/ConfigItem: Instantiated 2 HostGroups.
[2023-10-16 11:41:00 +0000] information/ConfigItem: Instantiated 1 Downtime.
[2023-10-16 11:41:00 +0000] information/ConfigItem: Instantiated 1 IcingaDB.
[2023-10-16 11:41:00 +0000] information/ConfigItem: Instantiated 1 FileLogger.
[2023-10-16 11:41:00 +0000] information/ConfigItem: Instantiated 3 Zones.
[2023-10-16 11:41:00 +0000] information/ConfigItem: Instantiated 1 CheckerComponent.
[2023-10-16 11:41:00 +0000] information/ConfigItem: Instantiated 1 ExternalCommandListener.
[2023-10-16 11:41:00 +0000] information/ConfigItem: Instantiated 1 Endpoint.
[2023-10-16 11:41:00 +0000] information/ConfigItem: Instantiated 1 ApiUser.
[2023-10-16 11:41:00 +0000] information/ConfigItem: Instantiated 1 NotificationComponent.
[2023-10-16 11:41:00 +0000] information/ConfigItem: Instantiated 1 ApiListener.
[2023-10-16 11:41:00 +0000] information/ConfigItem: Instantiated 249 CheckCommands.
[2023-10-16 11:41:00 +0000] information/ConfigItem: Instantiated 3 TimePeriods.
[2023-10-16 11:41:00 +0000] information/ConfigItem: Instantiated 1 UserGroup.
[2023-10-16 11:41:00 +0000] information/ConfigItem: Instantiated 1 User.
[2023-10-16 11:41:00 +0000] information/ConfigItem: Instantiated 155 Services.
[2023-10-16 11:41:00 +0000] information/ConfigItem: Instantiated 3 ServiceGroups.
[2023-10-16 11:41:00 +0000] information/ConfigItem: Instantiated 1 ScheduledDowntime.
[2023-10-16 11:41:00 +0000] information/ScriptGlobal: Dumping variables to file '/var/cache/icinga2/icinga2.vars'
[2023-10-16 11:41:01 +0000] information/cli: Finished validating the configuration file(s).

[moreamazingnick]

use if your passive check cron is set to send every 5m I would suggest this:

check_command = "passive"
max_check_attempts =1
check_interval = 10m
enable_active_checks = true
enable_passive_checks = true

I Icinga Director these settings can only be set in the Service Template
You check_interval must be larger than the RestApi request interval

here some posts you can read too:

Check Interval at 2hrs with cron job every 2 hrs - Service goes stale briefly before receiving input

I have a passive service that has a check_interval at 2h I am using a passive service to send disk space used in CRON every 2hrs, 0000, 0200, 0400 etc. The service is going purple/stale almost exactly at the same time the new data is published. I read the manual and it says the execution time is used in calculation. With this being a passive test I assume this means the execution time is 0 and it is exactly like it says, 2hrs exactly. I have resolved this by changing the check interval to 2…

Remote host - Passive Check - HARD state

Hello, We have passive service checks setup for a few hosts. The passive check result are sent from a remote host to Icinga api. However, the state is “SOFT;1” for “Critical” status. I would like to have it as “HARD”. Currently as shown in the log – CURRENT SERVICE STATE: ;;CRITICAL;SOFT;1; The service definition has these attributes check_command":“passive”, “enable_active_checks”:0 object Service "Service" { check_command = "passive" enable_active_checks = 0.000000 …

Passive Check Behavior

Hi all, I have a passive check configured where I’m getting unexpected behavior after service state != OK. Basically, I have a cron job that runs daily @ 02:00 and then sends a process-check-result. I want the freshness check to run between 9:00-9:15, where if no update in the past 24 hours then change to UNKNOWN. What I am seeing with the following code is cron updates service as WARNING fine, then almos 6 hours later freshness kicks in and changes to UNKNOWN. Any thoughts on what I’m doing wr…

Passive Check no timeout to status unknown

I have a passive check. In this passive check there come API calls with status very unregular and only if status changed. Now in my config is a problem so after i think 10 Min Status changed automatical to Unknown if no new API Call with status came. But i want only that API Call can Change Status. Here are Attribute List: Object ‘bgu-ZsvaPlaner!website_medorus_new’ of type ‘Service’: % declared in ‘/etc/icinga2/conf.d/services.conf’, lines 188:1-188:35 __name = “bgu-ZsvaPlaner!website_medor…

[j4n]

hi Moreamazingnick,

This is really useful. A question, though: the parameters you suggest are not all in the docs I’ve found for process-check-result; is there a more comprehensive document that lists all the relevant parameters? Perhaps I just need to read the existing docs more carefully.

[ncosta (Noé Costa)]

Hi @j4n

The options Moreamazingnick suggested are stored at the service and host configurations/templates level. That is also the reason why you have not found them in the API documentation.
You also do not pass them on your API request but when you are defining the service template.
This ensures that all of the created services that are referring to this template, are going to inherit the settings and you will not need to redefine them for each of your service objects.
I strongly recommend you to read through the two chapters (External Passive Check Results, Check Result Freshness) found in the Icinga 2 Configuration as they explain the result freshness calculations.

TL;DR
The check_interval defined at the service/host configuration level defines at which interval it expects a check to be running. If the check is provided by an external source, thus being a passive check, it will expect the external source to provide the next check result in that particular timeframe.
You can also provide a TTL (time-to-live duration) when submitting the check results from your external source, which then tells the Icinga 2 instance how long this check result is considered to be fresh. This overrides the configured check_interval on your service/host for this particular check.

Best regards,
Noé