How to send "passive" check results from an external Script

Hi,
I have a shell script on a remote Server witch check some status on it.
In icinga Vers.1.x I sent the result via

printf “$ICINGA\tbackup_lb\t0\t[OK] Some text \n” | usr/sbin/send_nsca > /dev/null 2>&1

How can I do that with Icinga2 ?

For better understanding. I will run a script via cron an would like to notify via icinga2 if the script output reports an error.

Thx for help
Joern

Hi, look here https://icinga.com/docs/icinga2/latest/doc/12-icinga2-api/#process-check-result :wink:

1 Like

OK that clear for me, but how configure it on the icinga Host?
As a Host? And if yes how because I have now IP address :frowning:

On the host which runs the cronjob, you either need a script or a full curl request targeting the Icinga master server with the REST API. That can either be an FQDN or an IP address. Keep in mind that in case of IP address changes, you’ll need to update all your clients.

Keep it simple, just play around with curl to produce a result once.

curl -k -s -u root:icinga -H 'Accept: application/json' \
 -X POST 'https://icinga-master1.localdomain:5665/v1/actions/process-check-result' \
-d '{ "type": "Service", "filter": "host.name==\"$ICINGA\" && service.name==\"tbackup_lb\"", "exit_status": 0, "plugin_output": "[OK] Some text \n" }'

Note: Replace or set the $ICINGA shell variable before.

I strongly recommend to also set the check_source to the local agent FQDN, this makes it easier to find out where this came from in your Icinga Web detail view.

Cheers,
Michael

1 Like

Hi Michael,
Ok the curl command now works fine for me but the result is not shown in the icinga GUI :frowning:
I think the error is that I have not setup a Service for that and this was my question before.

How configure the Icinga Host to display this check on the Icinga GUI?

Hi,

did you create a service to the host where you want to sent the request? You have to declar that the service can receive/handle passive check results. Look here for the Definition: https://icinga.com/docs/icinga2/latest/doc/09-object-types/#service

e.g. this is a Service Definition (in our case created from the director) to receive snmp traps:

apply Service “snmptrap” {
check_command = "passive"
max_check_attempts = “1”
check_period = “timeperiod_24x7”
enable_notifications = true
enable_active_checks = false
enable_passive_checks = true
enable_event_handler = false
enable_flapping = false
enable_perfdata = true
volatile = false
assign where host.vars.UeberwachungSNMPTrap == “Ja”
import DirectorOverrideTemplate
}

Check also your icinga2.log if icinga2 can handle the request or if the api reject it (maybe because of missing host/service definition).

1 Like

OK now my external Script is working well and shown in the icingaweb gui.
But now I have a Problem. My script runs only once a day. So I would like that icinga show me an error only if the last output from my script to the icinga host is older than 24hours.
How can I make this.
My Service definition:

apply Service “backup_lb” {
check_command = “dummy”
check_interval = 1440m

/* Set the state to UNKNOWN (3) if freshness checks fail. */
vars.dummy_state = 3

/* Use a runtime function to retrieve the last check time and more details. */
vars.dummy_text = {{
var service = get_service(macro("$host.name$"), macro("$service.name$"))
var lastCheck = DateTime(service.last_check).to_string()

return "No check results received. Last result time: " + lastCheck
}}
assign where host.vars.backup_lb == “yes”
}
}
But evertime this service runs my the check is unknown. But the check should only unknown if it older than 24hours :frowning:

This is the output of my external Script:
curl -k -s -u icingaweb2:xxxxxxxx -H ‘Accept: application/json’ -X POST ‘https://icinga-host:5665/v1/actions/process-check-result’ -d ‘{ “type”: “Service”, “filter”: “host.name==“Passive_Checks” && service.name==“backup_lb””, “exit_status”: 0, “plugin_output”: “[OK] Dies ist ein Test \n” }’

Could I set here the “TTL”? And if yes how???
Thx for help

This is because you defined a check_interval! A passive check normally don’t need this, because this is waiting for a passive check result from a transmitter (server, application etc.).
You need this only if you want to check, that every x minutes/hours/days the transmitter sends something. if Icinga don’t receive something, there is something wrong on the server, application etc.

Maybe it’s better for you to use the command “passive” (https://icinga.com/docs/icinga2/latest/doc/10-icinga-template-library/#passive). Also I’m missing this parameter in your snippset:

enable_active_checks = false
enable_passive_checks = true

BTW: you don’t should write passwords in a forum :wink:

Thx for promt answer:+1:
Ups for the passwd :grimacing:
OK for passive check, but how does the icinga server knows that the check is older than e.g. 24hours? Because my script runs only once a day?

That’s not entirely correct, the configuration from @joltmann is fine. It is just the 24h window which may be triggered too soon. The idea with the ttl attribute is good, I would try to add that with actually delaying the next expected check result from this point on.

Just add a JSON key ttl with the value in seconds, 86400 to the POST body.

Cheers,
Michael

ah ok, thanks @mfriedrich for the info. That’s a way I didn’t know/imagine.
We do the tests at the side from icinga2 not at the side of the script. Like Thomas and Lennart wrote in the icinga2 book at page 228. :wink:

@joltmann Our test case is to send a test snmptrap to the icinga master to check everything is working fine. Here we run logstash to recevie SNMP traps and pass the trap to icinga. This happens every hour.
This is the defination from our test snmptraps on our two masters (rendered from the director):

apply Service “snmptrap_test” {
import “template_generic-service_passive”
check_command = "passive
max_check_attempts = “1”
check_interval = 3800s
retry_interval = 70s
enable_active_checks = true
assign where host.name == “master1” || host.name == “master2”
vars.dummy_state = “1”
vars.dummy_text = “Kein SNMP-Trap empfangen”

 import DirectorOverrideTemplate 

}

Hi Stevie, hi Michi,
sorry but I am a little bit confused.
My Script (which is running on a remote Host) is running well and send Status OK or Staus NOK after execution every 24hours. This runs well and the iciinga-master display the Status well.
But what I need is the Status “unknown” if my script is not running (can`t send Status of e.g. Network or Server Problem).
Means if the check on the icinga-master is older than 24hour I would like the Status “unknown”.

Here my Service and my api post:
The interval is changed for test!

apply Service “backup_lb” {

check_command = “dummy”
max_check_attempts = “1”
check_interval = 60s
retry_interval = 70s
enable_active_checks = true

assign where host.vars.backup_lb == “yes”

vars.dummy_state = “1”
vars.dummy_text = “No Status received since 24hours”
import DirectorOverrideTemplate
}

curl -k -s -u icingaweb2:xxxxx -H ‘Accept: application/json’ -X POST ‘https://icinga-mon-master:5665/v1/actions/process-check-result’ -d '{ “attrs”: { “ttl” :“90” }, “type”: “Service”, “filter”: “host.name==“Passive_Checks” && service.name==“backup_lb””, “exit_status”: 0, “plugin_output”: “[OK] Dies ist ein Test \n”

Sorry for this confusion. If you want to check this case, like we do, this should work for you:

apply Service “backup_lb” {

check_command = “passive”
max_check_attempts = “1”
check_interval = 60s
retry_interval = 70s
enable_passive_checks = true
enable_active_checks = true
assign where host.vars.backup_lb == “yes”

vars.dummy_state = “1”
vars.dummy_text = “No Status received since 24hours”
}

The line “import DirectorOverrideTemplate” is rendered by the director. If you don’t use it, you don’t have to write this :wink:

What exactly the diffrence between the check commands “dummy” and “passive” is, I can’t explain. In the docs there is only written, that the passive check has appropriate default values. Hopefully Michael can bring a little bit light in the dark. For us the passive check works better in this case.

I never used the ttl options. So this is for me also new. You never stop learning :slight_smile:

Hi Stevie,
thanks for understanding but this doesn’t work for me because after 1min when the check is running again I get a warning. So everytime the check is running I got a warning. But I will get only a warning when the last information from my script is older than 24hours.
Maybe it is the wrong template? Could you post me your “template_generic-service_passive” template or I don`t need this?

Hi, this is our template:

template Service “template_generic-service_passive”
{
check_command = “passive”
max_check_attempts = “1”
check_period = “timeperiod_24x7”
enable_notifications = true
enable_active_checks = false
enable_passive_checks = true
enable_event_handler = false
enable_flapping = false
enable_perfdata = true
volatile = false
}

also created with the director

Strange doesn’t work for me :frowning:
Does anyone know when a check is set to unknown by icinga-master?

An idea: do you press “check now” at the check in the gui?

Both. Onetime yes and onetime wait until the next check is executed.
Both are the same result

if I press “check now” in the gui in our environment at one of our passive checks, this always get “unknown” as a check result.

check the icinga2 logs if you send something to the api

1 Like

Oh so sorry guys. It was a missunderstanding from me.
I runs well and fine. I interpred the out false.

Stevie your check runs well for me.

Thanks a lot for your help