Hello,
is it possible with icinga2 to resend notifications while Host is in UP status at reguliar intervals ?
Actualy working, with default configuration, for DOWN status and status change DOWN to UP, but I cannot find how to periodicaly inform of UP status.
Thanks,
Hi - welcome to the official community portal!
You could force the host in question to go to a different state than UP with an API command via a cronjob (assuming you’re running your Icinga on Linux or a lookalike); it would then go back to UP on the next check execution & should generate a notification.
You might want to add a preceding check in your cronjob whether the host is actually UP at that time; if not, terminate the job.
It may be possible to define a special state for this inbetween phase, which you could call UPCHECK or something like that. I’m not sure how to do this but am 99% certain that Icinga can do it… As a last resort, if checks occur often, just use WARNING.
This is messes up your sla history.
You can actually send notifications via icingaweb2 api:
curl -H "Accept: application/json" -u icingawebuser:icingawebpassword \
"https://icinga-master-node/icingaweb2/monitoring/host/send-custom-notification?host=YOURHOST" \
-F "forced=y" \
-F "comment=CRON-Notification"
Hi,
Thanks for the tip. I will look in that direction, but I’m afraid that DOWN notifications will be sent ?
And according to @moreamazingnick, It will break my SLA.
Bests,
Hello @moreamazingnick ,
Great, but I was looking for a way to use the icinga2 schedule engine. If it’s not possible, I will use a cronjob to retreive the state of the devices (there are plenty), and on UP state send notifications.
Knowing that notifications are using an external script wich triggers SNMPTraps, would you use icinga’s notification system or directly call the script wich sends the traps ?
My task is now :
1- retreive all eligibles devices on a criteria (a variable in the device configuration ?) using API
2- loop on devices to retreive their status using API
3- send notification for UP devices
Thanks for the advice, Bests,
No-one ever said anything about going DOWN…
Another option would be to have a special notifying script run for that host, which does the “cronjobbing” itself, in a sleep loop. It would get started once when the host first changes to UP, then terminate itself once it leaves that state. This would work irrespective of the op. system Icinga is running on…
Don’t grok this sentence since it seems self-referential. If you use a script that generates traps, it might be appropriate to use something like Trap Director to react to the arriving traps, incl. notification reactions within that.
Sorry for my poor English,
I monitor devices via icinga and send notifications via a custom Notification Object using a script wich use snmptrap Linux command to notify a third party. This third party wants snmptraps (UP or DOWN) every 15 minutes. For DOWN status, no problem.
For UP status, I have to schedule a cron job, but I was just wondering if it would be better to send the notifications via API call to Icinga or call my snmptrap script directly.
I think the first is better, because I will keep trace of notifications and will take advantage of mechanisms provided by icinga (no need to extract parameters like notification OIDs in the script etc.).
Anyways, thank for your responses,
Bests regards,
There are actualy about 20 devices needing this UP notifications, and it is possible that this number increase in the future. I’m “afraid” about launching long running processes, without controlling resources consumption.
Moreover, triggering theses processus by DOWN to UP status change, is a great idea, but what if I have to restart icinga ? I have to detach the processes from his ancestor to keep it running (nohup), and if the server is restarted, how to restart it ?
I prefer the cronjob solution, which may be less elegant, but seems more robust to me.
Re your last post: Forget the DOWN part, please! I never said DOWN; at most WARNING, but best would be to define a custom state. I’m sure that’s possible in Icinga.
You don’t seem to be aware that it’s normal to manually set a different state for a running host in Icinga for test purposes; it can be done via the GUI. This is the easiest way to test behaviour of a new plugin on a test host, for instance.
As for the long running processes, they would be sleeping most of the time. If you truly want to prevent having several “clogging” up your system, write a daemon program that gets started as a service, and reads a parameter file every x minutes (or whatever interval you require) which contains the names of hosts that are to be notified in this special way; the file could get hosts appended to it by a special UP notify script, and names of hosts no longer UP would be removed by the daemon. In a typical 2-process watchdog configuration this daemon could run continually, independent of any Icinga scheduling; and it would send notifications to all hosts in the processed parameter file, every time it becomes active.
Advantage: Only 2 processes would be running longer, one active, the other more or less passive (just checking that the other is still there, spawning a new one if not).
But this is another separate solution. My “another option” suggestion uses Icinga features and is relatively simple.
P.S.: Restarting Icinga would not be a problem since it will re-evaluate all statuses at that time.
So as I understand the return value is wrong.
if the UP state is the one YOU have to get active and fix something, you should invert that
Ok forget about UP and DOWN, create a host and use any other checkcommand like dummy or ping or whatever.
create a service for the host with a checkscript that does the following:
IF SNMPTRAP CASE technician must get active => return 2
ELSE IF SNMPTRAP CASE technician can get active => return 1
ELSE return 0
alternatively you could write a wrapper script that executes you first check and returns 2 if exitcode was 0
If you have such a valid check script icinga can resend notifications but only for non OK/UP events.
Hi,
First of all, problem is solved, thanks to @moreamazingnick and @bkai for your patience.
I realize that I was not clear during this discussion (sorry for my poor English).
I needed :
a personalized notification sending snmptraps for UP to DOWN and DOWN to UP hosts status change (with differents OIDs depending of the host and his status).
=> This was already working when I opened the thread using a custom script as notification command.
the same notification beeing resent every n minutes if the host is (and stays) DOWN
=> this was already working using Icinga’s parameter “interval=n*60” of Notification object
the same notification beeing resent every n minutes if the host is (and stays) UP
=> My question was mainly to know if it was possible to do that only by configuring Icinga, the response seems to be negative.
I solved that by using a cronjob, scheduled every n minutes which use the API :
Works like a charm, thanks again for your time. @moreamazingnick your post with API suggestion was the key, I saw a little piece of the power of this great API ! 
Bests regards, and happy new year. 