vSphere alarms with icinga

Icinga 2
1

I’d like to know how you guys handle vSphere alarms?

I’m currently using my own check plugin which returns all alarms at that time, but the downside of course is that notifications are only send when the state changes (which was discussed here and here.

However, the project request to have a notification for every single alarm instead. Using ELK just for this approach sounds like cracking nuts with a sledgehammer, am I wrong? What about using a passive check where my check plugin throws in any new alarm?

2

Hi.

Just brainstorming a little bit as i do not have a good straight forward idea:

Quick and dirty: save the output to a statefile and compare it on next run. If its identical do nothing, otherwise send a state change to “OK” (f.e. via api) and let the check continue. As the result will be critical, your message should triggered. (Not tested)

  1. let the vcenter trigger a passive check to get critical (f.e. via webhook), then set them back to okay after it. Use a second check to show “critical” in Icinga.
  2. No Idea if its possible, but as icinga can do everything ( :wink: ) I’am pretty sure its possible via API: record the last message again, but this time don’t change the state, but trigger the notification directly via api from inside your script.
  3. Don’t use icinga for it, use the vCenter notification system.
1 Like
3

Hi, thanks for your ideas.

  1. will send only one notification for example 3 new alarms
  2. Don’t know how vCenter Server could trigger something except mail, logfile or SNMP trap.
  3. I’ve found only send comments via API but no notification
  4. They want to have everything within one tool (which makes sense e.g. for downtimes and dependencies)
4
  1. Oh, okay, you want 3 Notifications if the status goes from ok to critical in ONE check, when the reasons are 3 Alarms. That was a misunderstanding from my side.
  2. vCenter can send webhooks as far as i know.
  3. https://icinga.com/docs/icinga2/latest/doc/12-icinga2-api/#send-custom-notification
5
  1. Cannot find anything about except vRealize which we don’t use.
  2. Is what I’ve mentioned and this would look different than normal notifications (have the full plugin output as additional info and anything as comment)

This might be a job for icinga-vspheredb, but currently it isn’t.

6

I found that one: https://www.greenreedtech.com/vsphere-alarms-with-slack-and-stackstorm/

7

Hmm, stackstorm looks like another software monster. :frowning:

8

I just mentioned it because of the curl webhook they are using from the vCenter, not because of the software. Instead of sending it to stackstorm you can send them to Icinga

#!/bin/bash

# environment variables
alarm_name=$VMWARE_ALARM_NAME  
alarm_id=$VMWARE_ALARM_ID  
alarm_target_name=$VMWARE_ALARM_TARGET_NAME  
alarm_description=$VMWARE_ALARM_EVENTDESCRIPTION

# Trigger StackStorm Web Hook
curl -k -X POST https://stackstorm01.grt.local/api/v1/webhooks/vsphere   
-H "St2-Api-Key: MTk4ZThiZWM3ZTc0ZWI0N2UxNDMzZGExOWZjNw" 
-H "Content-Type: application/json" 
--data '{"alarm_name": "value1", "alarm_id": "value2",
 "alarm_target_name": "value3", "alarm_description": "value4"}'
9

Hmm, I still cannot find anything about webhooks with vCenter Server only and your example is based on stackstorm. Hence, I’m not sure what you’re trying to tell me.