Monitoring IoT with Icinga2

Hi!

I would like to monitor some aspects of IoT-hardware. A lot of this hardware is based on OpenWRT and I need a lightweight solution to save space on the device.

I want to start with up/down status but ICMP is unavailable. Is it possible to ping some sort of API using wget / curl to get show the device as online? I remember that I saw something in the docs but I don’t remember where and I was unable to spot it again.

After up / down works, how to implement other checks like memory, load, space, etc.?
I am familiar with the setup if the agent is in use but there is none for OpenWRT (which is fine because it would take to much space).

Thanks!

Kevin

Hi,

Just replace the hostalive check_command for hosts with something else, e.g. http. This is described here.

Use SNMP, this can be enabled in OpenWRT and is a common standard. The linked wiki page also has snmpwalk as an example, this is where you can extract the required OIDs for check_snmp.

You can also try to use nwc_health or the manubulon snmp plugins.

“IoT” is a rather broad term, so once you’ve done all the basic embedded hardware monitoring stuff, you might have a look into MQTT and variants then as well.

Cheers,
Michael

SNMP is available but this hardware is installed in various locations that do not have Icinga2 nearby (thus no agent /check support). All masters / sat. are external / NAT. Thats why ICMP is unavailable (and HTTP also not possible).
These devices currently only connect to a provisioning API but I am thinking about implementing a Ping-API that shares that keepalive as external command to Icinga2 (https://icinga.com/docs/icinga2/latest/doc/12-icinga2-api/#icinga2-api-actions-process-check-result).

This would also allow me to drop SNMP in favor of other packages.

“process-check-result” should also be fine as an alternative to your approach, right?

Do these devices connect via a bus protocol, e.g. MQTT? You may setup an MQTT server and a local receiver with mqttwarn - and then process check results against the Icinga 2 API.

I wrote an article on that a while ago for the German iX magazine. Here’s the sample code/config I’ve used with mqttwarn.

You might also share more about the “provisioning API” to gather an idea whether there’s the possibility for event subscriptions.

Cheers,
Michael

Currently, no MQTT is in use. IoT might not be the correct term, it’s more some kind of small router / wifi mesh. It primarily connects to a central CMDB rather than LAN-interconnect. This CMDB-API provides the network configuration for these devices and collects network details for troubleshooting.
Currently these devices share data using Respondd, which also allows me to see wich devices are online. As we also manage all other network devices with Icinga2, I want a single dashboard.

I will look into mqttwarn, thanks for the hint. MQTT is on my to-do list but there was no time yet, seems like it should be the next topic to look into.

I am now using “passive” as up/down check for these “hosts”. I am pushing a heartbeat from my API to Icinga2 and it works perfectly fine.
As a “first make it work, then make it better”-approach, it’s well suited.

1 Like

Can you share some examples how you’ve integrated this with respondd and the CMDB-API? Others are likely interested as well :slight_smile:

I plan to provide docs in august, they will also include the information you requested.
Icinga2 is one of the core (external) components of the CMDB product I developed for several years now.

1 Like