Monitor availability of hosts from remote service's point of view

Dear all,

we want Icinga to monitor the availability of client hosts to a special service running on a remote host. (So checks to the client nodes themselves make no sense.)
For each client host its availability should be displayed.

For security reason, only ssh connections initiated from the Icinga host are permitted to the remote host.

My rough idea is

a) On remote host to dump locally a list consisting of of hosts and their availability status.
b) To fetch the list by Icinga host from remote host
c) To process the list with service checks for each client host

To Implement a) and c) seems to be trivial:

A) A check script on remote host called via ssh remote service check generates the list. The return code is used to display the availability of the special service itself.

C) On Icinga host execute for each client host a service check locally which takes the according state from the local list.

My question is:
Is it possible to implement b) so that fetching the list is triggered by A) ?
Is there a better, cleaner approach?

Because, EventCommand triggers only conditioanlly.
I didn’t find any way to just execute “auxillary” scripts or similar not related to a service which is related to a service/host.
Only one script can be called for a service.
A cron job (i.e. totally unrelated to Icinga) would introduce a time jitter to the data.

Many tanks!

Hi,
the easiest way of course would be to install an icinga agent on the Server. Whith a your own script (or the checks which are installed with icinga e.g. check_procs are enough) you could check this.
But if this is not possible you could check, if there is a API where icinga fetch the data (e.g. with a script on the icinga Servers)
An other possability would be that your server send this Information to the icinga api. Here you define a passive Service and icinga is waiting for data.

Hi Stieve,

thanks for your reply.

an Icinga service on the remote host is unfortunately not possible.

Yes, that’s my question. Does such API exists? I didn’t find any.

If connections from remote host to Icinga host are allowed passive checks on remote host can be used. For each client host the check reads the local list and sends the client state to Icinga host.
But unfortunately such connections are not allowed.

Hi,

Icinga has an API - https://icinga.com/docs/icinga2/latest/doc/12-icinga2-api/ So you could sent a curl-request from your client hosts every x Minute/hour/etc. to the API from icinga.

I meant if there is (Rest-)API at your clients host, where Icinga could fetch data. If the server or the installed software provide a API only know you or your client, not we in the community :wink:

So both ways are possible. Maybe you have to use the check_command check_by_ssh if icinga has to trigger the check and only the SSH-port is open for this.

Hi Stevie,

thank you for you input.
I finally implemented a more or less dirty solution.
In short, I use a main script on Icinga host doing a) b) c) in sequence.

Just want to recall that there are two constraints:

  1. I want to monitor if the service on service host considers the client nodes available in some sense. This is not exactly the same if the client nodes itself are healthy. On a client host I can’t determine its availability to the service.
    That’s why I connect only to the service host.
  2. The only connection type permitted between Icinga host and service host are outbound connections from Icinga host.

For each client host a passive service check is defined.
An active service check calling the main script is defined for the service on service host.
It uses check_by_ssh to create the list A) and scp to fetch the list B).
Then, the Icinga API is used locally to map the client’s states to the client hosts C).

Pros:

  • Clients’ checks are triggered synchronously right after the creation of the state list.

Contra:

  • Script is slow (takes more than 90s to finish)
  • not conform to Icinga’s ideas