Icinga Client check buffering

Hello,

We have an environment where Icinga satellites are communicating with Icinga clients that will occasionally lose network connectivity for some period of time but are still powered on. Is the client software able to buffer checks somehow, so that gaps in monitoring time-series data / status history can be filled in? If not, I’m thinking it could be possible to run the checks locally and save the data into a log file, but I’m not sure I could feed this data into the Icinga API afterward with historical information.

Hi, what you are looking for is “log_duration” - look here: https://icinga.com/docs/icinga2/latest/doc/09-object-types/#endpoint

Thanks Stevie. But how are these scheduled if the clients are not satellites?

Hi,

you can use the Top Down Config Sync configuration method not only for satellites.

You can configure a zone for your agent and put the configuration for this agent here. Some checks needs to be done from the parent node nonetheless e.g. ICMP checks.

Now if one agent lost network connectivity to its parent node, the agent will continue to execute checks, since they will be scheduled on the agent and not longer from the parent node. If the network connectivity resumes the agent will transmit its logs to the parent node - the so-called Replay Log. The gaps you’ll notice in the history and/or in a Graphing software like Grafana should be filled after a short time.

Best regards
Michael

Unfortunately, this is not supported for Windows agents.

What makes you think that? As far as I know satellite instances on Windows are not supported.

The documentation says:

The configuration files can be modified with your favorite editor e.g. Notepad++ or vim in Powershell (via chocolatey).

In order to use the top down agent configuration prepare the following steps.

You don’t need any local configuration on the agent except for CheckCommand definitions which can be synced using the global zone above. Therefore disable the inclusion of the conf.d directory in the icinga2.conf file.

Navigate to C:\ProgramData\icinga2\etc\icinga2 and open the icinga2.conf file in your preferred editor. Remove or comment ( // ) the following line:

[…]

https://icinga.com/docs/icinga2/latest/doc/06-distributed-monitoring/#agent-setup-on-windows

I read it that way that the Top Down mode is supported.

Best regards
Michael

@dnsmichi stated here: “The preferred method for agents is the command endpoint execution bridge, and this also is used inside the Icinga Director when making a host an agent.”