Icinga checks via SSH - Logging on Hosts

Hi, I am using icinga2 with IcingaDirector since a few years and have a running Environment with around 300 Hosts and several custom service check commands.

Every Command get the hosts IP Address as parameter and then performs a “ssh” command to the Host to execute whatever check I need. This is working fine. SSH login is performed via key.

My only Issue is: The logs on each Host fill up with a lot of logins from my Icinga2 server. (5-10 logins every 15 Minutes) Thereby other important Messages in /var/log/messages or /var/log/secure are very hard to find.

Did anyone faced the same/similar Issue and found a Workaround?
Is it possible to execute the check commands without a ssh login on the Host? (maybe via some local script or daemon running on each Host)

I am open to any Ideas on this Topic.
Many Thanks

My only Issue is: The logs on each Host fill up with a lot of logins from my Icinga2 server.

One option would be to add a custom Match[0] block to your sshd_config
against your Icinga 2 SSH-Agent user defining an incremented LogLevel[1].

This is just a hypothetical idea, which I have not actually tried and
would not recommend. In general, I would advise against suppressing
login information, as these are security-relevant records.

Alternatively, I would rather proceed with a corresponding “grep -v”
when viewing the log files.

Is it possible to execute the check commands without a ssh login on the Host?

There are other ways to perform agent-based checks. For example, you
could also install Icinga 2 on each node and configure it as an agent[3].

[0] https://man.openbsd.org/sshd_config#Match
[1] https://man.openbsd.org/sshd_config#LogLevel
[2]
Agent Based Monitoring - Icinga 2

You could modify your (r)syslog definition to exclude or move to dedicated logfile those messages. Or you configure persistent connections.

1 Like