i have a problem with the communication between my icinga master and one of my satellites.
All checks working but sometimes delayed and the “next check time” moves to a negative value (only on this satellite).
the following log is from the master (/var/log/icinga2/icinga2.log) and shows what happen if i restart the satellite.
[2020-02-05 11:16:54 +0100] information/ApiListener: New client connection for identity '**SATELLITE**' from [IP_SATELLITE]:51538
[2020-02-05 11:16:54 +0100] information/ApiListener: Sending config updates for endpoint '**SATELLITE**' in zone '**SATELLITE.ZONE**'.
[2020-02-05 11:16:54 +0100] information/ApiListener: Syncing configuration files for zone '**SATELLITE.ZONE**' to endpoint '**SATELLITE**'.
[2020-02-05 11:16:54 +0100] information/ApiListener: Syncing configuration files for global zone 'GLOBALZONE' to endpoint '**SATELLITE**'.
[2020-02-05 11:16:54 +0100] information/ApiListener: Finished sending config file updates for endpoint '**SATELLITE**' in zone '**SATELLITE.ZONE**'.
[2020-02-05 11:16:54 +0100] information/ApiListener: Syncing runtime objects to endpoint '**SATELLITE**'.
[2020-02-05 11:16:54 +0100] information/ApiListener: Finished syncing runtime objects to endpoint '**SATELLITE**'.
[2020-02-05 11:16:54 +0100] information/ApiListener: Finished sending runtime config updates for endpoint '**SATELLITE**' in zone '**SATELLITE.ZONE**'.
[2020-02-05 11:16:54 +0100] information/ApiListener: Sending replay log for endpoint '**SATELLITE**' in zone '**SATELLITE.ZONE**'.
[2020-02-05 11:16:54 +0100] information/ApiListener: Finished sending replay log for endpoint '**SATELLITE**' in zone '**SATELLITE.ZONE**'.
[2020-02-05 11:16:54 +0100] information/ApiListener: Finished syncing endpoint '**SATELLITE**' in zone '**SATELLITE.ZONE**'.
[2020-02-05 11:16:54 +0100] information/JsonRpcConnection: Received certificate request for CN '**SATELLITE**' signed by our CA.
[2020-02-05 11:16:54 +0100] information/JsonRpcConnection: The certificate for CN '**SATELLITE**' is valid and uptodate. Skipping automated renewal.
[2020-02-05 11:17:01 +0100] information/ApiListener: New client connection for identity '**SATELLITE**' from [IP_SATELLITE]:51540
[2020-02-05 11:17:01 +0100] warning/ApiListener: No data received on new API connection from [IP_SATELLITE]:51540 for identity '**SATELLITE**'. Ensure that the remote endpoints are properly configured in a cluster setup.
[2020-02-05 11:17:11 +0100] information/ApiListener: New client connection for identity '**SATELLITE**' from [IP_SATELLITE]:51548
[2020-02-05 11:17:11 +0100] warning/ApiListener: No data received on new API connection from [IP_SATELLITE]:51548 for identity '**SATELLITE**'. Ensure that the remote endpoints are properly configured in a cluster setup.
[2020-02-05 11:17:21 +0100] information/ApiListener: New client connection for identity '**SATELLITE**' from [IP_SATELLITE]:51550
icinga2 - The Icinga 2 network monitoring daemon (version: r2.11.2-1)
Copyright (c) 2012-2020 Icinga GmbH (https://icinga.com/)
License GPLv2+: GNU GPL version 2 or later <http://gnu.org/licenses/gpl2.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
System information:
Platform: Debian GNU/Linux
Platform version: 9 (stretch)
Kernel: Linux
Kernel version: 4.9.0-8-amd64
Architecture: x86_64
at first i used the icinga2 node wizard for the master + satellites and than i imported with the kickstart assistant the zone+endpoint config.
After this step i had problems with re-definied errors in the zones.conf and zones.d files.
I removed the satellite configs on the master at the /etc/icinga2/zones.conf and the problem was fixed.
All checks on the satellites working and i have no problems on the other satellites.
The delayed check problem is not the whole time.
Imho the master/satellite details should be stored in zones.conf and outside the infrastructure tab inside the Director. Their usage there is experimental and won’t necessarily work with deployments and cluster config syncs.
thanks for the link.
my cluster config is now stored at the local zone.conf.
the endpoints and zones were changed to external objects at the director.
but the problem was still there.
i reinstalled the satellite and now it works without problems or warnings at the log.