Random notification of Remote Icinga instance is not connected to 'icinga-mon.common.europages.com'

Icinga 2
1

Hi Community,
I have setup an running icinga2 version: r2.6.0-1 with a lot of icinga clients in different Subnetworks.
Everything runs fine! But for some Hosts I get from time to time a notification that the Service are unknown with additional Info:

Remote Icinga instance ‘server’ is not connected to ‘icinga master’.

After some minutes the Service are OK.
Any idea?
In the log of the Master I found the following:

[2021-11-24 17:19:23 +0100] information/JsonRpcConnection: No messages for identity 'server' have been received in the last 60 seconds.
[2021-11-24 17:19:23 +0100] warning/JsonRpcConnection: API client disconnected for identity 'server'
[2021-11-24 17:19:23 +0100] warning/JsonRpcConnection: API client disconnected for identity 'server'
[2021-11-24 17:19:23 +0100] warning/ApiListener: Removing API client for endpoint 'server'. 0 API clients left.
[2021-11-24 17:19:23 +0100] warning/ApiListener: Removing API client for endpoint 'server'. 0 API clients left.
[2021-11-24 17:20:16 +0100] information/JsonRpcConnection: Reconnecting to API endpoint 'server' via host 'server' and port '5665'
[2021-11-24 17:20:16 +0100] information/ApiListener: New client connection for identity 'server' to [xxx.xxx.xxx.xxx]:5665
[2021-11-24 17:20:16 +0100] information/ApiListener: Sending config updates for endpoint 'server'.
[2021-11-24 17:20:16 +0100] information/ApiListener: Syncing configuration files for global zone 'global-commands' to endpoint 'server'.
[2021-11-24 17:20:16 +0100] information/ApiListener: Syncing runtime objects to endpoint 'server'.
[2021-11-24 17:20:16 +0100] information/ApiListener: Finished syncing runtime objects to endpoint 'server'.
[2021-11-24 17:20:16 +0100] information/ApiListener: Finished sending config updates for endpoint 'server'.
[2021-11-24 17:20:16 +0100] information/ApiListener: Sending replay log for endpoint 'server'.
[2021-11-24 17:20:16 +0100] information/ApiListener: Finished sending replay log for endpoint 'server'.
[2021-11-24 17:24:08 +0100] information/JsonRpcConnection: No messages for identity 'server' have been received in the last 60 seconds.
[2021-11-24 17:24:08 +0100] warning/JsonRpcConnection: API client disconnected for identity 'server'
[2021-11-24 17:24:08 +0100] warning/JsonRpcConnection: API client disconnected for identity 'server'
[2021-11-24 17:24:08 +0100] warning/ApiListener: Removing API client for endpoint 'server'. 0 API clients left.
[2021-11-24 17:24:08 +0100] warning/ApiListener: Removing API client for endpoint 'server'. 0 API clients left.
[2021-11-24 17:24:16 +0100] information/JsonRpcConnection: Reconnecting to API endpoint 'server' via host 'server' and port '5665'
[2021-11-24 17:24:16 +0100] information/ApiListener: New client connection for identity 'server' to [xxx.xxx.xxx.xxx]:5665
[2021-11-24 17:24:16 +0100] information/ApiListener: Sending config updates for endpoint 'server'.
[2021-11-24 17:24:16 +0100] information/ApiListener: Syncing configuration files for global zone 'global-commands' to endpoint 'server'.
[2021-11-24 17:24:16 +0100] information/ApiListener: Syncing runtime objects to endpoint 'server'.
[2021-11-24 17:24:16 +0100] information/ApiListener: Finished syncing runtime objects to endpoint 'server'.
[2021-11-24 17:24:16 +0100] information/ApiListener: Finished sending config updates for endpoint 'server'.
[2021-11-24 17:24:16 +0100] information/ApiListener: Sending replay log for endpoint 'server'.
[2021-11-24 17:24:16 +0100] information/ApiListener: Finished sending replay log for endpoint 'server'.
[2021-11-24 17:28:38 +0100] information/JsonRpcConnection: No messages for identity 'server' have been received in the last 60 seconds.
[2021-11-24 17:28:38 +0100] warning/JsonRpcConnection: API client disconnected for identity 'server'
[2021-11-24 17:28:38 +0100] warning/JsonRpcConnection: API client disconnected for identity 'server'
[2021-11-24 17:28:38 +0100] warning/ApiListener: Removing API client for endpoint 'server'. 0 API clients left.
[2021-11-24 17:28:38 +0100] warning/ApiListener: Removing API client for endpoint 'server'. 0 API clients left.
[2021-11-24 17:29:16 +0100] information/JsonRpcConnection: Reconnecting to API endpoint 'server' via host 'server' and port '5665'
[2021-11-24 17:29:16 +0100] information/ApiListener: New client connection for identity 'server' to [xxx.xxx.xxx.xxx]:5665
[2021-11-24 17:29:16 +0100] information/ApiListener: Sending config updates for endpoint 'server'.
[2021-11-24 17:29:16 +0100] information/ApiListener: Syncing configuration files for global zone 'global-commands' to endpoint 'server'.
[2021-11-24 17:29:16 +0100] information/ApiListener: Syncing runtime objects to endpoint 'server'.
[2021-11-24 17:29:16 +0100] information/ApiListener: Finished syncing runtime objects to endpoint 'server'.
[2021-11-24 17:29:16 +0100] information/ApiListener: Finished sending config updates for endpoint 'server'.
[2021-11-24 17:29:16 +0100] information/ApiListener: Sending replay log for endpoint 'server'.
[2021-11-24 17:29:16 +0100] information/ApiListener: Finished sending replay log for endpoint 'server'.
2

Hi,

first of all version 2.6.0 is really quite old. In the meantime 2.13.2 is actual. There happend a lot of important fixes between your and the actual releases (incl. Network Stack).

What I’m missing is the information if you are running icinga2 as agent on your client. If yes, you should the check there as well. Mabe you start also the debug log if possible on such a client. Maybe there are more interesting log messages what can give you a hint.
But usally you’ll get this if the icinga agent loses connection to his parent. If this is the reason, you should check why. Possible is problems with the network, firewall etc.

3

Hi Stevie,
yes I use the Icinga2 client on my Server and no I can’t find anything in our Network.
I will try to switch on the debug level and will have a look.
And yes it’s time to update the Version :frowning:

1 Like