As of April 22 certification validation fails for some (not all) Windows endpoints.
Linux endpoints are not affected.
Server log shows:
warning/ApiListener: Certificate validation failed for endpoint 'xxx.yyyy.zzz': code 7: certificate signature failure
Server version:
icinga@master1:/var/log/icinga2# icinga2 --version
icinga2 - The Icinga 2 network monitoring daemon (version: r2.13.7-1)
Copyright (c) 2012-2023 Icinga GmbH (https://icinga.com/)
License GPLv2+: GNU GPL version 2 or later <https://gnu.org/licenses/gpl2.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
System information:
Platform: Ubuntu
Platform version: 20.04.6 LTS (Focal Fossa)
Kernel: Linux
Kernel version: 5.4.0-147-generic
Architecture: x86_64
Build information:
Compiler: GNU 9.4.0
Build host: runner-hh8q3bz2-project-575-concurrent-0
OpenSSL version: OpenSSL 1.1.1f 31 Mar 2020
Application information:
General paths:
Config directory: /etc/icinga2
Data directory: /var/lib/icinga2
Log directory: /var/log/icinga2
Cache directory: /var/cache/icinga2
Spool directory: /var/spool/icinga2
Run directory: /run/icinga2
Old paths (deprecated):
Installation root: /usr
Sysconf directory: /etc
Run directory (base): /run
Local state directory: /var
Internal paths:
Package data directory: /usr/share/icinga2
State path: /var/lib/icinga2/icinga2.state
Modified attributes path: /var/lib/icinga2/modified-attributes.conf
Objects path: /var/cache/icinga2/icinga2.debug
Vars path: /var/cache/icinga2/icinga2.vars
PID path: /run/icinga2/icinga2.pid
Client log shows:
C:\Program Files\ICINGA2\sbin>icinga2.exe -scm daemon
[2023-04-24 08:24:17 +0200] information/cli: Icinga application loader (version: v2.13.3)
[2023-04-24 08:24:17 +0200] information/cli: Loading configuration file(s).
[2023-04-24 08:24:17 +0200] information/ConfigItem: Committing config item(s).
[2023-04-24 08:24:17 +0200] information/ApiListener: My API identity: xxx.yyyy.zzz
...
[2023-04-24 08:24:17 +0200] information/ConfigItem: Instantiated 1 WindowsEventLogLogger.
[2023-04-24 08:24:17 +0200] information/ConfigItem: Instantiated 1 UserGroup.
[2023-04-24 08:24:17 +0200] information/ConfigItem: Instantiated 1 IcingaApplication.
[2023-04-24 08:24:17 +0200] information/ConfigItem: Instantiated 7 TimePeriods.
[2023-04-24 08:24:17 +0200] information/ConfigItem: Instantiated 19 ServiceGroups.
[2023-04-24 08:24:17 +0200] information/ConfigItem: Instantiated 14 HostGroups.
[2023-04-24 08:24:17 +0200] information/ConfigItem: Instantiated 8 NotificationCommands.
[2023-04-24 08:24:17 +0200] information/ConfigItem: Instantiated 4 Zones.
[2023-04-24 08:24:17 +0200] information/ConfigItem: Instantiated 349 CheckCommands.
[2023-04-24 08:24:17 +0200] information/ConfigItem: Instantiated 3 Endpoints.
[2023-04-24 08:24:17 +0200] information/ConfigItem: Instantiated 1 ApiListener.
[2023-04-24 08:24:17 +0200] information/ScriptGlobal: Dumping variables to file 'C:\ProgramData\icinga2\var\cache\icinga2/icinga2.vars'
[2023-04-24 08:24:17 +0200] information/ConfigObject: Restoring program state from file 'C:\ProgramData\icinga2\var\lib\icinga2/icinga2.state'
[2023-04-24 08:24:17 +0200] information/ConfigObject: Restored 408 objects. Loaded 0 new objects without state.
[2023-04-24 08:24:17 +0200] information/ConfigItem: Triggering Start signal for config items
[2023-04-24 08:24:17 +0200] information/ApiListener: 'api' started.
[2023-04-24 08:24:17 +0200] information/ApiListener: Started new listener on '[::]:5665'
[2023-04-24 08:24:17 +0200] information/ConfigItem: Activated all objects.
[2023-04-24 08:24:18 +0200] information/ApiListener: New client connection for identity 'master1.yyyy.zzz' from [::ffff:192.168.2.210]:44498
[2023-04-24 08:24:18 +0200] information/JsonRpcConnection: Requesting new certificate for this Icinga instance from endpoint 'master1.yyyy.zzz'.
[2023-04-24 08:24:18 +0200] information/ApiListener: Sending config updates for endpoint 'master1.yyyy.zzz' in zone 'zone1.yyyy.zzz'.
[2023-04-24 08:24:18 +0200] information/ApiListener: Finished sending config file updates for endpoint 'master1.yyyy.zzz' in zone 'zone1.yyyy.zzz'.
[2023-04-24 08:24:18 +0200] warning/JsonRpcConnection: API client disconnected for identity 'master1.yyyy.zzz'
[2023-04-24 08:24:18 +0200] information/ApiListener: Syncing runtime objects to endpoint 'master1.yyyy.zzz'.
[2023-04-24 08:24:18 +0200] warning/ApiListener: Removing API client for endpoint 'master1.yyyy.zzz'. 0 API clients left.
[2023-04-24 08:24:18 +0200] information/ApiListener: Finished syncing runtime objects to endpoint 'master1.yyyy.zzz'.
[2023-04-24 08:24:18 +0200] information/ApiListener: Finished sending runtime config updates for endpoint 'master1.yyyy.zzz' in zone 'zone1.yyyy.zzz'.
[2023-04-24 08:24:18 +0200] information/ApiListener: Sending replay log for endpoint 'master1.yyyy.zzz' in zone 'zone1.yyyy.zzz'.
[2023-04-24 08:24:18 +0200] information/ApiListener: Finished sending replay log for endpoint 'master1.yyyy.zzz' in zone 'zone1.yyyy.zzz'.
[2023-04-24 08:24:18 +0200] information/ApiListener: Finished syncing endpoint 'master1.yyyy.zzz' in zone 'zone1.yyyy.zzz'.
[2023-04-24 08:24:21 +0200] information/ApiListener: New client connection for identity 'master2.yyyy.zzz' from [::ffff:192.168.2.189]:33094
[2023-04-24 08:24:21 +0200] information/JsonRpcConnection: Requesting new certificate for this Icinga instance from endpoint 'master2.yyyy.zzz'.
[2023-04-24 08:24:21 +0200] information/ApiListener: Sending config updates for endpoint 'master2.yyyy.zzz' in zone 'zone1.yyyy.zzz'.
[2023-04-24 08:24:21 +0200] information/ApiListener: Finished sending config file updates for endpoint 'master2.yyyy.zzz' in zone 'zone1.yyyy.zzz'.
[2023-04-24 08:24:21 +0200] warning/JsonRpcConnection: API client disconnected for identity 'master2.yyyy.zzz'
[2023-04-24 08:24:21 +0200] information/ApiListener: Syncing runtime objects to endpoint 'master2.yyyy.zzz'.
[2023-04-24 08:24:21 +0200] warning/ApiListener: Removing API client for endpoint 'master2.yyyy.zzz'. 0 API clients left.
[2023-04-24 08:24:21 +0200] information/ApiListener: Finished syncing runtime objects to endpoint 'master2.yyyy.zzz'.
[2023-04-24 08:24:21 +0200] information/ApiListener: Finished sending runtime config updates for endpoint 'master2.yyyy.zzz' in zone 'zone1.yyyy.zzz'.
[2023-04-24 08:24:22 +0200] information/ApiListener: Sending replay log for endpoint 'master2.yyyy.zzz' in zone 'zone1.yyyy.zzz'.
[2023-04-24 08:24:22 +0200] information/ApiListener: Finished sending replay log for endpoint 'master2.yyyy.zzz' in zone 'zone1.yyyy.zzz'.
[2023-04-24 08:24:22 +0200] information/ApiListener: Finished syncing endpoint 'master2.yyyy.zzz' in zone 'zone1.yyyy.zzz'.
[2023-04-24 08:24:27 +0200] information/ApiListener: New client connection for identity 'master1.yyyy.zzz' from [::ffff:192.168.2.210]:48304
[2023-04-24 08:24:27 +0200] information/JsonRpcConnection: Requesting new certificate for this Icinga instance from endpoint 'master1.yyyy.zzz'.
[2023-04-24 08:24:27 +0200] information/ApiListener: Sending config updates for endpoint 'master1.yyyy.zzz' in zone 'zone1.yyyy.zzz'.
[2023-04-24 08:24:27 +0200] warning/JsonRpcConnection: API client disconnected for identity 'master1.yyyy.zzz'
[2023-04-24 08:24:27 +0200] information/ApiListener: Finished sending config file updates for endpoint 'master1.yyyy.zzz' in zone 'zone1.yyyy.zzz'.
[2023-04-24 08:24:27 +0200] warning/ApiListener: Removing API client for endpoint 'master1.yyyy.zzz'. 0 API clients left.
[2023-04-24 08:24:27 +0200] information/ApiListener: Syncing runtime objects to endpoint 'master1.yyyy.zzz'.
[2023-04-24 08:24:27 +0200] information/ApiListener: Finished syncing runtime objects to endpoint 'master1.yyyy.zzz'.
[2023-04-24 08:24:27 +0200] information/ApiListener: Finished sending runtime config updates for endpoint 'master1.yyyy.zzz' in zone 'zone1.yyyy.zzz'.
[2023-04-24 08:24:27 +0200] information/ApiListener: Sending replay log for endpoint 'master1.yyyy.zzz' in zone 'zone1.yyyy.zzz'.
[2023-04-24 08:24:27 +0200] information/ApiListener: Finished sending replay log for endpoint 'master1.yyyy.zzz' in zone 'zone1.yyyy.zzz'.
[2023-04-24 08:24:27 +0200] information/ApiListener: Finished syncing endpoint 'master1.yyyy.zzz' in zone 'zone1.yyyy.zzz'.
[2023-04-24 08:24:27 +0200] information/WorkQueue: #4 (ApiListener, RelayQueue) items: 0, rate: 0/s (0/min 0/5min 0/15min);
I upgraded one of the affected clients to 2.13.7 => No effect
PS C:\Program Files\ICINGA2\sbin> .\icinga2.exe -scm daemon
[2023-04-24 10:29:13 +0200] information/cli: Icinga application loader (version: v2.13.7)
[2023-04-24 10:29:13 +0200] information/cli: Loading configuration file(s).
[2023-04-24 10:29:14 +0200] information/ConfigItem: Committing config item(s).
[2023-04-24 10:29:14 +0200] information/ApiListener: My API identity: aaa.yyyy.zzz
...
[2023-04-24 10:29:14 +0200] information/ConfigItem: Instantiated 1 WindowsEventLogLogger.
[2023-04-24 10:29:14 +0200] information/ConfigItem: Instantiated 1 UserGroup.
[2023-04-24 10:29:14 +0200] information/ConfigItem: Instantiated 1 IcingaApplication.
[2023-04-24 10:29:14 +0200] information/ConfigItem: Instantiated 7 TimePeriods.
[2023-04-24 10:29:14 +0200] information/ConfigItem: Instantiated 19 ServiceGroups.
[2023-04-24 10:29:14 +0200] information/ConfigItem: Instantiated 3 Endpoints.
[2023-04-24 10:29:14 +0200] information/ConfigItem: Instantiated 1 ApiListener.
[2023-04-24 10:29:14 +0200] information/ConfigItem: Instantiated 4 Zones.
[2023-04-24 10:29:14 +0200] information/ConfigItem: Instantiated 14 HostGroups.
[2023-04-24 10:29:14 +0200] information/ConfigItem: Instantiated 8 NotificationCommands.
[2023-04-24 10:29:14 +0200] information/ConfigItem: Instantiated 349 CheckCommands.
[2023-04-24 10:29:14 +0200] information/ScriptGlobal: Dumping variables to file 'C:\ProgramData\icinga2\var\cache\icinga2/icinga2.vars'
[2023-04-24 10:29:14 +0200] information/ConfigObject: Restoring program state from file 'C:\ProgramData\icinga2\var\lib\icinga2/icinga2.state'
[2023-04-24 10:29:14 +0200] information/ConfigObject: Restored 408 objects. Loaded 0 new objects without state.
[2023-04-24 10:29:14 +0200] information/ConfigItem: Triggering Start signal for config items
[2023-04-24 10:29:14 +0200] information/ApiListener: 'api' started.
[2023-04-24 10:29:14 +0200] information/ApiListener: Started new listener on '[::]:5665'
[2023-04-24 10:29:14 +0200] information/ConfigItem: Activated all objects.
[2023-04-24 10:29:14 +0200] information/ApiListener: New client connection for identity 'master2.yyyy.zzz' from [::ffff:192.168.2.189]:45774
[2023-04-24 10:29:14 +0200] information/JsonRpcConnection: Requesting new certificate for this Icinga instance from endpoint 'master2.yyyy.zzz'.
[2023-04-24 10:29:14 +0200] information/ApiListener: Sending config updates for endpoint 'master2.yyyy.zzz' in zone1.yyyy.zzz.
[2023-04-24 10:29:14 +0200] information/ApiListener: Finished sending config file updates for endpoint 'master2.yyyy.zzz' in zone1.yyyy.zzz.
[2023-04-24 10:29:14 +0200] information/ApiListener: Syncing runtime objects to endpoint 'master2.yyyy.zzz'.
[2023-04-24 10:29:14 +0200] information/ApiListener: Finished syncing runtime objects to endpoint 'master2.yyyy.zzz'.
[2023-04-24 10:29:14 +0200] information/ApiListener: Finished sending runtime config updates for endpoint 'master2.yyyy.zzz' in zone1.yyyy.zzz.
[2023-04-24 10:29:14 +0200] information/ApiListener: Sending replay log for endpoint 'master2.yyyy.zzz' in zone1.yyyy.zzz.
[2023-04-24 10:29:14 +0200] warning/JsonRpcConnection: API client disconnected for identity 'master2.yyyy.zzz'
[2023-04-24 10:29:14 +0200] information/ApiListener: Finished sending replay log for endpoint 'master2.yyyy.zzz' in zone1.yyyy.zzz.
[2023-04-24 10:29:14 +0200] warning/ApiListener: Removing API client for endpoint 'master2.yyyy.zzz'. 0 API clients left.
[2023-04-24 10:29:14 +0200] information/ApiListener: Finished syncing endpoint 'master2.yyyy.zzz' in zone1.yyyy.zzz.
[2023-04-24 10:29:15 +0200] information/ApiListener: New client connection for identity 'master1.yyyy.zzz' from [::ffff:192.168.2.210]:57876
[2023-04-24 10:29:15 +0200] information/JsonRpcConnection: Requesting new certificate for this Icinga instance from endpoint 'master1.yyyy.zzz'.
[2023-04-24 10:29:15 +0200] information/ApiListener: Sending config updates for endpoint 'master1.yyyy.zzz' in zone1.yyyy.zzz.
[2023-04-24 10:29:15 +0200] information/ApiListener: Finished sending config file updates for endpoint 'master1.yyyy.zzz' in zone1.yyyy.zzz.
[2023-04-24 10:29:15 +0200] warning/JsonRpcConnection: API client disconnected for identity 'master1.yyyy.zzz'
[2023-04-24 10:29:15 +0200] information/ApiListener: Syncing runtime objects to endpoint 'master1.yyyy.zzz'.
[2023-04-24 10:29:15 +0200] warning/ApiListener: Removing API client for endpoint 'master1.yyyy.zzz'. 0 API clients left.
[2023-04-24 10:29:15 +0200] information/ApiListener: Finished syncing runtime objects to endpoint 'master1.yyyy.zzz'.
[2023-04-24 10:29:15 +0200] information/ApiListener: Finished sending runtime config updates for endpoint 'master1.yyyy.zzz' in zone1.yyyy.zzz.
[2023-04-24 10:29:15 +0200] information/ApiListener: Sending replay log for endpoint 'master1.yyyy.zzz' in zone1.yyyy.zzz.
[2023-04-24 10:29:15 +0200] information/ApiListener: Finished sending replay log for endpoint 'master1.yyyy.zzz' in zone1.yyyy.zzz.
[2023-04-24 10:29:15 +0200] information/ApiListener: Finished syncing endpoint 'master1.yyyy.zzz' in zone1.yyyy.zzz.
[2023-04-24 10:29:17 +0200] information/ApiListener: New client connection for identity 'master2.yyyy.zzz' from [::ffff:192.168.2.189]:52292
[2023-04-24 10:29:17 +0200] information/JsonRpcConnection: Requesting new certificate for this Icinga instance from endpoint 'master2.yyyy.zzz'.
[2023-04-24 10:29:17 +0200] information/ApiListener: Sending config updates for endpoint 'master2.yyyy.zzz' in zone1.yyyy.zzz.
[2023-04-24 10:29:17 +0200] information/ApiListener: Finished sending config file updates for endpoint 'master2.yyyy.zzz' in zone1.yyyy.zzz.
[2023-04-24 10:29:17 +0200] information/ApiListener: Syncing runtime objects to endpoint 'master2.yyyy.zzz'.
[2023-04-24 10:29:17 +0200] information/ApiListener: Finished syncing runtime objects to endpoint 'master2.yyyy.zzz'.
[2023-04-24 10:29:17 +0200] information/ApiListener: Finished sending runtime config updates for endpoint 'master2.yyyy.zzz' in zone1.yyyy.zzz.
[2023-04-24 10:29:17 +0200] information/ApiListener: Sending replay log for endpoint 'master2.yyyy.zzz' in zone1.yyyy.zzz.
[2023-04-24 10:29:17 +0200] information/ApiListener: Finished sending replay log for endpoint 'master2.yyyy.zzz' in zone1.yyyy.zzz.
[2023-04-24 10:29:17 +0200] information/ApiListener: Finished syncing endpoint 'master2.yyyy.zzz' in zone1.yyyy.zzz.
[2023-04-24 10:29:17 +0200] warning/JsonRpcConnection: API client disconnected for identity 'master2.yyyy.zzz'
[2023-04-24 10:29:17 +0200] warning/ApiListener: Removing API client for endpoint 'master2.yyyy.zzz'. 0 API clients left.
[2023-04-24 10:29:21 +0200] information/Application: Received request to shut down.
[2023-04-24 10:29:21 +0200] information/ApiListener: New client connection for identity 'master1.yyyy.zzz' from [::ffff:192.168.2.210]:60952
[2023-04-24 10:29:21 +0200] information/JsonRpcConnection: Requesting new certificate for this Icinga instance from endpoint 'master1.yyyy.zzz'.
[2023-04-24 10:29:21 +0200] information/ApiListener: Sending config updates for endpoint 'master1.yyyy.zzz' in zone1.yyyy.zzz.
[2023-04-24 10:29:21 +0200] information/ApiListener: Finished sending config file updates for endpoint 'master1.yyyy.zzz' in zone1.yyyy.zzz.
[2023-04-24 10:29:21 +0200] warning/JsonRpcConnection: API client disconnected for identity 'master1.yyyy.zzz'
[2023-04-24 10:29:21 +0200] information/ApiListener: Syncing runtime objects to endpoint 'master1.yyyy.zzz'.
[2023-04-24 10:29:21 +0200] warning/ApiListener: Removing API client for endpoint 'master1.yyyy.zzz'. 0 API clients left.
[2023-04-24 10:29:21 +0200] information/ApiListener: Finished syncing runtime objects to endpoint 'master1.yyyy.zzz'.
[2023-04-24 10:29:21 +0200] information/ApiListener: Finished sending runtime config updates for endpoint 'master1.yyyy.zzz' in zone1.yyyy.zzz.
[2023-04-24 10:29:21 +0200] information/ApiListener: Sending replay log for endpoint 'master1.yyyy.zzz' in zone1.yyyy.zzz.
[2023-04-24 10:29:21 +0200] information/ApiListener: Finished sending replay log for endpoint 'master1.yyyy.zzz' in zone1.yyyy.zzz.
[2023-04-24 10:29:22 +0200] information/ApiListener: Finished syncing endpoint 'master1.yyyy.zzz' in zone1.yyyy.zzz.
[2023-04-24 10:29:22 +0200] information/Application: Shutting down...
[2023-04-24 10:29:22 +0200] information/ApiListener: 'api' stopped.
[2023-04-24 10:29:22 +0200] information/ConfigObject: Dumping program state to file 'C:\ProgramData\icinga2\var\lib\icinga2/icinga2.state'
[2023-04-24 10:29:22 +0200] information/IcingaApplication: Icinga has shut down.
However, removing the Icinga installation on an affected client. Deleting the Icinga Director host object & publishing the configuration & reinstalling Icinga on that client fixes the issue.
But this is not a suitable solution…
Could this be related to:
If yes, is there an easy fix?
Best regards
JP