Certificate validation failed: code 7: certificate signature failure

Icinga 2
1

Hello everyone, we have icinga2 master node and many endpoints/agents. Some of the agents are failed now to connect to a master and I see the following error in icingaweb2.

Zone ‘clientXXX.de’ is not connected. Log lag: 2 days, 4 hours, 37 minutes and 15 seconds

I have tried to dig thru logs to find out what the issue this and found this:

[2022-10-21 15:41:50 +0200] information/ApiListener: Reconnecting to endpoint 'clientXXX.XX' via host 'XX.XX.XX.XX' and port '5665'
[2022-10-21 15:41:50 +0200] warning/JsonRpcConnection: API client disconnected for identity 'clientXXX.de'
[2022-10-21 15:41:50 +0200] warning/ApiListener: Certificate validation failed for endpoint 'clientXXX.de': code 7: certificate signature failure
[2022-10-21 15:41:50 +0200] information/ApiListener: New client connection for identity 'clientXXX.XX' to [XX.XX.XX.XX]:5665 (certificate validation failed: code 7: certificate signature failure)
[2022-10-21 15:41:50 +0200] information/ApiListener: Finished reconnecting to endpoint 'clientXXX.XX' via host 'XX.XX.XX.XX' and port '5665'
[2022-10-21 15:41:50 +0200] information/JsonRpcConnection: Received certificate request for CN 'clientXXX.de' signed by our CA.
[2022-10-21 15:41:50 +0200] information/JsonRpcConnection: The certificate for CN 'clientXXX.XX' is valid and uptodate. Skipping automated renewal.

I have tried to complete the steps described hier on master and on the agent:
docs/icinga-2/2.12/doc/06-distributed-monitoring/#signing-certificates-on-the-master

I have created ticket on master, then ran icinga2 node wizard on the agent, completed the wizard, gave the ticket on the agent and then signed the request from the agent on the master.

Restarted icinga agent and got the absolutely the same error. What did I wrong here?

2

What does the log on the agent say?

3 
[2022-11-07 14:55:59 +0100] information/ApiListener: Replayed 100534 messages.
[2022-11-07 14:56:02 +0100] information/ApiListener: Finished sending replay log for endpoint 'icinga' in zone 'master'.
[2022-11-07 14:56:02 +0100] information/ApiListener: Finished syncing endpoint 'icinga' in zone 'master'.
[2022-11-07 14:56:07 +0100] information/ApiListener: New client connection for identity 'icinga' from [::ffff:xx.xx.xx.xx]:46984
[2022-11-07 14:56:07 +0100] warning/JsonRpcConnection: API client disconnected for identity 'icinga'
[2022-11-07 14:56:07 +0100] warning/ApiListener: Removing API client for endpoint 'icinga'. 0 API clients left.
[2022-11-07 14:56:07 +0100] information/JsonRpcConnection: Requesting new certificate for this Icinga instance from endpoint 'icinga'.
[2022-11-07 14:56:07 +0100] information/ApiListener: Sending config updates for endpoint 'icinga' in zone 'master'.
[2022-11-07 14:56:07 +0100] information/ApiListener: Finished sending config file updates for endpoint 'icinga' in zone 'master'.
[2022-11-07 14:56:07 +0100] information/ApiListener: Syncing runtime objects to endpoint 'icinga'.
[2022-11-07 14:56:07 +0100] information/ApiListener: Finished syncing runtime objects to endpoint 'icinga'.
[2022-11-07 14:56:07 +0100] information/ApiListener: Finished sending runtime config updates for endpoint 'icinga' in zone 'master'.
[2022-11-07 14:56:07 +0100] information/ApiListener: Sending replay log for endpoint 'icinga' in zone 'master'.
[2022-11-07 14:56:09 +0100] information/ApiListener: Replayed 100534 messages.
[2022-11-07 14:56:12 +0100] information/ApiListener: Finished sending replay log for endpoint 'icinga' in zone 'master'.
[2022-11-07 14:56:12 +0100] information/ApiListener: Finished syncing endpoint 'icinga' in zone 'master'.
[2022-11-07 14:56:17 +0100] information/ApiListener: New client connection for identity 'icinga.' from [::ffff:xx.xx.xx.xx]:47106
[2022-11-07 14:56:17 +0100] warning/JsonRpcConnection: API client disconnected for identity 'icinga'
[2022-11-07 14:56:17 +0100] warning/ApiListener: Removing API client for endpoint 'icinga'. 0 API clients left.
[2022-11-07 14:56:17 +0100] information/JsonRpcConnection: Requesting new certificate for this Icinga instance from endpoint 'icinga'.
[2022-11-07 14:56:17 +0100] information/ApiListener: Sending config updates for endpoint 'icinga' in zone 'master'.
[2022-11-07 14:56:17 +0100] information/ApiListener: Finished sending config file updates for endpoint 'icinga' in zone 'master'.
[2022-11-07 14:56:17 +0100] information/ApiListener: Syncing runtime objects to endpoint 'icinga'.
[2022-11-07 14:56:17 +0100] information/ApiListener: Finished syncing runtime objects to endpoint 'icinga'.
[2022-11-07 14:56:17 +0100] information/ApiListener: Finished sending runtime config updates for endpoint 'icinga' in zone 'master'.
[2022-11-07 14:56:17 +0100] information/ApiListener: Sending replay log for endpoint 'icinga' in zone 'master'.
[2022-11-07 14:56:19 +0100] information/ApiListener: Replayed 100534 messages.
[2022-11-07 14:56:22 +0100] information/ApiListener: Finished sending replay log for endpoint 'icinga' in zone 'master'.
[2022-11-07 14:56:22 +0100] information/ApiListener: Finished syncing endpoint 'icinga' in zone 'master'.
[2022-11-07 14:56:27 +0100] information/ApiListener: New client connection for identity 'icinga' from [::ffff:xx.xx.xx.xx]:47306
[2022-11-07 14:56:27 +0100] warning/JsonRpcConnection: API client disconnected for identity 'icinga'
[2022-11-07 14:56:27 +0100] warning/ApiListener: Removing API client for endpoint 'icinga'. 0 API clients left.
4

Hm, looks like a sync loop on the agent.

Have you checked the log on the master at the same time?
Or does it only complain about the certificate problem?

Please also show the config object for the agent host on the master.
In case you are using the Icinga Director, be sure not to create endpoints and zones manually there.
This can result in a not-working cluster sync, as we currently are experiencing (though not with agents but satellite zones)…
If you use the Director be sure to follow the docs on the Agent configuration:
https://icinga.com/docs/icinga-director/latest/doc/24-Working-with-agents/

5

your certitficate on the client is wrong or not signed correctly