Dear all, I’m trying to set up a configuration with two masters. I set up the masters with the node wizard and then added each other in both /etc/icinga2/zones.conf files.
However, the Director elects a Config master, and deploys the configuration on only that master. If the config master goes down, or if some reason the “Active Endpoint” listed under Icinga Web 2’s Monitoring Heath page changes, the other master’s configuration is used. Is there a way to fix this so that the configuration stays synchronized between masters?
You can only have one config master, that being said the elected config master should sync its config to the other endpoint in its zone. What do your zones.conf and api.conf look like?
Looks normal to me. Keep an eye out for log messages from the ApiListener on your non config master. There should be a line like Applying config update from endpoint... or Ignoring config update for zone .... Log level information and warning respectively.
From the non config master:
[2019-06-27 15:34:33 +0200] information/ApiListener: Applying config update from endpoint ‘gnode3.ud.nohup.it’ of zone ‘master’.
[2019-06-27 15:34:33 +0200] warning/ApiListener: Ignoring config update for zone ‘director-global’ because we have an authoritative version of the zone’s config.
[2019-06-27 15:34:33 +0200] warning/ApiListener: Ignoring config update for zone ‘master’ because we have an authoritative version of the zone’s config.
From the config master:
[2019-06-27 15:34:35 +0200] information/ApiListener: Applying config update from endpoint ‘icinga2.ud.nohup.it’ of zone ‘master’.
[2019-06-27 15:34:35 +0200] warning/ApiListener: Ignoring config update for zone ‘director-global’ because we have an authoritative version of the zone’s config.
[2019-06-27 15:34:35 +0200] warning/ApiListener: Ignoring config update for zone ‘master’ because we have an authoritative version of the zone’s config.
Both endpoints either have configuration in /etc/icinga2/zones.d or in a deployed api config package which uses zones.d. I would guess that you’ve used the Director to deploy its package against both REST APIs. You can only do this for one endpoint, with a manual failover to the secondary host.
In order to clean things up, delete the director config package on the second master endpoint. This can be done with a curl request as explained in the API docs for Icinga 2.
Dear Michael, I removed the config package on the second master with a curl DELETE. The synchronization seems to work now (or at the very least, the following file is kept in sync: /var/lib/icinga2/api/zones/master/director/hosts.conf). However, when I disable the first master and the second master becomes the active endpoint, the list of hosts and services to check becomes empty. Do you have any idea?
This would imply that the secondary master does not have any configuration, or the IDO feature is not active on this endpoint. What’s the output of the monitoring health view in Icinga Web 2 for this?
So, nothing is synced from the first master to the secondary master. Check the logs, is there really an entry on the secondary master that the config is synced?
Hi, I looked at the logs but it seems that the configuration is properly synced. All files in /var/lib/icinga2/api/zones/director-global/director and /var/lib/icinga2/api/zones/master/director are the same on both masters.
It is as if the configuration inside /var/lib/icinga2/api/zones is ignored by the second master.
Please show the full tree of that directory, e.g. ls -lR /var/lib/icinga2/api/zones. Also, run icinga2 daemon -C -x notice and look for the included directories.
Glad you’ve solved it. Deleting the two files would have been enough, but I wanted to ensure that Icinga re-dumps everything from a cluster config sync without manual interaction.
Instead of editing the topic’s subject, please pick a reply and mark this as solution. This helps others to search for solved topics, and also grants a badge for those being helpful the most.
I have also set up icingaweb2 and director on both masters all sharing a MySQL DB. But the sync does not seem to be happening on the second master. I checked on the community site and a post that recommends this:-
In order to clean things up, delete the director config package on the second master endpoint. This can be done with a curl request as explained in the API docs for Icinga 2.
I understand how to execute a curl API request but what is the URL for deleting the directory config package on my second master endpoint?