I’m having issues with Dual HA Master setup with all clients on 2.12.2-1.
The /var/lib/icinga2/api/logs folder on Master1 is filling up with numbered logs and the 2nd Master was having OOM issues. With both Masters on, the stack used to work fine but now random zones will have late checks.
Running
var res = {}; for (s in get_objects(Service).filter(s => s.last_check < get_time() - 2 * s.check_interval)) { res[s.paused] += 1 }; res
on either master returns {} so I am unable to check if there is a split brain scenario happening between the two masters. Icingaweb2 seems to be fine and returning check results though.
My question is: While I’m trying to troubleshoot this issue, is it possible to downgrade the setup back to a Single master setup by just editing the zones.conf on the master and each pair of satellites in each zone to remove the 2nd master? Is there any other factors I should take into consideration here?