I have 2 data centres with a master node in each, I’m attempting to create a topology where if a datacentre is flooded, or goes offline the other master node should continue to monitor the entire network.
I’ve already setup Icinga2 with IcingaDB in a HA configuration, but where I’m stuck and seeking advice is on the IcingaDB system, as my current SQL server is a single point of failure.
Should I try using MariaDB with Galera and create a 3 node cluster to replicate data between all MariaDB instances? or is MariaDB’s inbuilt replication suitable?
If I do go down the route of multiple SQL servers, should each Icinga2 node write to the local SQL database?
As mentioned, I have infrastructure in 2 data centres, but the DC’s are some 600 kilometres apart from each other, I don’t really have a layer2 network between the two, so each icinga node is in a different subnet, most examples I’ve seen of DB clusters seem to have all the instances in the same local area network.
I’ve included a very basic topology diagram to help explain the topology I have to work with.
