Situation
The Standard way of installing Icingaweb2 on two servers is to save the configuration-files on a Samba-Share and link to that share. This wasn’t possible in my environment as the only way to share Files via a “Share-Directory” is a mounted MooseFS-Share. My Tests showd that simply adapting the Samba-Principle to my MooseFS-Solution didn’t work out so I had to forge a another way of keeping the servers in sync.
Also: Setting up Puppet and do it the right way ( 
) was out of scope of the project.
Prerequisites
- Webservers are behind HA-Proxy that distributes with standard leastconn-Algorythm (looking on the final Build, that’s just an Information that basically will not matter)
- Icingaweb will be served from two webservers which are running on nginx
- icinga2 HA-Cluster is set up
- both servers have the MooseFS share mounted (not part of this howto)
Conclusion what we will achieve
“Master-1” will serve Icingaweb and syncs its files constantly to the MooseFS-Share. “Master-2” constantly watches the webserver functionality of “Master-1” and syncs its config-files from the MooseFS-Share. The nginx-service on Master-2 is switched of.
If Master-1 fails, Master-2 recognizes and starts its webserver for as long as Master-1 is not reachable.
Let’s do it
Im using debian 9, keep that in mind
- Install lsyncd on both machines
apt-get install lsyncd - create file and folder structure
mkdir -p /var/log/lsyncd/
touch /var/log/lsyncd/{lsyncd.log,lsyncd.status}
mkdir -p /etc/lsyncd
touch /etc/lsyncd/lsyncd.conf.lua - write your configs:
++ lsyncd on master-1
master-1$: vi /etc/lsyncd/lsyncd.conf.lua
settings {
logfile = "/var/log/lsyncd/lsyncd.log",
statusFile = "/var/log/lsyncd/lsyncd.status"
}
sync {
default.rsync,
source = "/usr/share/icingaweb2",
target = "/path/to/moosefs/mount/usr/icingaweb2",
rsync = {
archive = true,
owner = true,
perms = true,
group = true,
compress = false,
whole_file = true,
temp_dir = "/path/to/moosefs/mount/master-1_temp",
}
}
sync {
default.rsync,
source = "/etc/icingaweb2",
target = "/path/to/moosefs/mount/etc/icingaweb2",
rsync = {
archive = true,
owner = true,
perms = true,
group = true,
compress = false,
whole_file = true,
temp_dir = "/path/to/moosefs/mount/master-1_temp",
}
}
sync {
default.rsync,
source = "/usr/share/php/Icinga",
target = "/path/to/moosefs/mount/usr/share/php/Icinga",
rsync = {
archive = true,
owner = true,
perms = true,
group = true,
compress = false,
whole_file = true,
temp_dir = "/path/to/moosefs/mount/master-1_temp",
}
}
- lsyncd-Config on master-2
master-2:$ vi /etc/lsyncd/lsyncd.conf.lua
settings {
logfile = "/var/log/lsyncd/lsyncd.log",
statusFile = "/var/log/lsyncd/lsyncd.status"
}
sync {
default.rsync,
source = "/path/to/moosefs/mount/usr/icingaweb2",
target = "/usr/share/icingaweb2",
rsync = {
archive = true,
owner = true,
perms = true,
group = true,
compress = false,
whole_file = true,
temp_dir = "/path/to/moosefs/mount/master-2_temp",
}
}
sync {
default.rsync,
source = "/path/to/moosefs/mount/etc/icingaweb2",
target = "/etc/icingaweb2",
rsync = {
archive = true,
owner = true,
perms = true,
group = true,
compress = false,
whole_file = true,
temp_dir = "/path/to/moosefs/mount/master-2_temp",
}
}
sync {
default.rsync,
source = "/path/to/moosefs/mount/usr/share/php/Icinga",
target = "/usr/share/php/Icinga",
rsync = {
archive = true,
owner = true,
perms = true,
group = true,
compress = false,
whole_file = true,
temp_dir = "/path/to/moosefs/mount/master-2_temp",
}
}
Implementing the Server-Check on master-2
- What we need:
++ a little bash-script that does the magic
++ a systemd-service
++ a systemd.timer
The bash-Script
#!/bin/bash
# File: /usr/local/bin/icinga-failover.sh
curl -kIs https://10.10.10.42 | head -n 1 | grep -q Found
if [ $? -eq 0 ] ; then
systemctl is-active --quiet nginx
if [ $? -eq 0 ] ; then
systemctl stop nginx
fi
else
systemctl start nginx
fi
The Systemd-Service
# File/etc/systemd/system/icinga-failover.service
[Unit]
Description=Icinga Failover Check via curl
[Service]
ExecStart=/usr/local/bin/icinga-failover.sh
[Install]
WantedBy=multi-user.target
The systemd timer
# File /etc/systemd/system/icinga-failover.timer
[Unit]
Description=Icinga Failover Timer
[Timer]
OnBootSec=10s
OnUnitInactiveSec=1s
[Install]
WantedBy=basic.target
That’s basically it. Enable the service and the timer with the systemctl command and you should be good to go.
You can add /etc/icinga2/ to t he synced folders, too. Then you don’t have to manually sync notification-scripts (as an example).
That’s my first howto, I hope it helps people and you liked it 
