Hello friends
I currently have one master and two satellites,which are working and there is no problem.
In the future I will add another satellite which will not be a problem either.
My question is how can I add a new master to this structure?
I want to install Galera on two masters to sync the databases.
But I think if the first master fails, the second master will have problems because it does not have the satellite keys!
Do you have a solution?
Because I have read the documents, everyone says that to set up a new master, I must first set the master as a satellite and then change it in my zones.
But this way the new master will not know about the satellites keys.
Hi
You need to edit the zone.conf files on the old master and the existing satellites to add the second master as additional endpoint of the master zone.
What do you mean by keys? Icinga uses a PKI to establish trust between the nodes but this is only for authentication as the roles are defined by the zone.conf (authorization) and features enabled.
As all nodes have the same CA they don’t need to know about the keys of the others as they can verify the certificate via the CA and in there zone.conf they can look up the role (parent, partner or child) in the cluster of the other node.
Also doesn’t Galera require 3 nodes and does it make sense to have the DB on the master nodes and not on dedicated hosts?
I once went through the icinga2 node wizard when I wanted to add satellites and at the end it asked me to create a PKI for the satellites to communicate with the master, right? Now my question is, do I need to do this again for each satellite and sign the new master (icinga2 pki ticket --cn satellitex0)?
AFAIK, you need to create the certificate for the endpoint only once, as long as you don’t rename the endpoint in the zone.conf in the same zone and the ones above or below. If you replace the CA on the config master you will also have to redo all the certificates.
So I don’t need to define them again on the new master?
Just run the icinga2 node wizard once on the new master and configure it as master.
Do I need to change anything else on the old master except zones.conf? (Like running the icinga2 node wizard)
I think setting it up as a satellite first and in a second step move it into the master zone in the zone.conf files of all the nodes, is the correct thing to do.
Okay, thanks.
I’ll try it and let you know here if everything is correct. 
1 Like
HI Dominik,
I initially configured the second master as a satellite for the first master (using icinga2 node wizard). Afterward (before restarting the icinga2 service), I edited the zones and placed the second master in the master’s zone:
object Endpoint "smon" {
host = "x"
port = "5665"
}
object Zone "smon" {
endpoints = [ "smon", "smon02" ]
}
object Zone "global-templates" {
global = true
}
object Zone "director-global" {
global = true
}
object Zone "satellite" {
endpoints = [ "satellite02", "satellite01" ]
parent = "smon"
}
object Endpoint "satellite02" {
host = "x"
port = "5665"
}
object Endpoint "satellite01" {
host = "x"
port = "5665"
}
object Endpoint "smon02" {
host = "x"
port = "5665"
}
Additionally, I have set up Galera between the two masters, and it is running successfully.
Previously, my setup included 1 master and 2 satellites. Now, when I restart the service, the dashboard becomes entirely empty—all services and hosts disappear. (They are still present in Icinga Director, but nothing appears on the dashboard).
What do you think the problem is?
I have kept the icinga-director service enabled only on Master 1 because if it’s enabled on both masters, it gets locked, and the logs indicate that it is locked.
The Icinga Web version for Master 1 is: icingaweb2-2.11.4, and for Master 2, it is: icingaweb2-2.12.2. However, I don’t think the issue is related to the version since the modules are identical on both.
I also noticed in the logs of Master 2 that there are issues with the certificates for the satellites.
Thanks
Yes, the director is not HA aware and the service should only run on one master.
My guess is, that you still have an error in the zone configs and or need to clear the Internal paths outside of /etc. You can get all the paths via icinga2 --version.
Master 1 :
icinga2 - The Icinga 2 network monitoring daemon (version: r2.14.2-1)
Copyright (c) 2012-2024 Icinga GmbH (https://icinga.com/)
License GPLv2+: GNU GPL version 2 or later <https://gnu.org/licenses/gpl2.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
System information:
Platform: SLES
Platform version: 15-SP5
Kernel: Linux
Kernel version: 5.14.21-150500.55.52-default
Architecture: x86_64
Build information:
Compiler: GNU 7.5.0
Build host: unknown
OpenSSL version: OpenSSL 1.1.1l-fips 24 Aug 2021 SUSE release 150500.17.25.1
Application information:
General paths:
Config directory: /etc/icinga2
Data directory: /var/lib/icinga2
Log directory: /var/log/icinga2
Cache directory: /var/cache/icinga2
Spool directory: /var/spool/icinga2
Run directory: /run/icinga2
Old paths (deprecated):
Installation root: /usr
Sysconf directory: /etc
Run directory (base): /run
Local state directory: /var
Internal paths:
Package data directory: /usr/share/icinga2
State path: /var/lib/icinga2/icinga2.state
Modified attributes path: /var/lib/icinga2/modified-attributes.conf
Objects path: /var/cache/icinga2/icinga2.debug
Vars path: /var/cache/icinga2/icinga2.vars
PID path: /run/icinga2/icinga2.pid
Master2:
icinga2 - The Icinga 2 network monitoring daemon (version: r2.14.3-1)
Copyright (c) 2012-2024 Icinga GmbH (https://icinga.com/)
License GPLv2+: GNU GPL version 2 or later <https://gnu.org/licenses/gpl2.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
System information:
Platform: openSUSE Leap
Platform version: 15.6
Kernel: Linux
Kernel version: 6.4.0-150600.23.25-default
Architecture: x86_64
Build information:
Compiler: GNU 7.5.0
Build host: unknown
OpenSSL version: OpenSSL 3.1.4 24 Oct 2023
Application information:
General paths:
Config directory: /etc/icinga2
Data directory: /var/lib/icinga2
Log directory: /var/log/icinga2
Cache directory: /var/cache/icinga2
Spool directory: /var/spool/icinga2
Run directory: /run/icinga2
Old paths (deprecated):
Installation root: /usr
Sysconf directory: /etc
Run directory (base): /run
Local state directory: /var
Internal paths:
Package data directory: /usr/share/icinga2
State path: /var/lib/icinga2/icinga2.state
Modified attributes path: /var/lib/icinga2/modified-attributes.conf
Objects path: /var/cache/icinga2/icinga2.debug
Vars path: /var/cache/icinga2/icinga2.vars
PID path: /run/icinga2/icinga2.pid
Zones Master 2 :
object Endpoint "smon" {
host = "193.170.214.132"
port = "5665"
}
object Zone "smon" {
endpoints = [ "smon","smon02" ]
}
object Zone "global-templates" {
global = true
}
object Zone "director-global" {
global = true
}
object Zone "satellite" {
endpoints = [ "satellite02","satellite01" ]
parent = "smon"
}
object Endpoint "satellite02" {
host = "x"
port = "5665"
}
object Endpoint "satellite01" {
host = "x"
port = "5665"
}
object Endpoint "smon02" {
host = "x"
port = "5665"
}
Zone Master1 :
object Endpoint "smon" {
host = "x"
port = "5665"
}
object Zone "smon" {
endpoints = [ "smon", "smon02" ]
}
object Zone "global-templates" {
global = true
}
object Zone "director-global" {
global = true
}
object Zone "satellite" {
endpoints = [ "satellite02", "satellite01" ]
parent = "smon"
}
object Endpoint "satellite02" {
host = "x"
port = "5665"
}
object Endpoint "satellite01" {
host = "x"
port = "5665"
}
object Endpoint "smon02" {
host = "x"
port = "5665"
}
Idid that to : systemctl stop influxdb.service
Thanks agein
Looks ok to me. Did you clear any of the internals of the second master?
Did you mean on the second master?
Is there any specific part I need to clear (like configurations or something)?
I’m still relatively new to Icinga, so apologies if I don’t fully understand.
Yes on the second master.
I would start with the “Internal paths:” you got from icinga2 --version.
Does that mean I should completely remove all these paths? ( Master2)
Internal paths:
- Package data directory: /usr/share/icinga2
- State path: /var/lib/icinga2/icinga2.state
- Modified attributes path: /var/lib/icinga2/modified-attributes.conf
- Objects path: /var/cache/icinga2/icinga2.debug
- Vars path: /var/cache/icinga2/icinga2.vars
- PID path: /run/icinga2/icinga2.pid
Not sure if all of them need to go but yes.
Maybe the /var/lib/icinga2/certs/ shouldn’t be removed or you need to resign the node in the PKI.
I removed those paths as suggested and restarted the service, but unfortunately, it didn’t help. The dashboard is still completely empty, just like before.
Maybe this has nothing to do with the second master.
The Dashboard is depending on the DB and isn’t directly connecting to icinga2.
What do you get if you talk to the icinga2 API or use icinga2 daemon -C --dump-objects && icinga2 object list?
I didn’t see any objects of type host or service.
icinga2 daemon -C --dump-objects && icinga2 object list --type host
icinga2 daemon -C --dump-objects && icinga2 object list --type service
Do you have them defined in /etc/icinga2/conf.d/ or in /etc/icinga2/zones.d/master/ or in your case /etc/icinga2/zones.d/smon/?
On Master2, nothing appears, which might be the reason why nothing is shown.
But since the database is synced, it should display the information.
The issue might be that data exchange between the two masters is not happening after starting the service.
When I run icinga2 daemon, the output is not the same as on Master1.
The logs I shared are also from Master2.
I use the Director module, so I add hosts and services through the graphical interface. All of them are located in this path:
/var/lib/icinga2/api/zones.
Did you enable HA and are both using the same DB?