Problem with two masters and Icingaweb2

Hello,

I have an problem with the ha master setup and icingaweb2.
I recently fixed my problem with the config sync between the masters and then always when the seconf master (not the config master) is online, it shows me nothing in icingaweb2.
When i shut down the second master and only the frist is active i can see all normally in icingaweb2.
Before the fix with the config sync both masters worked fine with icingaweb2 and nothing was changed in the config files.
Does anyone know what this could be?

The zones.conf:

/*
 * Generated by Icinga 2 node setup commands
 * on 2018-01-10 17:39:07 +0100
 */
object Endpoint "icinga02.picturemaxx.net" {
  host = "xxxxxxxxxxx"
}

object Endpoint "pr-be-ic-mas-02" {
  host = "xxxxxxxxxxxx"
}

#object Endpoint "ic-sat-qsc01" {
#  host = "xxxxxxxxxxxxx"
#  /* host = "xxxxxxxxxxx" */
#}

object Endpoint "ic-sat-qsc01-VM" {
  host = "xxxxxxxxxxxxxx"
}

#object Endpoint "icinga02-sat-off.picturemaxx.net" {
#  host = "xxxxxxxxxxxx"
#}

#object Endpoint "icinga02-sat-aws.picturemaxx.net" {
#  host = "xxxxxxxxxxxx"
#}

object Endpoint "ic-sat-aws-apne1-01" {
  host = "xxxxxxxxxxxxx"
}

object Endpoint "ic-sat-aws-use1-01" {
  host = "xxxxxxxxxxxxx"
}

object Endpoint "ic-sat-aws-euw1-01" {
  host = "xxxxxxxxxxxxx"
}

object Endpoint "ic-sat-aws-euc1-01" {
  host = "xxxxxxxxxxxx"
}

object Endpoint "pr-be-ic-sat-off-01" {
  host = "xxxxxxxxxxx"
}

object Zone "ic-mas-qsc01" {
  endpoints = [ "icinga02.picturemaxx.net", "pr-be-ic-mas-02" ]
}

object Zone "ic-sat-qsc01" {
  endpoints = [ "ic-sat-qsc01-VM" ]
  parent = "ic-mas-qsc01"
}

object Zone "ic-sat-off01" {
  endpoints = [ "pr-be-ic-sat-off-01" ]
  parent = "ic-mas-qsc01"
}

#object Zone "ic-sat-aws01" {
#  endpoints = [ "icinga02-sat-aws.picturemaxx.net" ]
#  parent = "ic-mas-qsc01"
#}

object Zone "ic-sat-aws-apne1-01" {
  endpoints = [ "ic-sat-aws-apne1-01" ]
  parent = "ic-mas-qsc01"
}

object Zone "ic-sat-aws-euw1-01" {
  endpoints = [ "ic-sat-aws-euw1-01" ]
  parent = "ic-mas-qsc01"
}

object Zone "ic-sat-aws-euc1-01" {
  endpoints = [ "ic-sat-aws-euc1-01" ]
  parent = "ic-mas-qsc01"
}

object Zone "ic-sat-aws-use1-01" {
  endpoints = [ "ic-sat-aws-use1-01" ]
  parent = "ic-mas-qsc01"
}

object Zone "global-config" {
	global = true
}

object Zone "global-templates" {
  global = true
}

The api.conf:

/**
 * The API listener is used for distributed monitoring setups.
 */
object ApiListener "api" {
  accept_config = true
  accept_commands = true

  ticket_salt = TicketSalt
}
object ApiUser "client-pki-ticket" {
  password = "xxxxxxxxxxxxxxx"
  permissions = [ "actions/generate-ticket" ]
}
object ApiUser "director" {
  password = "xxxxxxxxxxxxxxx"
  permissions = [ "*" ]
}
object ApiUser "dashing" {
  password = "xxxxxxxxxxxxxxxx"
  permissions = [ "status/query", "objects/query/*" ]
}

The ido-mysql.conf:

library "db_ido_mysql"

object IdoMysqlConnection "ido-mysql" {
  user = "icinga2",
  password = "xxxxxxxxxxxx",
  host = "10.49.96.196",
  database = "icinga2"

  cleanup = {
    contactnotifications_age = 30d
    downtimehistory_age = 48h
    servicechecks_age = 96h
    statehistory_age = 48h
    flappinghistory_age = 48h
  }
}

The Icingaweb2 command transport config:

[icinga2-api]
transport = "api"
host = "10.49.80.25"
port = "5665"
username = "director"
password = "xxxxxxxxxxxxxxx"

[icinga2-api-2nd-master]
transport = "api"
host = "10.49.80.250"
port = "5665"
username = "director"
password = "xxxxxxxxxxxxxx"

[icinga2]
transport = "local"
path = "/var/run/icinga2/cmd/icinga2.cmd"

The Icingaweb2 resources config:

[icingaweb_db]
type = "db"
db = "mysql"
#host = "localhost"
host = "10.49.96.196"
port = ""
dbname = "icingaweb2"
username = "icingaweb2"
password = "xxxxxxxxxxxxx"
#charset = ""
charset = "latin1"
persistent = "0"
use_ssl = "0"

[icinga_ido]
type = "db"
db = "mysql"
#host = "localhost"
host = "10.49.96.196"
port = ""
dbname = "icinga2"
username = "icinga2"
password = "xxxxxxxxxxx"
charset = "latin1"
#charset = ""
persistent = "0"
use_ssl = "0"

[ad]
type = "ldap"
hostname = "10.49.88.8"
port = "389"
root_dn = "DC=muc,DC=pm"
bind_dn = "CN=svc_SSO,OU=Agents,OU=PM-MUC,DC=muc,DC=pm"
bind_pw = "xxxxxxxxxxxxxx"

[icingadb]
type = "db"
db = "mysql"
host = "127.0.0.1"
dbname = "icingadb"
username = "icingadb"
password = "icingadb"
use_ssl = "0"
charset = "latin1"
  • Version used (icinga2 --version):
icinga2 - The Icinga 2 network monitoring daemon (version: r2.13.7-1)

Copyright (c) 2012-2023 Icinga GmbH (https://icinga.com/)
License GPLv2+: GNU GPL version 2 or later <https://gnu.org/licenses/gpl2.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

System information:
  Platform: Debian GNU/Linux
  Platform version: 11 (bullseye)
  Kernel: Linux
  Kernel version: 5.10.0-23-amd64
  Architecture: x86_64

Build information:
  Compiler: GNU 10.2.1
  Build host: runner-hh8q3bz2-project-575-concurrent-0
  OpenSSL version: OpenSSL 1.1.1n  15 Mar 2022

Application information:

General paths:
  Config directory: /etc/icinga2
  Data directory: /var/lib/icinga2
  Log directory: /var/log/icinga2
  Cache directory: /var/cache/icinga2
  Spool directory: /var/spool/icinga2
  Run directory: /run/icinga2

Old paths (deprecated):
  Installation root: /usr
  Sysconf directory: /etc
  Run directory (base): /run
  Local state directory: /var

Internal paths:
  Package data directory: /usr/share/icinga2
  State path: /var/lib/icinga2/icinga2.state
  Modified attributes path: /var/lib/icinga2/modified-attributes.conf
  Objects path: /var/cache/icinga2/icinga2.debug
  Vars path: /var/cache/icinga2/icinga2.vars
  PID path: /run/icinga2/icinga2.pid
  • Operating System and version: Debian 11.6
  • Enabled features (icinga2 feature list):
Disabled features: compatlog debuglog elasticsearch gelf icingadb influxdb influxdb2 opentsdb perfdata statusdata syslog
Enabled features: api checker command graphite ido-mysql livestatus mainlog notification

Greetings
dcz01

Icingaweb2 relies on the DB for content but the above doesn’t look like it uses a HA-DB.
Does it work in the classical monitoring module and only the module icingadb is empty?

Thanks for your answer.
Well this record in the config file was added with an update of icinga2 automatically.
We’re not using icingadb yet and only the classic monitoring module is active in icingaweb2.
There isn’t any module named icingadb.

And i forgot to write that with both masters active no services where shown in icingaweb2.
The hosts created via the API where there and could be found.
Also the initial sync ( Distributed Monitoring - Icinga 2 ) of the important files where made another time just to try but it didn’t change anything.

Now i tried it with a clean state second master by removing the runtime files rm -rf /var/lib/icinga2/api/{zones,zones-stage}/* and then a fresh inital sync another time.
That worked fine and now the second master is showing me again the checks in icingaweb2.
But now its rechecking all services and removeing itself the acks:

Why is that happening?