Duplicated Hosts, Services etc in Master/Master Setup

Hello Everyone
I am currently running a Master/Master Setup in Icinga2, the Problem is that all my Hosts are getting duplicated in the icingadb, which really doesnt look great. This happens with everything i put in the Zones.d/master folder. I think it has something to do with the comunication between the two nodes, but i really cant figure out what the exact problem is.

My Icinga Version is 2.13.6-1
My Zones.conf:
object Endpoint “UBUICI2” {
zone = “master”
}

object Endpoint “ubuici3” {
host = “192.168.XXX.XXX”
zone = “master”
port = 5665
}

object Zone “master” {
endpoints = [ “UBUICI2”,“ubuici3”]
}

object Zone “global-templates” {
global = true
}

object Zone “director-global” {
global = true
}
If you need something else please let me know
Best regards xl3von

Hello,

please share a pair of rows representing a duplicated host from the database.

Our IDs are binary, so please use mysql --binary-as-hex -h…

Best,
A/K

The first one is ID (sorry it didnt fit in the picture and it is the host table of icingadb), which attributes do you need to see?
Best regards xl3von

The more the better. At least name, IDs and checksums.

Sorry for the bad layout.

I meant all IDs – *_id. The layout may be fixed with \G instead ; at the end.

id: 0xFDEC392DFA2FCB7ECA9B8A778B1CF099D4F5B305
environment_id: 0x90854C6CC1AA2617991AAA09DE73AAECF6296C1E
name_checksum: 0x852BD40A142F7A66C0422483A075BE259EDEB367
properties_checksum: 0x3F5775DB1CB9B226DEF7ECAF26CB2810D7D07AC7
name: UBU-SHARE
name_ci: UBU-SHARE
display_name: UBU-SHARE
address: 192.168.XXX.XXX
address6:
address_bin: 0xC0A808E2
address6_bin: NULL
checkcommand_name: hostalive
checkcommand_id: 0x6439F5DCE0657C90F4B1276C4A9B3A8F4D9816A0
max_check_attempts: 3
check_timeperiod_name:
check_timeperiod_id: NULL
check_timeout: 0
check_interval: 60
check_retry_interval: 30
active_checks_enabled: y
passive_checks_enabled: y
event_handler_enabled: y
notifications_enabled: y
flapping_enabled: n
flapping_threshold_low: 25
flapping_threshold_high: 30
perfdata_enabled: y
eventcommand_name:
eventcommand_id: NULL
is_volatile: n
action_url_id: NULL
notes_url_id: NULL
notes:
icon_image_id: NULL
icon_image_alt:
zone_name: master
zone_id: 0x97688BD0E5F847B81AB24D341A7627FC4070BA6F
command_endpoint_name: UBUICI2
command_endpoint_id: 0xD427535D7BC035F6F86068F6423FA9F9AA896B5E


id: 0xD66CBC9D77F25412EED9C509D57DC7A12ECFE0BC
environment_id: 0x0A94F3152EC49C6AB8ABEF5A39DB0DE259F5F7E1
name_checksum: 0x852BD40A142F7A66C0422483A075BE259EDEB367
properties_checksum: 0xA5009838096253B468AA33744BE21864238C8BB0
name: UBU-SHARE
name_ci: UBU-SHARE
display_name: UBU-SHARE
address: 192.168.XXX.XXX
address6:
address_bin: 0xC0A808E2
address6_bin: NULL
checkcommand_name: hostalive
checkcommand_id: 0xC8ED3E10988C2A691152C100B38DB2702A83B775
max_check_attempts: 3
check_timeperiod_name:
check_timeperiod_id: NULL
check_timeout: 0
check_interval: 60
check_retry_interval: 30
active_checks_enabled: y
passive_checks_enabled: y
event_handler_enabled: y
notifications_enabled: y
flapping_enabled: n
flapping_threshold_low: 25
flapping_threshold_high: 30
perfdata_enabled: y
eventcommand_name:
eventcommand_id: NULL
is_volatile: n
action_url_id: NULL
notes_url_id: NULL
notes:
icon_image_id: NULL
icon_image_alt:
zone_name: master
zone_id: 0x5F8E91C936BEA3B4F226BEB57EF197EC4CFBB293
command_endpoint_name: UBUICI2
command_endpoint_id: 0x3E602366648B669BE9862B49969C68D8C6E88128

This is a pair of duplicated hosts. And thats all attributes.

Do your masters’ /var/lib/icinga2/icingadb.env-s differ?

Yes they are completely different, should they be the same?

Exactly. Try to delete them and restart Icinga.

Alright, this solved the duplicated hosts but the services, users etc are still doubled. And i have this message under the services.

Anything suspicious in the logs of the instance the shown complaints are about?

UBUICI3:
[2023-02-06 14:43:24 +0100] information/ConfigObject: Dumping program state to file ‘/var/lib/icinga2/icinga2.state’
[2023-02-06 14:43:33 +0100] information/WorkQueue: #8 (ApiListener, RelayQueue) items: 0, rate: 0.383333/s (23/min 114/5min 247/15min);
[2023-02-06 14:43:33 +0100] information/WorkQueue: #9 (ApiListener, SyncQueue) items: 0, rate: 0/s (0/min 0/5min 0/15min);
[2023-02-06 14:43:34 +0100] information/IcingaDB: Pending queries: 0 (Input: 1/s; Output: 1/s)
[2023-02-06 14:43:40 +0100] information/IdoMysqlConnection: Pending queries: 0 (Input: 3/s; Output: 3/s)
[2023-02-06 14:48:24 +0100] information/ConfigObject: Dumping program state to file ‘/var/lib/icinga2/icinga2.state’
[2023-02-06 14:48:33 +0100] information/WorkQueue: #9 (ApiListener, SyncQueue) items: 0, rate: 0/s (0/min 0/5min 0/15min);
[2023-02-06 14:48:33 +0100] information/WorkQueue: #8 (ApiListener, RelayQueue) items: 0, rate: 0.416667/s (25/min 125/5min 361/15min);
[2023-02-06 14:48:34 +0100] information/IcingaDB: Pending queries: 0 (Input: 1/s; Output: 1/s)
[2023-02-06 14:48:40 +0100] information/IdoMysqlConnection: Pending queries: 0 (Input: 3/s; Output: 3/s)

UBUICI2:
[2023-02-06 14:50:20 +0100] information/IcingaDB: Pending queries: 0 (Input: 1/s; Output: 1/s)
[2023-02-06 14:50:31 +0100] information/RemoteCheckQueue: items: 0, rate: 0/s (6/min 30/5min 90/15min);
[2023-02-06 14:50:49 +0100] information/WorkQueue: #8 (ApiListener, RelayQueue) items: 0, rate: 0.366667/s (22/min 104/5min 320/15min);
[2023-02-06 14:50:49 +0100] information/WorkQueue: #9 (ApiListener, SyncQueue) items: 0, rate: 0/s (0/min 0/5min 0/15min);
[2023-02-06 14:51:41 +0100] information/RemoteCheckQueue: items: 0, rate: 0/s (6/min 30/5min 90/15min);
[2023-02-06 14:52:41 +0100] information/RemoteCheckQueue: items: 0, rate: 0/s (6/min 30/5min 90/15min);
[2023-02-06 14:53:41 +0100] information/RemoteCheckQueue: items: 0, rate: 0/s (6/min 30/5min 90/15min);
[2023-02-06 14:54:41 +0100] information/RemoteCheckQueue: items: 0, rate: 0/s (6/min 30/5min 90/15min);
[2023-02-06 14:55:10 +0100] information/ConfigObject: Dumping program state to file ‘/var/lib/icinga2/icinga2.state’
[2023-02-06 14:55:20 +0100] information/IcingaDB: Pending queries: 0 (Input: 1/s; Output: 1/s)

Looks fine to me. Should i maybe try to delete the duplicated services users etc?

If now the env files don’t differ, feel free to delete everything which doesn’t mach that env from all tables with an environment_id ex. *history* and *sla*.

Nice, thank you very much. If you have the time could you please explain what went wrong and maybe why if you know it. From my understanding i think that the .env file is like a session variable and that it may happened because i created one master way before the other one. Is this mainly correct or am i completely wrong?

Wish you a good day

Did you setup API and Icinga DB before connecting the nodes to a cluster?

Yes, and i forgot to tell you but i have now deleted the Attribute with the wrong env_id and they didnt get restored and everything works fine.


Ok i think i was wrong its not everything allright. This randomly happens and fixes itself after like 10mins.