Icinga2 HA Setup some questions

Hey,

me again.

I decided to practise around with a HA-Master setup in icinga.

So fresh virtual envirment for testing. Setup:

  • 3x Debian 12 (ic-master1.lab.nk, ic-master2.lab.nk, ic-mariadb.lab.nk)
  • All in same subnet with working dns (internally)

A quick overview what I did from the original icinga2 documentation:

  • Installed basic Icinga2 package on master1 + master2 with monitoring-plugins

  • Installed icingadb-redis (changed listening interface + protected mode to “no” on both masters

  • Installed icingadb and enabled feature on both masters

  • Installed mariadb-server on server ic-mariadb (act as database server and test client/agent for checks in this setup) and imported icinga-schema [created a database for icingadb, icingaweb and director]

  • Installed icingadb-web package on both masters

  • Configured both masters web-setup (…/icingaweb2/setup) entered on both the icingadb database, the same icingaweb database for authentication. Entered on both servers the redis primary server the ic-master1.lab.nk and as secondery on both the second one (may wrong?!), entered for each command-transport there own local api-user (right?!)

  • Both web installers runs accross with no errors. Works fine. Both websites shown pre-defined icinga2 checks on ic-master1.lab.nk. Login with same user works.

  • Followed 1:1 these steps: How to set up High-Availability Masters (Just used only dns names instead of IPs)

  • Installed icinga-director package module on ic-master1.nk.lab via apt. Configured director with database and did the first deployment without errors.

Now the step where I’m lost:

Created a host- and service template. Created the host “ic-mariadb.nk.lab” and added a “icinga2” agent check for the host. Now I see this check two times on each webinterface. So its because both nodes looks into the same database? If I shutdown the master1, the check cant be executed manually from master2 interface because of missing permissions?!..

Please help… :slight_smile:

Hello Phylipp,

can you maybe provide some additional Information like:

icinga2 Daemon -C

on both masters!

and the zones configs and so on?

Im Kinda confused where this Setup went wrong :smiley:

Greetings

Tell me what u need, I can provide all of this test-environment.

ic-master1.lab.nk:

grafik

ic-master2.lab.nk:

grafik

################

Hey, thank you, can you share the Service Configurations please?

Greetings

This one?

No one any idea? :smiley:

Is there may somewhere a 3rd party installation tutorial for HA in icingadb with redis etc.?

what’s your icingadb config on both servers?

master1:

master2:

an now your icingadb.conf from

/etc/icinga2/features-enabled/

On both:

I dont unterstand, why I get this service two times on the screen. I just created one single check on the db-server:

what’s the content of: /var/lib/icinga2/icingadb.env?
maybe they are “out of sync” on the master nodes?
https://icinga.com/docs/icinga-db/latest/doc/05-Distributed-Setups/

For high-availability setups, it is a good idea to enable the Icinga DB feature on the secondary master after you have successfully connected from/to the primary master so that the certificates are set up properly. The secondary master will then generate the same environment ID since it is working with the same CA certificate.

In any case make sure that the file /var/lib/icinga2/icingadb.env does not change over time and is the same on all Icinga 2 master nodes per environment.

1 Like

Master1:

Master2:

That should be the same? - Yes, how can I force to reinitialize them?

In any case make sure that the file /var/lib/icinga2/icingadb.env does not change over time and is the same on all Icinga 2 master nodes per environment.

copy them from the primary master/ config master to the other machine and restart icinga2.

I would also cleanup the database.

1 Like

Let my try. U are a genius. :smiley:

This works so far!

I deleted the database on my db-server and created it new and imported the schema.

Recreated host & service via director, works! I see the service-object now only one time on both dashboards of the masters.

Check-Scheduler was master1, I powered off master1, check-scheduler changed instantly to master2. Can view results on masters2 dashboard while master1 is offline.

Next little problem:

If I want to manually use the button “Check now” from masters2 dashboard, I get this error-message (check result is every 10s updated and works):

what’s the content of /etc/icinga2/conf.d/api-users.conf on both masters?

On both root with * permission but each master has a unique generated password.

Should be also the same?

If so, what is the option in the director → icinga infrastucture → endpoints → “api user” field? Should I enter the details in the director for the both masters or in the local file that u called?

for the director you use only ONE endpoint, your config master
the command transport in

icingaweb2/icingadb/command-transport

should use the correct credentials referenced in /etc/icinga2/conf.d/api-users.conf

1 Like

Yes, I understand. I installed director only on one master.

I’ll check this option! Thanks a lot!