Adding Master to active setup - will it resolve false positives

Icinga 2
1

Hi community,

I have my monitoring setup and it’s working great. Straight to my question about false positives where my current master sometimes gives an alert but, for example, ping from another server somewhere in the world there isnt that much going on but it was probably because of an routing issues from my master to the agent.

Will a second active master resolve those false positives or how is this meant to work when setting up two active masters ? Anyone having experience in this ?

Would love to setup two active masters so one can always “be right” :slight_smile:
Any other ideas are welcome!

Thank you!

2

Hello @Creamers158,
Adding a second master server will not resolve the problem of ping command failures because of network problems.

Once you add a second master server, all check command are load balanced between both master servers. If master 1 is pinging server xyz it will continue to ping server xyz until the Icinga application is stopped on master 1 or master 1 is hard down. Once Icinga on master 1 is offline then Icinga on master 2 will take over all check command that were running on master 1.

I have the same problem you are descripting in my dual master HA Icinga setup. I have 2 masters servers at two different sites. I can ping the server xyz from master 1 but not from master 2. In Icinga it say that server xyz is down because master 2 is the “check source” for server xyz. I asked my network team about this and they said it is a routing issue. The network packets are coming back a different route which is cause the ping failure. :frowning_face:

I hope this help.
Alex

BTW - have a dual master setup is nice when upgrading Icinga to a new version or adding new modules or features.

3

Hi @aclark6996 ,

You describe exactly what I mean. I wish there was some kind workaround to get rid of those exact false positives, especially those pings as you describe then. When being away and not at my desk I would like to be assured that there is nothing wrong.
Maybe setup some addition pings from agents to agents, but that gets messy :wink:

Or some logic that using the ping results maybe…makes me think :wink:

Thanks for your reply, let’s see if there are also other people with the exact same issue.

Any idea’s are welcome!

4

Hello @Creamers158!

Have you tried this one?

Best,
AK

5

This sounds almost to good to be true :wink: Will check it out!

Any references to an icmp example in the meantime ?

6

Check out the new article from @nhilverling !!! Gonna try this and see if it works and how it works.
Thanks @nhilverling !!!

7

If I pin a ICMP service to the first and second master then I would have two icmp result from each
master1-hostA and master2-hostA
master1-hostBand master2-hostB
master1-hostC and master2-hostC
etc

Is there a way to create result of each combination programmatically like the above example ?