My company has a python script that they run a check for an Apache URL API that checks for warnings, errors, etc. The problem I am running into is this API floats between 2 different nodes based on server load.
The script has the URL hardcoded, so when it is on a certain node, of course the checks work, if it moves to another node the service check fails of course.
Is there a way to have a check for each (somenode-1 and somenode-2) that monitor the dashboard from the inside, but only raise it as an alarm if both fail in other words, we’d expect 1 to always be in a failed state - but 2 would be bad.
I would expect that if you have an API which floats between two nodes there is
a single address which resolves to the floating address - in other words, it’s
a single address which connects to whichever node is currently “live”.
The simplest solution is to make your service check connect to this floating
address instead of either of the “real server” addresses, and then:
a) you’ll get a response if either API is running, and
b) you’ll be checking the same address that any genuine client would be using,
so you’re performing a more realistic and thorough check of the service
availability.
I was wrong about the IP staying the same. Actually there is a zookeeper process that choose the next server to become the master. That being said it flips between 2 nodes.
Hi I hope you solved this issue. I was searching something else and have stumbled on this post. There is simple solution. You can use Business process for that and there you can specify condition when both of “hearbeat” check are critical to raise alert.