Service to other host service dependency

staypekos · January 20, 2021, 11:13am

Thanks for the answer.
I already tried with parent_host_name = "interconnect" in apply rule, but it didn’t work.

Sorry for confusion, I removed the domain to shorter the log and forgot to remove it in shown config.

Could it possibly be the problem with some interference with host dependency?

apply Dependency "disable-host-service-checks" to Service {
  disable_checks = true
  ignore_soft_states = false
  assign where true
}

apply Dependency "interconnect" to Host {
  parent_host_name = "interconnect"
  assign where host.vars.location == "main"
}

log1c · January 20, 2021, 2:52pm

you need a ignore where host.name == "interconnect" for the host dependency.
Otherwise the dependency would be a loop on itself, I think.
Or does the host “interconnect” not have the variable location set to “main”?

staypekos · January 20, 2021, 2:55pm

As far as it’s network between both locations, the location variable is not set on this host.

log1c · January 21, 2021, 1:39pm

One thing I missed earlier:

Is the parent service named “ping4-interconnect” or just “ping4”?

Can you post your current dependency you are testing with the correct names, I am confused now

staypekos · January 22, 2021, 11:45am

Host interconnect has basic ping4 service. I named the dependency ping4-interconnect.

Here is the actual running configuration, I comment out other hosts and tried only with these two

object Host "interconnect" {
  import "generic-host"
  address = "192.168.255.2"
  retry_interval = 15s
}

object Host "testing.tacr.local" {
  import "generic-host"
  display_name = "testing"
  address = "10.10.25.223"
  vars.os = "Linux"
}

dependency definition

apply Dependency "disable-host-service-checks" to Service {
  disable_checks = true
  ignore_soft_states = false
  assign where true
}

apply Dependency "interconnect" to Host {
  parent_host_name = "interconnect"
  assign where host.name == "testing.tacr.local"
}

object Dependency "ping4-interconnect" {
  parent_host_name = "interconnect"
  parent_service_name = "ping4"
  child_host_name = "testing.tacr.local"
  child_service_name = "ping4"
  states = [ OK ]
  disable_checks = true
  ignore_soft_states = false
}

/* #alternative with apply rule
apply Dependency "ping4-interconnect" to Service {
  parent_host_name = "interconnect"
  parent_service_name = "ping4"
  states = [ OK ]
  disable_checks = true
  ignore_soft_states = false
  assign where service.check_command == "ping4"
  ignore where host.name == "interconnect"
}
*/

and the service

apply Service "ping4" {
  import "generic-service"
  check_command = "ping4"
  if (host.name == "interconnect") {
    max_check_attempts = 3
  }
  assign where host.address
}

log1c · January 22, 2021, 12:12pm

Apart from missing states in you host dependency apply rule, though I’m not sure if there is a default, all other dependecies are looking good to me and should work.

Just in case you have the commented apply rule like you posted in your config:
be aware that you need * in front of every line for a multi-line comment:

/* #alternative with apply rule
*apply Dependency "ping4-interconnect" to Service {
*  parent_host_name = "interconnect"
*  parent_service_name = "ping4"
*  states = [ OK ]
*  disable_checks = true
*  ignore_soft_states = false
*  assign where service.check_command == "ping4"
*  ignore where host.name == "interconnect"
*}
*/

staypekos · January 22, 2021, 1:34pm

Yes, it has default values, according to documentation
Configuration Attributes: States - Defaults to [ OK, Warning ] for services and [ Up ] for hosts.

I used

/*
_some_
_multiline_
_configuration_
*/

through all config files and there is no problem with validation and other host disapear from web GUI.
Anyvay I tried to remove the commented code from dependencies configuration but without change.

staypekos · January 22, 2021, 2:22pm

Ok, I probably found the problem. When I omit disable-host-service-checks dependency, everything works good. I think it’s because the ping4-interconnect dependency fails correctly, but after that disable-host-service-checks dependency check testing host, which is OK and the dependency passes, so notification is sent.

[2021-01-22 14:54:51 +0100] notice/Dependency: Dependency 'testing.tacr.local!interconnect' passed: Parent host 'interconnect' matches state filter.
[2021-01-22 14:54:51 +0100] notice/Dependency: Dependency 'interconnect!ping4!disable-host-service-checks' failed: Parent host 'interconnect' is in a soft state.
[2021-01-22 14:54:51 +0100] notice/Dependency: Dependency 'interconnect!ping4!disable-host-service-checks' passed: Parent host 'interconnect' matches state filter.
-> [2021-01-22 14:54:51 +0100] notice/Dependency: Dependency 'testing.tacr.local!ping4!ping4-interconnect' failed. Parent service 'interconnect!ping4' is CRITICAL
-> [2021-01-22 14:54:51 +0100] notice/Dependency: Dependency 'testing.tacr.local!ping4!disable-host-service-checks' failed: Parent host 'testing.tacr.local' is in a soft state.
-> [2021-01-22 14:54:51 +0100] notice/Dependency: Dependency 'testing.tacr.local!ping4!disable-host-service-checks' passed: Parent host 'testing.tacr.local' matches state filter.
[2021-01-22 14:54:51 +0100] notice/Dependency: Dependency 'testing.tacr.local!interconnect' passed: Parent host 'interconnect' matches state filter.

But I’m afraid, I cannot find solution with both dependencies

log1c · January 22, 2021, 2:33pm

good catch, I missed that.

Out of interest, why did you implement the “disable-host-service-checks” dependency?
There already is a default implicit dependency of hosts to their own services. So if a host is down no notification will be sent for the services of that host. The only difference is that the checks are still running.

Apart from that I don’t really know how to implement both without interefering with each other.
You could try setting the host check command of your “interconnect” host to something else than a ping check, so that the host can be UP, while the ping4 service is CRITICAL.

staypekos · January 22, 2021, 2:50pm

I received notifications when host went back to up state, but it’s services were still critical.
When no checks are running if the host is down, then in the time it’s back up, there is still time at least to one check on services (because they are in soft state) and they get ok.
If the check are done during the host down state, when the host is back up, all services are already in hard state and notifications are sent immediately.

I found this solution here

It’s not the problem. In this situation the host interconnect remains up, even the ping service is critical, because it’s still reachable - host reachability RTA values (C,W) 100.00 ms, 200.00 ms vs ping service 3.00 s, 5.00 s

log1c · January 25, 2021, 7:21am

Could you send the state history of the hosts/services (from the webinterface) at the time of the log messages of your previous post:

Maybe we both overlooked something.

staypekos · January 25, 2021, 3:00pm

log1c · January 26, 2021, 10:24am

I am missing the hosts states. those are more relevant atm, as the host dependency seems to be your problem (if I understood correctly)

Looking at this screenshot I don’t see anything wrong.

ping4 on interconnect is critical and sends a notification.
ping4 on testing is still in a soft state and did not send a notification.

staypekos · January 26, 2021, 2:36pm

I think it’s because the the hosts states didn’t changed.
Or I don’t know where to find them.

Yes, I think, I canceled the test, because the ping4 on testing has been still checked.
When I removed the disable-host-service-checks the ping4 on testing turned gray and was no more checked.

log1c · January 27, 2021, 7:48am

Both host and service have a “History” tab in the webinterface.
Service:

Host:

This would mean:
one screenshot per host history
one screenshot per service history.
four in total:

host interconnect
host testing
service ping4 on interconnect
service ping4 on testing

staypekos · January 27, 2021, 2:16pm

Ah, okey. I made new simulation (with the same result)

Host interconnect - no state change

Host testing - no state change

Service ping4 on interconnect

Service ping4 on testing

I attach debug_filtered.log (163.6 KB), from which lines with ApiListener, IdoMysqlConnection and DbConnection are removed.

staypekos · January 27, 2021, 2:33pm

Here is, how it looks like, when I remove the disable-host-service-checks dependency

The ping4 service on testing goes to Overdue state and no more checks are done (already on soft state of ping4 on interconnect)

Service ping4 on interconnect

Service ping4 on testing

log1c · January 27, 2021, 4:37pm

Thx for the screen shots.
It really looks like the disable-host-service-checks depedency somehow interferes with the whole setup.
But I don’t know why.

I found recent issuse that maybe have something to do with your problem, but I’m not sure.
Maybe check those if they look like they might fit.

Other than that I’m out of ideas.

Maybe someone you/of the team can have a “quick” look at this, @theFeu ^^?

staypekos · March 17, 2021, 11:32am

Yes, 2.12 came with new solution for multiparent dependency. I though it’s only for hosts, but same behaviour is for services too.

I setup distributed monitoring last week. Master on previous server in remote location and satellite in main location and everything seems fine.

log1c · March 17, 2021, 3:13pm

Glad you got it working

Can you pick any post as solution so the topic gets marked “solved”?

Cheers and have fun with Icinga