Agent host is down but no errors are shown

I have two machines both running linux. For now i just want to set up 1 master and 1 agent. The issue is that my agent machine shows up on the icinga web interface but it remains in the DOWN state. If i add a service to be monitored on the client it just stays in “Pending” mode. It’s difficult to troubleshoot this issue because both the client and master logs are not showing any errors. In fact on my masters logs it isn’t showing any mention of my client at all. Isn’t this odd? Maybe this points to an issue with api. The api feature is enabled on both machines as well as check and mainlog. I believe there should be some messages along the lines of "New api identity for ‘client hostname’ but none of these messages appear.
icinga2 daemon -C on both machines finishes without any errors. On the master it shows that there are 2 hosts/endpoints/zones which is correct.
Any ideas on how to troubleshoot and issue that is not giving any errors? Thank you.

What check_command are you using? In case of hostalive the DOWN state means your master cannot ping the agent machine.

I am using cluster_zone but I have tried hostalive as well. I can ping the client machine from the master and vice versa and there is no firewall involved.

Slight update . Checked the logs on the client after restarting the icinga2 service. Saw a message “warning/PluginCheckTask: Check command for object ‘client.hostname’ (PID: 71850, arguments” ‘/usr/lib/nagios/plugins/check_ping’ ‘-H’ ‘127.0.0.1’ ‘-c’ ‘500,100%’ ‘w’ ‘3000,80%’) terminated with exit code 128, output: < Terminated by signal 15 (Terminated).>

So I’m assuming this has something to do with why the ping service is showing up as pending on the icinga web interface. If anyone has any suggestions I’d really appreciate it .

So the client runs check_ping against himself? Could you share your config?

Sure . I’m running an older version of icinga so this config is valid. I have another setup with the same version/config so I know it is valid.

Client zones.conf

object Endpoint “master.hostname.com” {
host = “master.hostname.com
port = “5665”
}
object Zone “master” {
endpoints = [ “master.hostname.com” ]
}

The master zones.conf does not have anything in it.
I have defined the client hosts,endpoints, and zones under to repository.d directory on the master.

/etc/icinga2/repository.d/hosts/client.conf

object Host “client.hostname.conf” {
import “satellite-host”
check_command = “hostalive”
vars.os = “com”
}

/etc/icinga2/repository.d/client/ping4.conf

object Service “ping4” {
import “satellite-service”
check_command = “hostalive”
host_name = “client.hostname.com
zone = “client.hostname.com
}

/etc/icinga2/repository.d/endpoints/clienthostname.conf

object Endpoint “client.hostname.com” {
}

/etc/icinga2/repository.d/zones/client.conf

object Zone “client.hostname.comf” {
endpoints = [ “client.hostname.com” ]
parent = “client.hostname.com
}

icnga2 daemon -C on both machines returns no errors. On the master it validate 2 hosts,zones and endpoints which is correct. Features enabled on the master machine " api command debuglog ido-mysql mainlog notificaton"
. Features enabled on the client “api mainlong notification checker deubglog”

Zone and endpoint object for the client is missing in the client’s zones.conf.

zonedefinition here is wrong, please remove it.

Starting with V2.11 your config will become invalid since zone and endpoint objects needs to be configured in zones.conf only.

Thank you for you reply. I will try your suggestions when i get a chance. I just wanted some clairfication. So my clients zone.conf is currently -

object Endpoint “master.hostname.com” {
host = “master.hostname.com
port = “5665”
}
object Zone “master” {
endpoints = [ “master.hostname.com” ]
}

When you say the zone and endpoint objects are missing in the clients zone.conf what would i add to the file?
I will remove the line you specified from the ping.conf file.

I have removed the zone definition like you suggested. I have not made any changes to the clients zones.conf yet as I’m not sure what to add. One thing i noticed in the masters log file is a message "debug/ApiListener: Not connecting to Endpoint ‘client.hostname.com’ because the host/port attributes are missing’.

Also in the clients log I see a message "debug/ApiListener: Not connecting to zone ‘master’ because it’s not in the same zone, a parent or a child zone.’