“no Endpoint object found for identity”

SK_2019 · July 2, 2020, 2:33pm

Hello,

i would like to make a monitore some Agents over the Satellite…

Icinga versions:
master(Ubuntu): r2.11.4-1
satellite(Ubuntu): r2.11.4-1
agent (Windows): v2.12.0-rc1-52-g04704a49a

Master:

/etc/icinga2/icinga2.conf

/**
 * Icinga 2 configuration file
 * - this is where you define settings for the Icinga application including
 * which hosts/services to check.
 *
 * For an overview of all available configuration options please refer
 * to the documentation that is distributed as part of Icinga 2.
 */

/**
 * The constants.conf defines global constants.
 */
include "constants.conf"

/**
 * The zones.conf defines zones for a cluster setup.
 * Not required for single instance setups.
 */
include "zones.conf"

/**
 * The Icinga Template Library (ITL) provides a number of useful templates
 * and command definitions.
 * Common monitoring plugin command definitions are included separately.
 */
include <itl>
include <plugins>
include <plugins-contrib>
include <manubulon>

/**
 * This includes the Icinga 2 Windows plugins. These command definitions
 * are required on a master node when a client is used as command endpoint.
 */
include <windows-plugins>

/**
 * This includes the NSClient++ check commands. These command definitions
 * are required on a master node when a client is used as command endpoint.
 */
include <nscp>

/**
 * The features-available directory contains a number of configuration
 * files for features which can be enabled and disabled using the
 * icinga2 feature enable / icinga2 feature disable CLI commands.
 * These commands work by creating and removing symbolic links in
 * the features-enabled directory.
 */
include "features-enabled/*.conf"

/**
 * Although in theory you could define all your objects in this file
 * the preferred way is to create separate directories and files in the conf.d
 * directory. Each of these files must have the file extension ".conf".
 */
include_recursive "conf.d"

/etc/icinga2/zones.conf

object Endpoint "monitoring.[...]" {
        host = "monitoring.[...]"
}

object Endpoint "monitoring-dev.[...]" {
        host = "monitoring-dev.[...]"
}

object Zone "master" {
        endpoints = [ "monitoring.[...]" ]
}

object Zone "monitoring-dev.[...]" {
        endpoints = [ "monitoring-dev.[...]" ]
 parent = "master"
}

object Zone "global-templates" {
        global = true
}

object Zone "director-global" {
        global = true
}

object Zone "windows-commands" {
        global = true
}

object ApiListener "api" {
  accept_config = true
  accept_commands = true
}

/etc/icinga2/zones.d/monitoring-dev.[…]/hosts_winTest.conf

object Endpoint "<Name>" {
        host = "<Name>"
        log_duration = 0
}

object Zone "<Name>" {
        endpoints = [ "<Name>" ]
        parent = "monitoring-dev.[...]"
}

object Host <Name> {
  import "generic-host"
  address = <IP Address>
  display_name = "Win2019-IcingaTEST"
  vars.os = "Windows"
  zone = "monitoring-dev.[...]"
  command_endpoint = "monitoring-dev.[...]"
  vars.wintest = "true"
  vars.notification["mail"] = {
    groups = [ "icingaadmins" ]
  }
}

/etc/icinga2/zones.d/global-templates/services_satellite.conf

apply Service "AdminWindows:Disk" {
  import "generic-service"
  display_name = "Satellite Disk"
  check_command = "disk-windows"
  command_endpoint = host.name
  assign where (host.vars.wintest)
}

Satellite:

api.conf

object ApiListener "api" {
  accept_config = true
  accept_commands = true
}

/etc/icinga2/zones.conf

object Endpoint "monitoring.[...]" {
        host = "monitoring.[...]"
        port = "5665"
}

object Endpoint "monitoring-dev.[...]" {
}

object Zone "master" {
        endpoints = [ "monitoring.[...]" ]
}

object Zone "monitoring-dev[...]" {
        endpoints = [ "monitoring-dev.[...]" ]
        parent = "master"
}

object Zone "global-templates" {
        global = true
}

object Zone "director-global" {
        global = true
}

Agent:

zones.conf

object Endpoint "monitoring-dev.[...]" {
        host = "monitoring-dev.[...]"
        port = "5665"
}

object Zone "master" {
        endpoints = [ "monitoring-dev.[...]" ]
}

object Endpoint "<Name>" {
}

object Zone "<Name>" {
        endpoints = [ "<Name>" ]
        parent = "master"
}

object Zone "global-templates" {
        global = true
}

object Zone "director-global" {
        global = true
}

Error Messages:

icinga Web:
Remote Icinga instance ‘’ is not connected to ‘monitoring.[…]’

Satellite:
icinga2.log:

[2020-07-02 16:22:26 +0200] information/FileLogger: 'main-log' started.
[2020-07-02 16:22:26 +0200] information/ApiListener: 'api' started.
[2020-07-02 16:22:26 +0200] information/ApiListener: Started new listener on '[0.0.0.0]:5665'
[2020-07-02 16:22:26 +0200] information/ExternalCommandListener: 'command' started.
[2020-07-02 16:22:26 +0200] warning/ExternalCommandListener: This feature is DEPRECATED and will be removed in future releases. Check the roadmap at https://github.com/Icinga/icinga2/milestones
Context:
        (0) Activating object 'command' of type 'ExternalCommandListener'

[2020-07-02 16:22:26 +0200] information/CheckerComponent: 'checker' started.
[2020-07-02 16:22:26 +0200] information/ConfigItem: Activated all objects.
[2020-07-02 16:22:26 +0200] information/ApiListener: Reconnecting to endpoint 'monitoring.[...]' via host 'monitoring.[...]' and port '5665'[2020-07-02 16:22:26 +0200] information/ApiListener: New client connection for identity 'monitoring.[...]' to [IP-Address]:5665
[2020-07-02 16:22:26 +0200] information/ApiListener: Requesting new certificate for this Icinga instance from endpoint 'monitoring.[...]'.
[2020-07-02 16:22:26 +0200] information/ApiListener: Sending config updates for endpoint 'monitoring.[...]' in zone 'master'.
[2020-07-02 16:22:26 +0200] information/ApiListener: Finished sending config file updates for endpoint 'monitoring.[...]' in zone 'master'.
[2020-07-02 16:22:26 +0200] information/ApiListener: Syncing runtime objects to endpoint 'monitoring.[...]'.
[2020-07-02 16:22:26 +0200] information/ApiListener: Finished syncing runtime objects to endpoint 'monitoring.[...]'.
[2020-07-02 16:22:26 +0200] information/ApiListener: Finished sending runtime config updates for endpoint 'monitoring.[...]' in zone 'master'.
[2020-07-02 16:22:26 +0200] information/ApiListener: Sending replay log for endpoint 'monitoring.[...]' in zone 'master'.
[2020-07-02 16:22:26 +0200] information/ApiListener: Finished sending replay log for endpoint 'monitoring.[...]' in zone 'master'.
[2020-07-02 16:22:26 +0200] information/ApiListener: Finished syncing endpoint 'monitoring.[...]' in zone 'master'.
[2020-07-02 16:22:26 +0200] information/ApiListener: Finished reconnecting to endpoint 'monitoring.[...]' via host 'monitoring.[...]' and port '5665'
[2020-07-02 16:22:26 +0200] information/ApiListener: Applying config update from endpoint 'monitoring.[...]' of zone 'master'.
[2020-07-02 16:22:26 +0200] information/ApiListener: Received configuration for zone 'global-templates' from endpoint 'monitoring.[...]'. Comparing the timestamp and checksums.
[2020-07-02 16:22:26 +0200] information/ApiListener: Stage: Updating received configuration file '/var/lib/icinga2/api/zones-stage/global-templates//_etc/services_satellite.conf' for zone 'global-templates'.
[2020-07-02 16:22:26 +0200] information/ApiListener: Applying configuration file update for path '/var/lib/icinga2/api/zones-stage/global-templates' (1473 Bytes).
[2020-07-02 16:22:26 +0200] information/ApiListener: Received configuration for zone 'monitoring-dev.[...]' from endpoint 'monitoring.[...]'. Comparing the timestamp and checksums.
[2020-07-02 16:22:26 +0200] information/ApiListener: Stage: Updating received configuration file '/var/lib/icinga2/api/zones-stage/monitoring-dev.[...]//_etc/hosts_winTest.conf' for zone 'monitoring-dev.[...]'.
[2020-07-02 16:22:26 +0200] information/ApiListener: Applying configuration file update for path '/var/lib/icinga2/api/zones-stage/monitoring-dev.[...]' (3585 Bytes).
[2020-07-02 16:22:26 +0200] information/ApiListener: Received configuration updates (2) from endpoint 'monitoring.[...]' are different to production, triggering validation and reload.
[2020-07-02 16:22:26 +0200] critical/ApiListener: Config validation failed for staged cluster config sync in '/var/lib/icinga2/api/zones-stage/'. Aborting. Logs: '/var/lib/icinga2/api/zones-stage//startup.log'
**[2020-07-02 16:22:28 +0200] information/ApiListener: New client connection for identity '<Name>' from [IP-Address]:61153 (no Endpoint object found for identity)**
[2020-07-02 16:22:28 +0200] information/JsonRpcConnection: Received certificate request for CN '<Name>' signed by our CA.
[2020-07-02 16:22:28 +0200] information/JsonRpcConnection: The certificate for CN '<Name>' is valid and uptodate. Skipping automated renewal.
[2020-07-02 16:22:36 +0200] information/WorkQueue: #6 (ApiListener, SyncQueue) items: 0, rate:  0/s (0/min 0/5min 0/15min);
[2020-07-02 16:22:36 +0200] information/WorkQueue: #5 (ApiListener, RelayQueue) items: 0, rate:  0/s (0/min 0/5min 0/15min);

-> [2020-07-02 16:22:28 +0200] information/ApiListener: New client connection for identity ‘’ from [IP-Address]:61153 (no Endpoint object found for identity)

Why I get an Error ? What’s wrong?

Firewall isn’t a problem.

Can you help me please?

Thank you

theFeu · July 3, 2020, 7:10am

Hello!

You posted a lot of config in quotes there, would it be possible to reformat your post with code blocks instead for better readability?
We have some handy guidelines on how to do that

Sorry for chiming in without providing an answer and good luck!
Feu

rsx · July 3, 2020, 7:32am

First, this is unsupported.

Second, using V2.11 every zone and endpint object needs to be stored in zones.conf only.

SK_2019 · July 3, 2020, 8:03am

Thank you for yor answer.

But the Documentation says, that the zone and endpoint from the host should be in the host-file.

SK_2019 · July 3, 2020, 8:30am

version on the agent is v2.11.3 now

rsx · July 3, 2020, 9:11am

This will not work since it was change with V2.11. Detailed information could be found here or here or here.

And it is the reason why you are getting:

[2020-07-02 16:22:28 +0200] information/ApiListener: New client connection for identity ‘’ from [IP-Address]:61153 (no Endpoint object found for identity)

SK_2019 · July 3, 2020, 10:05am

So I have changed the config from Endpoint and Zone on the Master

/etc/icinga2/zones.conf

object Endpoint "monitoring.[...]" {
        host = "monitoring.[...]"
}

object Endpoint "monitoring-dev.[...]" {
        host = "monitoring-dev.[...]"
}

object Endpoint "<Name>" {
        log_duration = 0
}

object Zone "master" {
        endpoints = [ "monitoring.[...]" ]
}

object Zone "monitoring-dev.[...]" {
        endpoints = [ "monitoring-dev.[...]" ]
 parent = "master"
}

object Zone "<Name>" {
  endpoints = [ "<Name>" ]
  parent = "monitoring-dev.[...]" 
 
}

object Zone "global-templates" {
        global = true
}

object Zone "director-global" {
        global = true
}

object Zone "windows-commands" {
        global = true
}

but the same Error:
[2020-07-03 12:04:02 +0200] information/ApiListener: New client connection for identity ‘’ from [IP-Address]:57062 (no Endpoint object found for identity)

rsx · July 3, 2020, 10:08am

You need to change this on every node in the chain, means master - satellite [- satellite etc.] - agent. You could verify if all required objects are existing on a particular node using icinga object list ....

SK_2019 · July 8, 2020, 7:54am

Thank you for your Answer.

I changed the Order at the master, satellite and agent.

It looks now:

master

/etc/icinga2/zones.conf

object Endpoint "monitoring.[...]" {
        host = "monitoring.[...]"
}

object Endpoint "monitoring-dev.[...]" {
        host = "monitoring-dev.[...]"
}

object Endpoint "<Name>" {
        log_duration = 0
}

object Zone "master" {
        endpoints = [ "monitoring.[...]" ]
}

object Zone "monitoring-dev.[...]" {
        endpoints = [ "monitoring-dev.[...]" ]
 parent = "master"
}

object Zone "<Name>" {
  endpoints = [ "<Name>" ]
  parent = "monitoring-dev.[...]" 
 
}

object Zone "global-templates" {
        global = true
}

object Zone "director-global" {
        global = true

Satellite:
zones.conf:

object Endpoint "monitoring.[...]" {
        host = "monitoring.[...]"
        port = "5665"
}

object Endpoint "monitoring-dev.[...]" {
}

object Zone "master" {
        endpoints = [ "monitoring.[...]" ]
}

object Zone "monitoring-dev[...]" {
        endpoints = [ "monitoring-dev.[...]" ]
        parent = "master"
}

object Zone "global-templates" {
        global = true
}

object Zone "director-global" {
        global = true
}

Agent:
zones.conf:

object Endpoint "monitoring-dev.[...]" {
        host = "monitoring-dev.[...]"
        port = "5665"
}

object Endpoint "<Name>" {
}

object Zone "master" {
        endpoints = [ "monitoring-dev.[...]" ]
}

object Zone "<Name>" {
        endpoints = [ "<Name>" ]
        parent = "master"
}

object Zone "global-templates" {
        global = true
}

object Zone "director-global" {
        global = true
}

But unfortunately it show the same error Message:

[2020-07-08 09:44:24 +0200] information/ApiListener: New client connection for identity ‘’ from [IP-Address]:57247 (no Endpoint object found for identity)

rsx · July 8, 2020, 8:04am

The order doesn’t matter. In your case one or more object(s) is/are missing at some place. To check this out you can use icinga2 object list.

SK_2019 · July 9, 2020, 7:41am

On the master is everyone object there.

On the Satellite the Endpoint and Zone from Agent are missing.

On the Agent are Endpoit and Zone form monitororing are missing. The zone master is there.
The zone master has as attribut from “endpoints” the value “monitoring-dev.[…]” (Satellite)

What is wrong here?

Thanks

rsx · July 9, 2020, 8:05am

That means you need to add these objects to satellite’s zones.conf.

Yeah, setup wizard creates misleading names. master is here just a name of the parent zone BUT it’s not your master zone. In your case it would be less confusing if it were monitoring-dev.[…]. But it will work anyway.

This is subsequently correct.

homerjay · July 9, 2020, 12:33pm

Hi.

Additionally, an example.
Important: global zones have been omitted for better readability

In the following example:
The zone of the master is: master
the zone of the satellite is: satellite-zone (yes, I’m creative )

Parts of zones.conf from master, satellite and agent

####################################################
# [master-zone] <-> [satellite-zone] <-> [agent-in-satellite-zone]  #
####################################################

######################
# The Icinga2-Master
######################

object Endpoint "name.of.master"  {
}

object Zone "master"  {
  endpoints = [ "name.of.master", ]
}

object Endpoint "your.satellite.host" {
        host = "your.satellite.host"
}

object Zone "satellite-zone" {
        endpoints = [ "your.satellite.host" ]
        parent = "master"
}

# ---------------------------------

######################
# The satellite
######################

object Endpoint "name.of.master" {
        host = "name.of.master"
        port = "5665"
}

object Zone "master" {
        endpoints = [ "name.of.master" ]
}

object Endpoint "your.satellite.host" {
}

object Zone "satellite-zone" {
        endpoints = [ "your.satellite.host" ]
        parent = "master"
}

# ---------------------------------

# An agent in the "satellite-zone"
# Note: The agent knows nothing about the master-host or the master zone
# and both are not part of the zones.conf

object Endpoint "your.satellite.host" {
        host = "your.satellite.host";
        port = "5665";
}

object Zone "satellite-zone" {
        endpoints = [ "your.satellite.host" ]
}

object Endpoint "<Name>" {
}

object Zone "<Name>" {
        endpoints = [ "<Name>" ]
        parent = "satellite-zone"
}

Hope this helps.

Greetings

SK_2019 · July 15, 2020, 9:23am

Thank you for your help.

So I configurated the Endpoint and Zone from the Agent on the Sateliite too.
After that the error-Message in log is gone.

I created a Service for example for Check Disk on Linux Agent

/etc/icinga2/zones.d/global-templates/satellite-services.conf

apply Service "Check Linux Disk" {
   import "disk-linux"
   check_command = "disk"
   command_endpoint = host.vars.agent_endpoint
   assign where host.zone == "monitoring-dev.[...]" && host.vars.agent_endpoint && host.vars.os == "Linux"
}

/etc/icinga2/zones.d/monitoring-dev[…]/hosts_linux.conf

object Host "<Name>" {
  import "generic-host"
  check_command = "hostalive"
  address = "IP-Address"
  vars.os = "Linux"
  vars.agent_endpoint = name
  zone = "monitoring-dev.[...]"
  vars.notification["mail"] = {
    groups = [ "icingaadmins" ]
  }
}

So the problem is now, when I stopping the Icinga2-Service on the Sattelite (or shutting down the Satellite), then the Service(-s) is/are sitll working.

The text “DISK OK - free space…” is something grayed out, but it still working and reachable. There is no Error-Status like “No connection to monitoring-dev[…] from …” on Icingaweb2.

Have you an advice for me? Is something wrong by the zones or by services-config?

rsx · July 15, 2020, 9:30am

For monitoring connected zones I prefer to have checks using cluster-zone.

SK_2019 · July 15, 2020, 9:56am

I changed it to cluster_zone, but still the same problem.

homerjay · July 15, 2020, 1:17pm

Hi.

This seems to be a bit tricky.

As the master and satellite are not in the same zone, cluster-zone would not work as expected as it would be executed on the satellite (please feel free to correct if I am wrong).
If the satellite-agent is down, services are greyed out in icingaweb2 when they are overdue (see the clock in your picture).

Instead of cluster-zone, you could add a service with CheckCommand “cluster” (part of the ITL). It checks for all connected endpoints.

This would add it to your master (as an example):

apply Service "cluster" {
  import "generic-service"

  check_command = "cluster"

  // ...
  // e.g.
  assign where "your.icinga2.master" in host.templates
}

Well, that doesn’t help with displaying the status in icingaweb2, but it points to the reason.

Hope this helps.

Greetings

SK_2019 · July 17, 2020, 12:31pm

If I do Dependency…

/etc/icinga2/zones.d/monitoring-dev[…]/parents.conf

apply Dependency "TEST" to Host {
  parent_host_name = "monitoring-dev.[...]"
  disable_checks = false
  disable_notifications = true
  ignore_soft_states = false
  states = [ Up, Down ]
  assign where host.name == "<Name>"

}


apply Service "ping4" {
  import "generic-service"
  check_command = "ping4"
  assign where host.address
}


apply Dependency "TEST" to Service {
  parent_host_name = "monitoring-dev.[...]"
  parent_service_name = "ping4"
  disable_checks = false
  states = [ OK ]

  assign where host.name == "<Name>"
}

… is nothing happens, no changes.
The idea behind this, is when the Satellite is DOWN, it should be that the alert show the status for the Agent and their Services as DOWN or UNKNOWN or like that too. But it’s show’s now the status OK.
Also when the Satellite is off, then the Service ping4 from Sateliite is CRITICAL and then it must show Agent (and their Services) as Critical / Unkwnown…
You cannot see clearly, that the Agent is unreachable now (if Satellite off).

homerjay · July 17, 2020, 1:53pm

Maybe you should open a new thread for the new problem.
The title of this thread is a bit misleading.

Greetings.

SK_2019 · July 17, 2020, 2:23pm

I have created a new thread.