Host created by satellite api successfull but does not get displayed in icinagweb2 on master

Icinga 2
1

Hi Icinga Team,

my icinga installation is HA Master <=> Master with a number of satellites.

Icinga2 latest (version: 2.11.4-1)

I created a new host using the API and directed the request to a satellite icinga instance.
The curl request was initiated from a node in a zone that cannot reach the master directly, but it can reach the satellite.

Here is the curl request to the API:-

curl -k -s -u 'user-name:password' -H 'Accept: application/json' \
-X PUT 'https://satellite01.somedomain.net:5665/v1/objects/hosts/pgApiTest2FromSat.testdomain' \
 -d '{ "templates": [ "K8S Kube Icinga" ], "attrs": { "address": "192.168.5.1", "check_command": "hostalive", "vars.os" : "Linux" }, "pretty": true }'
{
  "results": [
      {
        "code": 200.0,
        "status": "Object was created"
      }
  ]
}

Successfull creation of the host!

But wait… The host never turns up in the Icingaweb2 UI?
Also, the IP address I gave it will fail the ‘hostalive’ check, This was intentional to see if I get an alert.

So after a successfull creation of a new host that should now be alerting, I see nothing in my Icingaweb2 UI, no node, no alerts?

As a final check to see if the node was really created I called for a list of hosts from the satellite node.

curl -k -u "username:password" https://https://satellite01.somedomain.net:5665/v1/objects/hosts?attrs=address
{"results":[
...OTHER HOSTS REMOVED FOR BREVITY..
{"attrs":{"address":"192.168.1.1"},"joins":{},"meta":{},"name":"pgApiTestFromSat.localdomain","type":"Host"}]}

So the host is being reported as present.

The status now is.

  1. A new host has been created.

  2. The new host is not reporting any errors (although its IP address is not reachable for the ‘hostalive’ check)

  3. The new host does not appear anywhere on IcingaWeb2

  4. A host exists in our Icinga monitoring system that does not appear or alert?

    Icinga Web 2 Version 2.8.1
    Git commit 233bd29e4104125b4e5ef631e8c16dde33dadd9a
    PHP Version 7.3.11
    Git commit date 2020-06-29

Can any explain why the host never really gets created although it is present in the API

Thank you
Peter

2

Who should report? Your masters? The can’t as they don’t know about this new host.

Icingaweb2 on your masters? That’s ok as the masters don’t know about this new host.

To make it work you need to create the new host on your master(s) as well.

3

Hi Roland,

thank you for your comments.

I would have thought that when the master connects to the satellite it would get any updates, is that not how the monitoring results are returned, so why not return that a new host has been added to the satellite?

Our HA Master <=> Master => Satellites => Agents uses top down as the examples in the ‘Distributed Monitoring’ on the Icinga WWW Site. By allowing the API to add hosts to a Satellite but not to inform the Primary Master means the monitoring can get out of sync.

I wanted to use a monitoring plugin for Kubernetes that automatically adds pods as hosts to icinga (https://github.com/gyselroth/kube-icinga) using an API user.

Our K8S clusters are in different zones, reporting to satellites. It means this plugin will never work which is a shame.

It’s not all bad news really, our new IcingaWeb2, Director, Certificate Monitoring, Virtualization module are all working very well. The K8S module was the ‘icing on the cake’.

Finally I have to say this…

‘Icinga Rocks!!!’

It’s been a pleasure learning, installing, fixing, learning a bit more, and thanks to the icinga website examples and the community for the help

4

Why don’t you use the masters api endpoint to create the hosts and put them in the correct zone?
As the master is the only system sedning out notifications, this explains the missing alerts as well.

5

Hello @log1c,

yes good point, we considered this, but that means opening up firewall from agent to master. This is something we do not want to do as we have a satellite in the middle.

I am working on a solution using the API from the satellite to master.

e.g.
‘Agent informs Satellite, Satellite makes API call to the MASTER and adds node in correct zone.’

6

I’m running into something similar to the topic creator. I’m not using icinagweb2 but I did install icinga2 on a satellite server to connect to a master. The satellite is running Ubuntu 16.04.2 with icinga version r2.12.0-1 and the master is running CentOS 7 with icinga version v2.6.3. Is there any compatibility issues between these two operating systems and icinga versions?

7

It is a bit hidden in the distributed monitoring docs, but there is a passage about versions:
https://icinga.com/docs/icinga2/latest/doc/06-distributed-monitoring/#versions-and-upgrade

If the CentOS 7 machine really has 2.6.3 installed this is a really old version and you will have to update.
icinga2 --version will tell you the exact version of your icinga2 core.