Hello,
I am trying to test Icinga2 to monitor cloud related entities, like VMs in a private/public cloud and containers running on K8s (private or public clusters).
The infra design has 2 masters, 2 satellites and all monitored entities should connect to the satellites.
Since the entities are volatile (some VMs/containers appear and dissappear all the time), we will need to use the Director to declare the Hosts.
The Services would be also be uploaded via Director based on what checks should be performed.
From this point forward, let’s stick to VMs (containers are another story).
I managed to automate the install of the agents, and they successfully join the “communication network” (mTLS setup).
Having read most of the documentation, for the life of me, I cannot figure out a way to declare the Host objects. The reason is that we (who control Icinga) do not control the monitored VMs. In the current monitoring solution we want to replace, the agents once connected report what microservice they belong to and the monitoring solution gives them checks to run based on that their service is.
I could not find a way to setup a variable or something on the agent that would be sent to the masters or somewhere from where I could read it and tell the Director to create the Host objects setting a vars with that value.
For instance if I know “agent1” is part of “service1”, I could declare the Host with a vars.service = “service1” then the Services would use where vars.service is “service1”.
Has anybody used Icinga in an environment where we do not know what the monitored entities do, and expect them to report it to us so we can set them up?
Thank you.
PS. One last question since it’s not clear to me: if I have an Agent successfully connected to a Satellite (via mTLS), should I be able to see it in the Satellite object list? (i ask because I do not, I only see what that Satellite has loaded from zones.conf). If not is there any way to see what Agents are connected and online?
- Version used (
icinga2 --version): r2.13.5-1
- Operating System and version: Ubuntu 20.04
- Enabled features (
icinga2 feature list): not relevant
- Icinga Web 2 version and modules (System - About): 2.11.1-1.focal
- Config validation (
icinga2 daemon -C): not relevant
- If you run multiple Icinga 2 instances, the
zones.conf file (or icinga2 object list --type Endpoint and icinga2 object list --type Zone) from all affected nodes: Not yet, that’s why I’m asking the questions
67 views and no reply later, I’ll answer my own question with what I figured out in the meantime.
If anybody has a better answer than what is written below, please feel free to comment.
So, long story short, no, I was not able to find a way to specify “something” on an agent that would be passed upwards to satellites->masters in order to magically know what microservice an agent belongs to.
The workaround for me was to use the configuration management solution’s API to fetch attributes of the node/VM on which the agent runs, to figure out what Services the Host (of the agent’s) should have.
This works fine, but it only covers VMs using that configuration management solution.
For a VM which does not use the configuration management, I do have a plan, but still need to test it, once I try it and if it works I will update here, maybe it will help someone someday.
And for the “how do I know what agents are connected to a satellite (using cli or something)”, I still have no answer, the only way would be to check in icingaweb which Hosts are DOWN (all the Hosts have cluster-zone check_command, so it runs on the satellites which report if a Zone (with the Endpoint of the agent the only thing in that Zone) is not connected.
Thank you.
an agent belongs to a zone, if left blank in your case, icinga director chooses the zone for you.
if you setup an satellite (sat01) which is an agent and create the appropriate zone (EU) for it with the parent zone (icinga-master) and define an endpoint (sat01) in icinga director for that zone which is once again the satellite (sat01).
In Icinga director you set the zone of the host to eg EU.
Now it depends on the way your config is written. If your agent connects to your master or satelite, you define endpoint and IP in your zones.conf of the agent. If your Satellite connects to your agent, you define just the endpoint without the IP.
I fully agree with the above.
My context is as follows:
- 2 masters - connect to each other
- 2 satellites (for now) - connect to each other and both masters
- agents on leaf nodes - connect to both satellites
My issues is that:
- i have no idea when a new VM will be installed and have the agent on it (I do not deploy it)
- i have no idea to which microservice a new VM belongs to
Using Icinga Director and the masters’ certificates list I am able to register the Host via Icinga Director.
The issue now is how to figure out what Service should get applied to that Host since I do not know what it does. That’s was I was interested in a way for icinga on an agent to send some info to the masters (upstream).
My workaround was to use configuration management API (which applies to every new VM) to get attributes of that node which that node explicitly sets on itself. Thus I can find out what that agent should check, thus being able to assign the Services.
For containers it is going to be another story since the same from above apply, but there is no configuration management for them.
Will post here if my planned workaround will work.
Thank you.