Best practices on host-templates, apply for, host groups, service sets and so on using Director

Icinga Director
, , , ,
1

Hi everybody,

I am now building up my third Icinga2 based monitoring environment. I was never 100% happy with the previous ones based on experiences I made here and there. As I am strive to optimize I was looking for some best practices and experiences of others how to organize monitoring objects the best way using templates, apply rules, host groups, service sets and so on as there are - in my opinion - too many ways to achieve the same things.
But if you go down the road these ways are not really identical, some have pros here but cons there and the next way to realize something has it other way round.

One contraint: I am focussing on managing the environment manually (without CMDB, Puppet, …) with Director.

So I am really looking for some open discussion and exchange of experiences how you organize your objects.

To start with I am sharing some of my thoughts that were the reason to start this thread:

Templates or not?
Templates make life easier…always…nearly…
In the past (using Nagios, Centreon and then Icinga2) I always used them to have some kind of scaffolding for my hosts delivering the services.
So basically I created some kind of “hierarchical” naming for my host templates and to have separate settings. E.g.

  • host-ext for external hosts, only using dummy host check as the host itself is not under my control (e.g. subscribed SaaS services, maybe not even pingable).
  • host-int as the opposite for internal
  • host-int-srv for internal server objects
  • host-int-generic for internal generic objects (e.g. not having an OS)
  • …

But how to proceed then to include / scaffold information as OS or network-based parameters (longer PING thresholds)?
Are you using multiple host-templates that must be imported together? On the one hand I understand this concept, on the other hand I find it quite “unhandy” as there are no ways to assure that for instance always one OS host-template and one location host-template is used. Additionally you have to import them in the right order.
Yes there are choices available as well, but here again it just adds even more complexity which makes an environment harder to understand and to control, right?
Bringing all those informations into one host template is even more a mess as it will automatically create duplicates: you have some Ubu20.04 host in location1 and in location2.

Anybody using Service Sets?
One of the features I instantly loved Icinga for when it was released were service apply rules!
Then I came along Service Sets…honestly: is anybody using them? And if yes: can you please explain me why?
I cant use apply for in service sets, right? And the dynamics assignment is possible with Service apply rules as well?

…and host (and service) groups
And not to forget: then we have host and service groups. What are you using them for? Just for logical separation? Are you using them additionally for apply rules?
Somehow I have the feeling they are something from old-school-nagios-times that always have been there and just have not been removed? Or is there anything I can achieve with them and with nothing else?

I am really looking forward for your opinion, experiences and feedback!

Best and have a healthy new year!
Matt

2

Hi,

best practice is difficult to say. Because everybody will handle the director in diffrent ways. It depends how your wokflow looks like.

At first some points you should think about:

  • How do you get your hosts into the director?
    -> Do you create them manually?
    -> Can you import them from an internal CMDB?
    -> Can you create the hosts via tools from like Ansible, SaltStack, HashiCorp etc.?
    -> Do you have your own Deployment Tools which is catching also some information about the server and with them you create the hosts?

  • What informations you can import from the mentioned tools above?

  • How your infrastructure looks in generally? Do you want to use Agents? Do you have at your icinga setup alse satellites?

  • Also think about the diffrent types of hardare you want to check (server (maybe diffrent os and versions), switches and router (maybe diffrent manufacturer) etc.)

Templates
if you want to work with the Director you have to be prepared anyway that templates are required in many places. How many templates you have to create depends on how many information you can import as custom var.
If you have diffrent types of devices, hardware, manufacturer, os, os version etc and you put every possible custom var in one host template, the information icingaweb will show will beconfusing. Here it makes maybe sense to create diffrent templates for the diffrent device types. Maybe even through inheritance and hirachical names as you wrote.
Also multiple templates are possible. E.g. on the server is installed icinga as an agent, this one get an extra template for this.
We are doing a way like this.

Service Sets:
Service Sets can make life easier and also and also clearer in administration. It’s such a group of service check. Here you create a set of services which a special group of devices should get. Like base check every windows server should have (e.g. load, memory, disk etc.). For them you have to create only one rule to assign. This ensures that a certain type of server, hardware, os, usage type or whatever get certain checks in any case.

Of course you also every assign rule which is defined in a service set seperate. But doesn’t it get confusing here at some point?

And if some services should also have some additional checks, you can create apply rules. And yes with service sets apply fore are not possible, you have to use apply rules. For this there is an example in the docs.

service/host groups:
yes it’s just for logical seperation. Like every device which check x, every device which is a router, every device is on location x etc. etc.
But you can also create a sperate view in icingaweb. It’s how you like. And of course you can create this views bases on service groups.

That are the good things from icingaweb and the director, they are flexible.

Other users will have other ideas.

1 Like
3

Hi @stevie-sy
thanks for your input. I forgot to mention that we are focussing on maintaining the environment manually with Director (no CMDB, …) :wink:

What do you do? Do you have any automation things in place and you just import everything or are your staff maintaining monitoring objects manually in Director?

Templates
So you use multiple templates? Are you using additional choices? How to you guarantee the right order of template imports when using multiple ones?

Service Sets
Well I feel somehow that they are a little too confusing or adding too much complexity by adding another layer and loosing the apply for logic.
So I can not just have one disk check defined in a service set and then loop through a host.vars.disks in service sets. I have to add disk-check X and disk-check Y
Thus I somehow feel that Service Apply Rules are everything I need (in combination with “assign where” to vars or groups or …)

Groups
Thanks for the input on creating different views…thats really something we haven’t looked at so far. So the thread was already worth it for us :wink:

Thanks and appreciating other feedback from other users as well :wink:

Best,
M.

4

Hi,

we import our devices from an internal CMDB and with that and the power of the import function from the director we can automate a lot incl. the templates :wink:
And because of that, we decided not to use choices.

That’s not a problem with the disk checks (linux and windows as well) and custom vars. If you think about to handle this manually you only have to choose the correct parmeter and set the correct vars. We are doing this also. But of course, if you want to have one service for every partition or drive than you need apply for rules.

5

So if you don’t want to use service sets, do you use any apply rules at all to assign services to hosts? We use service sets quite a bit, and find them helpful!

6

@bkai
I used service sets before but we really re-think this at the moment.
We use apply-rules (with and without apply-for) but that works fine and I don’t see any benefit to bundle them in those service sets.
Sure, you could use both, but I want to streamline our processes and avoid if-a-then-this, if-b-then-that decissions to be done by administrators (we don’t use imports and maintain completely manually).

7

Hi,
what do you guys think about assigning services in general: apply-rules or assigning via a host-template which are imported?
Best,
M.

8

If you use apply rules that’s fine. Service sets are basically just a special form of apply rules; we use them and no other apply rules (except for notifications & dependencies). So we “order” our stuff via service sets attached to host variables defined in non-exclusive host profiles…

So: I wish you success! :slight_smile:

1 Like
9

In my last setup i went with templates (for services) and 2 basic service sets only.
Basic service set were assigned with the host template used (linux with agent or windows with agent, also had some host without agents for … reasons).
All other services i assigned manually to the hosts, and i build a template tree for those so there were barely any values i needed to set manually.
That enabled me to add a host and all serices needed in about 5 minutes.

I’ll build a new setup in my new company soon and will probably do it similarily again…

1 Like