GIT repo for config file

Icinga 2 Automation & Infrastructure
1

Hi,

Out of curiosity, anyone is using GIT to manage the folder /etc/icinga2/…

Right now, we changing any files manually directly on the server and I’m curious to see how others deal with config file without using “Director”.

2

As the Icinga config is only present on the master and automatically gets synced on the other master (and/or satellite) node(s) we don’t have any additional syncing in place. As we also use the Director as well, there is a basic version control already.

What we have is a Azure blob storage (via blobfuse) for both our webservers, that is mounted to /etc/icingaweb2 to only need to modify the/a config once.

But no GIT/version control here either as both (/etc/icinga2 and /etc/icingaweb2) (could) hold passwords in clear text.

3

I have all of my Icinga config in a git repository. While I have Icinga director setup, I have not made much use of it as yet. So my config is all files underneath the /etc/icinga2 directory. This has been a git repository since just after getting Icinga2 working.

As the config is sent out from the master to the satellites and agents, it makes it easy to grab all the changes.

I do all the Icinga changes at my work, so the work flow is simplistic at this time.
I actually work on the Icinga master node at the moment. Yes I know that is not the best method.

The git repository is on a locally hosted GitLab system.
I intend to create some scripts and a hook so you can work remotely and when the repository is updated it triggers a deployment script that will;

  1. Retrieve the latest git repository contents on the Icinga master
  2. Do a “icinga2 daemon -C” and only if there are no errors will it
  3. Place the new repository changes into operation.
  4. This also involves transferring custom checks and sudo entries to the satellites and agents via rsync.

I have script that does the last few steps on the Icinga master.

The password problem has been dealt with for things like PostgreSQL databases by having a role that Icinga uses to do its queries that has read-only ability to certain tables. So having those passwords in the git repository is considered less of a problem. But passwords are definitely an issue. I use command locked SSH keys for some checks to avoid passwords in a few cases.

Icinga could do with some sort of of call out script function to obtain information (Eg: passwords) from somewhere (Hashicorp Vault or even Ansible vault). The value obtained would be passed to the agent from the master and then used for the specific check.

2 Likes
4

thanks @neil

Are you having the entire /etc/icinga2/ directory in GIT ? Or you have some files/directory ignored ?

We are also having a local GITLAB system that’s not accessible from the outside and totally private, so I’m not having an issue also with “password” and we are also using the same kind of approach by having database user using read only access.

5

I have all sub-directories of /etc/icinga2 included except for a /etc/icinga2/notused.d/ directory which holds some some experimental Icinga config files and is also used when I need to remove a host from monitoring for a period. Decommissioning or major work on host comes to mind when I do not want “Service problems” inundated with downtime or acknowledged entries which will hang around for many days or longer.

6

Hi, we dont use curently git for Icinga2 configuration. Unfortunatelly we are using Icinga2 director but maybe some replacment for it in the futere could be git and some CI/CD princeples … using git for some icinga2 stuff is good idea … for example if you are using Icinga2 Business Process module and you have HA setup, 2 masters, you need actually manually copy file on second master as well … this could be easy solved with git and some of CI/CD princeples … especially if you are using gitlab, CI/CD is already there … :slight_smile: … sky is the limit!