Pipelining Icinga2 configuration check, merge and pull

Hi,

I hesitated between posting in this category or the more configuration specific Icinga2 category.

We’re using Icinga2 2.13.2-1 and we want to automate the following:

  • When a configuration change is pushed to a temporary branch in BitBucket, a pipeline has to check the configuration validation (are there no errors)
  • After a validation with no errors it’s allowed to merge to master
  • When the merge is complete a trigger will run to pull the configuration change on the servers with Icinga2 monitoring and restart Icinga2

At this time a configuration issue will result in a not working Icinga2, by automating the above steps it will be safer and will cost us less work in the end (only the setup of the above will take some extra time).

Anyone has experience with the above? Any thoughts / solutions / examples that can help me further?

Hello Optima!

Indeed. If you put invalid files on the disk and for some reason you stop Icinga 2, you won’t be able to start it again until you fix the config.

But if you feed in the config in a dedicated package like below via your CI, Icinga should handle it well.

https://icinga.com/docs/icinga-2/latest/doc/12-icinga2-api/#configuration-management

Best,
A/K

you can also do something similar with an extra Icinga instance with the disabled service, and you use it just for the “deamon --validate” command.

Then, as you said you create your CI pipeline to:

  • each time you commit on the temporary branch you trigger a git pull of that branch on the validate server
  • if the validation ends up ok, your script will simply merge and push to master, which will trigger the pull and reload on the master.

having an extra instance will avoid the problem that icinga fails in the rare case that it get restarted having the bad config already deployed