Hide credentials from icinga2 config files

Icinga 2
1

Hi
We have ~800 hosts and about half of them need some checks over a REST API running locally on the checked host, that are done by self-written plugins (called by ssh). These are using credetials to access the API, which are different on each host and stored in an external database. The icinga2 configuration is saved in our internal gitlab repo, so I would prefer not to save all these passwords in clear text.
These could either be written by cron in a separate file that is outside of the git checkout (in variables containing the host name), or maybe even directly queried from there by a script…
Does anyone have a similar situation? How would you go about getting these credentials into the icinga2 config? (I know these will land in all kind of files and in the postgres db on the monitoring hosts, but then at least they are somehow protected by the system rights and not stored in the gitlab repo)

2

As Icinga does not need the credentials but the plugins, I would first look for an option to get the credentials directly in the plugin.
If the risk is too high that you can get all the passwords then from all hosts, I would create a wrapper plugin on the Icinga system that gets the credentials from the database and then performs the original plugin via ssh.
If this is too much load on the database, slows down the plugin too much or something similar with negative performance impact, I would introduce a cache for the credentials in some way.

3

Hi,

contrary to the suggestion of @dgoetz we decided to do this with hidden files and keep the the scripts clean from passwords. Every script has to catch the credentials first and after run the check. The hidden file can be protected with file permission and SELinux.
With this way we can use diffrent passwords and also we can transfer to other servers and send it to other colleagues

5

@stevie-sy How did you achieve “Every script has to CATCH the credentials”? Are you using a wrapper script, specifying in .conf file for the check? Any direction would be appreciated.

6

No, just a read file function with your prefered scripting language

7

Since it’s a local http(s) check, I ended up using curl’s -n switch (credentials stored in a ~/.netrc file).
This file can/will be deployed by ansible which gets the credentials from the password manager’s API.
I know, it’s not the most elegant solution, but the user the check runs as (per ssh) has more rights then these credentials would grant him anyway, so I can live with it…

8

We had the same challenge. Our Solution is the following…

icinga2.conf → include /dir/outside/gitrepo/.constantsCOMPANY.conf

File → .constantsCOMPANY.conf

const CredPasswordA= "foot"
const CredPassHttpCompany = "user:password"
const CredApiKey = "ladskjfaiejkdjdjd"

And the Service for that… example…

apply Service for (http_vhost => config in host.vars.http_vhosts) {
  import "generic-service"

  check_command = "http"
  vars.http_proxy_auth_pair = CredPassHttpCompany

  vars += config
}

I would be happy to receive feedback on our solution

1 Like
9

For some scripts we use files with strict permissions, but lately we started shoving things inside of environment vars and Consul.

10

Hi @wattebausch
how are you syncing the file in a cluster? Seems, it does not work, when it is only defined on the master and has to be copied to all satellites.

11

for the .constantsCOMPANY.conf we use a simple rsync one-liner in cron with public/priv key