Monitoring disk under privileges

Service Monitoring
1

Hi,
Maybe this question is old but I don’t find any clear answer.
I’m using docker compose example (docker-compose Icinga) to deploy icinga.
My question is clear : How to configure check_disk service to check space of a protected directory (need to have rights to access it) on a remote host.

Here is the error I got :
DISK CRITICAL - /var/lib/docker is not accessible: No such file or directory

This is the command as displayed in the icinga web under services/source :
‘/usr/lib/nagios/plugins/check_disk’ ‘-c’ ‘10%’ ‘-w’ ‘20%’ ‘-X’ ‘none’ ‘-X’ ‘tmpfs’ ‘-X’ ‘sysfs’ ‘-X’ ‘proc’ ‘-X’ ‘configfs’ ‘-X’ ‘devtmpfs’ ‘-X’ ‘devfs’ ‘-X’ ‘mtmfs’ ‘-X’ ‘tracefs’ ‘-X’ ‘cgroup’ ‘-X’ ‘fuse.gvfsd-fuse’ ‘-X’ ‘fuse.gvfs-fuse-daemon’ ‘-X’ ‘fuse.portal’ ‘-X’ ‘fdescfs’ ‘-X’ ‘overlay’ ‘-X’ ‘nsfs’ ‘-X’ ‘squashfs’ ‘-p’ ‘/var/lib/docker’

Inside the icinga docker container I noticed icinga2 process runs with “icinga” user.

In an older post they advice to change the user icinga runs with…but how? what user shall I use?

Thank you in advance for your support

2

Hi,

We have solved this issue with two different approaches - I don’t know which is better, or if there is a best practice.

First you need to check what are the needed privileges to read (why would you need to write, btw?) from /var/lib/docker

First approach: grant the icinga user read permission on /var/lib/docker
You will need to include the icinga user in the docker group, probably.

Second approach: grant the icinga user the permission to run the check_disk command as root (or any user that has the permission to read /var/lib/docker)
You will need to edit the sudoers file.
You will also need to either make a new Icinga command to include “sudo” in front of “check_disk”, or to define a small script that wraps the check_disk command within a new plugin (check_disk_as_root) that you then define as a new command within Icinga.
You will need to adapt your service definition to based on the new command.

Hope this helps,

Jean

3

Hi jean,
Thank you for your support I highly appreciate.
Still some mist in my undertanding

First approach :

  • I created icinga user and set its password on remote host
  • I changed /var/lib/docker group to docker
  • I added icinga user to docker group
  • I also added icinga to wheel group to accept ssh connection
  • at this point icinga user successfuly access /var/lib/docker and I can connect via ssh from icinga master using password.

This does not solve the problem.
How the icinga would connect to host with icinga user and not asking for password??? where to put the user password??

Second approach:
“grant the icinga user the permission to run the check_disk command as root” is this to do on icinga master or on the remote machine?

same question here, no need to mention password?

thank you again!

4

My mistakes :

1- I thought icinga will monitor disk of other machine without installing any agent on the target machine (I’m a bit dumb)

2- I deployed the icinga agent on target machine in a docker container

Feedback : When installing icinga agent (do not use docker) on target machine it will setup the icinga user and all the needed privileges…so nothing to add it will work by default.