error:0200100D:system library:fopen:Permission denied

Icinga 2
1

Hello all,

I am trying to install icinga in a centos 7 machine and I am getting this weird error:

Error: Systemd start for icinga2 failed!
journalctl log for icinga2:
-- Logs begin at Fri 2019-07-05 11:55:00 CEST, end at Tue 2021-10-12 13:27:56 CEST. --
Oct 12 13:26:19 lxlicen13b.cern.ch systemd[1]: Unit icinga2.service cannot be reloaded because it is inactive.
Oct 12 13:26:27 lxlicen13b.cern.ch systemd[1]: Starting Icinga host/service/network monitoring system...
Oct 12 13:26:27 lxlicen13b.cern.ch icinga2[7093]: [2021-10-12 13:26:27 +0200] information/cli: Icinga application loader (version: 2.13.1-1)
Oct 12 13:26:27 lxlicen13b.cern.ch icinga2[7093]: [2021-10-12 13:26:27 +0200] information/cli: Loading configuration file(s).
Oct 12 13:26:27 lxlicen13b.cern.ch icinga2[7093]: [2021-10-12 13:26:27 +0200] information/ConfigItem: Committing config item(s).
Oct 12 13:26:27 lxlicen13b.cern.ch icinga2[7093]: [2021-10-12 13:26:27 +0200] critical/SSL: Error on bio X509 AUX reading pem file '/var/lib/icinga2/certs//lxlicen13b.cern.ch.crt': 33558541, "error:0200100D:system library:fopen:Permission denied"
Oct 12 13:26:27 lxlicen13b.cern.ch icinga2[7093]: [2021-10-12 13:26:27 +0200] critical/config: Error: Cannot get certificate from cert path: '/var/lib/icinga2/certs//lxlicen13b.cern.ch.crt'.
Oct 12 13:26:27 lxlicen13b.cern.ch icinga2[7093]: Location: in /etc/icinga2/features-enabled/api.conf: 3:1-3:24
Oct 12 13:26:27 lxlicen13b.cern.ch icinga2[7093]: /etc/icinga2/features-enabled/api.conf(1): # This file is managed by Puppet. DO NOT EDIT.
Oct 12 13:26:27 lxlicen13b.cern.ch icinga2[7093]: /etc/icinga2/features-enabled/api.conf(2):
Oct 12 13:26:27 lxlicen13b.cern.ch icinga2[7093]: /etc/icinga2/features-enabled/api.conf(3): object ApiListener "api" {
Oct 12 13:26:27 lxlicen13b.cern.ch icinga2[7093]: ^^^^^^^^^^^^^^^^^^^^^^^^
Oct 12 13:26:27 lxlicen13b.cern.ch icinga2[7093]: /etc/icinga2/features-enabled/api.conf(4):   accept_commands = true
Oct 12 13:26:27 lxlicen13b.cern.ch icinga2[7093]: /etc/icinga2/features-enabled/api.conf(5):   accept_config = true
Oct 12 13:26:27 lxlicen13b.cern.ch icinga2[7093]: [2021-10-12 13:26:27 +0200] critical/config: 1 error
Oct 12 13:26:27 lxlicen13b.cern.ch icinga2[7093]: [2021-10-12 13:26:27 +0200] critical/cli: Config validation failed. Re-run with 'icinga2 daemon -C' after fixing the config.
Oct 12 13:26:27 lxlicen13b.cern.ch systemd[1]: icinga2.service: main process exited, code=exited, status=1/FAILURE
Oct 12 13:26:27 lxlicen13b.cern.ch systemd[1]: Failed to start Icinga host/service/network monitoring system.
Oct 12 13:26:27 lxlicen13b.cern.ch systemd[1]: Unit icinga2.service entered failed state.
Oct 12 13:26:27 lxlicen13b.cern.ch systemd[1]: icinga2.service failed.
Oct 12 13:27:56 lxlicen13b.cern.ch systemd[1]: Starting Icinga host/service/network monitoring system...
Oct 12 13:27:56 lxlicen13b.cern.ch icinga2[7774]: [2021-10-12 13:27:56 +0200] information/cli: Icinga application loader (version: 2.13.1-1)
Oct 12 13:27:56 lxlicen13b.cern.ch icinga2[7774]: [2021-10-12 13:27:56 +0200] information/cli: Loading configuration file(s).
Oct 12 13:27:56 lxlicen13b.cern.ch icinga2[7774]: [2021-10-12 13:27:56 +0200] information/ConfigItem: Committing config item(s).
Oct 12 13:27:56 lxlicen13b.cern.ch icinga2[7774]: [2021-10-12 13:27:56 +0200] critical/SSL: Error on bio X509 AUX reading pem file '/var/lib/icinga2/certs//lxlicen13b.cern.ch.crt': 33558541, "error:0200100D:system library:fopen:Permission denied"
Oct 12 13:27:56 lxlicen13b.cern.ch icinga2[7774]: [2021-10-12 13:27:56 +0200] critical/config: Error: Cannot get certificate from cert path: '/var/lib/icinga2/certs//lxlicen13b.cern.ch.crt'.
Oct 12 13:27:56 lxlicen13b.cern.ch icinga2[7774]: Location: in /etc/icinga2/features-enabled/api.conf: 3:1-3:24
Oct 12 13:27:56 lxlicen13b.cern.ch icinga2[7774]: /etc/icinga2/features-enabled/api.conf(1): # This file is managed by Puppet. DO NOT EDIT.
Oct 12 13:27:56 lxlicen13b.cern.ch icinga2[7774]: /etc/icinga2/features-enabled/api.conf(2):
Oct 12 13:27:56 lxlicen13b.cern.ch icinga2[7774]: /etc/icinga2/features-enabled/api.conf(3): object ApiListener "api" {
Oct 12 13:27:56 lxlicen13b.cern.ch icinga2[7774]: ^^^^^^^^^^^^^^^^^^^^^^^^
Oct 12 13:27:56 lxlicen13b.cern.ch icinga2[7774]: /etc/icinga2/features-enabled/api.conf(4):   accept_commands = true
Oct 12 13:27:56 lxlicen13b.cern.ch icinga2[7774]: /etc/icinga2/features-enabled/api.conf(5):   accept_config = true
Oct 12 13:27:56 lxlicen13b.cern.ch icinga2[7774]: [2021-10-12 13:27:56 +0200] critical/config: 1 error
Oct 12 13:27:56 lxlicen13b.cern.ch icinga2[7774]: [2021-10-12 13:27:56 +0200] critical/cli: Config validation failed. Re-run with 'icinga2 daemon -C' after fixing the config.
Oct 12 13:27:56 lxlicen13b.cern.ch systemd[1]: icinga2.service: main process exited, code=exited, status=1/FAILURE
Oct 12 13:27:56 lxlicen13b.cern.ch systemd[1]: Failed to start Icinga host/service/network monitoring system.
Oct 12 13:27:56 lxlicen13b.cern.ch systemd[1]: Unit icinga2.service entered failed state.
Oct 12 13:27:56 lxlicen13b.cern.ch systemd[1]: icinga2.service failed.

Do you may know what is causing it? First time I’m seeing it. The permissions on the certificates seem ok to me. I have no clue what could be the problem.

Icinga Version: 2.13

2

If permissions are fine, how about SELinux context of the file (assuming SELinux is set to enforcing)?
ls -lZ /var/lib/icinga2/certs/lxlicen13b.cern.ch.crt

3

The results of the command:

-rw-rw-r--. icinga icinga unconfined_u:object_r:var_lib_t:s0 /var/lib/icinga2/certs/lxlicen13b.cern.ch.crt
4

With the icinga2 policy installed, it should have the type icinga2_var_lib_t. So if you run restorecon -Rv /var/lib/icinga2/certs/ it should change the context so icinga2 has access to it.

2 Likes
5

Ah yes that fixed it thanks.
But I am curious to why this happened since icinga module on puppet manages SELinux.

6

SELinux context is stored in an extended attribute so a file will keep its context when moved. Puppet has still some problems with this and so I think the puppet-icinga2 module as there is no option to manage the context only if selinux is enabled. And most modules do not manage the context because of this in the hope puppet creates files in the correct context. But in case a transition is only done when coming from a specific context or a file is created somewhere else and then moved by puppet errors happen.

2 Likes
7

Thank you very much Dirk, that was very helpful.
Have a nice day
Cheers!