Director not talking to background process

Hi All,
Hoping someone might have hit this before, or know where I could dig deeper. The error messages aren’t hugely helpful :frowning:

This is also logged at Github (Icinga Director Background Daemon not reported as running by web interface (but service is running...) · Issue #2963 · Icinga/icingaweb2-module-director · GitHub)

Expected Behaviour:

Icinga Director Daemon page reports connection to background daemon, and as such jobs process etc.

Actual Behaviour

https:///director/daemon page reports:

The Icinga Director Background Daemon is not running. Please check our [documentation](https://<node>/doc/module/director/chapter/Background-Daemon) in case you need step by step instructions showing you how to fix this.

The service does appear to be running however.

[node]# systemctl status icinga-director.service
● icinga-director.service - Icinga Director - Monitoring Configuration
     Loaded: loaded (/etc/systemd/system/icinga-director.service; enabled; preset: disabled)
    Drop-In: /usr/lib/systemd/system/service.d
             └─10-timeout-abort.conf, 50-keep-warm.conf
     Active: active (running) since Tue 2025-03-18 14:10:13 AEDT; 988ms ago
 Invocation: e3149d7b9d3f48a59f3703883e483c90
       Docs: https://icinga.com/docs/director/latest/
   Main PID: 2746990 (icingacli)
     Status: "running, db: no configuration"
      Tasks: 1 (limit: 38421)
     Memory: 11.2M (peak: 11.4M)
        CPU: 46ms
     CGroup: /system.slice/icinga-director.service
             └─2746990 "icinga::director: running, db: no configuration"

Mar 18 14:10:13 node.fqdn systemd[1]: Starting icinga-director.service - Icinga Director - Monitoring Configuration...
Mar 18 14:10:13 node.fqdn systemd[1]: Started icinga-director.service - Icinga Director - Monitoring Configuration.
Mar 18 14:10:13 node.fqdn icingadirector[2746990]: DB configuration is not valid

This is after a full uninstall/reinstall of the icinga-director packages, and clearing out the database and user and recreating it all as we hit a similar problem where the page was not reporting the service running (although, it was).

Other Info

OS: Fedora 41 (Stable) 6.13.4-200.fc41.x86_64 #1 SMP PREEMPT_DYNAMIC Sat Feb 22 16:09:10 UTC 2025 x86_64 GNU/Linux

Icinga installed from ICINGA repos (https://packages.icinga.com/fedora/$releasever/release/)

[node]# dnf list --installed | grep -i icinga
icinga-cube.noarch                          1.3.3-1.fc41                     icinga-stable-release
icinga-cube-web.noarch                      1.3.3-1.fc41                     icinga-stable-release
icinga-director.noarch                      1.11.3-1.fc41                    icinga-stable-release
icinga-director-daemon.noarch               1.11.3-1.fc41                    icinga-stable-release
icinga-director-php.noarch                  1.11.3-1.fc41                    icinga-stable-release
icinga-director-web.noarch                  1.11.3-1.fc41                    icinga-stable-release
icinga-l10n.noarch                          1.3.0-1.fc41                     icinga-stable-release
icinga-php-incubator.noarch                 0.22.0-1.fc41                    icinga-stable-release
icinga-php-library.noarch                   0.14.1-1.fc41                    icinga-stable-release
icinga-php-thirdparty.noarch                0.12.1-1.fc41                    icinga-stable-release
icinga-selinux-common.noarch                1.0.0-1.fc41                     icinga-stable-release
icinga2.x86_64                              2.14.5-1.fc41                    icinga-stable-release
icinga2-bin.x86_64                          2.14.5-1.fc41                    icinga-stable-release
icinga2-common.x86_64                       2.14.5-1.fc41                    icinga-stable-release
icinga2-selinux.x86_64                      2.14.5-1.fc41                    icinga-stable-release
icingacli.noarch                            2.12.2-1.fc41                    icinga-stable-release
icingadb.x86_64                             1.2.1-1.fc41                     icinga-stable-release
icingadb-redis.x86_64                       7.2.7-1.fc41                     icinga-stable-release
icingadb-web.noarch                         1.1.3-1.fc41                     icinga-stable-release
icingaweb2.noarch                           2.12.2-1.fc41                    icinga-stable-release
icingaweb2-common.noarch                    2.12.2-1.fc41                    icinga-stable-release
icingaweb2-selinux.noarch                   2.12.2-1.fc41                    icinga-stable-release
php-icinga.noarch                           2.12.2-1.fc41                    icinga-stable-release
# icinga2 --version
icinga2 - The Icinga 2 network monitoring daemon (version: r2.14.5-1)

Copyright (c) 2012-2025 Icinga GmbH (https://icinga.com/)
License GPLv2+: GNU GPL version 2 or later <https://gnu.org/licenses/gpl2.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

System information:
  Platform: Fedora Linux
  Platform version: 41 (Forty One)
  Kernel: Linux
  Kernel version: 6.13.4-200.fc41.x86_64
  Architecture: x86_64

Build information:
  Compiler: GNU 14.2.1
  Build host: unknown
  OpenSSL version: OpenSSL 3.2.4 11 Feb 2025

Application information:

General paths:
  Config directory: /etc/icinga2
  Data directory: /var/lib/icinga2
  Log directory: /var/log/icinga2
  Cache directory: /var/cache/icinga2
  Spool directory: /var/spool/icinga2
  Run directory: /run/icinga2

Old paths (deprecated):
  Installation root: /usr
  Sysconf directory: /etc
  Run directory (base): /run
  Local state directory: /var

Internal paths:
  Package data directory: /usr/share/icinga2
  State path: /var/lib/icinga2/icinga2.state
  Modified attributes path: /var/lib/icinga2/modified-attributes.conf
  Objects path: /var/cache/icinga2/icinga2.debug
  Vars path: /var/cache/icinga2/icinga2.vars
  PID path: /run/icinga2/icinga2.pid
# icinga2 feature list
Disabled features: command compatlog debuglog elasticsearch gelf graphite influxdb influxdb2 journald livestatus opentsdb perfdata syslog
Enabled features: api checker icingadb mainlog notification
# icinga2 daemon -C
[2025-03-18 14:21:58 +1100] information/cli: Icinga application loader (version: r2.14.5-1)
[2025-03-18 14:21:58 +1100] information/cli: Loading configuration file(s).
[2025-03-18 14:21:59 +1100] information/ConfigItem: Committing config item(s).
[2025-03-18 14:21:59 +1100] information/ApiListener: My API identity: hum-imas-01.on.act.gov.au
[2025-03-18 14:22:01 +1100] warning/ApplyRule: Apply rule 'ping6' (in /etc/icinga2/conf.d/services.conf: 34:1-34:21) for type 'Service' does not match anywhere!
[2025-03-18 14:22:01 +1100] warning/ApplyRule: Apply rule 'VDSL Sync Speed (C1117)' (in /var/lib/icinga2/api/packages/director/d754c4d8-c6fd-4797-8bdb-61be0edb0029/zones.d/director-global/service_apply.conf: 12:1-12:39) for type 'Service' does not match anywhere!
[2025-03-18 14:22:01 +1100] warning/ApplyRule: Apply rule 'VDSL Sync Speed (C1127)' (in /var/lib/icinga2/api/packages/director/d754c4d8-c6fd-4797-8bdb-61be0edb0029/zones.d/director-global/service_apply.conf: 25:1-25:39) for type 'Service' does not match anywhere!
[2025-03-18 14:22:01 +1100] warning/ApplyRule: Apply rule 'VDSL Sync State (C1117)' (in /var/lib/icinga2/api/packages/director/d754c4d8-c6fd-4797-8bdb-61be0edb0029/zones.d/director-global/service_apply.conf: 38:1-38:39) for type 'Service' does not match anywhere!
[2025-03-18 14:22:01 +1100] warning/ApplyRule: Apply rule 'VDSL Sync State (C1127)' (in /var/lib/icinga2/api/packages/director/d754c4d8-c6fd-4797-8bdb-61be0edb0029/zones.d/director-global/service_apply.conf: 49:1-49:39) for type 'Service' does not match anywhere!
[2025-03-18 14:22:01 +1100] information/ConfigItem: Instantiated 4 NotificationCommands.
[2025-03-18 14:22:01 +1100] information/ConfigItem: Instantiated 50852 Notifications.
[2025-03-18 14:22:01 +1100] information/ConfigItem: Instantiated 1 IcingaApplication.
[2025-03-18 14:22:01 +1100] information/ConfigItem: Instantiated 432 HostGroups.
[2025-03-18 14:22:01 +1100] information/ConfigItem: Instantiated 13734 Hosts.
[2025-03-18 14:22:01 +1100] information/ConfigItem: Instantiated 1 Downtime.
[2025-03-18 14:22:01 +1100] information/ConfigItem: Instantiated 12 Comments.
[2025-03-18 14:22:01 +1100] information/ConfigItem: Instantiated 1 IcingaDB.
[2025-03-18 14:22:01 +1100] information/ConfigItem: Instantiated 1 FileLogger.
[2025-03-18 14:22:01 +1100] information/ConfigItem: Instantiated 3 Zones.
[2025-03-18 14:22:01 +1100] information/ConfigItem: Instantiated 1 CheckerComponent.
[2025-03-18 14:22:01 +1100] information/ConfigItem: Instantiated 1 Endpoint.
[2025-03-18 14:22:01 +1100] information/ConfigItem: Instantiated 2 ApiUsers.
[2025-03-18 14:22:01 +1100] information/ConfigItem: Instantiated 2 Users.
[2025-03-18 14:22:01 +1100] information/ConfigItem: Instantiated 1 ApiListener.
[2025-03-18 14:22:01 +1100] information/ConfigItem: Instantiated 1 NotificationComponent.
[2025-03-18 14:22:01 +1100] information/ConfigItem: Instantiated 246 CheckCommands.
[2025-03-18 14:22:01 +1100] information/ConfigItem: Instantiated 1 UserGroup.
[2025-03-18 14:22:01 +1100] information/ConfigItem: Instantiated 3 ServiceGroups.
[2025-03-18 14:22:01 +1100] information/ConfigItem: Instantiated 3 TimePeriods.
[2025-03-18 14:22:01 +1100] information/ConfigItem: Instantiated 1 ScheduledDowntime.
[2025-03-18 14:22:01 +1100] information/ConfigItem: Instantiated 13748 Services.
[2025-03-18 14:22:01 +1100] information/ScriptGlobal: Dumping variables to file '/var/cache/icinga2/icinga2.vars'
[2025-03-18 14:22:01 +1100] information/cli: Finished validating the configuration file(s).
# cat /etc/systemd/system/icinga-director.service
[Unit]
Description=Icinga Director - Monitoring Configuration
Documentation=https://icinga.com/docs/director/latest/
Wants=network.target

[Service]
EnvironmentFile=-/etc/default/icinga-director
EnvironmentFile=-/etc/sysconfig/icinga-director
ExecStart=/usr/bin/icingacli director daemon run
ExecReload=/bin/kill -HUP ${MAINPID}
User=icingadirector
SyslogIdentifier=icingadirector
Type=notify

NotifyAccess=main
WatchdogSec=10
RestartSec=30
Restart=always

[Install]
WantedBy=multi-user.target
# icingacli module list
MODULE         VERSION   STATE     DESCRIPTION
cube           1.3.3     enabled   Cube for Icinga Web
director       1.11.3    enabled   Director - Config tool for Icinga 2
doc            2.12.2    enabled   Documentation module
icingadb       1.1.3     enabled   Icinga DB Web
icingalegacytheme 1.0.0     enabled   This module brings back the icinga theme from icingaweb2 2.9
icingaweb2-theme-solarized 1.0.0     enabled   Solarized Theme
incubator      0.22.0    enabled   Incubator provides bleeding-edge libraries
netbox         4.0.8.1   enabled   Import devices, sites and other objects from Netbox
# cat /etc/icingaweb2/resouces.ini
<snip>...

[director]
type = db
db = mysql
host = localhost
dbname = director
username = directoruser
password = <snip>
charset = utf8
use_ssl = 0
# cat /etc/icingaweb2/modules/director/config.ini
[db]
resource = "director"

Data from /about

Icinga Web 2 Version	2.12.2
Git commit	7f9217b0521f19f770ad59ca17d7d4b3514e54e5
PHP Version	8.3.17
Git commit date	2024-11-13

Loaded Libraries
icinga/icinga-php-library	0.14.1
icinga/icinga-php-thirdparty	0.12.1

Loaded Modules
icingadb		        1.1.3	
cube		                1.3.3	
director		        1.11.3	
doc		                2.12.2
icingalegacytheme		1.0.0
icingaweb2-theme-solarized	1.0.0
incubator		        0.22.0
netbox		                4.0.8.1

Check if you have any special characters in your .ini file that mess up the syntax.

1 Like

That seems to have been the issue, @log1c - thank you! I’ll make a note on the github ticket - mayhaps the error message could be improved in someway.

1 Like