IcingaDB error during Icinga2 initial install: "invalid configuration: Redis host missing"

Problem Description:
I am doing an initial install of Icinga on Ubuntu 22.04.1 using the apt repo as the package source. I am using the Icinga documentation for the installation process, not third-party sources.

I am at the stage were I am setting up icingadb, and I get an error starting the service, which I can’t seem to resolve. It is acting like there is a config file parsing error.

Specific systemd error for IcingaDB service
Aug 25 17:22:20 icinga2 systemd[1]: Starting Icinga DB...
Aug 25 17:22:20 icinga2 icingadb[33218]: invalid configuration: Redis host missing
Aug 25 17:22:20 icinga2 systemd[1]: icingadb.service: Main process exited, code=exited, status=2/INVALIDARGUMENT
Aug 25 17:22:20 icinga2 systemd[1]: icingadb.service: Failed with result 'exit-code'.
Aug 25 17:22:20 icinga2 systemd[1]: Failed to start Icinga DB.

I am using the default file (/etc/icingadb/config.yml) but modified it to reflect MySQL server password, and changed nothing else, but made sure that redis lines were present and not commented out.

Prior to this error:

  1. SQL service is confirmed operational (using mariadb)
  2. Redis is confirmed operational.
  3. Icinga2 service is running fine.

I just can’t seem to get past the above error.

I tried:

  1. running icingadb directly to see if I could get more insights, but it didn’t provide anything new.
  2. I also tried a new config.yml with no comments in it, just manually entered config lines, didn’t fix it.
  3. Verified the the systemd service was calling the correct config.yml
  4. Added intentional error to config.yml SQL section to make sure it was loading correct file.
  5. Checked to make sure the feature was enabled in icinga

Icinga is connecting to REDIS, as the debug log shows:

[2022-08-25 16:49:22 +0000] notice/IcingaDB: Trying to connect to Redis server (async) on host '127.0.0.1:6380'
[2022-08-25 16:49:22 +0000] notice/IcingaDB: Connected to Redis server
icingadb config.yml
# This is the configuration file for Icinga DB.

# Connection configuration for the database to which Icinga DB synchronizes monitoring data.
# This is also the database used in Icinga DB Web to view and work with the data.
# In high availability setups, all Icinga DB instances must write to the same database.
database:
  # Database type. Either 'mysql' for MySQL or 'pgsql' for PostgreSQL.
  # Defaults to 'mysql'.
  type: mysql

  # Database host or absolute Unix socket path.
  host: localhost

  # Database port. By default, the MySQL or PostgreSQL port, depending on the database type.
  port: 3306

  # Database name.
  database: icingadb

  # Database user.
  user: icingadb

  # Database password.
password: *<REDACTED>*

# Connection configuration for the Redis server where Icinga 2 writes its configuration, state and history items.
# This is the same connection as configured in the 'icingadb' feature of the corresponding Icinga 2 node.
# High availability setups require a dedicated Redis server per Icinga 2 node and
# therefore a dedicated Icinga DB instance that connects to it.
redis:
  # Redis host or absolute Unix socket path.
  host: localhost

  # Redis port.
  # Defaults to '6380' since the Redis server provided by the 'icingadb-redis' package listens on that port.
  port: 6380

  # Redis password.
#  password:

System details:
Platform: Ubuntu Linux 22.04.1 amd64 (Jammy)
APT repo used: icinga-jammy (deb https://packages.icinga.com/ubuntu icinga-jammy main)
CPUs: 4
RAM: 8GB
Disk: 15GB

APT Packages Installed Details:
icinga2 2.13.5-1.jammy [ubuntu icinga-jammy/icinga-jammy main]
icinga2-bin 2.13.5-1.jammy [ubuntu icinga-jammy/icinga-jammy main]
icinga2-common 2.13.5-1.jammy [ubuntu icinga-jammy/icinga-jammy main]
icinga2-doc 2.13.5-1.jammy [ubuntu icinga-jammy/icinga-jammy main]
icingadb 1.0.0-1.jammy [ubuntu icinga-jammy/icinga-jammy main]
icingadb-redis 7.0.2-1.jammy [ubuntu icinga-jammy/icinga-jammy main]
icingadb-redis-server 7.0.2-1.jammy [ubuntu icinga-jammy/icinga-jammy main]
icingadb-redis-tools 7.0.2-1.jammy [ubuntu icinga-jammy/icinga-jammy main]
Output of "icinga2 object list --type IcingaDB"
sudo icinga2 object list --type IcingaDB
Object 'icingadb' of type 'IcingaDB':
  % declared in '/etc/icinga2/features-enabled/icingadb.conf', lines 1:0-1:25
  * __name = "icingadb"
  * ca_path = ""
  * cert_path = ""
  * cipher_list = "ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:AES256-GCM-SHA384:AES128-GCM-SHA256"
  * connect_timeout = 15
  * crl_path = ""
  * db_index = 0
  * enable_tls = false
  * host = "127.0.0.1"
    % = modified in '/etc/icinga2/features-enabled/icingadb.conf', lines 2:3-2:20
  * insecure_noverify = false
  * key_path = ""
  * name = "icingadb"
  * package = "_etc"
  * password = ""
  * path = ""
  * port = 6380
    % = modified in '/etc/icinga2/features-enabled/icingadb.conf', lines 3:3-3:13
  * source_location
    * first_column = 0
    * first_line = 1
    * last_column = 25
    * last_line = 1
    * path = "/etc/icinga2/features-enabled/icingadb.conf"
  * templates = [ "icingadb" ]
    % = modified in '/etc/icinga2/features-enabled/icingadb.conf', lines 1:0-1:25
  * tls_protocolmin = "TLSv1.2"
  * type = "IcingaDB"
  * zone = ""
Content of icingadb.conf
object IcingaDB "icingadb" {
  host = "127.0.0.1"
  port = 6380
}
Icinga version output
icinga2 - The Icinga 2 network monitoring daemon (version: r2.13.5-1)

Copyright (c) 2012-2022 Icinga GmbH (https://icinga.com/)
License GPLv2+: GNU GPL version 2 or later <https://gnu.org/licenses/gpl2.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

System information:
  Platform: Ubuntu
  Platform version: 22.04.1 LTS (Jammy Jellyfish)
  Kernel: Linux
  Kernel version: 5.15.0-46-generic
  Architecture: x86_64

Build information:
  Compiler: GNU 11.2.0
  Build host: runner-hh8q3bz2-project-298-concurrent-0
  OpenSSL version: OpenSSL 3.0.2 15 Mar 2022

Application information:

General paths:
  Config directory: /etc/icinga2
  Data directory: /var/lib/icinga2
  Log directory: /var/log/icinga2
  Cache directory: /var/cache/icinga2
  Spool directory: /var/spool/icinga2
  Run directory: /run/icinga2

Old paths (deprecated):
  Installation root: /usr
  Sysconf directory: /etc
  Run directory (base): /run
  Local state directory: /var

Internal paths:
  Package data directory: /usr/share/icinga2
  State path: /var/lib/icinga2/icinga2.state
  Modified attributes path: /var/lib/icinga2/modified-attributes.conf
  Objects path: /var/cache/icinga2/icinga2.debug
  Vars path: /var/cache/icinga2/icinga2.vars
  PID path: /run/icinga2/icinga2.pid
Icinga feature lists output
sudo icinga2 feature list
Disabled features: command compatlog elasticsearch gelf graphite influxdb influxdb2 livestatus opentsdb perfdata statusdata syslog
Enabled features: api checker debuglog icingadb mainlog notification

I think I figured this out…the password for mysql db in the icingadb can’t have exclamation points (!) in it. Once I changed that password to have only letters and numbers in it, the issue resolved. The exclamation points in the password must have been throwing off the parsing of the config file so that later lines in the file were not read.

Is there a way to escape the password so that special characters can be used? I tried quotes, but it didn’t work.

Hello Nicholas,

Given your password is C0rrectHorseBatteryStaple!which quotes didn’t work? 'C0rrectHorseBatteryStaple!'? "C0rrectHorseBatteryStaple!"? Or even

password: |-
  C0rrectHorseBatteryStaple!

Best,
A/K