My story about KickStart Director

This is not a question; it is a story. I hope no one is offended. No harm intended.

When starting the kickstart wizard, I kept receiving this error:

I found no Endpoint object called "icinga.jumano.salt" on 192.168.0.90:5665 (KickstartHelper.php:360)

I entered my hosts domain name as shown here:

$ hostnamectl
   Static hostname: icinga.jumano.salt
         Icon name: computer-vm
           Chassis: vm
        Machine ID: 97270b089a234e20b51e49149840d47f
           Boot ID: 79c84a46ace54267b8f071d57b667888
    Virtualization: kvm
  Operating System: CentOS Stream 8
       CPE OS Name: cpe:/o:centos:centos:8
            Kernel: Linux 4.18.0-338.el8.x86_64
      Architecture: x86-64

I was sure I must have missed some step because I see the note is associated with the Endpoint Name info:

This is the name of the Endpoint object (and certificate name) you created for your ApiListener object. In case you are unsure what this means please make sure to read the documentation first

So I went back through the install pages for Icinga2, Icingaweb2, and Director and I could not find any references to the words, certificate, or endpoint, or ApiListener. I looked around on the web and found a mention that the Endpoint Name was the same as the fqdn of the host name. So I typed in icinga.jumano.salt (my internal network domain). I typed in the credentials for the API User as I did when I added an additional user into the api-users.conf file in the Icinga2 Install instructions (under the sub-title ā€˜Setting Up Icinga2 REST APIā€™). But the error came back, again, as above.

I tried many versions of the api root password, the director password, and on and on. But no joy.

Finally after looking over the web and not finding my problem I decided I must have done something wrong in the icinga2 api setup step. So I ran it again, and received this info:

# icinga2 api setup
information/cli: Generating new CA.
warning/cli: CA files '/var/lib/icinga2/ca//ca.crt' and '/var/lib/icinga2/ca//ca.key' already exist.
warning/cli: Found CA, skipping and using the existing one.
information/cli: Private key file '/var/lib/icinga2/certs//localhost.localdomain.key' already exists, not generating new certificate.
information/cli: API user config file `/etc/icinga2/conf.d/api-users.conf` already exists, not creating config file.
information/cli: Reading '/etc/icinga2/icinga2.conf'.
information/cli: Enabling the 'api' feature.
warning/cli: Feature 'api' already enabled.
information/cli: Updating 'NodeName' constant in '/etc/icinga2/constants.conf'.
information/cli: Backup file '/etc/icinga2/constants.conf.orig' already exists. Skipping backup.
information/cli: Updating 'ZoneName' constant in '/etc/icinga2/constants.conf'.
information/cli: Backup file '/etc/icinga2/constants.conf.orig' already exists. Skipping backup.
Done.

Now restart your Icinga 2 daemon to finish the installation!

And what do you know. Buried in the mumbo jumbo is an already existing certificate named localhost.localdomain.key. And it decided that it will not replace it.

When I first installed all the components, I had not reset my hostname or configured dns or a static ip for this host. I was just installing to see if it would work OK on centos 8 stream. Later, when I discovered that I should have a meaningful FQDN for my Director config, I reset it from localhost.localdomain to icinga.jumano.salt. I can see it in my terminal prompt.

So I deleted the certs associated with localhost.localdomain and ran the icinga2 api setup again. DOH! It created a cert with the same name.

So I looked further at where it might be finding the host name.

root@icinga:/home/chrisj $ hostname
icinga.jumano.salt

No not thereā€¦ So I looked in /etc/hosts

# cat /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6

Aha! Maybe this is where the code it getting localhost.localdomain. I changed it to this:

# cat /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.0.90 icinga.jumano.salt icinga

Nope.

Aha! Maybe this is where it is getting localhost.localdomain.

# cat /etc/icinga2/constants.conf
.
.
.
/* Our local instance name. By default this is the server's hostname as returned by `hostname --fqdn`.
 * This should be the common name from the API certificate.
 */
const NodeName = "localhost.localdomain"

/* Our local zone name. */
const ZoneName = "localhost.localdomain"

So I changed them to my new FQDN, but no. In fact after changing them and running icinga2 api setup, they were changed back to the original localhost.localdomain.

So still looking for the source of why icinga2 api setup keeps setting the certs to localhost.localdomain.

Now doing a google search, I find a reference to deleting the cert dirs and an icinga cache dir This is what they look like:

rm /var/lib/icinga2/ca/*
rm /var/lib/icinga2/certs/*
rm /var/cache/icinga2/*

I look in the cache dir in /var/cache/icinga2/icinga2.vars and found what looks like a version of the constants.conf file in json form. Here are the references from that file:

{"name":"NodeName","value":"localhost.localdomain"}
{"name":"ZoneName","value":"localhost.localdomain"}

I run the rm commands and then run icinga2 api setup again:

Hallelujah! I now have properly named certs and the constants.conf file is magically updated with the fqdn of my host.

I restart Icinga2 service. All good; it does not fail.

Now back to the Director page and I fill in my info:

  • DB Resource: director
  • Endpoint Name: icinga.jumano.salt
  • Icinga host: 192.168.0.90
  • Port: [default = 5665]
  • API User: icingaweb2
  • Password: (from the same as in 2nd user in api-users.conf)

I click on Run Import - although, at this point, Iā€™m not sure what Iā€™m importing.

ARG! Another Error.

ā€œWhat in the wide, wide, world of sports is-a goinā€™ on here?ā€ (look up that quote, its hilarious)

The requested path 'v1/console/execute-script' could not be found or the request method is not valid for this path. (RestApiResponse.php:111)

Another google search reveals it might be an issue with the permissions for the user in the api-users.conf. But I used the default values. That looks like an exception message. It must be serious if they never caught it with a reference to the associated documentation that says,

ā€œYouā€™re api user permissions do not allow this action. Please go to page 436 and read how to define permissions. The permission for this action is AllowKickstartImportā€.

So because I donā€™t know all the permissions, I change the user permissions in this file, /etc/icinga2/conf.d/api-users.conf:

from this, the default:
permissions = [ "status/query", "actions/*", "objects/modify/*", "objects/query/*" ]

to this:
permissions = [ "*" ]

I restart Icinga andā€¦ I see an icinga floating logo ā€¦ andā€¦ I think itā€™s working. At least it did not throw an error. So I think itā€™s good.

Ah, but, well thereā€™s a red tab and it says:

The Icinga Director Background Daemon is not running. Please check our documentation in case you need step by step instructions showing you how to fix this.

But that is a story for another bed time. Good night.


the versions and info on my setup:

Icinga Web 2 Version - 2.9.3
Git commit - c757a17530c326c7d6daebd6171486cb8167a8a1
PHP Version - 7.3.20
Git commit date - 2021-08-10
director	1.8.1
incubator	0.6.0
monitoring	2.9.3
reactbundle	0.9.0

# icinga2 -V
icinga2 - The Icinga 2 network monitoring daemon (version: 2.13.1-1)

Copyright (c) 2012-2021 Icinga GmbH (https://icinga.com/)
License GPLv2+: GNU GPL version 2 or later <https://gnu.org/licenses/gpl2.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

System information:
  Platform: CentOS Stream
  Platform version: 8
  Kernel: Linux
  Kernel version: 4.18.0-338.el8.x86_64
  Architecture: x86_64

Build information:
  Compiler: GNU 8.4.1
  Build host: runner-hh8q3bz2-project-507-concurrent-0
  OpenSSL version: OpenSSL 1.1.1k  FIPS 25 Mar 2021

Application information:

General paths:
  Config directory: /etc/icinga2
  Data directory: /var/lib/icinga2
  Log directory: /var/log/icinga2
  Cache directory: /var/cache/icinga2
  Spool directory: /var/spool/icinga2
  Run directory: /run/icinga2

Old paths (deprecated):
  Installation root: /usr
  Sysconf directory: /etc
  Run directory (base): /run
  Local state directory: /var

Internal paths:
  Package data directory: /usr/share/icinga2
  State path: /var/lib/icinga2/icinga2.state
  Modified attributes path: /var/lib/icinga2/modified-attributes.conf
  Objects path: /var/cache/icinga2/icinga2.debug
  Vars path: /var/cache/icinga2/icinga2.vars
  PID path: /run/icinga2/icinga2.pid

Thank you for sharing your story and struggles with us!
Itā€™s a learning experience, for you struggling through it, everyone who reads it and learns from what you did, but also for us as the development team, because it helps us understand better how people interact with our software!

1 Like