Set up Icinga2 automatically via Puppet

No, don’t do it this way. This was just to compare what’s going on. For some reason, the ticket calculation doesn’t work … and it will be cumbersome to do this for any new host. As said, the Puppet module should work OOTB with generating and implemented the ticket into the Puppet run for every new host.

Question aside - did you setup Icinga manually once before diving into Puppet already? This helps to understand how zones, endpoints and TLS certificates work before abstracting this into automation.

The reload output looks like a wrong configuration now being deployed, you can run icinga2 daemon -C for a full validation output.

Cheers,
Michael

I’ve setup Icinga manually but it’s a few month ago and I think I’ve forgot a little bit :(…

When I ran icinga2 daemon -C I got the following:

root@worker-template:~# icinga2 daemon -C
[2019-05-16 18:29:45 +0200] information/cli: Icinga application loader (version: r2.10.4-1)
[2019-05-16 18:29:45 +0200] information/cli: Loading configuration file(s).
[2019-05-16 18:29:45 +0200] information/ConfigItem: Committing config item(s).
[2019-05-16 18:29:45 +0200] information/ApiListener: My API identity: worker-template.local
[2019-05-16 18:29:45 +0200] critical/config: Error: Validation failed for object 'master' of type 'Zone'; Attribute 'endpoints': Object 'icinga2master.vorlage.local' of type 'Endpoint' does not exist.
Location: in /etc/icinga2/zones.conf: 12:3-12:48
/etc/icinga2/zones.conf(10):
/etc/icinga2/zones.conf(11): object Zone "master" {
/etc/icinga2/zones.conf(12):   endpoints = [ "icinga2master.vorlage.local", ]
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
/etc/icinga2/zones.conf(13): }
/etc/icinga2/zones.conf(14):

[2019-05-16 18:29:45 +0200] critical/config: 1 error
root@worker-template:~#

It seems that I had to define an Endpoint for “icinga2master.vorlage.local”, right?

EDIT:

I’ve tried this and now my manifest looks like this:

# include icinga2

class { '::icinga2':
        manage_repo => true,
        manage_package => true,
        confd     => false,
        features  => ['checker','mainlog','notification','statusdata','compatlog','command'],
        constants => {
                'ZoneName' => 'master',
        },
}

class { '::icinga2::feature::api':
        pki             => 'icinga2',
        ca_host         => 'icinga2master.vorlage.local',
        ticket_salt     => '<very-save-ticket_salt>',
        ensure          => 'present',
        accept_config   => true,
        accept_commands => true,
        endpoints       => {
           $facts['fqdn'] => {
              'host' => $facts['ipaddress'],
           }, 'icinga2master.vorlage.local' => {
                  'host' => '192.168.117.30',
           },
        },
        zones           => {
           'master' => {
              'endpoints' => ['icinga2master.vorlage.local']
           },
     }
}

icinga2::object::zone { 'global-templates':
  global => true,
}

I’m wondering why the resource doesn’t create the Endpoint object, is it still defined like this?

...
        endpoints       => {
                'NodeName'                   => {},
                'icinga2master.vorlage.local' => {
                  'host' => '192.168.117.30',
                }
        },
        zones           => {
                'NodeName' => {
                   'endpoints' => ["${facts['fqdn']}"],
                   'parent'    => 'master',
                },
                'master' => {
                   'endpoints' => ['icinga2master.vorlage.local']
                }

        }

Can you extract the report from Puppet to see whether this actually happened?

Cheers,
Michael

I just added the entry manually :

endpoints       => {
           $facts['fqdn'] => {
              'host' => $facts['ipaddress'],
           }, 'icinga2master.vorlage.local' => {
                  'host' => '192.168.117.30',
           },
        },

Do you mean with ‘extract the report’ that I paste the Output of my ‘puppet agent -t --debug’?

Only the relevant parts which include creating icinga objects. I haven’t used the module for a while now, maybe 2.1.0 broke something or the behaviour changed. So to speak, could be a bug as well.

I hope this are the relavant parts :slight_smile: :

Debug: Executing: '/bin/systemctl start icinga2'
Debug: Running journalctl command to get logs for systemd start failure: journalctl -n 50 --since '5 minutes ago' -u icinga2 --no-pager
Debug: Executing: 'journalctl -n 50 --since '5 minutes ago' -u icinga2 --no-pager'
Error: Systemd start for icinga2 failed!
journalctl log for icinga2:
-- Logs begin at Thu 2019-05-16 14:03:32 CEST, end at Thu 2019-05-16 19:02:07 CEST. --
May 16 19:02:07 worker-template systemd[1]: Starting Icinga host/service/network monitoring system...
May 16 19:02:07 worker-template icinga2[25989]: [2019-05-16 19:02:07 +0200] information/cli: Icinga application loader (version: r2.10.4-1)
May 16 19:02:07 worker-template icinga2[25989]: [2019-05-16 19:02:07 +0200] information/cli: Loading configuration file(s).
May 16 19:02:07 worker-template icinga2[25989]: [2019-05-16 19:02:07 +0200] information/ConfigItem: Committing config item(s).
May 16 19:02:07 worker-template icinga2[25989]: [2019-05-16 19:02:07 +0200] information/ApiListener: My API identity: worker-template.local
May 16 19:02:07 worker-template icinga2[25989]: [2019-05-16 19:02:07 +0200] critical/config: Error: Endpoint 'worker-template.local' does not belong to a zone.
May 16 19:02:07 worker-template icinga2[25989]: Location: in /etc/icinga2/zones.conf: 7:1-7:47
May 16 19:02:07 worker-template icinga2[25989]: /etc/icinga2/zones.conf(5): }
May 16 19:02:07 worker-template icinga2[25989]: /etc/icinga2/zones.conf(6):
May 16 19:02:07 worker-template icinga2[25989]: /etc/icinga2/zones.conf(7): object Endpoint "worker-template.local" {
May 16 19:02:07 worker-template icinga2[25989]:                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 16 19:02:07 worker-template icinga2[25989]: /etc/icinga2/zones.conf(8):   host = "192.168.117.25"
May 16 19:02:07 worker-template icinga2[25989]: /etc/icinga2/zones.conf(9): }
May 16 19:02:07 worker-template icinga2[25989]: [2019-05-16 19:02:07 +0200] critical/config: 1 error
May 16 19:02:07 worker-template systemd[1]: icinga2.service: Main process exited, code=exited, status=1/FAILURE
May 16 19:02:07 worker-template systemd[1]: Failed to start Icinga host/service/network monitoring system.
May 16 19:02:07 worker-template systemd[1]: icinga2.service: Unit entered failed state.
May 16 19:02:07 worker-template systemd[1]: icinga2.service: Failed with result 'exit-code'.

Error: /Stage[main]/Icinga2::Service/Service[icinga2]/ensure: change from 'stopped' to 'running' failed: Systemd start for icinga2 failed!
journalctl log for icinga2:
-- Logs begin at Thu 2019-05-16 14:03:32 CEST, end at Thu 2019-05-16 19:02:07 CEST. --
May 16 19:02:07 worker-template systemd[1]: Starting Icinga host/service/network monitoring system...
May 16 19:02:07 worker-template icinga2[25989]: [2019-05-16 19:02:07 +0200] information/cli: Icinga application loader (version: r2.10.4-1)
May 16 19:02:07 worker-template icinga2[25989]: [2019-05-16 19:02:07 +0200] information/cli: Loading configuration file(s).
May 16 19:02:07 worker-template icinga2[25989]: [2019-05-16 19:02:07 +0200] information/ConfigItem: Committing config item(s).
May 16 19:02:07 worker-template icinga2[25989]: [2019-05-16 19:02:07 +0200] information/ApiListener: My API identity: worker-template.local
May 16 19:02:07 worker-template icinga2[25989]: [2019-05-16 19:02:07 +0200] critical/config: Error: Endpoint 'worker-template.local' does not belong to a zone.
May 16 19:02:07 worker-template icinga2[25989]: Location: in /etc/icinga2/zones.conf: 7:1-7:47
May 16 19:02:07 worker-template icinga2[25989]: /etc/icinga2/zones.conf(5): }
May 16 19:02:07 worker-template icinga2[25989]: /etc/icinga2/zones.conf(6):
May 16 19:02:07 worker-template icinga2[25989]: /etc/icinga2/zones.conf(7): object Endpoint "worker-template.local" {
May 16 19:02:07 worker-template icinga2[25989]:                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 16 19:02:07 worker-template icinga2[25989]: /etc/icinga2/zones.conf(8):   host = "192.168.117.25"
May 16 19:02:07 worker-template icinga2[25989]: /etc/icinga2/zones.conf(9): }
May 16 19:02:07 worker-template icinga2[25989]: [2019-05-16 19:02:07 +0200] critical/config: 1 error
May 16 19:02:07 worker-template systemd[1]: icinga2.service: Main process exited, code=exited, status=1/FAILURE
May 16 19:02:07 worker-template systemd[1]: Failed to start Icinga host/service/network monitoring system.
May 16 19:02:07 worker-template systemd[1]: icinga2.service: Unit entered failed state.
May 16 19:02:07 worker-template systemd[1]: icinga2.service: Failed with result 'exit-code'.
 (corrective)
Debug: Class[Icinga2::Service]: Resource is being skipped, unscheduling all events
Notice: /Stage[main]/Icinga2/Anchor[::icinga2::end]: Dependency Service[icinga2] has failures: true
Warning: /Stage[main]/Icinga2/Anchor[::icinga2::end]: Skipping because of failed dependencies
Debug: /Stage[main]/Icinga2/Anchor[::icinga2::end]: Resource is being skipped, unscheduling all events
Debug: Class[Icinga2]: Resource is being skipped, unscheduling all events
Debug: Stage[main]: Resource is being skipped, unscheduling all events
Debug: Finishing transaction 29383120

Hmm, that’s too far in the future. I am looking for the endpoint/zone creation bits, specifically the calls here.

Cheers,
Michael

Hm, okay… Where can I find this calls?

…and I’ve another question…

How can I automatize this scenario with puppet:

"If the host ‘puppet-node1.customerA.local’ is signed at the Puppet Master, it should get e.g. the WorkerName ‘worker-customerA.local’. and the WorkerZone ‘customerA’. "

Is this also possible to automatize this with puppet and the module?

Hmm, I’ve reset my VM where I my Icinga2-Node via Puppet…
When I run now an ‘puppet agent -t’ Icinga2 will be installed properly…

When I do an service icinga2 status I got the following:

root@worker-template:~# service icinga2 status
● icinga2.service - Icinga host/service/network monitoring system
   Loaded: loaded (/lib/systemd/system/icinga2.service; enabled; vendor preset: enabl
  Drop-In: /etc/systemd/system/icinga2.service.d
           └─limits.conf
   Active: active (running) since Fr 2019-05-17 11:07:01 CEST; 5min ago
 Main PID: 18914 (icinga2)
   CGroup: /system.slice/icinga2.service
           ├─18914 /usr/lib/x86_64-linux-gnu/icinga2/sbin/icinga2 --no-stack-rlimit d
           └─18948 /usr/lib/x86_64-linux-gnu/icinga2/sbin/icinga2 --no-stack-rlimit d

Mai 17 11:07:01 worker-template icinga2[18914]: [2019-05-17 11:07:01 +0200] i
Mai 17 11:07:01 worker-template icinga2[18914]: [2019-05-17 11:07:01 +0200] i
Mai 17 11:07:01 worker-template icinga2[18914]: [2019-05-17 11:07:01 +0200] i
Mai 17 11:07:01 worker-template icinga2[18914]: [2019-05-17 11:07:01 +0200] i
Mai 17 11:07:01 worker-template icinga2[18914]: [2019-05-17 11:07:01 +0200] i
Mai 17 11:07:01 worker-template icinga2[18914]: [2019-05-17 11:07:01 +0200] i
Mai 17 11:07:01 worker-template icinga2[18914]: [2019-05-17 11:07:01 +0200] i
Mai 17 11:07:01 worker-template icinga2[18914]: [2019-05-17 11:07:01 +0200] i
Mai 17 11:07:01 worker-template icinga2[18914]: [2019-05-17 11:07:01 +0200] i
Mai 17 11:07:01 worker-template systemd[1]: Started Icinga host/service/netwo
root@worker-template:~#

…and when I do after that an service icinga2 restart I got the following:

root@worker-template:~# service icinga2 restart
Job for icinga2.service failed because the control process exited with error code. Se                                                                                        e "systemctl status icinga2.service" and "journalctl -xe" for details.
root@worker-template:~# service icinga2 status
● icinga2.service - Icinga host/service/network monitoring system
   Loaded: loaded (/lib/systemd/system/icinga2.service; enabled; vendor preset: enabl
  Drop-In: /etc/systemd/system/icinga2.service.d
           └─limits.conf
   Active: failed (Result: exit-code) since Fr 2019-05-17 11:12:43 CEST; 8s ago
  Process: 20471 ExecStart=/usr/sbin/icinga2 daemon --close-stdio -e ${ICINGA2_ERROR_
  Process: 20460 ExecStartPre=/usr/lib/icinga2/prepare-dirs /etc/default/icinga2 (cod
 Main PID: 20471 (code=exited, status=1/FAILURE)

Mai 17 11:12:43 worker-template icinga2[20471]: /etc/icinga2/zones.conf(6):
Mai 17 11:12:43 worker-template icinga2[20471]: /etc/icinga2/zones.conf(7): o
Mai 17 11:12:43 worker-template icinga2[20471]:                             ^
Mai 17 11:12:43 worker-template icinga2[20471]: /etc/icinga2/zones.conf(8):
Mai 17 11:12:43 worker-template icinga2[20471]: /etc/icinga2/zones.conf(9): }
Mai 17 11:12:43 worker-template icinga2[20471]: [2019-05-17 11:12:43 +0200] c
Mai 17 11:12:43 worker-template systemd[1]: icinga2.service: Main process exi
Mai 17 11:12:43 worker-template systemd[1]: Failed to start Icinga host/servi
Mai 17 11:12:43 worker-template systemd[1]: icinga2.service: Unit entered fai
Mai 17 11:12:43 worker-template systemd[1]: icinga2.service: Failed with resu
lines 1-19/19 (END)
● icinga2.service - Icinga host/service/network monitoring system
   Loaded: loaded (/lib/systemd/system/icinga2.service; enabled; vendor preset: enabled)
  Drop-In: /etc/systemd/system/icinga2.service.d
           └─limits.conf
   Active: failed (Result: exit-code) since Fr 2019-05-17 11:12:43 CEST; 8s ago
  Process: 20471 ExecStart=/usr/sbin/icinga2 daemon --close-stdio -e ${ICINGA2_ERROR_LOG} (code=exited, status=1/FAILURE)
  Process: 20460 ExecStartPre=/usr/lib/icinga2/prepare-dirs /etc/default/icinga2 (code=exited, status=0/SUCCESS)
 Main PID: 20471 (code=exited, status=1/FAILURE)

Mai 17 11:12:43 worker-template icinga2[20471]: /etc/icinga2/zones.conf(6):
Mai 17 11:12:43 worker-template icinga2[20471]: /etc/icinga2/zones.conf(7): object Endpoint "worker-template.local" {
Mai 17 11:12:43 worker-template icinga2[20471]:                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mai 17 11:12:43 worker-template icinga2[20471]: /etc/icinga2/zones.conf(8):   host = "192.168.117.25"
Mai 17 11:12:43 worker-template icinga2[20471]: /etc/icinga2/zones.conf(9): }
Mai 17 11:12:43 worker-template icinga2[20471]: [2019-05-17 11:12:43 +0200] critical/config: 1 error
Mai 17 11:12:43 worker-template systemd[1]: icinga2.service: Main process exited, code=exited, status=1/FAILURE
Mai 17 11:12:43 worker-template systemd[1]: Failed to start Icinga host/service/network monitoring system.
Mai 17 11:12:43 worker-template systemd[1]: icinga2.service: Unit entered failed state.
Mai 17 11:12:43 worker-template systemd[1]: icinga2.service: Failed with result 'exit-code'.

When I look in my from puppet generated /etc/icinga2/zones.conf it has the following content:

# This file is managed by Puppet. DO NOT EDIT.

object Endpoint "icinga2master.vorlage.local" {
  host = "192.168.117.30"
}

object Endpoint "worker-template.local" {
  host = "192.168.117.25"
}

object Zone "global-templates" {
  global = true
}

object Zone "master" {
  endpoints = [ "icinga2master.vorlage.local", ]
}

I think it looks good but I don’t understand why I got an error anymore.

I’ve anymore the problem, that the generated ticket from my new Icinga2-Node (which is setup via Puppet) seems to be different than this on my Icinga2-Master. After the run of ‘puppet agent -t --debug’ I got the following Error:

Notice: /Stage[main]/Icinga2::Feature::Api/Exec[icinga2 pki request]/returns: information/cli: Writing CA certificate to file '/var/lib/icinga2/certs/ca.crt'.
Notice: /Stage[main]/Icinga2::Feature::Api/Exec[icinga2 pki request]/returns: critical/cli: !!! Invalid ticket for CN 'worker-template.local'.
Error: '"/usr/sbin/icinga2" pki request --host icinga2master.vorlage.local --port 5665 --ca /var/lib/icinga2/certs/ca.crt --key /var/lib/icinga2/certs/worker-template.local.key --cert /var/lib/icinga2/certs/worker-template.local.crt --trustedcert /var/lib/icinga2/certs/trusted-cert.crt --ticket <ticketsalt>' returned 1 instead of one of [0]
Error: /Stage[main]/Icinga2::Feature::Api/Exec[icinga2 pki request]/returns: change from 'notrun' to ['0'] failed: '"/usr/sbin/icinga2" pki request --host icinga2master.vorlage.local --port 5665 --ca /var/lib/icinga2/certs/ca.crt --key /var/lib/icinga2/certs/worker-template.local.key --cert /var/lib/icinga2/certs/worker-template.local.crt --trustedcert /var/lib/icinga2/certs/trusted-cert.crt --ticket <ticketsalt>' returned 1 instead of one of [0]
Debug: Executing: 'diff -u /etc/icinga2/features-available/checker.conf /tmp/puppet-file20190517-17846-179ftqa'

…and when I have a look on my Icinga2-Master into /var/log/icinga2/icinga2.log I got the following errors:

[2019-05-17 09:05:30 +0200] information/WorkQueue: #7 (IdoMysqlConnection, ido-mysql) items: 7, rate: 3.01667/s (181/min 933/5min 2811/15min);
[2019-05-17 09:05:40 +0200] information/WorkQueue: #7 (IdoMysqlConnection, ido-mysql) items: 7, rate: 3.01667/s (181/min 929/5min 2807/15min);
[2019-05-17 09:06:00 +0200] information/ConfigObject: Dumping program state to file '/var/lib/icinga2/icinga2.state'
[2019-05-17 09:06:00 +0200] information/WorkQueue: #6 (ApiListener, SyncQueue) items: 0, rate:  0/s (0/min 0/5min 0/15min);
[2019-05-17 09:06:00 +0200] information/WorkQueue: #7 (IdoMysqlConnection, ido-mysql) items: 0, rate: 3.13333/s (188/min 938/5min 2818/15min);
[2019-05-17 09:06:00 +0200] information/WorkQueue: #5 (ApiListener, RelayQueue) items: 0, rate: 0.65/s (39/min 192/5min 582/15min);
[2019-05-17 09:06:10 +0200] information/WorkQueue: #7 (IdoMysqlConnection, ido-mysql) items: 1, rate: 3.01667/s (181/min 933/5min 2813/15min);
[2019-05-17 09:06:40 +0200] information/WorkQueue: #7 (IdoMysqlConnection, ido-mysql) items: 1, rate: 3.01667/s (181/min 929/5min 2807/15min);
[2019-05-17 09:07:08 +0200] information/ApiListener: New client connection from [192.168.117.25]:51044 (no client certificate)
[2019-05-17 09:07:08 +0200] warning/TlsStream: TLS stream was disconnected.
[2019-05-17 09:07:08 +0200] information/ApiListener: No data received on new API connection. Ensure that the remote endpoints are properly configured in a cluster setup.
[2019-05-17 09:07:08 +0200] information/ApiListener: New client connection for identity 'worker-template.local' from [192.168.117.25]:51046 (certificate validation failed: code 9: certificate is not yet valid)
[2019-05-17 09:07:08 +0200] information/JsonRpcConnection: Received certificate request for CN 'worker-template.local' not signed by our CA.
[2019-05-17 09:07:08 +0200] warning/JsonRpcConnection: Ticket '<ticketsalt>' for CN 'worker-template.local' is invalid.
[2019-05-17 09:07:08 +0200] warning/TlsStream: TLS stream was disconnected.
[2019-05-17 09:07:08 +0200] warning/JsonRpcConnection: API client disconnected for identity 'worker-template.local'
[2019-05-17 09:07:10 +0200] information/WorkQueue: #7 (IdoMysqlConnection, ido-mysql) items: 6, rate: 2.98333/s (179/min 931/5min 2811/15min);
[2019-05-17 09:07:18 +0200] information/WorkQueue: #11 (JsonRpcConnection, #0) items: 0, rate: 0.0166667/s (1/min 1/5min 1/15min);

Therefore I reexecuted @dnsmichi command icinga2 pki ticket --cn sws-noc-worker-template.local and got the following:

root@icinga2master:~# icinga2 pki ticket --cn worker-template.local
<ticketsalt>
root@icinga2master:~#

I don’t understand, why the generation of the ticket doesn’t seem to work properly :sleepy:!?

(By the way, @dnsmichi I’ve change the Salt-Ticket-IDs :slight_smile: :wink: )

Hi,

this message appears if

  1. the certificate name mismatches the associated endpoint name.
  2. the ticketsalt is wrong, has to be the same on master and agent/satellite.
  3. the optional fingerprint isn’t correct, it’s the fingerprint of the certificate of your CA master and not from the CA itself.

The procedure of configuration is described in the documentation and as complete code in examples/ini_master.pp and init_slave.pp (init_slave_validate.pp).

You get the fingerprint by
openssl x509 -noout -fingerprint -sha1 -inform pem -in master.crt

2 Likes

Hmm…

  1. the certificate name mismatches the associated endpoint name.

The relevant endpoint name is the FQDN (in that case worker-template.local) of the Icinga2-Node, right?

  1. the ticketsalt is wrong, has to be the same on master and agent/satellite.

Where can I find the ticketsalt? Is it the one, which is on the master in the /etc/icinga2/constants.conf:

/* Secret key for remote node tickets */

const TicketSalt = “very_secure_ticket_salt”

  1. the optional fingerprint isn’t correct, it’s the fingerprint of the certificate of your CA master and not from the CA itself.

I don’t use an optional fingerprint in my Test-Setup ;)… So I think then this shouldn’t be the problem, right?

1-2) right
3) you posted code and fingerprint was set

Please post yout complete code for master and slave without ticketsalt is shown.

@lbetz: Thanks for your fast reply. I’ve changed it back to ticketsalt because of this points from @dnsmichi: https://github.com/Icinga/puppet-icinga2/issues/495

My code on my Puppet-Master for the Icinga2-Slave is this:

$master_cert = 'icinga2master.vorlage.local'
$master_ip = '192.168.117.30'

# get it on CA host 'openssl x509 -noout -fingerprint -sha1 -inform pem -in /var/lib/icinga2/certs/master.localdomain.crt'
$fingerprint = 'CC:90:41:24:D0:92:CA:C3:C8:A4:75:97:C7:32:CB:91:45:78:6H:28'


class { '::icinga2':
        manage_repo => true,
        manage_package => true,
        confd     => false,
        features  => ['checker','mainlog','notification','statusdata','compatlog','command'],
        constants => {
                'NodeName' => $facts['fqdn'],
        },
}

class { '::icinga2::feature::api':
        pki             => 'icinga2',
        ca_host         => $master_ip,
        ticket_salt     => '<ticketsalt>',
        ensure          => 'present',
        accept_config   => true,
        accept_commands => true,
        endpoints       => {
          'NodeName'       => {},
          "${master_cert}" => {
             'host' => $master_ip,
          }
        },
        zones           => {
          'ZoneName' => {
            'endpoints' => [ 'NodeName' ],
            'parent'    => 'master',
          },
          'master' => {
            'endpoints' => [ $master_cert ],
          },
        },
        fingerprint     => $fingerprint,
}

icinga2::object::zone { 'global-templates':
  global => true,
}

( ticket_salt => ’ <ticketsalt>’ is from /etc/icinga2/constants.conf )
(fingerprint ==> ‘CC:90:41:24:D0:92:CA:C3:C8:A4:75:97:C7:32:CB:91:45:78:6H:28’ is that value which I get when I execute ‘openssl x509 -noout -fingerprint -sha1 -inform pem -in /var/lib/icinga2/certs/master.localdomain.crt’)

…when I do then an ‘puppet agent -t’ the following /etc/icinga2/zones.conf will be created on my Slave:

# This file is managed by Puppet. DO NOT EDIT.

object Endpoint NodeName {
}

object Endpoint "icinga2master.vorlage.local" {
  host = "192.168.117.30"
}

object Zone ZoneName {
  endpoints = [ NodeName, ]
  parent = "master"
}

object Zone "global-templates" {
  global = true
}

object Zone "master" {
  endpoints = [ "icinga2master.vorlage.local", ]
}

(I don’t know the correct Syntax how to ask “ZoneName” and “NodeName” correct from hiera, do you have an idea? )

When I reexecute the command from @dnsmichi icinga2 pki ticket --cn worker-template.local
I got the same as this morning:

root@icinga2master:~# icinga2 pki ticket --cn worker-template.local
<ticketsalt>
root@icinga2master:~#

But I also have to say that I signed my Icinga2 node - which I created via puppet - on my Icinga2 master by hand and it worked.
Then I reset the Icinga2 node to test it again but the old Status of the VM is signed anymore on my Icinga2-Master…

Hope that’s all understandable what I’ve written…

@lbetz, @dnsmichi: Do you have any ideas? At the weekend I’ve tested again by resetting my Test-VMs to an old Snapshot an I’ve used the TicketSalt from my Icinga2-Master (/etc/icinga2/constants.conf):

/* Secret key for remote node tickets */
const TicketSalt = “<ticketsalt>”`

…and the Fingerprint from my Icinga2-Master (which I get via ‘openssl x509 -noout -fingerprint -sha1 -inform pem -in /var/lib/icinga2/certs/icinga2master.vorlage.local.crt’):

root@icinga2master:~# openssl x509 -noout -fingerprint -sha1 -inform pem -in /var/lib/icinga2/certs/icinga2master.vorlage.local.crt
SHA1 Fingerprint=CC:90:41:24:D0:92:CA:C3:C8:A4:75:97:C7:32:CB:91:45:78:6H:28
root@icinga2master:~#

When I have now an look in my Logs on the Icinga2-Master (/var/log/icinga2/icinga2.log) I can see the following at the time of my Puppet-Run / trying to connect from the Icinga2-Satelitte to the Icinga2-Master:

[2019-05-19 21:54:26 +0200] information/WorkQueue: #7 (IdoMysqlConnection, ido-mysql) items: 6, rate: 2.98333/s (179/min 931/5min 2761/15min);
[2019-05-19 21:54:46 +0200] information/ConfigObject: Dumping program state to file '/var/lib/icinga2/icinga2.state'
[2019-05-19 21:54:56 +0200] information/WorkQueue: #5 (ApiListener, RelayQueue) items: 0, rate: 0.7/s (42/min 197/5min 594/15min);
[2019-05-19 21:54:56 +0200] information/WorkQueue: #6 (ApiListener, SyncQueue) items: 0, rate:  0/s (0/min 0/5min 0/15min);
[2019-05-19 21:54:56 +0200] information/WorkQueue: #7 (IdoMysqlConnection, ido-mysql) items: 6, rate: 3.01667/s (181/min 933/5min 2823/15min);
[2019-05-19 21:55:01 +0200] information/ApiListener: New client connection from [192.168.117.25]:43078 (no client certificate)
[2019-05-19 21:55:01 +0200] warning/TlsStream: TLS stream was disconnected.
[2019-05-19 21:55:01 +0200] information/ApiListener: No data received on new API connection. Ensure that the remote endpoints are properly configured in a cluster setup.
[2019-05-19 21:55:02 +0200] information/ApiListener: New client connection for identity 'worker-template.local' from [192.168.117.25]:43080 (certificate validation failed: code 18: self signed certificate)
[2019-05-19 21:55:02 +0200] information/JsonRpcConnection: Received certificate request for CN 'worker-template.local' not signed by our CA.
[2019-05-19 21:55:02 +0200] information/JsonRpcConnection: Sending certificate response for CN 'worker-template.local' to endpoint 'worker-template.local' (auto-signing ticket).
[2019-05-19 21:55:02 +0200] warning/TlsStream: TLS stream was disconnected.
[2019-05-19 21:55:02 +0200] warning/JsonRpcConnection: API client disconnected for identity 'worker-template.local'
[2019-05-19 21:55:03 +0200] information/ApiListener: New client connection for identity 'worker-template.local' from [192.168.117.25]:43082 (no Endpoint object found for identity)
[2019-05-19 21:55:03 +0200] information/JsonRpcConnection: Received certificate request for CN 'worker-template.local' signed by our CA.
[2019-05-19 21:55:03 +0200] information/JsonRpcConnection: The certificate for CN 'worker-template.local' is valid and uptodate. Skipping automated renewal.
[2019-05-19 21:55:06 +0200] information/WorkQueue: #7 (IdoMysqlConnection, ido-mysql) items: 6, rate: 2.98333/s (179/min 933/5min 2824/15min);
[2019-05-19 21:55:12 +0200] information/WorkQueue: #11 (JsonRpcConnection, #0) items: 0, rate: 0.05/s (3/min 3/5min 3/15min);
[2019-05-19 21:55:16 +0200] information/WorkQueue: #7 (IdoMysqlConnection, ido-mysql) items: 6, rate: 2.98333/s (179/min 931/5min 2820/15min);
[2019-05-19 21:55:26 +0200] information/WorkQueue: #7 (IdoMysqlConnection, ido-mysql) items: 6, rate: 2.95/s (177/min 929/5min 2820/15min);

Did I understand correctly that the connection worked and the Icinga2-Satelitte is connected with the Icinga2-Master?

Hi,

while I appreciate all the details, I’ve lost track on what exactly is the problem now. Is it still the wrong PKI ticket being generated?

Yet again, you’ve shared your private TicketSalt again. Please avoid that in future postings, replace that with something like <ticketsalt>.

The logs look sane, the satellite wants a certificate and gets it auto-signed.

[2019-05-19 21:55:02 +0200] information/ApiListener: New client connection for identity 'worker-template.local' from [192.168.117.25]:43080 (certificate validation failed: code 18: self signed certificate)
[2019-05-19 21:55:02 +0200] information/JsonRpcConnection: Received certificate request for CN 'worker-template.local' not signed by our CA.
[2019-05-19 21:55:02 +0200] information/JsonRpcConnection: Sending certificate response for CN 'worker-template.local' to endpoint 'worker-template.local' (auto-signing ticket).

The only issue with connecting is that there’s no Endpoint object on the master yet.

[2019-05-19 21:55:03 +0200] information/ApiListener: New client connection for identity 'worker-template.local' from [192.168.117.25]:43082 (no Endpoint object found for identity)
[2019-05-19 21:55:03 +0200] information/JsonRpcConnection: Received certificate request for CN 'worker-template.local' signed by our CA.
[2019-05-19 21:55:03 +0200] information/JsonRpcConnection: The certificate for CN 'worker-template.local' is valid and uptodate. Skipping automated renewal.

That being said, TLS works, but the secondary level of trust is not yet configured - thus generating the Endpoint object for the satellite on the master. Similar thing as if you would manually edit zones.conf with a manual setup.

I would investigate into this direction if I were you.

Cheers,
Michael

1 Like

@dnsmichi: Thanks a lot for your fast reply :).

Hi,
while I appreciate all the details, I’ve lost track on what exactly is the problem now. Is it still the wrong PKI ticket being generated?

No, that seems to work fine now after I’ve used the Fingerprint at the TicketSalt from my Icinga2-Master (If I understood you correctly, does the output from the file /var/log/icinga2/icinga2.log mean that too, right?)

The only issue with connecting is that there’s no Endpoint object on the master yet.

[2019-05-19 21:55:03 +0200] information/ApiListener: New client connection for identity 'worker-template.local' from [192.168.117.25]:43082 (no Endpoint object found for identity)
[2019-05-19 21:55:03 +0200] information/JsonRpcConnection: Received certificate request for CN 'worker-template.local' signed by our CA.
[2019-05-19 21:55:03 +0200] information/JsonRpcConnection: The certificate for CN 'worker-template.local' is valid and uptodate. Skipping automated renewal.

That being said, TLS works, but the secondary level of trust is not yet configured - thus generating the Endpoint object for the satellite on the master. Similar thing as if you would manually edit zones.conf with a manual setup.

Okay, that sounds good :). But how can I configure the secondary level of trust that the Endpoint object will generated for the satellite on the master?
I don’t want to edit zones.conf manually but via Puppet…

Hope, everything is clear what I wanna do :)…

Thanks a lot and best regards,
Matthias

Similar to how you’ve created the profile for your satellite, the master also needs to have the zones and endpoints hash configured inside the API feature. @lbetz already pointed you towards the examples directory on GitHub, specifically this one:

Might also be worthwhile to look for a workshop, configuring Puppet and discussing things online really is hard.

Cheers,
Michael

hi @maze-m,

have a look on your other thread from me.

cu denny