Hello,
I would like to connect my icinga2 server to Thruk IHM, but i am facing some TLS issues, (no client certificate).
Please someone could help ?
My errors message
- On thruk IHM, when i tried to connect to icinga2 by http type, there is my error on icinga2 server:
[2023-04-20 09:52:50 +0000] information/ApiListener: New client connection from [::ffff:172.18.0.3]:55270 (no client certificate)
[2023-04-20 09:52:50 +0000] information/HttpServerConnection: Request: POST /thruk/cgi-bin/remote.cgi (from [::ffff:172.18.0.3]:55270), user: root, agent: thruk, status: Bad Request).
[2023-04-20 09:52:50 +0000] information/HttpServerConnection: HTTP client disconnected (from [::ffff:172.18.0.3]:55270)
[2023-04-20 09:52:51 +0000] information/ApiListener: New client connection from [::ffff:172.18.0.3]:55272 (no client certificate)
[2023-04-20 09:52:51 +0000] information/HttpServerConnection: Request: POST /thruk/cgi-bin/remote.cgi (from [::ffff:172.18.0.3]:55272), user: root, agent: thruk, status: Bad Request).
[2023-04-20 09:52:51 +0000] information/HttpServerConnection: HTTP client disconnected (from [::ffff:172.18.0.3]:55272)
[2023-04-20 09:52:57 +0000] information/ApiListener: New client connection from [::ffff:172.18.0.3]:55274 (no client certificate)
[2023-04-20 09:52:57 +0000] information/HttpServerConnection: Request: POST /thruk/cgi-bin/remote.cgi (from [::ffff:172.18.0.3]:55274), user: root, agent: thruk, status: Bad Request).
[2023-04-20 09:52:57 +0000] information/HttpServerConnection: HTTP client disconnected (from [::ffff:172.18.0.3]:55274)
[2023-04-20 09:52:57 +0000] information/ApiListener: New client connection from [::ffff:172.18.0.3]:55276 (no client certificate)
[2023-04-20 09:52:57 +0000] information/HttpServerConnection: Request: POST /thruk/cgi-bin/remote.cgi (from [::ffff:172.18.0.3]:55276), user: root, agent: thruk, status: Bad Request).
[2023-04-20 09:52:57 +0000] information/HttpServerConnection: HTTP client disconnected (from [::ffff:172.18.0.3]:55276)
- On thruk IHM, when i tried to connect to icinga2 by livestatus type, there is my error on icinga2 server:
[2023-04-20 10:09:53 +0000] information/ApiListener: New client connection from [::ffff:172.18.0.3]:55434 (no client certificate)
[2023-04-20 10:09:53 +0000] information/HttpServerConnection: HTTP client disconnected (from [::ffff:172.18.0.3]:55434)
[2023-04-20 10:09:53 +0000] information/ApiListener: New client connection from [::ffff:172.18.0.3]:55436 (no client certificate)
[2023-04-20 10:09:53 +0000] information/HttpServerConnection: Request: POST /thruk/cgi-bin/remote.cgi (from [::ffff:172.18.0.3]:55436), user: root, agent: thruk, status: Bad Request).
[2023-04-20 10:09:53 +0000] information/HttpServerConnection: HTTP client disconnected (from [::ffff:172.18.0.3]:55436)
[2023-04-20 10:09:53 +0000] critical/ApiListener: Client TLS handshake failed (from [::ffff:172.18.0.3]:55438): http request
[2023-04-20 10:09:54 +0000] critical/ApiListener: Client TLS handshake failed (from [::ffff:172.18.0.3]:55440): http request
[2023-04-20 10:09:55 +0000] critical/ApiListener: Client TLS handshake failed (from [::ffff:172.18.0.3]:55444): http reques
Platform caracteristics
I am working on docker containers, i created two containers, One is icinga-master via icinga2 image and the other is Thruk via custom image builded. Both of them ran on the same bridge docker network.
Icinga configuration
-
Launching icinga containers:
docker network create icinga docker run -d --network icinga --name icinga-master -h icinga-master -p 9000:5665 -v icinga-master:/data -e ICINGA_MASTER=1 icinga/icinga2:latest -
icinga2 version :
icinga@icinga-master:/$ icinga2 --version icinga2 - The Icinga 2 network monitoring daemon (version: v2.13.7) Copyright (c) 2012-2023 Icinga GmbH (https://icinga.com/) License GPLv2+: GNU GPL version 2 or later <https://gnu.org/licenses/gpl2.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. System information: Platform: Debian GNU/Linux Platform version: 11 (bullseye) Kernel: Linux Kernel version: 3.10.0-1160.88.1.el7.x86_64 Architecture: x86_64 Build information: Compiler: GNU 10.2.1 Build host: buildkitsandbox OpenSSL version: OpenSSL 1.1.1n 15 Mar 2022 -
Operating System and version
icinga@icinga-master:/$ cat /etc/os-release PRETTY_NAME="Debian GNU/Linux 11 (bullseye)" NAME="Debian GNU/Linux" VERSION_ID="11" VERSION="11 (bullseye)" VERSION_CODENAME=bullseye ID=debian HOME_URL="https://www.debian.org/" SUPPORT_URL="https://www.debian.org/support" BUG_REPORT_URL="https://bugs.debian.org/" -
Icinga2 Container kernel
icinga@icinga-master:/$ uname -a Linux icinga-master 3.10.0-1160.88.1.el7.x86_64 #1 SMP Tue Mar 7 15:41:52 UTC 2023 x86_64 GNU/Linux -
Enabled features (
icinga2 feature list)icinga@icinga-master:/$ icinga2 feature list Disabled features: debuglog elasticsearch gelf graphite icingadb ido-mysql ido-pgsql influxdb influxdb2 mainlog opentsdb perfdata syslog Enabled features: api checker notification -
Config validation (icinga2 daemon -C)
icinga@icinga-master:/$ icinga2 daemon -C [2023-04-20 09:15:01 +0000] information/cli: Icinga application loader (version: v2.13.7) [2023-04-20 09:15:01 +0000] information/cli: Loading configuration file(s). [2023-04-20 09:15:01 +0000] information/ConfigItem: Committing config item(s). [2023-04-20 09:15:01 +0000] information/ApiListener: My API identity: icinga-master [2023-04-20 09:15:02 +0000] information/ConfigItem: Instantiated 1 Host. [2023-04-20 09:15:02 +0000] information/ConfigItem: Instantiated 2 NotificationCommands. [2023-04-20 09:15:02 +0000] information/ConfigItem: Instantiated 1 Downtime. [2023-04-20 09:15:02 +0000] information/ConfigItem: Instantiated 12 Notifications. [2023-04-20 09:15:02 +0000] information/ConfigItem: Instantiated 1 IcingaApplication. [2023-04-20 09:15:02 +0000] information/ConfigItem: Instantiated 2 HostGroups. [2023-04-20 09:15:02 +0000] information/ConfigItem: Instantiated 3 Zones. [2023-04-20 09:15:02 +0000] information/ConfigItem: Instantiated 1 Endpoint. [2023-04-20 09:15:02 +0000] information/ConfigItem: Instantiated 1 NotificationComponent. [2023-04-20 09:15:02 +0000] information/ConfigItem: Instantiated 1 CheckerComponent. [2023-04-20 09:15:02 +0000] information/ConfigItem: Instantiated 1 ApiUser. [2023-04-20 09:15:02 +0000] information/ConfigItem: Instantiated 1 ApiListener. [2023-04-20 09:15:02 +0000] information/ConfigItem: Instantiated 244 CheckCommands. [2023-04-20 09:15:02 +0000] information/ConfigItem: Instantiated 3 TimePeriods. [2023-04-20 09:15:02 +0000] information/ConfigItem: Instantiated 1 UserGroup. [2023-04-20 09:15:02 +0000] information/ConfigItem: Instantiated 1 User. [2023-04-20 09:15:02 +0000] information/ConfigItem: Instantiated 11 Services. [2023-04-20 09:15:02 +0000] information/ConfigItem: Instantiated 3 ServiceGroups. [2023-04-20 09:15:02 +0000] information/ConfigItem: Instantiated 1 ScheduledDowntime. [2023-04-20 09:15:02 +0000] information/ScriptGlobal: Dumping variables to file '/var/cache/icinga2/icinga2.vars' [2023-04-20 09:15:02 +0000] information/cli: Finished validating the configuration file(s).
Thruk Configuration
- Thruk Dockerfile:
FROM ubuntu:22.04 # https://download.thruk.org/pkg/v3.04/ubuntu22.04/amd64/ COPY packages /tmp/ COPY docker-entrypoint.sh /usr/local/bin/ RUN apt-get update -y RUN apt-get install apache2 libapache2-mod-fcgid -y RUN dpkg -i /tmp/libthruk_3.00_ubuntu22.04_amd64.deb \ /tmp/thruk-base_3.04-1_ubuntu22.04_amd64.deb \ /tmp/thruk_3.04-1_ubuntu22.04_amd64.deb \ /tmp/thruk-plugin-reporting_3.04-1_ubuntu22.04_amd64.deb || true RUN apt-get install -f -y EXPOSE 80 # ENTRYPOINT ["/etc/init.d/apache2", "start"] ENTRYPOINT ["/usr/local/bin/docker-entrypoint.sh"] - Launching Thruk containers:
docker run -dit --name thruk -h thruk --network icinga -p 80:80 --restart=always my-thruk-image - Thruk version
root@thruk:/# thruk --version thruk v3.04 - Copying icinga2 certificates to thruk containers
[icinga@devopsbox icinga2]$ docker cp icinga-master:/var/lib/icinga2/certs/ca.crt . Successfully copied 3.584kB to /home/icinga/icinga2/. [icinga@devopsbox icinga2]$ docker cp icinga-master:/var/lib/icinga2/certs/icinga-master.crt . Successfully copied 3.584kB to /home/icinga/icinga2/. [icinga@devopsbox icinga2]$ docker cp icinga-master:/var/lib/icinga2/certs/icinga-master.key . Successfully copied 5.12kB to /home/icinga/icinga2/. [icinga@devopsbox icinga2]$ docker cp ./ca.crt thruk:/tmp/ Successfully copied 3.584kB to thruk:/tmp/ [icinga@devopsbox icinga2]$ docker cp ./icinga-master.crt thruk:/tmp/ Successfully copied 3.584kB to thruk:/tmp/ [icinga@devopsbox icinga2]$ docker cp ./icinga-master.key thruk:/tmp/ Successfully copied 5.12kB to thruk:/tmp/ - Permission on icinga certificates on thruk containers (Read permissions seted)
root@thruk:/# ls /tmp/ -ltotal 29556 -rw-r--r--. 1 5665 5665 1720 Apr 20 09:09 ca.crt -rw-r--r--. 1 5665 5665 1757 Apr 20 09:09 icinga-master.crt -rw-r--r--. 1 5665 5665 3243 Apr 20 09:09 icinga-master.key - I added Icinga CA to trustore of thruk containers :
root@thruk:/# cp /tmp/ca.crt /usr/local/share/ca-certificates/ root@thruk:/# update-ca-certificates Updating certificates in /etc/ssl/certs... rehash: warning: skipping ca-certificates.crt,it does not contain exactly one certificate or CRL 1 added, 0 removed; done. Running hooks in /etc/ca-certificates/update.d... done. - I Enabled Icinga feature on Thruk config (/etc/thruk/thruk.conf)
root@thruk:/# grep enable_icinga_features /etc/thruk/thruk.conf #enable_icinga_features = 0 enable_icinga_features = 1 - I created an agent ticket for icinga API
root@thruk:/# curl -k -s -S -i -u root:828d95c660d274c3 -H 'Accept: application/json' \ -X POST 'https://icinga-master:5665/v1/actions/generate-ticket' \ -d '{ "cn": "icinga-agent", "pretty": true }' HTTP/1.1 200 OK Server: Icinga/v2.13.7 Content-Type: application/json Content-Length: 258 { "results": [ { "code": 200, "status": "Generated PKI ticket '963ad4231128b81e2f25a5e9a1fc0709d0ec45a5' for common name 'icinga-agent'.", "ticket": "963ad4231128b81e2f25a5e9a1fc0709d0ec45a5" } ] } - Here is my Thruk config (/etc/thruk/thruk_local.conf)
root@thruk:/# cat /etc/thruk/thruk_local.conf ssl_verify_hostnames = 0 <Component Thruk::Backend> <peer> name = icinga2-livestatus id = 36a9e type = livestatus <options> peer = tls://icinga-master:5665 auth = 963ad4231128b81e2f25a5e9a1fc0709d0ec45a5 </options> </peer> <peer> name = icinga-http type = http <options> peer = https://root:828d95c660d274c3@icinga-master:5665 # apiser credentials are: login: root, password: 828d95c660d274c3 auth = 963ad4231128b81e2f25a5e9a1fc0709d0ec45a5 # ticket generated ca_file = /tmp/ca.crt cert = /tmp/icinga-master.crt key = /tmp/icinga-master.key </options> </peer>