IcingaDB cannot connect to Redis using TLS

Steps to recreate:

  1. Create a set of TLS certificates for Redis and IcingaDB, place them in ‘.ssl’
  2. Create Redis configuration
# /etc/redis.conf
bind 0.0.0.0
protected-mode no
port 0
loglevel notice
tls-port 6379
tls-cert-file /usr/local/etc/redis/ssl/redis.pem
tls-key-file /usr/local/etc/redis/ssl/redis-key.pem
tls-ca-cert-file /usr/local/etc/redis/ssl/allcacerts.pem
proc-title-template "{title} {listen-addr} {server-mode}"
  1. Create Redis service container
docker run --rm -d \
	-v ./ssl:/data/ssl/ \
	-v /etc/redis.conf:/usr/local/etc/redis/redis.conf \
	-p 6379:6379 \
	--name redis \
	-h redis04 \
	redis:alpine3.15 \
	redis-server /usr/local/etc/redis/redis.conf
  1. Create IcingaDB configuration
# /etc/icingadb/config.yml
 database:
   type: mysql
   host: mysql04
   port: 3306
   database: icingadb
   user: icinga
   password: T3stCred$
 redis:
   address: redis04:6379
   tls: true
   cert: /data/ssl/icinga.pem
   key: /data/ssl/icinga-key.pem
   ca: /data/ssl/allcacerts.pem
 logging:
  level: debug
  output: console
 history-retention:
   days: 2
   options:
  1. Following the Installation Instructions, get the repo (Ubuntu) key
wget -O /data/icingadb/icinga.key https://packages.icinga.com/icinga.key 
echo "deb https://packages.icinga.com/ubuntu icinga-focal-testing main" > /data/icingadb/focal-icinga-testing.list
  1. The IcingaDB Dockerfile
# /data/icingadb.Dockerfile
FROM ubuntu:focal AS focal-b
RUN apt update;\
        apt -y install apt-utils apt-transport-https wget gnupg;\
        apt clean
ADD focal-icinga-testing.list /etc/apt/sources.list.d/focal-icinga-testing.list
ADD icinga.key icinga.key
RUN apt-key add icinga.key;\
        apt update

FROM focal-b AS focal-icingadb
RUN mkdir /etc/icingadb
RUN chown -vR 101:101 /etc/icingadb
RUN apt install --no-install-recommends -y icingadb;\
        apt clean
CMD ["icingadb", "--config", "/etc/icingadb/config.yml"]
  1. Build the Dockerfile image
docker build /data/icingadb -f /data/icingadb/icingadb.Dockerfile -t focal-icingadb
  1. Run the IcingaDB container
docker run --rm \
	-v /data/icingadb/config.yml:/etc/icingadb/config.yml \
	-v .ssl:/data/ssl/ \
	focal-icingadb:latest \
	icingadb -c /etc/icingadb/config.yml

The container will start but then will exit with the following errors

2022-05-01T00:50:47.543Z        INFO    icingadb        Starting Icinga DB
2022-05-01T00:50:47.543Z        INFO    icingadb        Connecting to database
2022-05-01T00:50:47.554Z        INFO    icingadb        Connecting to Redis
2022-05-01T00:50:55.758Z        FATAL   icingadb        read tcp 172.19.0.4:36988->172.19.0.2:6379: read: connection reset by peer
can't connect to Redis
main.run
        /builds/packaging/deb-icingadb/build/icingadb/cmd/icingadb/main.go:83
main.main
        /builds/packaging/deb-icingadb/build/icingadb/cmd/icingadb/main.go:34
runtime.main
        /usr/local/go/src/runtime/proc.go:225
runtime.goexit
        /usr/local/go/src/runtime/asm_amd64.s:1371

According to the Redis service container, there is disagreement about how SSL is processed

docker logs -n 10 redis04

1:M 01 May 2022 00:50:53.695 # Error accepting a client connection: error:1408F10B:SSL routines:ssl3_get_record:wrong version number
1:M 01 May 2022 00:50:54.209 # Error accepting a client connection: error:1408F10B:SSL routines:ssl3_get_record:wrong version number
1:M 01 May 2022 00:50:54.725 # Error accepting a client connection: error:1408F10B:SSL routines:ssl3_get_record:wrong version number

And yet when confirmed (using the IcingaDB set of TLS certificates) Redis is properly configured and functioning

docker run --rm  \
	-v .ssl:/data/ssl/ \
	redis:alpine3.15 \
	redis-cli --tls \
		--cert /data/ssl/icinga.pem \
		--key /data/ssl/icinga-key.pem \
		--cacert /data/ssl/allcacerts.pem \
		-h redis04 SET last 04-29-2022_1552
OK
docker run --rm  \
	-v .ssl:/data/ssl/ \
	redis:alpine3.15 \
	redis-cli --tls \
		--cert /data/ssl/icinga.pem \
		--key /data/ssl/icinga-key.pem \
		--cacert /data/ssl/allcacerts.pem \
		-h redis04 GET last
04-29-2022_1552

The idea was to get around the current IcingaDB docker TLS issue, but even this method proven illusive. Short of using Redis without TLS for IcingaDB (not desirable, even for development) are there any other options?

You’re installing the RC2 of Icinga DB. This is still affected by this issue. Why don’t you extend the icinga/icingadb:master image?

Please forgive my lack of understanding to what this suggestion means. The icinga/icingadb:master image has not been updated since the docker version of IcingaDB with TLS enabled Redis was reported to not function as intended.

The problem you had in the other thread was about the environment variables not working as intended. Your attempt here is to get around this, so you’re installing icingadb by package in your own docker image.

What I’m suggesting now is, that you don’t install icingadb by package but that your own docker image extends icinga/icingadb:master. You can then override the entrypoint and command so that your own config is loaded, bypassing any environment variable mechanics.