Installing Icinga2 to a Raspberry PI crashes with SIGSEGV

Hello!

I run into a Segmentation Fault error using Icinga2/Icingaweb2 on a Raspberry PI running Raspbian (just FYI, the downloads page does not have the official Raspbian Repository listed).

I followed these installation steps:

  1. https: // icinga.com/docs/icinga2/latest/doc/02-getting-started/
  2. https: // icinga.com/docs/icingaweb2/latest/doc/02-Installation/
  3. Then I followed the instructions on the /icingaweb2/setup page and configured the application to use LDAP authentication.
  4. Confirmed that the Icinga2/Icingaweb2 is up and running by logging in to the web interface and checked the local host being monitored and all services were in OK state.

After the default configuration was up and running, I tried to reconfigure this host as a satellite.
I keep all the hostnames and other info in ENV variables, so the vars here are not just for posting the request here.

The commands executed:

  1. SIGN_REQUEST_TICKET=$( curl --insecure --quiet --user client-pki-ticket:$ICINGA_PKI_PASS --header ‘Accept: application/json’ --request POST “https://$ICINGA_MASTER_IP:5665/v1/actions/generate-ticket” --data “{ “cn”: “$ICINGA_SATELLITE_FQDN” }” | jq --raw-output ‘.results[0].ticket’ )
  2. icinga2 pki save-cert --host $ICINGA_MASTER_IP --port 5665 --trustedcert /var/lib/icinga2/certs/$ICINGA_MASTER_FQDN.crt
  3. icinga2 node setup --cn $ICINGA_SATELLITE_FQDN --zone $ICINGA_ZONE --endpoint $ICINGA_MASTER_FQDN --parent_zone $ICINGA_MASTER_ZONE --parent_host $ICINGA_MASTER_IP --accept-config --accept-commands --disable-confd --ticket $SIGN_REQUEST_TICKET --trustedcert /var/lib/icinga2/certs/$ICINGA_MASTER_FQDN.crt

The last step failed with the following output:

information/cli: Requesting certificate with ticket ‘SIGN_REQUEST_TICKET’.
information/cli: Verifying parent host connection information: host ‘ICINGA_MASTER_IP’, port ‘5665’.
information/cli: Using the following CN (defaults to FQDN): ‘ICINGA_SATELLITE_FQDN’.
information/cli: Created backup file ‘/var/lib/icinga2/certs//ICINGA_SATELLITE_FQDN.key.orig’.
information/cli: Created backup file ‘/var/lib/icinga2/certs//ICINGA_SATELLITE_FQDN.crt.orig’.
information/base: Writing private key to ‘/var/lib/icinga2/certs//ICINGA_SATELLITE_FQDN.key’.
information/base: Writing X509 certificate to ‘/var/lib/icinga2/certs//ICINGA_SATELLITE_FQDN.crt’.
information/cli: Verifying trusted certificate file ‘/var/lib/icinga2/certs/ICINGA_MASTER_FQDN.crt’.
information/cli: Requesting a signed certificate from the parent Icinga node.
information/cli: Writing CA certificate to file ‘/var/lib/icinga2/certs//ca.crt’.
information/cli: Writing signed certificate to file ‘/var/lib/icinga2/certs//ICINGA_SATELLITE_FQDN.crt’.
information/cli: Disabling the Notification feature.
Disabling feature notification. Make sure to restart Icinga 2 for these changes to take effect.
information/cli: Updating the ApiListener feature.
warning/cli: Feature ‘api’ already enabled.
information/cli: Created backup file ‘/etc/icinga2/features-available/api.conf.orig’.
information/cli: Generating zone and object configuration.
Segmentation fault

I have no experience at all debugging problems resulting a SIGSEGV, but here is the end of an strace log of the command:

2915 open("/etc/icinga2/features-available/api.conf.9A6oKQ", O_RDWR|O_CREAT|O_EXCL, 0600) = 37
2915 open("/etc/icinga2/features-available/api.conf.9A6oKQ", O_WRONLY|O_CREAT|O_TRUNC|O_LARGEFILE, 0666) = 38
2915 close(37) = 0
2915 chmod("/etc/icinga2/features-available/api.conf.9A6oKQ", 0644) = 0
2915 write(38, “/**\n * The API listener is used “…, 150) = 150
2915 close(38) = 0
2915 rename(”/etc/icinga2/features-available/api.conf.9A6oKQ”, “/etc/icinga2/features-available/api.conf”) = 0
2915 write(1, “\33[32minformation\33[0m/cli: Genera”…, 68) = 68
2915 — SIGSEGV {si_signo=SIGSEGV, si_code=SEGV_MAPERR, si_addr=NULL} —
2990 <… epoll_wait resumed> <unfinished …>) = ?
2989 <… epoll_wait resumed> <unfinished …>) = ?
2991 <… epoll_wait resumed> <unfinished …>) = ?
2988 <… epoll_wait resumed> <unfinished …>) = ?
2975 <… epoll_wait resumed> <unfinished …>) = ?
2960 <… epoll_wait resumed> <unfinished …>) = ?
2959 <… epoll_wait resumed> <unfinished …>) = ?
2958 <… epoll_wait resumed> <unfinished …>) = ?
2946 <… futex resumed>) = ?
2945 <… futex resumed>) = ?
2944 <… futex resumed>) = ?
2943 <… futex resumed>) = ?
2942 <… futex resumed>) = ?
2991 +++ killed by SIGSEGV +++
2990 +++ killed by SIGSEGV +++
2989 +++ killed by SIGSEGV +++
2988 +++ killed by SIGSEGV +++
2975 +++ killed by SIGSEGV +++
2960 +++ killed by SIGSEGV +++
2959 +++ killed by SIGSEGV +++
2958 +++ killed by SIGSEGV +++
2946 +++ killed by SIGSEGV +++
2945 +++ killed by SIGSEGV +++
2944 +++ killed by SIGSEGV +++
2943 +++ killed by SIGSEGV +++
2942 +++ killed by SIGSEGV +++
2953 <… recvmsg resumed> {msg_name=NULL, msg_namelen=0, msg_iov=[{iov_base="", iov_len=4}], msg_iovlen=1, msg_controllen=0, msg_flags=0}, 0) = 0
2915 +++ killed by SIGSEGV +++
2953 exit_group(0) = ?
2953 +++ exited with 0 +++

gdb output:

Thread 1 “icinga2” received signal SIGSEGV, Segmentation fault.
0x005b6de4 in ?? ()
(gdb) backtrace
#0 0x005b6de4 in ?? ()
#1 0x005b6f10 in ?? ()
#2 0x00193d6c in icinga::Dictionary::Dictionary(std::initializer_list<std::pair<icinga::String const, icinga::Value> >) ()
#3 0x003d7990 in icinga::NodeUtility::GenerateNodeIcingaConfig(icinga::String const&, icinga::String const&, icinga::String const&, std::vector<std::__cxx11::basic_string<char, std::char_traits, std::allocator >, std::allocator<std::__cxx11::basic_string<char, std::char_traits, std::allocator > > > const&, std::vector<icinga::String, std::allocatoricinga::String > const&) ()
#4 0x003de098 in icinga::NodeSetupCommand::SetupNode(boost::program_options::variables_map const&, std::vector<std::__cxx11::basic_string<char, std::char_traits, std::allocator >, std::allocator<std::__cxx11::basic_string<char, std::char_traits, std::allocator > > > const&) ()
#5 0x003e4c2c in icinga::NodeSetupCommand::Run(boost::program_options::variables_map const&, std::vector<std::__cxx11::basic_string<char, std::char_traits, std::allocator >, std::allocator<std::__cxx11::basic_string<char, std::char_traits, std::allocator > > > const&) const ()
#6 0x001d666c in ?? ()
#7 0x001787d4 in main ()
(gdb)

Any ideas ?
Peter

There is a known issue about icinga2 segfaulting on armhf platforms https://github.com/Icinga/icinga2/issues/6966

1 Like

Yep. Affects Debian Stretch, and is a gcc compiler regression on armhf. No working solutions yet, subscribe to the linked issue.

1 Like

We have a lot of raspberries running icinga2 on Debian Stretch. The errors seems to appear only in the initial setup (see the issue) and shouldn’t affect the running daemon. Try to run the command again, which runs without a flaw (in our setup).

Cheers Nicolai

Thank you all for the replies.
I checked out the issue and subscribed to it so I get notified when it gets solved.

Nicolai, unfortunately in my case, not all the config files are generated before crashing and it crashes all the time, so I have to create them manually.
I do that and continue to use them.

Cheers,
Peter

1 Like