Icinga process not listening on 5665

Icinga 2
#1

Hi

I have successfully setup Icinga server (Ubuntu 20.04.1) + windows client.
But when it comes to server based on Red Hat 7, i hit into troubles.

First, after ‘icinga2 node wizard’ for master, there were some 139 memory violation codes.
But turning off/on the API module finally got the service running.

However, compared to the Ubuntu installation, no Icinga process is listening on 5665 port.
How would one troubleshoot that please?

#2

The feature which is required to be enabled and configured is api, so if icinga2 starts successfully but does not listen on 5665 the feature is not enabled or configured to bind to another port. If it is enabled and configured icinga2 should throw an error during config validation or latest at start.

#3

Hello Dirk

This is true, previously i mis-informed that API flopping is helping, but this is not true, i forgot to restart service after.

So the current problem description may be summed up this way:

  • when API is enabled, service wont start, main process fails with code 139.
  • config validation shows no error or warnings
  • there are no errors and one warnings for Listener in /var/log/icinga2:
# egrep -r -i error\|warn /var/log/icinga2/*
/var/log/icinga2/icinga2.log:[2020-09-10 16:32:24 +0000] warning/ExternalCommandListener: This feature is DEPRECATED and will be removed in future releases. Check the roadmap at https://github.com/Icinga/icinga2/milestones
/var/log/icinga2/icinga2.log:[2020-09-10 16:33:27 +0000] warning/ExternalCommandListener: This feature is DEPRECATED and will be removed in future releases. Check the roadmap at https://github.com/Icinga/icinga2/milestones

Whole RedHat server has been restarted.
I reinstalled icinga2, together with dropping DBs, but httpd left untouched.
In that process erased /etc/icinga2 (but not /var/lib/icinga2)

#4

Returncode 139 is a Segfault, so probably Icinga dies before it can hand over to logging, but in this case it should write in /var/log/icinga2/crash and/or /var/log/icinga2/start.err (or something similar, writing this from my head :wink: ). You can also trying running it in the foreground with icinga2 daemon -x debug.

#5

You have another thread for the same problem, where you seem to have got it working:

What is not clear to me is what you are trying to achieve.
Should the RHEL7 machine be a master or should it be a satellite/agent connected to the Ubuntu master?
Please be more specific on that, because the two threads contradict themselves.

Another thing the comes to my mind when seeing RHEL: Is SELinux enabled? If yes, is the icinga2-selinux package installed?

#6

@dgoetz: Hmm, the /var/log/icinga2/crash subdir is empty

@log1c: I started this thread thinking the issue is different now to previous 139 code - which is no process listening on 5665. I just realized that after service restart i again get 139 code;
So yes, now see the thread is a double, sorry

As to Ubuntu:
I have RedHat 7 server and Windows clients here.
I mentioned that i did a separate test installation with Ubuntu 20.04 acting as a server - and it worked fine with Windows laptop as a client connected over WIFI.

#7

No worries, just trying to figure out how/where to help :wink:

So the RHEL7 machine should be a master as well.

What about SELinux?

#8

Both icinga2-selinux and icingaweb2-selinux are there

#9

Many thanks for your hint, i disabled SE Linux and Iciga process started with 5665 port enabled :slight_smile:

#10

icinga2-selinux should label the port, so can you show the output of semanage port -l | grep icinga. It should look like icinga2_port_t tcp 5665. Also this should not cause a segfault but this is something for the developers.

#11

Hello Dirk
The output of ‘semanage’ command is exactly what you pointed to:

# semanage port -l | grep icinga
icinga2_port_t                 tcp      5665

But what i can see now, after OS reboot SE Linux remained ‘enforcing’.
So possibly what helped, was not the SE disabling, but some SE reload?

   # sestatus
   SELinux status:                 enabled
   SELinuxfs mount:                /sys/fs/selinux
   SELinux root directory:         /etc/selinux
   Loaded policy name:             targeted
   Current mode:                   enforcing
   Mode from config file:          enforcing
   Policy MLS status:              enabled
   Policy deny_unknown status:     allowed
   Max kernel policy version:      31
#12

If it now runs fine with SELinux in enforcing mode, the port labeled, the module loaded and icinga2 running in icinga2_t, then I am ok with it. I am always looking for improvements of the SELinux policy, this why I was asking.

#13

Hi

I again got 139 codes, so decided to:

  • reinstall icinga and icinga web package sets (with prior deleting /etc/icinga2 and/var/lib/icinga2 dirs),
  • drop and recreate both DBs.

This time i decided to turn off SE Linux on the server and not install icinga/web selinux packages.

And again, whenever trying to turn on API feature, i run into 139 code, no single trace of problems in logs…