Very high load after updating to icingadb 1.1.1

Hi at all,

since our last update of some modules and php to version 8.1 (SLES doesn’t provide 8.2 now), the load of our icinga2 master literally exploded. We updated following packages from repo:
*icingadb to 1.1.1
*icingadb-web to 1.1.1
*business process module to 2.5

We also updated the mysql schema for icingadb

The load per CPU increased from 0.2 to 2.5. Mainly affected on the mysqld, which is using round about 1700% of CPU usage. When disabling icingadb-web the load drops back to 0.2.
additionally we see errors like below on users who are in many persmission roles.

Deprecated: preg_replace(): Passing null to parameter #3 ($subject) of type array|string is deprecated in /usr/share/icinga-php/vendor/vendor/shardj/zf1-future/library/Zend/Db/Statement.php on line 212

  • Icinga DB Web version (System - About): 1.1.1
  • Icinga Web 2 version (System - About): 1.1.1
  • Web browser: Firefox
  • Icinga 2 version (icinga2 --version): 2.14.0
  • Icinga DB version (icingadb --version): 1.1.1
  • PHP version used (php --version): 8.1
  • Server operating system and version: SLES 15 SP5
  • icinga-php-library: 0.13.1
  • icinga-php-thirdparty on 0.12.1

Has anyone ideas to this?

what does the icingaweb2 application log say?
can you set it to debug first?

can you disable modules to check which one causes the deprecation message?

Hi nicolas,
the debuglog only comes up with some ldap messages “Issuing LDAP search” and LDAP query result does not provide the requested field “member:1.2.840.113556.1.4.1941:”, but nothing in context of icingadb, even not when enabling and disabling the icingadb-web module. In the messages syslog there are entries of icingadb itself for syncing elements, but these are independently of the icingadb-web module. The Icingadb-web is working when enabled, but is kicking the mysqld to its max. Also, when adding some CPUs to the vm, after 3 seconds mysqld is using them all and again causing a load of about 2,5 and remaining there. It would be interesting what icingadb-web is doing there in mysql.

I disabled all modules one after another, and only by disabling the monitoring module, the Deprecated message disapeared. Maybe I have to make the symptom more presice.
Admin Users who see everything, don’t get the message. Even users who are only in a a few roles don’t get it. But we have some Operator users (teamleaders for example) who are in about 120 roles. And they get the message. By kicking them out of the half of the roles (it doesn’t matter what roles), the message disappears. The counting level ist somewhere between 50 and 70 and floating between.

So why are they in so many roles? Because we have a lot of business processes and the consultants of those should only see the ones which they are responsible for. So we made one role for every Business Process, and only allowed objects they are allowed to see. The teamleaders although are allowed to see all objects of their team members, so they are in a lot of roles included.

Before updating php to 8.1 this wasn’t a problem at all. But now they thet shown this message, and nagstamon is freaking out on this and don’t show anything anymore.

are roles written by the gui or “by hand”/ansible…?
maybe some of the roles have no values in some fields of the monitoring module rather that “”.
But since you are already in the process to transition to icingadb here are some other ideas…

  • identify the issue in the monitoring module role setup and create a github ticket

or

  • stop using the monitoring module

or as a last and not fully recommended:

  • remove the possibility for icingaweb2 to change the errorlevel for now
    and set the global error reporting to error_reporting = E_ALL & ~E_STRICT
    https://webmasters.stackexchange.com/a/92838
    that should remove the deprecated message in php8

or

  • patch the 2 files webrouter.php ApplicationBootstrap.php with the error_reporting settings from above. do A filesystem search to locate the files, I think webrouter.php is in another directory.

usually we edit the roles by hand in /etc/icingaweb2/roles.ini to add new roles or to edit the filters. What do you mean with “fields”? there are no empty entries. If the filter is set, there are host and service definitions on which to filter. When adding a user to the one half of all roles, the message is gone, also when adding only the other half. If there would be one buggy role, i would have been able to isolate it. But it seems to be a problem of the amount of roles. I will create a github ticket on this one.

Well, if icingadb would be running without problems, the step to disable monitoring and only using icingadb would be our favorite one. But there is still the load problem, and out Destkop Notification Tool Nagstamon is still not fully compatible with icingadb.