Icingaweb 2.8.2 - Very slow Service View when Limit > 200 (compared to 2.7.4)

Icinga Web 2
#1

After installing a new server with icingaweb 2.8.2 the Service Problems view ist not usable anymore, if there are a lot of errors. The Web-UI freezes so hard that is even impossible to click something on the left menu. To get the freeze it can be enough to scroll up and down fast enough (but maybe thats the autorefreh). Tested with Firefox and google.

  • A user with fewer Service Problem is mostly ok.
  • The user (admin) with the problems has around 400 problems (lot of unknowns)
  • Other Views like host, which has a loot more informations is completly fine.

Anyone has performance problems too?

#2

Don’t have those problems.
Just checked a system in development, where /icingaweb2/monitoring/list/services?service_problem=1&sort=service_severity&dir=desc has 21 pages with 25 services each (8 crit, mostly unknown or pending) and it works as fas and responsive as other pages.

Have you activated the benchmark function of the webinterface? Or tried with the browser console (F12)?
For comparison:

image
image1691×244 11 KB
image

1 Like
#3

If i limit the view to 25 services its okay. but not on 500 f.e. I even can’t work with the browser console anymore as it stucks.

2.8.2:

image

2.7.3:

image

EDIT: Just Found out, that i have the same Problem, when i go to Overview > Services and select limit 200. up to limit 100 everything is fine. It will not recover unitl reload after i go to limit 200

CPU Load of browser is going very high every 5 sec., even if i just have a dashboard open. Its the same on Firefox. With 2.7.4 its max 20%. Could that be a serverside (apache/php) misconfiguration?

grafik

This icinga.min.js on 2.7.4 is 537 Lines long. On 2.8.2 its 4651 (!)

#4

Some more informations:

This it whats takes so long:

grafik
grafik1254×867 25.1 KB

I even tried 2.8.1.

My PHP Version is: 7.4.9 on Ubuntu 20.04.1 LTS

I uploaded both icinga.min.js from 2.7.4 and 2.8.1 for comparison. The one of 2.74. is only 537 lines long, the other one is 4651 (!) Not sure if that has anything to do with it, as i have no skills in web things.

icinga.min.js-2.7.4.txt (351.2 KB) icinga.min.js-2.8.4.txt (584.3 KB)

#5

The size of the minified JS is not the problem here. But it’s that big because it seems your 2.8.x installation has some additional module which loads a huge dependency in. (CodeMirror)

The actual issue here is that your browser constantly re-renders the page. Looking at the call tree, it’s no wonder why. But it seems pretty exaggerated in your case.

Have you many checks that return percentual performance data? (e.g. Many rows in the service overview with little pie charts?)

The reason why this doesn’t occur with v2.7.4, is probably because we significantly changed the way with v2.8 how Icinga Web 2’s layout works.

#6

Hi,

there are not many pie charts in the view. I think not more then 5% of the services in the view are rendering pie charts. But some of them have more then 5 pie charts (if i open the service)

I now disabled all modules, but the monitoring modul, the problem is still there.

On my side, every service has an icon configured, thats the only thing which (maybe) is not standard that i can think of, but on the Host tab everything is workin fine, so it has clearly to do with the service view.

Is there a way to disable pi charts for testing?

#7

Just stumbled upon a github issue which reminded me of this thread.


Could this be the root cause here as well?
1 Like
#8

Hi,

as I’am not having the problem on 2.7.4 (I see some delay, but its not unusable as it is on 2.8.x) i’am not sure if its the same problem, but the Symtoms are the same as far as i can see it.

#9

Hi,
Looks like I have the same problem.
550 hosts, 12000+ services.

When I try to display more than 500 services on the same page from the WebUI (IcingaWeb 2.8.2), the browser (Chrome/Opera/Firefox) is loading to 100% CPU usage.
Don’t seem to have the issue on 2.7+.

As Rafael asked, I would liked to know if we can diseable the pi chart for testing ?
Or if there is anything else we can do ?

1 Like
#10

Disabling the pie charts is only possible by adjusting the code. I’ve attached a patch that does this.

Apply the patch with patch -p3 < no-sparkline-pies.txt at either /usr/share/php/Icinga (package installation) or library/Icinga (git clone) depending on how you’ve installed Icinga Web 2.
no-sparkline-pies.txt (868 Bytes)

Once done, any browser you test this with needs to perform a hard refresh. (e.g. Ctrl+F5)

If the issue is not solved, please show again what’s requiring too much time in the call tree.

1 Like
#11

Thanks for your support,

i need to patch two files to get rid of the pie charts:

Icinga/Web/JavaScript.php
and
Icinga/library/Icinga/Web/JavaScript.php

After that its a LOT better. Still its not as fast as 2.7, but its produktive ready this way:

  1. try:

image
image951×800 22.9 KB

  1. try

image
image930×426 10.5 KB

#12

At which limit? 200 again?

And, I wouldn’t call this production ready. It’s disabling the pie charts completely. Maybe try the PR I’ve just opened which changes this properly. (Still shows the pie charts, but renders them differently) https://github.com/Icinga/icingaweb2/pull/4235

#13

Its at Limit 500.

I just meant produktive ready for the performance, not for the missing charts :slight_smile:

#14

Well, with this large amount of data to render it will obviously take a bit more time. I think 1.5 seconds are also not that much in this case. It’s also native render time now, and not caused by Javascript anymore.

So I’d see this as fixed then, if the linked PR doesn’t introduce another performance issue. :smiley:

1 Like
#15

I copied the files manualy from the pull request, as iam using packaged version. Looks good!

1 Like
#16

I’ve tried both the patch and the PR. Both looks good.
CPU’s usage have reduced a lot. Browser not crashing anymore :slight_smile:

1 Like