Icinga reloads config slowly? Try jemalloc


for detailed instructions.

Please share your before/after time measurements here.

Hi @Al2Klimov,
here are my results in version 2.13.2 and 2.13.6:
Unfortunately somewhat sobering. Bernd Erk has mentioned in this year’s osmc that with the new version 2.13.6 the config reload has become about 30% faster.
Which is unfortunately not the case with me.

time icinga2 daemon -C
[2022-11-22 11:26:27 +0100] information/cli: Icinga application loader (version: r2.13.2-1)
real    0m13.148s
user    0m55.685s
sys     0m9.926s

time env LD_PRELOAD=/lib/x86_64-linux-gnu/libjemalloc.so.2 icinga2 daemon -C
[2022-11-22 11:28:23 +0100] information/cli: Icinga application loader (version: r2.13.2-1)
real    0m11.556s
user    0m49.510s
sys     0m9.848s

time icinga2 daemon -C
[2022-11-23 11:15:07 +0100] information/cli: Icinga application loader (version: r2.13.6-1)
real    0m13.642s
user    0m53.440s
sys     0m10.781s

time env LD_PRELOAD=/lib/x86_64-linux-gnu/libjemalloc.so.2 icinga2 daemon -C
[2022-11-23 11:15:25 +0100] information/cli: Icinga application loader (version: r2.13.6-1)
real    0m12.006s
user    0m49.974s
sys     0m10.586s

My Setup:

62824 Notifications
1 IcingaApplication
26740 Dependencies
69 HostGroups
1819 Hosts
1 FileLogger
21092 Downtimes
1 Influxdb2Writer
1 IdoMysqlConnection
1 CheckerComponent
333 Zones
333 Endpoints
5 ApiUsers
1 ApiListener
1 NotificationComponent
311 CheckCommands
22 UserGroups
19 ServiceGroups
11 TimePeriods
55 Users
19935 ScheduledDowntimes
29385 Services
4 NotificationCommands

Tried it on RHEL7.7 system with a quite outdated icinga2 (update incoming :wink: ):

Icinga application loader (version: 2.11.2-1)

 # icinga2 feature list
Disabled features: compatlog debuglog elasticsearch gelf graphite opentsdb perfdata statusdata syslog
Enabled features: api checker command ido-mysql influxdb livestatus mainlog notification

40509 ScheduledDowntimes.
30012 Dependencies.
23182 Notifications.
17390 Services.
2576 Hosts.
1053 Downtimes.
341 CheckCommands.
229 Users.
129 ServiceGroups.
88 UserGroups.
42 HostGroups.
10 ApiUsers.
9 NotificationCommands.
6 TimePeriods.
4 Zones.
3 Endpoints.
1 NotificationComponent.
1 LivestatusListener.
1 InfluxdbWriter.
1 IdoMysqlConnection.
1 IcingaApplication.
1 FileLogger.
1 ExternalCommandListener.
1 CheckerComponent.
1 ApiListener.


real    0m22.024s
user    4m25.635s
sys     2m47.647s

with jemalloc

# time env LD_PRELOAD=/lib64/libjemalloc.so.1 icinga2 daemon -C

real    0m19.754s
user    4m13.474s
sys     3m15.749s

seems to not make any difference in this case.
will do another test once the system is updated :slight_smile:

@berk has also clearly mentioned the non-reliability of benchmarks as such. Just to be clear (as he spoke based on my info and I’ve even made the performance improvement in the first place): The more apply rules like assign where host.name == "h" && service.name == "s" you have, the faster 2.13.6 is compared to 2.13.5.

Also your load time isn’t even 1/2 minute, why you’re reading the Icinga starts/restarts/reloads very slowly section? :smiley:

Yes I know, in his eyes its about 90% faster :smiley:
I have about 200 of your described assign where conditions. Yes our systemd reload is not that slow. But the wohle relad process is slow, see: Remote Icinga instance is not connectet to master - for a few minutes after reload
So I thought it should be at least a little bit faster and I’ll write it in here.

Instantiated 1 NotificationComponent.
Instantiated 1 LivestatusListener.
Instantiated 1 ExternalCommandListener.
Instantiated 1 CheckerComponent.
Instantiated 4 TimePeriods.
Instantiated 2 Users.
Instantiated 84430 Services.
Instantiated 75474 ScheduledDowntimes.
Instantiated 2 Zones.
Instantiated 4 NotificationCommands.
Instantiated 162678 Notifications.
Instantiated 2977 Hosts.
Instantiated 1 IcingaApplication.
Instantiated 18 HostGroups.
Instantiated 84430 Dependencies.
Instantiated 300 Comments.
Instantiated 10 EventCommands.
Instantiated 421799 Downtimes.
Instantiated 2 Endpoints.
Instantiated 1 FileLogger.
Instantiated 12 ApiUsers.
Instantiated 253 CheckCommands.
Instantiated 1 ApiListener.
Instantiated 1 IcingaDB.


real	4m7.549s
user	14m33.763s
sys	1m42.542s

With jemalloc:

real	2m15.408s
user	7m11.452s
sys	2m55.066s

Around 50% shaved off the startup time. Nice :smiley: