Icinga reloads config slowly? Try jemalloc

See

for detailed instructions.

Please share your before/after time measurements here.

Hi @Al2Klimov,
here are my results in version 2.13.2 and 2.13.6:
Unfortunately somewhat sobering. Bernd Erk has mentioned in this year’s osmc that with the new version 2.13.6 the config reload has become about 30% faster.
Which is unfortunately not the case with me.

time icinga2 daemon -C
[2022-11-22 11:26:27 +0100] information/cli: Icinga application loader (version: r2.13.2-1)
real    0m13.148s
user    0m55.685s
sys     0m9.926s

time env LD_PRELOAD=/lib/x86_64-linux-gnu/libjemalloc.so.2 icinga2 daemon -C
[2022-11-22 11:28:23 +0100] information/cli: Icinga application loader (version: r2.13.2-1)
real    0m11.556s
user    0m49.510s
sys     0m9.848s

time icinga2 daemon -C
[2022-11-23 11:15:07 +0100] information/cli: Icinga application loader (version: r2.13.6-1)
real    0m13.642s
user    0m53.440s
sys     0m10.781s

time env LD_PRELOAD=/lib/x86_64-linux-gnu/libjemalloc.so.2 icinga2 daemon -C
[2022-11-23 11:15:25 +0100] information/cli: Icinga application loader (version: r2.13.6-1)
real    0m12.006s
user    0m49.974s
sys     0m10.586s

My Setup:

62824 Notifications
1 IcingaApplication
26740 Dependencies
69 HostGroups
1819 Hosts
1 FileLogger
21092 Downtimes
1 Influxdb2Writer
194 Comments
1 IdoMysqlConnection
1 CheckerComponent
333 Zones
333 Endpoints
5 ApiUsers
1 ApiListener
1 NotificationComponent
311 CheckCommands
22 UserGroups
19 ServiceGroups
11 TimePeriods
55 Users
19935 ScheduledDowntimes
29385 Services
4 NotificationCommands

Tried it on RHEL7.7 system with a quite outdated icinga2 (update incoming :wink: ):

Icinga application loader (version: 2.11.2-1)

 # icinga2 feature list
Disabled features: compatlog debuglog elasticsearch gelf graphite opentsdb perfdata statusdata syslog
Enabled features: api checker command ido-mysql influxdb livestatus mainlog notification

40509 ScheduledDowntimes.
30012 Dependencies.
23182 Notifications.
17390 Services.
2576 Hosts.
1053 Downtimes.
341 CheckCommands.
229 Users.
129 ServiceGroups.
88 UserGroups.
81 Comments.
42 HostGroups.
10 ApiUsers.
9 NotificationCommands.
6 TimePeriods.
4 Zones.
3 Endpoints.
1 NotificationComponent.
1 LivestatusListener.
1 InfluxdbWriter.
1 IdoMysqlConnection.
1 IcingaApplication.
1 FileLogger.
1 ExternalCommandListener.
1 CheckerComponent.
1 ApiListener.

without:

real    0m22.024s
user    4m25.635s
sys     2m47.647s

with jemalloc

# time env LD_PRELOAD=/lib64/libjemalloc.so.1 icinga2 daemon -C

real    0m19.754s
user    4m13.474s
sys     3m15.749s

seems to not make any difference in this case.
will do another test once the system is updated :slight_smile:

@berk has also clearly mentioned the non-reliability of benchmarks as such. Just to be clear (as he spoke based on my info and I’ve even made the performance improvement in the first place): The more apply rules like assign where host.name == "h" && service.name == "s" you have, the faster 2.13.6 is compared to 2.13.5.

Also your load time isn’t even 1/2 minute, why you’re reading the Icinga starts/restarts/reloads very slowly section? :smiley:

Yes I know, in his eyes its about 90% faster :smiley:
I have about 200 of your described assign where conditions. Yes our systemd reload is not that slow. But the wohle relad process is slow, see: Remote Icinga instance is not connectet to master - for a few minutes after reload
So I thought it should be at least a little bit faster and I’ll write it in here.

Instantiated 1 NotificationComponent.
Instantiated 1 LivestatusListener.
Instantiated 1 ExternalCommandListener.
Instantiated 1 CheckerComponent.
Instantiated 4 TimePeriods.
Instantiated 2 Users.
Instantiated 84430 Services.
Instantiated 75474 ScheduledDowntimes.
Instantiated 2 Zones.
Instantiated 4 NotificationCommands.
Instantiated 162678 Notifications.
Instantiated 2977 Hosts.
Instantiated 1 IcingaApplication.
Instantiated 18 HostGroups.
Instantiated 84430 Dependencies.
Instantiated 300 Comments.
Instantiated 10 EventCommands.
Instantiated 421799 Downtimes.
Instantiated 2 Endpoints.
Instantiated 1 FileLogger.
Instantiated 12 ApiUsers.
Instantiated 253 CheckCommands.
Instantiated 1 ApiListener.
Instantiated 1 IcingaDB.

Without:

real	4m7.549s
user	14m33.763s
sys	1m42.542s

With jemalloc:

real	2m15.408s
user	7m11.452s
sys	2m55.066s

Around 50% shaved off the startup time. Nice :smiley: