Any way to reduce the number of threads in icinga?

I’ve tried looking around a bunch but it seems that this value is tied to the number of cores/threads the host running icinga has. Ideally, we would like to reduce icinga to just run off of 1 thread if possible.

It seems that this used to be tied to a value in init.conf but was removed. So I’m assuming it would be something that would go in /etc/sysconfig/icinga2 now.

Example value I’d like to reduce
/# icinga2 console -e Concurrency
18

Thank you for help :slight_smile:

The threads can not be influenced by now which can cause problems with a high cpu count. But is this what you really want, Icinga 2 will not run with just one thread as it at least consists of 3 processes and has to run plugins to be useful. I think you want to run only one check, so this could be done with constant MaxConcurrentChecks, but I have not experimented with a value of 1.

1 Like

Hmm I don’t want to limit the amount of checks though, its more like, i just don’t want icinga spawning its initial threads based on cpu counts which seems to be the case currently. Threads from checks I would expect, but a box with no checks currently will spawn a value of threads which correlates to its cpu/cputhread count

Out of interest, what problem does it cause you if icinga spawns a number of
threads which are not actually performing checks?

Antony.

1 Like

Its a concern of our users that icinga is spawning so many threads

It may be better to explain the “why” as opposed to lower the amount of threads.

1 Like

Lower latency environment, and it causes concerns that a process without anything running may have initially 18 or more threads. And if with more once checks run

Is the number of 18 threads based on something specific? I’d be curious how many threads you have running now, have you checked?

I know we talked about this some days or weeks ago internally as a customer has upgraded a system to such a high number of cpus that he ran into the limit for threads and Icinga 2 failed to start without a meaningful error message. But I do not see an issue created for it, so I would say open an issue over at github to bring this back to developers attention and to express your concerns.

There already exists one: https://github.com/Icinga/icinga2/issues/9099

1 Like

I was searching for tasks which I would have expected in the title, and I forget about the issue number! So thank you!

So initial threads are based off of core count on the server. That 18 is based on a 18 core server. If you run that same check on something smaller, it will show less.

FYI - I opened this issue in gitlab incase the conversation moves that way

Sure, I’m just not sure what problem you are having. The value returned from icinga2 console -e Concurrency does not mean that is how many threads are currently running. e.g. is your server having performance issues you believe to be caused by the number of threads started by Icinga2?

On the master:

$ lscpu
CPU(s):                2

$ for p in `pgrep icinga2`; do ps h -T -p $p; done | wc -l
25

$ cat /proc/loadavg
0.50 0.64 0.87 5/497 1331

Server A:

$ lscpu 
CPU(s):              8

$ for p in `pgrep icinga2`; do ps h -T -p $p; done | wc -l
58

$ cat /proc/loadavg 
5.30 5.08 5.25 7/348 61287

I dont really care about the number of threads currently running. It’s the number of threads running at startup that bothers me. I can accept an extra thread for a check running, but when a 128 core server, causes 128 threads to start at startup with no checks running, then thats an issue.

just to follow up on this thread now that version 2.14 has been released in case anyone else is looking for this information. The fix moving forward with version 2.14 is to add the option -DConcurrency=1 to the ExecStart start up command (after daemon) in the systemd startup file

ExecStart=/usr/sbin/icinga2 daemon -DConcurrency=1 --close-stdio -e ${ICINGA2_ERROR_LOG}