Hi all,
Currently we have an Icinga setup which collects monitoring data from several upstream systems. Those systems all add hosts/ config using API calls, and afterwards trigger a deploy command. Since they are separate entities, there are situations when changes/ additions render 2 deploy actions to be received concurrently (less than 1 min apart). In those cases, we observed the IDO DB data gets in an inconsistent state, i.e. not all added hosts appear in a given hostgroup, even though config is correct).
At the moment we have disabled API deploy permissions and set a deploy job to run on a schedule, but looking for options of handling parallel API calls triggering deploy commands, so that the config is not rendered inconsistent. Is there any option to handle concurrent deploy calls and wait for a running deploy to complete before starting work on another one?
In the current Deploy log, we had about 5-6 failed (?) deploys per day due to this before switching to the deploy job.
Some system details below. Thanks for the help!
- Director version (System - About): 1.8.1
- Icinga Web 2 version and modules (System - About): 2.9.3
- Icinga 2 version (
icinga2 --version): 2.13.0-1 - Operating System and version: Ubuntu 20.04
- Webserver, PHP versions: