Hi, I am using the REST API to schedule host downtimes with “all_services=true” and a filter based on a custom host variable (e.g.: host.vars[“SYSNR”] == “123456”).
In many cases this single API call takes several minutes to complete, even though the filter matches for only one or two hosts with a few services.
In the icinga2.log you can see that every single service downtime creation takes 20-30 seconds in this example:
[2021-04-20 11:41:39 +0000] information/ConfigObjectUtility: Created and activated object 'HOSTNAME01!d0ddabde-1fa9-422e-9d3f-621fa1d71a2e' of type 'Downtime'.
[2021-04-20 11:41:39 +0000] information/Downtime: Added downtime 'HOSTNAME01!d0ddabde-1fa9-422e-9d3f-621fa1d71a2e' between '2021-04-14 05:28:01' and '2099-12-31 23:59:59', author: 'Downtime API', fixed
[2021-04-20 11:42:15 +0000] information/ConfigObjectUtility: Created and activated object 'HOSTNAME01!SERVICE01!cc709056-f2c5-40a2-ab22-57b86b0c161f' of type 'Downtime'.
[2021-04-20 11:42:15 +0000] information/Downtime: Added downtime 'HOSTNAME01!SERVICE01!cc709056-f2c5-40a2-ab22-57b86b0c161f' between '2021-04-14 05:28:01' and '2099-12-31 23:59:59', author: 'Downtime API', fixed
[2021-04-20 11:42:48 +0000] information/ConfigObjectUtility: Created and activated object 'HOSTNAME01!SERVICE02!b83e7d1e-034e-4f25-9285-10c2d809eadb' of type 'Downtime'.
[2021-04-20 11:42:48 +0000] information/Downtime: Added downtime 'HOSTNAME01!SERVICE02!b83e7d1e-034e-4f25-9285-10c2d809eadb' between '2021-04-14 05:28:01' and '2099-12-31 23:59:59', author: 'Downtime API', fixed
[2021-04-20 11:43:19 +0000] information/ConfigObjectUtility: Created and activated object 'HOSTNAME01!SERVICE03!82466a0f-ea63-4cbd-9cbe-211ba0405240' of type 'Downtime'.
[2021-04-20 11:43:19 +0000] information/Downtime: Added downtime 'HOSTNAME01!SERVICE03!82466a0f-ea63-4cbd-9cbe-211ba0405240' between '2021-04-14 05:28:01' and '2099-12-31 23:59:59', author: 'Downtime API', fixed
[2021-04-20 11:43:56 +0000] information/ConfigObjectUtility: Created and activated object 'HOSTNAME01!SERVICE04!06cd4b4f-3f22-41f8-b3e1-30478bb28314' of type 'Downtime'.
[2021-04-20 11:43:56 +0000] information/Downtime: Added downtime 'HOSTNAME01!SERVICE04!06cd4b4f-3f22-41f8-b3e1-30478bb28314' between '2021-04-14 05:28:01' and '2099-12-31 23:59:59', author: 'Downtime API', fixed
[2021-04-20 11:44:24 +0000] information/ConfigObjectUtility: Created and activated object 'HOSTNAME01!SERVICE05!dd077adc-3ea4-4f25-88f2-f32c3f3ab630' of type 'Downtime'.
[2021-04-20 11:44:24 +0000] information/Downtime: Added downtime 'HOSTNAME01!SERVICE05!dd077adc-3ea4-4f25-88f2-f32c3f3ab630' between '2021-04-14 05:28:01' and '2099-12-31 23:59:59', author: 'Downtime API', fixed
[2021-04-20 11:44:57 +0000] information/ConfigObjectUtility: Created and activated object 'HOSTNAME01!SERVICE06!7c406b72-2c8b-4802-9c57-a8e15870a3aa' of type 'Downtime'.
[2021-04-20 11:44:57 +0000] information/Downtime: Added downtime 'HOSTNAME01!SERVICE06!7c406b72-2c8b-4802-9c57-a8e15870a3aa' between '2021-04-14 05:28:01' and '2099-12-31 23:59:59', author: 'Downtime API', fixed
[2021-04-20 11:45:29 +0000] information/ConfigObjectUtility: Created and activated object 'HOSTNAME01!SERVICE07!21294d6b-9c04-4b09-a68c-46a0cee88a98' of type 'Downtime'.
[2021-04-20 11:45:29 +0000] information/Downtime: Added downtime 'HOSTNAME01!SERVICE07!21294d6b-9c04-4b09-a68c-46a0cee88a98' between '2021-04-14 05:28:01' and '2099-12-31 23:59:59', author: 'Downtime API', fixed
[2021-04-20 11:45:58 +0000] information/ConfigObjectUtility: Created and activated object 'HOSTNAME01!SERVICE08!1443a2be-2784-4ba4-be8c-0d74aafd81d1' of type 'Downtime'.
[2021-04-20 11:45:58 +0000] information/Downtime: Added downtime 'HOSTNAME01!SERVICE08!1443a2be-2784-4ba4-be8c-0d74aafd81d1' between '2021-04-14 05:28:01' and '2099-12-31 23:59:59', author: 'Downtime API', fixed
[2021-04-20 11:46:35 +0000] information/ConfigObjectUtility: Created and activated object 'HOSTNAME01!SERVICE09!ac09817a-3633-407b-a213-aa5f54d96f53' of type 'Downtime'.
[2021-04-20 11:46:35 +0000] information/Downtime: Added downtime 'HOSTNAME01!SERVICE09!ac09817a-3633-407b-a213-aa5f54d96f53' between '2021-04-14 05:28:01' and '2099-12-31 23:59:59', author: 'Downtime API', fixed
[2021-04-20 11:47:15 +0000] information/ConfigObjectUtility: Created and activated object 'HOSTNAME01!SERVICE10!ba0f5398-d6a7-495d-b747-0da0552d5d24' of type 'Downtime'.
[2021-04-20 11:47:15 +0000] information/Downtime: Added downtime 'HOSTNAME01!SERVICE10!ba0f5398-d6a7-495d-b747-0da0552d5d24' between '2021-04-14 05:28:01' and '2099-12-31 23:59:59', author: 'Downtime API', fixed
[2021-04-20 11:47:52 +0000] information/ConfigObjectUtility: Created and activated object 'HOSTNAME01!SERVICE11!26284542-45f2-4acb-bb7a-c2336d1e2751' of type 'Downtime'.
[2021-04-20 11:47:52 +0000] information/Downtime: Added downtime 'HOSTNAME01!SERVICE11!26284542-45f2-4acb-bb7a-c2336d1e2751' between '2021-04-14 05:28:01' and '2099-12-31 23:59:59', author: 'Downtime API', fixed
[2021-04-20 11:48:23 +0000] information/ConfigObjectUtility: Created and activated object 'HOSTNAME01!SERVICE12!428cf5e3-f935-431a-a1b1-365ac4988ff8' of type 'Downtime'.
[2021-04-20 11:48:23 +0000] information/Downtime: Added downtime 'HOSTNAME01!SERVICE12!428cf5e3-f935-431a-a1b1-365ac4988ff8' between '2021-04-14 05:28:01' and '2099-12-31 23:59:59', author: 'Downtime API', fixed
[2021-04-20 11:49:03 +0000] information/ConfigObjectUtility: Created and activated object 'HOSTNAME01!SERVICE13!8b85a400-c167-4517-a5c6-2ba403d063ff' of type 'Downtime'.
[2021-04-20 11:49:03 +0000] information/Downtime: Added downtime 'HOSTNAME01!SERVICE13!8b85a400-c167-4517-a5c6-2ba403d063ff' between '2021-04-14 05:28:01' and '2099-12-31 23:59:59', author: 'Downtime API', fixed
[2021-04-20 11:49:41 +0000] information/ConfigObjectUtility: Created and activated object 'HOSTNAME01!SERVICE14!f8cb7c97-fc6d-4377-a55b-a10e136dae50' of type 'Downtime'.
[2021-04-20 11:49:41 +0000] information/Downtime: Added downtime 'HOSTNAME01!SERVICE14!f8cb7c97-fc6d-4377-a55b-a10e136dae50' between '2021-04-14 05:28:01' and '2099-12-31 23:59:59', author: 'Downtime API', fixed
[2021-04-20 11:50:16 +0000] information/ConfigObjectUtility: Created and activated object 'HOSTNAME01!SERVICE15!3dfbf2e2-40fc-4c07-9ab6-ef0b1d87edd9' of type 'Downtime'.
[2021-04-20 11:50:16 +0000] information/Downtime: Added downtime 'HOSTNAME01!SERVICE15!3dfbf2e2-40fc-4c07-9ab6-ef0b1d87edd9' between '2021-04-14 05:28:01' and '2099-12-31 23:59:59', author: 'Downtime API', fixed
[2021-04-20 11:50:47 +0000] information/ConfigObjectUtility: Created and activated object 'HOSTNAME01!SERVICE16!fa4badbe-1a9b-44f0-8beb-339974fd8647' of type 'Downtime'.
[2021-04-20 11:50:47 +0000] information/Downtime: Added downtime 'HOSTNAME01!SERVICE16!fa4badbe-1a9b-44f0-8beb-339974fd8647' between '2021-04-14 05:28:01' and '2099-12-31 23:59:59', author: 'Downtime API', fixed
[2021-04-20 11:51:28 +0000] information/ConfigObjectUtility: Created and activated object 'HOSTNAME01!SERVICE17!f1366a3d-ca04-4380-81ef-c7ffc5a6c6d1' of type 'Downtime'.
[2021-04-20 11:51:28 +0000] information/Downtime: Added downtime 'HOSTNAME01!SERVICE17!f1366a3d-ca04-4380-81ef-c7ffc5a6c6d1' between '2021-04-14 05:28:01' and '2099-12-31 23:59:59', author: 'Downtime API', fixed
[2021-04-20 11:51:52 +0000] information/ConfigObjectUtility: Created and activated object 'HOSTNAME01!8901078d-4118-41af-b21a-734ca9c3bcc0' of type 'Downtime'.
[2021-04-20 11:51:52 +0000] information/Downtime: Added downtime 'HOSTNAME01!8901078d-4118-41af-b21a-734ca9c3bcc0' between '2021-04-14 05:28:01' and '2099-12-31 23:59:59', author: 'Downtime API', fixed
The Icinga version is r2.11.8-1, but I also experienced similar slow processing in earlier versions when “all_services” was not yet available and I had to trigger host and service downtimes separately.
What is going on internally in Icinga that makes this API call so slow?
What can be done to speed this up?
Currently this makes it unfeasible to use this Downtime API for our environment with approx. 100k Host objects and 1M services.
Update:
I just noticed that the server is busy deleting expired downtimes over and over again:
[2021-04-20 13:01:56 +0000] information/ConfigObjectUtility: Deleted object 'HOSTNAME02!SERVICE42!976acd34-21a7-416a-aa63-736637497655' of type 'Downtime'.
[2021-04-20 13:01:56 +0000] information/Downtime: Removed downtime 'HOSTNAME02!SERVICE42!976acd34-21a7-416a-aa63-736637497655' from checkable 'HOSTNAME02!SERVICE42' (Reason: expired at 2021-04-20 13:00:00 +0000).
[2021-04-20 13:02:01 +0000] information/ConfigObjectUtility: Deleted object 'HOSTNAME02!SERVICE42!ff6a5e2e-5f45-4dde-8bd0-25483f0a1b57' of type 'Downtime'.
[2021-04-20 13:02:01 +0000] information/Downtime: Removed downtime 'HOSTNAME02!SERVICE42!ff6a5e2e-5f45-4dde-8bd0-25483f0a1b57' from checkable 'HOSTNAME02!SERVICE42' (Reason: expired at 2021-04-20 13:00:00 +0000).
[2021-04-20 13:04:05 +0000] information/ConfigObjectUtility: Deleted object 'HOSTNAME02!SERVICE42!d0db6400-4baf-4a2c-af20-080130e5c320' of type 'Downtime'.
[2021-04-20 13:04:05 +0000] information/Downtime: Removed downtime 'HOSTNAME02!SERVICE42!d0db6400-4baf-4a2c-af20-080130e5c320' from checkable 'HOSTNAME02!SERVICE42' (Reason: expired at 2021-04-20 13:00:00 +0000).
[2021-04-20 13:05:13 +0000] information/ConfigObjectUtility: Deleted object 'HOSTNAME02!SERVICE42!10b2302b-7a09-43a0-b050-b52d395aaf73' of type 'Downtime'.
[2021-04-20 13:05:13 +0000] information/Downtime: Removed downtime 'HOSTNAME02!SERVICE42!10b2302b-7a09-43a0-b050-b52d395aaf73' from checkable 'HOSTNAME02!SERVICE42' (Reason: expired at 2021-04-20 13:00:00 +0000).
[2021-04-20 13:05:30 +0000] information/ConfigObjectUtility: Deleted object 'HOSTNAME02!SERVICE42!99922de1-45d0-49ae-a6f4-dcff6dd7f27b' of type 'Downtime'.
[2021-04-20 13:05:30 +0000] information/Downtime: Removed downtime 'HOSTNAME02!SERVICE42!99922de1-45d0-49ae-a6f4-dcff6dd7f27b' from checkable 'HOSTNAME02!SERVICE42' (Reason: expired at 2021-04-20 13:00:00 +0000).
[2021-04-20 13:05:42 +0000] information/ConfigObjectUtility: Deleted object 'HOSTNAME02!SERVICE42!35408290-68cf-4b61-a269-b3f4e8e09736' of type 'Downtime'.
[2021-04-20 13:05:42 +0000] information/Downtime: Removed downtime 'HOSTNAME02!SERVICE42!35408290-68cf-4b61-a269-b3f4e8e09736' from checkable 'HOSTNAME02!SERVICE42' (Reason: expired at 2021-04-20 13:00:00 +0000).
[2021-04-20 13:05:43 +0000] information/ConfigObjectUtility: Deleted object 'HOSTNAME02!SERVICE42!b88658c6-8347-44ad-905a-406e2792401c' of type 'Downtime'.
[2021-04-20 13:05:43 +0000] information/Downtime: Removed downtime 'HOSTNAME02!SERVICE42!b88658c6-8347-44ad-905a-406e2792401c' from checkable 'HOSTNAME02!SERVICE42' (Reason: expired at 2021-04-20 13:00:00 +0000).
[2021-04-20 13:06:29 +0000] information/ConfigObjectUtility: Deleted object 'HOSTNAME02!SERVICE42!d4401541-a403-49ae-beb1-b050a6b3e9cf' of type 'Downtime'.
[2021-04-20 13:06:29 +0000] information/Downtime: Removed downtime 'HOSTNAME02!SERVICE42!d4401541-a403-49ae-beb1-b050a6b3e9cf' from checkable 'HOSTNAME02!SERVICE42' (Reason: expired at 2021-04-20 13:00:00 +0000).
[2021-04-20 13:06:38 +0000] information/ConfigObjectUtility: Deleted object 'HOSTNAME02!SERVICE42!db0edd2a-f10c-4b74-a0f8-f2283967e485' of type 'Downtime'.
[2021-04-20 13:06:38 +0000] information/Downtime: Removed downtime 'HOSTNAME02!SERVICE42!db0edd2a-f10c-4b74-a0f8-f2283967e485' from checkable 'HOSTNAME02!SERVICE42' (Reason: expired at 2021-04-20 13:00:00 +0000).
[2021-04-20 13:06:56 +0000] information/ConfigObjectUtility: Deleted object 'HOSTNAME02!SERVICE42!6e70bdac-8e1a-436f-b0a6-f0470c972df6' of type 'Downtime'.
[2021-04-20 13:06:56 +0000] information/Downtime: Removed downtime 'HOSTNAME02!SERVICE42!6e70bdac-8e1a-436f-b0a6-f0470c972df6' from checkable 'HOSTNAME02!SERVICE42' (Reason: expired at 2021-04-20 13:00:00 +0000).
[2021-04-20 13:07:06 +0000] information/ConfigObjectUtility: Deleted object 'HOSTNAME02!SERVICE42!c55f3244-d648-4b9b-aa29-a4683e6f887e' of type 'Downtime'.
[2021-04-20 13:07:06 +0000] information/Downtime: Removed downtime 'HOSTNAME02!SERVICE42!c55f3244-d648-4b9b-aa29-a4683e6f887e' from checkable 'HOSTNAME02!SERVICE42' (Reason: expired at 2021-04-20 13:00:00 +0000).
[2021-04-20 13:07:18 +0000] information/ConfigObjectUtility: Deleted object 'HOSTNAME02!SERVICE42!579f52a8-ab9f-489d-b627-dbe3b220dbc3' of type 'Downtime'.
[2021-04-20 13:07:18 +0000] information/Downtime: Removed downtime 'HOSTNAME02!SERVICE42!579f52a8-ab9f-489d-b627-dbe3b220dbc3' from checkable 'HOSTNAME02!SERVICE42' (Reason: expired at 2021-04-20 13:00:00 +0000).
[2021-04-20 13:07:44 +0000] information/ConfigObjectUtility: Deleted object 'HOSTNAME02!SERVICE42!c3aa8b51-401a-4b08-9a34-2d2e6dbc4d30' of type 'Downtime'.
[2021-04-20 13:07:44 +0000] information/Downtime: Removed downtime 'HOSTNAME02!SERVICE42!c3aa8b51-401a-4b08-9a34-2d2e6dbc4d30' from checkable 'HOSTNAME02!SERVICE42' (Reason: expired at 2021-04-20 13:00:00 +0000).
[2021-04-20 13:08:36 +0000] information/ConfigObjectUtility: Deleted object 'HOSTNAME02!SERVICE42!f5372061-24f8-46b8-8ad6-f77a00d68cfa' of type 'Downtime'.
[2021-04-20 13:08:36 +0000] information/Downtime: Removed downtime 'HOSTNAME02!SERVICE42!f5372061-24f8-46b8-8ad6-f77a00d68cfa' from checkable 'HOSTNAME02!SERVICE42' (Reason: expired at 2021-04-20 13:00:00 +0000).
[2021-04-20 13:08:45 +0000] information/ConfigObjectUtility: Deleted object 'HOSTNAME02!SERVICE42!00877771-3336-4dc6-8b90-9147485299cb' of type 'Downtime'.
[2021-04-20 13:08:45 +0000] information/Downtime: Removed downtime 'HOSTNAME02!SERVICE42!00877771-3336-4dc6-8b90-9147485299cb' from checkable 'HOSTNAME02!SERVICE42' (Reason: expired at 2021-04-20 13:00:00 +0000).
[2021-04-20 13:09:02 +0000] information/ConfigObjectUtility: Deleted object 'HOSTNAME02!SERVICE42!9d43e887-0111-4d3a-8393-2c37a0008235' of type 'Downtime'.
[2021-04-20 13:09:02 +0000] information/Downtime: Removed downtime 'HOSTNAME02!SERVICE42!9d43e887-0111-4d3a-8393-2c37a0008235' from checkable 'HOSTNAME02!SERVICE42' (Reason: expired at 2021-04-20 13:00:00 +0000).
[2021-04-20 13:09:35 +0000] information/ConfigObjectUtility: Deleted object 'HOSTNAME02!SERVICE42!aff9a7cc-7994-47a3-a32a-6143dfc7295b' of type 'Downtime'.
[2021-04-20 13:09:35 +0000] information/Downtime: Removed downtime 'HOSTNAME02!SERVICE42!aff9a7cc-7994-47a3-a32a-6143dfc7295b' from checkable 'HOSTNAME02!SERVICE42' (Reason: expired at 2021-04-20 13:00:00 +0000).
[2021-04-20 13:11:03 +0000] information/ConfigObjectUtility: Deleted object 'HOSTNAME02!SERVICE42!3aad0265-a9ef-49b3-a2f1-6c35de6d72e1' of type 'Downtime'.
[2021-04-20 13:11:03 +0000] information/Downtime: Removed downtime 'HOSTNAME02!SERVICE42!3aad0265-a9ef-49b3-a2f1-6c35de6d72e1' from checkable 'HOSTNAME02!SERVICE42' (Reason: expired at 2021-04-20 13:00:00 +0000).
[2021-04-20 13:11:44 +0000] information/ConfigObjectUtility: Deleted object 'HOSTNAME02!SERVICE42!6615f005-87e2-49cb-8a09-a233895ea4b0' of type 'Downtime'.
[2021-04-20 13:11:44 +0000] information/Downtime: Removed downtime 'HOSTNAME02!SERVICE42!6615f005-87e2-49cb-8a09-a233895ea4b0' from checkable 'HOSTNAME02!SERVICE42' (Reason: expired at 2021-04-20 13:00:00 +0000).
[2021-04-20 13:12:31 +0000] information/ConfigObjectUtility: Deleted object 'HOSTNAME02!SERVICE42!2cb019a6-54c5-454d-ad8e-62b80170e700' of type 'Downtime'.
[2021-04-20 13:12:31 +0000] information/Downtime: Removed downtime 'HOSTNAME02!SERVICE42!2cb019a6-54c5-454d-ad8e-62b80170e700' from checkable 'HOSTNAME02!SERVICE42' (Reason: expired at 2021-04-20 13:00:00 +0000).
[2021-04-20 13:12:49 +0000] information/ConfigObjectUtility: Deleted object 'HOSTNAME02!SERVICE42!4a1ef5cb-b4ac-42a3-8ff8-9639b07f87db' of type 'Downtime'.
[2021-04-20 13:12:49 +0000] information/Downtime: Removed downtime 'HOSTNAME02!SERVICE42!4a1ef5cb-b4ac-42a3-8ff8-9639b07f87db' from checkable 'HOSTNAME02!SERVICE42' (Reason: expired at 2021-04-20 13:00:00 +0000).
[2021-04-20 13:13:32 +0000] information/ConfigObjectUtility: Deleted object 'HOSTNAME02!SERVICE42!fb890ec7-4653-4ef5-bc07-05100dd05609' of type 'Downtime'.
[2021-04-20 13:13:32 +0000] information/Downtime: Removed downtime 'HOSTNAME02!SERVICE42!fb890ec7-4653-4ef5-bc07-05100dd05609' from checkable 'HOSTNAME02!SERVICE42' (Reason: expired at 2021-04-20 13:00:00 +0000).
[2021-04-20 13:14:18 +0000] information/ConfigObjectUtility: Deleted object 'HOSTNAME02!SERVICE42!a1783e17-8323-441a-bd47-b6aa8e27f3d7' of type 'Downtime'.
[2021-04-20 13:14:18 +0000] information/Downtime: Removed downtime 'HOSTNAME02!SERVICE42!a1783e17-8323-441a-bd47-b6aa8e27f3d7' from checkable 'HOSTNAME02!SERVICE42' (Reason: expired at 2021-04-20 13:00:00 +0000).
[2021-04-20 13:14:48 +0000] information/ConfigObjectUtility: Deleted object 'HOSTNAME02!SERVICE42!cd51827e-f780-412d-add8-e3488ed8a91b' of type 'Downtime'.
[2021-04-20 13:14:48 +0000] information/Downtime: Removed downtime 'HOSTNAME02!SERVICE42!cd51827e-f780-412d-add8-e3488ed8a91b' from checkable 'HOSTNAME02!SERVICE42' (Reason: expired at 2021-04-20 13:00:00 +0000).
[2021-04-20 13:15:03 +0000] information/ConfigObjectUtility: Deleted object 'HOSTNAME02!SERVICE42!f183746d-aa8c-4458-b021-b2da44433bfb' of type 'Downtime'.
[2021-04-20 13:15:03 +0000] information/Downtime: Removed downtime 'HOSTNAME02!SERVICE42!f183746d-aa8c-4458-b021-b2da44433bfb' from checkable 'HOSTNAME02!SERVICE42' (Reason: expired at 2021-04-20 13:00:00 +0000).
[2021-04-20 13:15:47 +0000] information/ConfigObjectUtility: Deleted object 'HOSTNAME02!SERVICE42!4c69bdad-0b37-48fa-a2eb-e04c48eadd3c' of type 'Downtime'.
[2021-04-20 13:15:47 +0000] information/Downtime: Removed downtime 'HOSTNAME02!SERVICE42!4c69bdad-0b37-48fa-a2eb-e04c48eadd3c' from checkable 'HOSTNAME02!SERVICE42' (Reason: expired at 2021-04-20 13:00:00 +0000).
[2021-04-20 13:16:08 +0000] information/ConfigObjectUtility: Deleted object 'HOSTNAME02!SERVICE42!59b55aad-b95e-4369-a195-fdae9a066ba5' of type 'Downtime'.
[2021-04-20 13:16:08 +0000] information/Downtime: Removed downtime 'HOSTNAME02!SERVICE42!59b55aad-b95e-4369-a195-fdae9a066ba5' from checkable 'HOSTNAME02!SERVICE42' (Reason: expired at 2021-04-20 13:00:00 +0000).
[2021-04-20 13:16:36 +0000] information/ConfigObjectUtility: Deleted object 'HOSTNAME02!SERVICE42!a92591b1-022a-4c2f-a0e1-0080ae84a1bd' of type 'Downtime'.
[2021-04-20 13:16:36 +0000] information/Downtime: Removed downtime 'HOSTNAME02!SERVICE42!a92591b1-022a-4c2f-a0e1-0080ae84a1bd' from checkable 'HOSTNAME02!SERVICE42' (Reason: expired at 2021-04-20 13:00:00 +0000).
[2021-04-20 13:17:09 +0000] information/ConfigObjectUtility: Deleted object 'HOSTNAME02!SERVICE42!d2132dd2-e1f8-43b8-89c8-ad772561f87e' of type 'Downtime'.
[2021-04-20 13:17:09 +0000] information/Downtime: Removed downtime 'HOSTNAME02!SERVICE42!d2132dd2-e1f8-43b8-89c8-ad772561f87e' from checkable 'HOSTNAME02!SERVICE42' (Reason: expired at 2021-04-20 13:00:00 +0000).
[2021-04-20 13:17:34 +0000] information/ConfigObjectUtility: Deleted object 'HOSTNAME02!SERVICE42!7a6f4f11-aea9-4be4-99c4-c0b183fdf9b2' of type 'Downtime'.
[2021-04-20 13:17:34 +0000] information/Downtime: Removed downtime 'HOSTNAME02!SERVICE42!7a6f4f11-aea9-4be4-99c4-c0b183fdf9b2' from checkable 'HOSTNAME02!SERVICE42' (Reason: expired at 2021-04-20 13:00:00 +0000).
[2021-04-20 13:18:05 +0000] information/ConfigObjectUtility: Deleted object 'HOSTNAME02!SERVICE42!568cc57b-249c-4abe-8f9b-329325e96266' of type 'Downtime'.
[2021-04-20 13:18:05 +0000] information/Downtime: Removed downtime 'HOSTNAME02!SERVICE42!568cc57b-249c-4abe-8f9b-329325e96266' from checkable 'HOSTNAME02!SERVICE42' (Reason: expired at 2021-04-20 13:00:00 +0000).
[2021-04-20 13:18:22 +0000] information/ConfigObjectUtility: Deleted object 'HOSTNAME02!SERVICE42!5ab3d2a7-fc19-46b7-b00a-c09fd75554c9' of type 'Downtime'.
[2021-04-20 13:18:22 +0000] information/Downtime: Removed downtime 'HOSTNAME02!SERVICE42!5ab3d2a7-fc19-46b7-b00a-c09fd75554c9' from checkable 'HOSTNAME02!SERVICE42' (Reason: expired at 2021-04-20 13:00:00 +0000).
This looks a lot like the old downtime/comment loop bug, but this server is not part of a cluster.