Delete acknowledgements after one week

Icinga 2
1

Hi, it has happened to us too often now that some people acknowledge alerts and they remain in the system without the problem being tackled in any way.
I tried to use the API to remove acknowledgements which are older than a week. How would I do that, as I failed…?

Alternatively, I think I would have to run a SQL script, right?

Thanks
Klaus

2

Hi,

acknowledgements can set an expire time, and AFAIK there is a web config flag to enforce this field being set all the time.

In terms of deleting the acknowledgements, you must tell Icinga 2 as core about it. Manually purging them from the DB backend will not remove them from the core preventing notifications.

The REST API action you’re looking for is remove-acknowledgement.

Cheers,
Michael

3

Thanks Michael. I cannot find a config flag for it, and even if I force that field to be set, I need to allow a maximum value of 7 days probably.

In other words, removing the acknowledgements from the database and restarting icinga core should work, shouldn’t it?

The REST API would be great, but there is no filter, I can add, I think… I find the API documentation really confusing unfortunately…

4

Hi,

here’s the setting I was referring to: https://icinga.com/docs/icingaweb2/latest/modules/monitoring/doc/03-Configuration/#default-settings

No, exactly this does not work. You really need to do that with interacting with the core.

What exactly makes it confusing? The API action allows to use filters, and depending on the scripting language you’ll use for this kind of implementation, you can go further.

Do you have a concrete set of hosts/service and a matching pattern you’d like to remove acknowledgements from?

Cheers,
Michael

5

Ah, I see. That’s great.
I don’t have a specific set in mind, it should remove ALL acknowledgements which are older than 7 days. I will try to implement it via the filter. Many thanks so far.

6

Hi,

ok when it comes to a time based filter, this will be trickier since you need to pre-select the host/service objects which match before. Since an acknowledgement involves a special Comment type, that should be doable with a query against the /v1/objects/comments URL endpoint, specifically checking for

  • entry_type == 4 (CommentAcknowledgement)
  • entry_time < now - 7d

Once you’ve extracted host_name and service_name you can use that as filter for remove-acknowledgement.

Cheers,
Michael

1 Like
7

Cheers for that. I would have hoped to calculate the epoch time of now - 7 days and use it as a filter to remove-acknowledgement. Something like:

service_last_state_change<1582284000

Unfortunately this does not work… I will try your suggestion though. Thanks a lot

2 Likes
8

It is an interesting idea, I have never thought about this. If we would store the acknowledgement_time on the host/service when it happens, that could be used as a filter for remove-acknowledgement later.

I was looking into the code, and now I see that IcingaDB and 2.12 will add acknowledgement_last_change. It doesn’t help you now but in future releases this will be easier then.

2 Likes