# Clean up downtime objects due to high memory usage

**URL:** <https://community.icinga.com/t/clean-up-downtime-objects-due-to-high-memory-usage/11555>\
**Category:** Icinga 2\
**Created:** [March 7, 2023, 5:03pm UTC](https://community.icinga.com/t/clean-up-downtime-objects-due-to-high-memory-usage/11555 "2023-03-07T17:03:25Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![Erik](https://community.icinga.com/user_avatar/community.icinga.com/erik/32/6098_2.png) [@Erik](https://community.icinga.com/u/Erik)\
**Post date:** [March 7, 2023, 5:03pm UTC](https://community.icinga.com/t/clean-up-downtime-objects-due-to-high-memory-usage/11555/1 "2023-03-07T17:03:25Z")

</div>

**Description**

Hi all,

The memory usage of our primary Icinga2 instance has increased significantly since the beginning of this month. We’re running it HA with 2 master nodes.

The icinga2 process on the primary master is using 18G of memory. The secondary is only using 3,5G. Digging in deeper, I suspect that it might be the high amount of downtime objects causing this, since this seems really high: 421,799

 ![Screenshot 2023-03-07 at 17.46.12](https://community.icinga.com/uploads/default/original/2X/9/97496affce7974c6943dd838daeb9af817e1afaf.png)

Now I’m trying to clean those up. I reduced the retention period in IcingaDB from 365 to 90 days. This looked promising after it ran the cleanup:

```auto
retention: Removed 636977 old downtime history items

```

However, after restarting Icinga it still showed me 421k downtime objects instantiated:

```auto
 information/ConfigItem: Instantiated 421915 Downtimes

```

It seems like they are still present somewhere. How can I clean this up?

DB:

```auto
MariaDB [icingadb-prod]> select count(*) from downtime_history;
+----------+
| count(*) |
+----------+
| 911381 |
+----------+
1 row in set (0.27 sec)

MariaDB [icingadb-prod]> select count(*) from downtime;
+----------+
| count(*) |
+----------+
| 557 |
+----------+
1 row in set (0.00 sec)

```

**System version & info**

```auto
Icinga2 - version: r2.13.6-1

System information:
  Platform: CentOS Linux
  Platform version: 7 (Core)
  Kernel: Linux
  Kernel version: 3.10.0-1127.18.2.el7.x86_64
  Architecture: x86_64

```

```auto
 $ sudo icinga2 feature list
Disabled features: compatlog debuglog elasticsearch gelf graphite ido-mysql influxdb influxdb2 opentsdb perfdata statusdata syslog
Enabled features: api checker command icingadb livestatus mainlog notification

```

```auto
Instantiated 1 NotificationComponent.
Instantiated 1 LivestatusListener.
Instantiated 1 ExternalCommandListener.
Instantiated 1 CheckerComponent.
Instantiated 4 TimePeriods.
Instantiated 2 Users.
Instantiated 84430 Services.
Instantiated 75474 ScheduledDowntimes.
Instantiated 2 Zones.
Instantiated 4 NotificationCommands.
Instantiated 162678 Notifications.
Instantiated 2977 Hosts.
Instantiated 1 IcingaApplication.
Instantiated 18 HostGroups.
Instantiated 84430 Dependencies.
Instantiated 300 Comments.
Instantiated 10 EventCommands.
Instantiated 421799 Downtimes.
Instantiated 2 Endpoints.
Instantiated 1 FileLogger.
Instantiated 12 ApiUsers.
Instantiated 253 CheckCommands.
Instantiated 1 ApiListener.
Instantiated 1 IcingaDB.

```

---

<div class="post-metadata">

**Author:** ![Erik](https://community.icinga.com/user_avatar/community.icinga.com/erik/32/6098_2.png) [@Erik](https://community.icinga.com/u/Erik)\
**Post date:** [March 7, 2023, 5:31pm UTC](https://community.icinga.com/t/clean-up-downtime-objects-due-to-high-memory-usage/11555/2 "2023-03-07T17:31:01Z")

</div>

I found out the downtime objects where configured in a temporary folder from the API:

`/var/lib/icinga2/api/packages/_api/<uuid>/conf.d/downtimes `

```auto
$ sudo find /var/lib/icinga2/api/packages/_api/<uuid>/conf.d/downtimes | wc -l
421369

```

I shutdown Icinga on both nodes, together with icingadb. Removed old downtime objects from this directory and started the node again.

Memory usage went down to 2.7G… remarkable!