Performance data is discarded from Graphite after 6 hours

Hello community,
I’m currently stumbling upon issues with visualizing performance data from Icinga2 with Graphite + Grafana.

Performance data older than 6 hours are deleted from Graphite. Leveraging performance data up to 6 hours of age works perfectly.
I’m using thes storage-schemas.conf from the official Icinga2 vagrant boxes.

Setup / Envrionment:
Icinga2 is running on a virtualized Debian Jessie machine.
Beside Icinga2, also Docker CE is installed. Graphite and Grafana are running in containers.


pattern = ^carbon\.
retentions = 10s:6h,1m:90d

pattern = .*
retentions = 10s:6h,1m:6d,10m:1800d

pattern = ^icinga2\..*\.(max_check_attempts|reachable|current_attempt|execution_time|latency|state|state_type)
retentions = 5m:7d

pattern = ^icinga2\.
retentions = 5m:10d,30m:90d,360m:4y

docker-compose.yml for Graphite / Grafana:

version: '2'

    image: graphiteapp/graphite-statsd
    restart: always
      - "80:80"
      - "2003:2003"
      - "2004:2004"
      - "8125:8125/udp"
      - "8126:8126"
      - storage:/opt/graphite/storage
      - ./storage-schemas.conf:/opt/graphite/conf/storage-schemas.conf:ro

    image: grafana/grafana
    restart: always
      - "3000:3000"


Icinga 2 - 2.10.4-1.jessie
Docker Community Edition - 18.06.3~ce~3-0~debian
Graphite container image - 1d7b81067e37 (March 2019)
Grafana container image - d9bdb6044027 (April 2019)

Do you have any hints how to process performance data that is older than 6 hours?
Thanks a lot in advance!

Best wishes,

Quoting from another thread

Not sure if this is the entire reason why the perf data are only kept for 6 hours, but it’s a start.

1 Like

Yeah it looks like a few things here.

Your default is definitely hitting before Icinga is, but the 6h entry I see is regex’d for carbon, so that wouldn’t apply to Icinga’s perfdata.

Did you, by chance, copy/paste some entries, modify them, reload carbon, and haven’t resized the whisper files? Either way, definitely move that Icinga entry up, I’d say go ahead and nuke your icinga2 folder in the whisper storage since it’ll be gone in 6 hours anyway, start carbon back up and try again.

Use and to make sure your whisper files are generating right.

Also, don’t use the ones from the Graphite box, that’s just a demo. Refer to what’s written in the documentation.