This one is used to tell graphite how much quality we expect in our time series database to make a clean aggregation between archives.
Small Example!
First retention tells Graphite when it should expect a metric, in your case every minute.
1w:4w
(This is our raw data)
00:01:00 icinga2.value 2.0
00:02:00 icinga2.value 10.0
00:03:00 icinga2.value 1.0
00:04:00 icinga2.value 49.0
00:05:00 icinga2.value 38.0
00:06:00 icinga2.value 4.0
Second retention says, okay we want to store 5 minutes for 23 weeks. Now that means Graphite will summarize five 1 minute points to one 5 minute point.
5m:23w
(This is our already aggregated data in a perfect world)
00:05:00 icinga2.value 20.0 = ((2+10+1+49+38)/5) <- Because we want the avearage
00:06:00 icinga2.value 20.4 = ((10+1+49+38+4)/5) <- Because we want the avearage
Why perfect world? We learned already that sometimes a point (or several) can get lost because check was killed or point was to early / late. Then Graphite will set a default value like 0
for this point.
Than in the real world it could give situations like these:
1w:4w
00:01:00 icinga2.value 0 <- dead check
00:02:00 icinga2.value 0 <- dead check
00:03:00 icinga2.value 0 <- dead check
00:04:00 icinga2.value 49 <- admin fixed the check
00:05:00 icinga2.value 38
00:06:00 icinga2.value 4
5m:23w
00:05:00 icinga2.value 56.6 = ((0+0+0+49+38)/5) <- Before 20.0 difference + 36.6
00:06:00 icinga2.value 20.2 = ((0+0+49+38+4)/5) <- Before 20.4 difference - 0.02
To prevent such rollercoaster an aggregation schema will be applied to this. Default one says:
[default]
pattern = .*
xFilesFactor = 0.5
aggregationMethod = [average](https://graphite.readthedocs.io/en/latest/config-carbon.html?highlight=aggregationMethod#storage-aggregation-conf)
Pattern says: âWe want to match everythingâ.
xFileFactor says: "Okay if we should aggregate clean between our archives we want 0.5 = 50% of data not 0
".
aggregationMethod says: âUse the average value for aggregationâ
Now we expect 50% of data to be not 0
, then our real world scenario looks like this:
1w:4w
00:01:00 icinga2.value 0 <- dead check
00:02:00 icinga2.value 0 <- dead check
00:03:00 icinga2.value 0 <- dead check
00:04:00 icinga2.value 49 <- admin fixed the check
00:05:00 icinga2.value 38
00:06:00 icinga2.value 4
5m:23w
00:05:00 icinga2.value 0 = ((0+0+0+49+38)/5) <- 3 of 5 = 0 | 60% of 100% = 0
00:06:00 icinga2.value 20.2 = ((0+0+49+38+4)/5) <- 2 of 5 = 20.2 | 40% of 100% = 20.2
WHAT!? yeah.
Since xFileFactor is set to 0.5 we expect less then 50% of our data to be not 0
. Means if ±50% of our data are 0
Graphite will aggregate a 0
into the next archive.
Here is some lecture about this: http://obfuscurity.com/2012/04/Unhelpful-Graphite-Tip-9