Graphite won't keep history

Icinga Web Modules
1

Hi guys.

I have an Icinga2 + Graphite setup working. But my Graphite doesn’t have a history longer than 1 day.

Screenshot%20from%202019-04-04%2013-59-30
Screenshot%20from%202019-04-04%2013-59-30635×573 66.5 KB

If I select for 1 week or 1 month, the graph won’t change. This happens for any service/host.

/etc/carbon/storage-schemas.conf:

[carbon]
pattern = ^carbon\.
retentions = 60:90d

[default_1min_for_1day]
pattern = .*
retentions = 60s:1d

[icinga_internals]
pattern = ^icinga\..*\.(max_check_attempts|reachable|current_attempt|execution_time|latency|state|state_type)  
retentions = 5m:7d

[icinga_default]
pattern = ^icinga\. 
retentions = 1m:2d,5m:10d,30m:90d,360m:4y

Does anybody know where to start looking? Thanks!

2

Are you using the official Graphite module?

The time selectors look different to mine/those from the official module.

3

Yes I am using the official Graphite module. I couldn’t find the time selectors from the URL you provided. Can you point me them, please?

Thanks!

4

I meant those:
image

I had this problem too, but I cab’t remeber exactly what I did to fix it.

Please try setting the character set of your ido-db resource to latin1 in the icingaweb2 resource configureation and see if that helps

1 Like
5

Hmm you’re right… it is different. Maybe mine is the latest version?

Tried the latin1, it didn’t help. :frowning:

6

Are you 100% sure that you have installed v1.1.0 from https://github.com/Icinga/icingaweb2-module-graphite ?

because your graphs and interface look like those from this module: https://github.com/findmypast/icingaweb2-module-graphite

The latest official module version should look like:
image

Compare your issue to https://github.com/Icinga/icingaweb2-module-graphite/issues/176 where someone else also had no changes between the different time selections due to using the unofficial module.

2 Likes
7

You are correct. I was not using the official https://github.com/Icinga/icingaweb2-module-graphite.
I am now trying to set up the official, and am getting errors. The documentation is not good.

But that’s stuff for another thread.

8

In storage-schemas.conf in your carbon config folder, first match wins. So in this case, your default_1min_for_1day entry is hitting before your Icinga entries. You’ll want to move those up.

Something that might catch you off guard if you’re new to graphite is that your whisper database will not update the retention automatically as that’s set in the metadata of the whisper files. You can see this by using whisper-info.py on any given value.wsp. If it’s a new setup, I’d just go ahead and start over. Otherwise, the whisper-resize.py script is available.

1 Like
9

So, I have installed the version recommended by @log1c here. However, it did not solve my problem.

It looks like that @blakehartshorn’s idea is valid. I’ve made the change, but now apparently I need to use the whisper-resize script to apply the change to existing graphs.

The /etc/carbon/storage-schemas.conf file is now:

[icinga_internals]
pattern = ^icinga\..*\.(max_check_attempts|reachable|current_attempt|execution_time|latency|state|state_type)  
retentions = 5m:7d

[icinga_default]
pattern = ^icinga\. 
retentions = 1m:2d,5m:10d,30m:90d,360m:4y 

[default_1min_for_1day]
pattern = .*
retentions = 60s:1d

[carbon]
pattern = ^carbon\.
retentions = 60:90d

So, I’ve done this:
find /var/lib/graphite/whisper/ -type f -name '*.wsp' -exec /usr/bin/whisper-resize --nobackup {} 1m:2d 5m:10d 30m:90d 360m:4y \;

It works, I can now select 1 week, 1 month or 3 hours. Even though there is no data, it works!

Thanks!

1 Like