Baseline deviation alerts monitoring

Hello ,
I have scenario that certain servers, type dev builders that usually compiling code every night, and short period during the day.
Those servers are really intensive work subsystem during there jobs tasks, i would like to implement instead of simple warn crit values trip alerts ,i would like to do kind of “smart” thresholds , i mean that i can profile every server workload for example for week period and take for example CurrentDiskQeueu metric behavior and if its during the period selected for basline deviation change behavior will trigger alerts
Is that possible to achieve with Icinga or any plugin module?
Please advice
Thanks

Shalom Tal
What you are trying to achieve is based of metrics gathering and evaluation, Icinga was not built as a metrics platform, although it has integration with graphite (a metric gathering engine).


So you can gather those metrics in graphite and then write a plugin that will read that data and create the alert based on the properties you need.

Another way is to use Prometheus tools and use it’s exporter tools to access those metrics and use a plugin to read them,I wrote a one that can be incorporated to Icinga, but it is a rudimentary one and might not answer your explicit needs.

Stay Safe.

1 Like

Shalom Assaf,
Thanks for the answer, i don’t know if your solution/suggestion is applicable to my config, since i am writing the metric data into InfluxDb , any other suggestion ? , i was starting of thinking implementing Kapacitor , but its something i would need to learn how to use.
Thanks stay safe and HAG SAMEAH :slight_smile:

1 Like

Hi Tal
You can try using this influxdb plugin

https://exchange.icinga.com/Mikanoshi/InfluxDB%20data%20monitoring%20plugin
Or

and there are more plugins to query influxdb covered in this topic:

Hag Sameah

2 Likes