I have scenario that certain servers, type dev builders that usually compiling code every night, and short period during the day.
Those servers are really intensive work subsystem during there jobs tasks, i would like to implement instead of simple warn crit values trip alerts ,i would like to do kind of “smart” thresholds , i mean that i can profile every server workload for example for week period and take for example CurrentDiskQeueu metric behavior and if its during the period selected for basline deviation change behavior will trigger alerts
Is that possible to achieve with Icinga or any plugin module?
What you are trying to achieve is based of metrics gathering and evaluation, Icinga was not built as a metrics platform, although it has integration with graphite (a metric gathering engine).
So you can gather those metrics in graphite and then write a plugin that will read that data and create the alert based on the properties you need.
Another way is to use Prometheus tools and use it’s exporter tools to access those metrics and use a plugin to read them,I wrote a one that can be incorporated to Icinga, but it is a rudimentary one and might not answer your explicit needs.
Thanks for the answer, i don’t know if your solution/suggestion is applicable to my config, since i am writing the metric data into InfluxDb , any other suggestion ? , i was starting of thinking implementing Kapacitor , but its something i would need to learn how to use.
Thanks stay safe and HAG SAMEAH
You can try using this influxdb plugin
and there are more plugins to query influxdb covered in this topic: