How can I confure, monitor and send notifications on high CPU steal usage?

Service Monitoring
1

Hello Guys,

I’m new to Icinga2 and this is my very first question to the icinga community people.

I want to configure the alerts whenever CPU steal usage is high or above certain percentage. How can I accomplish this? I have checked some existing plugins such as iostat (doesn’t seem to give us CPU steal usage)

Unfortunately, I am not really aware of how I can achieve this? Can someone help me and explain which plugins can be used and how can I integrate to the services (neither did I see any pre-existing service template)?

2

It looks like there are a few out there…I’m not using any of them myself.

https://exchange.icinga.com/cheko/check_cpu_usage

Seems bugged if you specify --names though, e.g.:
./check_cpu_usage --details --names steal Linux CPU Usage OK - CPU (steal=100.00), CPU0 (steal=100.00), CPU1 (steal=100.00) | cpu_steal=100%;; cpu0_steal=100%;; cpu1_steal=100%;;

There are different ways to add your own plugin depending on if you’re using Director or not. There’s a lot of documentation for both ways and you can always look at the included configs for examples (e.g. /usr/share/icinga2/include/command-plugins.conf). If you’re using Director it’s pretty simple to create a Service Template once the Command has been imported or created.

1 Like
3

Thanks so much for the explanation @leeclemens.

Do you mind pointing towards the correct documentation for both the ways?

4

This may be a good place to start: Configuration - Icinga 2

5

Hi @leeclemens,

I was able to made changes to the custom plugin to monitor CPU steal usage for all the CPUs in the instance/VM. Now, I want that custom plugin to be visible in the icinga web UI. However, I’m kind of stuck with it right now even after following the documentation ( Configuration - Icinga 2)

Steps followed so far:

  • Copied the custom plugin under PluginDir(/usr/lib64/nagios/plugins/)
  • Define checkcommand object under /etc/icinga2/conf.d/commands.conf
  • Define the service using the above command under /etc/icinga2/conf.d/services.conf

After following the above steps, I restarted the icinga service and check the configs using icinga2 daemon -C. All okay at this point but I’m still not able see the custom plugin in the web UI.

It’s kind of confusing as not all the existing commands are defined under /etc/icinga2/conf.d/commands.conf rather it’s defined under less /usr/share/icinga2/include/command-plugins.conf, which I have already tried to make the changes to but still no luck.

Can you please share the steps by step instructions or please advise what I’m missing here?

Thanks,
Siddarth

6

I would not modify the CheckCommands in /usr/share/icinga2 directly as those should be managed by the package. If you’re using Director you’ll need to import or kickstart to get the custom CheckCommand in the interface to utilize. Otherwise you’ll need to Apply Service with your custom command (and template, iirc - I prefer Director).

7

Already reverted the changes made to the /usr/share/icinga2. Yes, I’m using icinga director. How can I import it in the interface?

In order to import I go to Icinga Director → Commands → External Commands → Add, I don’t see any template related to my custom plugin(which I’ve defined under /etc/icinga2/conf.d/templates.conf) to import from.

Also, Apply Service isn’t working. Below are the configs with which I had given a shot:

In /etc/icinga2/conf.d/commands.conf:

object CheckCommand "steal" {
        command = [ PluginDir + "/check_steal_stats" ]

        arguments = {
                "-w" = {
                        value = "$steal_warning$"
                        description = "Exit with WARNING status if steal percentage exceeds 50%"
                }
                "-c" = {
                        value = "$steal_critical$"
                        description = "Exit with CRITICAL status if steal percentage exceeds  60%"
                }

        }

        vars.steal_warning = "50"
        vars.steal_critical = "60"

}

In /etc/icinga2/conf.d/services.conf

apply Service "steal" {
  import "generic-service"  ## using this generic service template from 
                                          ##   /etc/icinga2/conf.d/templates.conf

  check_command = "steal"

  assign where host.name == NodeName
}