Ifw - Invoke-IcingaCheckPartitionSpace - Wrong output: "Size is greater than threshold"

Icinga 2
1

Hello,
i’ve setup an icinga installation and adding windows systems to it. One of the windows sserver 2016 system have a false output of the “Invoke-IcingaCheckPartitionSpace” ifw plugin.

Free Partition Space: 1 Critical 3 Ok Partition E: (99.95% (499.70GiB))
_ Partition E: 99.95% (499.70GiB) is greater than threshold 95% (474.95GiB)

That does not make sense. The threshold were defined for 95 % critical and 80 % warning.

Hope someone can help.

Edit: I add more systems to icinga2 and the error appears on multiple systems. Till know there is no things thats are equal above the systems.

2

Hi @teqqy ,
First things first: Is the E: filesystem 99.95% full or empty?

3

Hi. Thx for the reply. It’s 99,95 % free.

4

I think there is a command line parameter which works as a switch to decide which way this works.
Or you could just use Invoke-IcingaCheckUsedPartition

5

But shold this issue not be on all systems if the check is configuried wrong? I have about 40 windows vm in my icinga instance and just about 5 have this issue.

6

hm, could you add the exectued Command here, so I can see what is actually happening?
If you are using IcingaDB you find that in the “Source” tab of the service.

7

Of course. Thats no problem. I reactivated the thresholds.

Output in icingaweb2

 Free Partition Space: 1 Critical 1 Ok  Partition D: (98.04% (98.03GiB))
\_  Partition C: 74.33% (73.13GiB)
\_  Partition D: 98.04% (98.03GiB) is greater than threshold 95% (94.98GiB)

Execute command due to icingaweb2
'C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe' '-NoProfile' '-NoLogo' '-ExecutionPolicy' 'ByPass' '-C' 'try { Use-Icinga -Minimal; } catch { Write-Output '\''The Icinga PowerShell Framework is either not installed on the system or not configured properly. Please check https://icinga.com/docs/windows for further details'\''; Write-Output '\''Error:'\'' $($_.Exception.Message)Components:rn$( Get-Module -ListAvailable '\''icinga-powershell-*'\'' )rn'\''Module-Path:'\''rn$($Env:PSModulePath); exit 3; }; Exit-IcingaExecutePlugin -Command '\''Invoke-IcingaCheckPartitionSpace'\'' ' '-Warning' '90%' '-Critical' '95%' '-Include' '@()' '-Exclude' '@()' '-Verbosity' '2' '-IgnoreEmptyChecks' '-SkipUnknown'

8

I still have the issue on some systems after switching to the api checks

'curl' '--verbose' '--tlsv1.2' '--fail-with-body' '--connect-to' 'vmdoccap01.tim.de:5668:localhost:5668'
'--ciphers' 'ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:DHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256'
'--cert' 'C:\ProgramData\icinga2\var\lib\icinga2/certs//vmdoccap01.example.com.crt'
'--key' 'C:\ProgramData\icinga2\var\lib\icinga2/certs//vmdoccap01.example.com.key'
'--cacert' 'C:\ProgramData\icinga2\var\lib\icinga2/certs//ca.crt' '--request' 'POST' '--url' 'https://vmdoccap01.excample.com:5668/v1/checker?command=Invoke-IcingaCheckPartitionSpace'
'--user-agent' 'Icinga/v2.14.0' '--header' 'Accept: application/json' '--header' 'Content-Type: application/json'
'--data-raw' '{"Critical":"95%","IgnoreEmptyChecks":true,"SkipUnknown":true,"Verbosity":"2","Warning":"90%"}'
9

Hi Christian,

I am struggling with a similar problem.

In my case all IfW checks work but do not have any thresholds - so the CPU check does not alarm, when the the load is high - which is bad.

This happened after I switched to the API forwarder (with the legacy way the thresholds are working fine, but the load goes up when the checks are executed).

While reading your post I am wondering about one thing:

Your curl command shows some parameters at the very end - but my curl does not show that:

  • ‘curl’ ‘–verbose’ ‘–tlsv1.2’ ‘–fail-with-body’ ‘–connect-to’ ‘SERVERNAME:5668:localhost:5668’ ‘–ciphers’ ‘ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:DHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256’ ‘–cert’ ‘C:\ProgramData\icinga2\var\lib\icinga2/certs//SERVERNAME.crt’ ‘–key’ ‘C:\ProgramData\icinga2\var\lib\icinga2/certs//SERVERNAME.key’ ‘–cacert’ ‘C:\ProgramData\icinga2\var\lib\icinga2/certs//ca.crt’ ‘–request’ ‘POST’ ‘–url’ ‘https://SERVERNAME:5668/v1/checker?command=Invoke-IcingaCheckCPU’ ‘–user-agent’ ‘Icinga/v2.14.3’ ‘–header’ ‘Accept: application/json’ ‘–header’ ‘Content-Type: application/json’ ‘–data-raw’ ‘{}’

If you found any solution for your problem I would be interested to know.

Best regards
Volker

PS: And a Happy New Year!

10

Sorry @teqqy, I didn’t follow up for some reason. I would recommend to try Invoke-IcingaCheckUsedPartitionSpace which (as the name suggests) should evaluate the used volume instead of the free one.

@vpdose It is hard to tell why, but the parameters are obviously not encoded in this command.
Could describe here how the command is configured in your system? So, Director or Icinga DSL, where did you get the CheckCommand from (IfW export or manual) and how your Service definition looks like.

11

Hi @lorenz - thanks for replying !

I am using Icinga Director to render the Icinga config. The Invoke-* commands were imported as basket (and exported in IfW via Get-IcingaCheckCommandConfig -OutDirectory 'C:\Users\myuser\Documents').

The service check for the CPU load without ifw-api looks like this:

‘C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe’ ‘-NoProfile’ ‘-NoLogo’ ‘-C’ ‘try { Use-Icinga -Minimal; } catch { Write-Output ‘'‘The Icinga PowerShell Framework is either not installed on the system or not configured properly. Please check Introduction - Icinga for Windows for further details’'’; Write-Output ‘'‘Error:’'’ $($_.Exception.Message)Components:rn$( Get-Module -ListAvailable ‘'‘icinga-powershell-*’'’ )rn’'‘Module-Path:’''rn$($Env:PSModulePath); exit 3; }; Exit-IcingaExecutePlugin -Command ‘'‘Invoke-IcingaCheckCPU’'’ ’ ‘-Warning’ ‘70’ ‘-Critical’ ‘85’ ‘-Core’ ‘_Total’ ‘-Verbosity’ ‘1’

At the end one finds the parameters I am using.

When I import the ifw-api template to Powershell-Base I am missing the thresholds and parameters - see above.

It is the same picture with the Invoke-IcingaCheckUsedPartitionSpace - with ifw-api the command is like this

‘curl’ ‘–verbose’ ‘–tlsv1.2’ ‘–fail-with-body’ ‘–connect-to’ ‘DE-REN-SR-1492:5668:localhost:5668’ ‘–ciphers’ ‘ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:DHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256’ ‘–cert’ ‘C:\ProgramData\icinga2\var\lib\icinga2/certs//DE-REN-SR-1492.crt’ ‘–key’ ‘C:\ProgramData\icinga2\var\lib\icinga2/certs//DE-REN-SR-1492.key’ ‘–cacert’ ‘C:\ProgramData\icinga2\var\lib\icinga2/certs//ca.crt’ ‘–request’ ‘POST’ ‘–url’ ‘https://DE-REN-SR-1492:5668/v1/checker?command=Invoke-IcingaCheckUsedPartitionSpace’ ‘–user-agent’ ‘Icinga/v2.14.3’ ‘–header’ ‘Accept: application/json’ ‘–header’ ‘Content-Type: application/json’ ‘–data-raw’ ‘{}’

Again, no parameters after --data-raw ‘{}’

I do have parameters for the check:

Custom Variables

|IcingaCheckUsedPartitionSpace_Array_Include (Array)| 1 item|
[0] |C|
|IcingaCheckUsedPartitionSpace_Int32_Verbosity| 2|
|IcingaCheckUsedPartitionSpace_Object_Critical| 95%|
|IcingaCheckUsedPartitionSpace_Object_Warning| 85%|
|IcingaCheckUsedPartitionSpace_Switchparameter_IgnoreEmptyChecks| true|
|IcingaCheckUsedPartitionSpace_Switchparameter_SkipUnknown |false|

Is my problem perhaps, that the import of the Invoke commands is too “old” - I did this a while ago …

Best regards
Volker

12

Hint: It would helpful for readers, if you could format copy and paste content in a more readable way, for example as a code block.

I am not sure where this went wrong, but yes, you could reimport the configuration again (of course a freshly generated config from IfW). Might help

13

Hi lorenz,

export the Invoke-* commands like this:

Get-IcingaCheckCommandConfig -OutDirectory 'Y:\Volker D\Icinga2-Windows\Baskets\'

and importing the json file as basket did the trick, now the thresholds and parameters are used in the rest api forwarding command:

'curl' '--verbose' '--tlsv1.2' '--fail-with-body' '--connect-to' '<HOSTNAME>:5668:localhost:5668' '--ciphers' 'ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:DHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256' '--cert' 'C:\ProgramData\icinga2\var\lib\icinga2/certs//<HOSTNAME>.crt' '--key' 'C:\ProgramData\icinga2\var\lib\icinga2/certs//<HOSTNAME>.key' '--cacert' 'C:\ProgramData\icinga2\var\lib\icinga2/certs//ca.crt' '--request' 'POST' '--url' 'https://<HOSTNAME>:5668/v1/checker?command=Invoke-IcingaCheckCPU' '--user-agent' 'Icinga/v2.14.3' '--header' 'Accept: application/json' '--header' 'Content-Type: application/json' '--data-raw' '{"Core":"_Total","Critical":"95","Verbosity":"2","Warning":"80"}'

Thank you so much for being able to discuss this with you!!

Best regards
Volker