Monitoring windows remotely through WMI

Author: @gkoutsog

Revision: v0.20

Tested with:

  • Icinga 2 v2.10.x
  • Icinga Web 2 v2.6.x
  • Windows Server 2012 and later

Introduction

In case someone does not want to install the Icinga agent on Windows machines and wants to go the agentless way, the use of the WMI layer can provide a solution. Windows Management Instrumentation (aka WMI) is the Microsoft implementation of Web-Based Enterprise Management (WBEM). Other solutions for monitoring Windows machines without an agent include Powershell, SSH (for 2019 and later versions) and SNMP. Here we will focus on WMI and let others describe their solutions for their respective tools.

Further information on the WMI Home Page

Requirements

In order to be able to get the required information from Windows to Icinga, a plugin is needed, named check_wmi_plus. Plugin is writen in Perl and is dependent on a couple of other Perl modules. Also needed is an underlying tool named wmic which is the WMI client for Linux that most often needs to be compiled from source. Last but not least, WMI needs to be enabled on Windows machines (usually on by default) and a separate user created that has access to WMI but keep in mind to limit that user’s other privileges.

Installation

Linux

WMIC

This can be either installed by compiling from source or by using pre-packaged binaries. Compiling can be problematic since the code has not seen maintenance for some time and probably never will again.

Source

Go to http://edcint.co.nz/checkwmiplus/download/zenoss-wmi-source-v1-3-14/ and download the source code and store it somewhere like /usr/local/src/. Uncompress it and then go into Samba/source : cd Samba/source and run the following:

./autogen.sh
./configure
make
make install #might not be needed

In case you see an error like the following:

Can't use 'defined(@array)' (Maybe you should just omit the defined()?) at ./pidl/pidl line 583.

Just comment out that define in the line mentioned and run make again.

You can also safely ignore the following:

make: *** No rule to make target `wmi/wmiq.o', needed by `bin/wmiq'.  Stop.

If you get another error, try changing compiler directives, like:

make "CPP=gcc -E -ffreestanding"

If the compilation finished without errors, it’s time to test it.
Start with a simple command: wmic -U [domain/]adminuser%password //host_or_IP "select TotalPhysicalMemory from Win32_ComputerSystem"
and the response should look like:

CLASS: Win32_ComputerSystem
Name|TotalPhysicalMemory
hostname|412180664

For more possible values, run:
wmic -U [domain/]adminuser%password //host_or_IP "select * from Win32_ComputerSystem"

You can probably compile this on one machine and transfer the binary to another but I haven’t tested this.

Packages

For the record, binaries are not your best bet but maybe worth a try if you find something that is close enough to your distribution version. There are a couple of sources suggested below but you are welcome to scourge the web for more and suggest them in the comment section.

or search rpm.pbone.net

Icinga/Nagios Plugin

Since check_wmi_plus is a plugin written in Perl, there are certain modules that need to be installed alongside. This can be done either through the package manager of your system (apt for Debian/Ubuntu, yum/dnf for Fedora/RedHat/Centos, pacman for Arch etc etc) or through CPAN. Do note that the plugin is (kind of) sensitive in the versions of the modules so an installation from CPAN is advisable if the repositories of your Linux distribution contain much newer versions than those required.

This is how it looks currently on a CentOS 7 server that we run, with all Perl modules coming from the official CentOS repository.

MODULE NAME INSTALLED VERSION DESIRED VERSION
Config::IniFiles 2.79 2.58
Getopt::Long 2.4 2.38
DateTime 1.04 0.66
Number::Format 1.73 1.73
Data::Dumper 2.145 2.125
Scalar::Util 1.27 1.22
Storable 2.45 2.22
Perl Version 5.016003 5.01

Installation from CPAN is easy as e.g: cpan install Number::Format . But if you need a specific version, you need to provide the full module distribution filename including the author, e.g: cpan SHLOMIF/Config-IniFiles-2.58.tar.gz

Next step is to download the plugin itself. Go to Releases and download the latest release. Unpack it somewhere, ideally where your Icinga variable CustomPuginDir points to but under a catalog of its own. Then you need to make some changes in the main Perl script check_wmi_plus.pl.

  • Set the correct location of utils.pm. By default it points to /usr/lib/nagios/plugins but more likely to be in /usr/lib64/nagios/plugins.
  • Set the $base_dir variable to the directory where check_wmi_plus.pl is installed.
  • Set the location of wmic binary in the $wmic_command variable.
  • (Optional) Set the variable $wmi_ini_dir to the location of all the ini files
  • (Optional) Set the variabele $tmp_dir in order to change the default location of temporary files. Default is /tmp/

Read more in the plugin homepage

Windows

In order for our monitoring solution to be able to login to Windows machines and fetch all the metrics we need, an account is needed. The process of adding the user is as follows in your domain controller.

  1. Open the WMI Control console: Click Start , click Run , type wmimgmt.msc and then click OK .
  2. In the console tree, right-click WMI Control , and then click Properties .
  3. Click the Security tab.
  4. Select the namespace for which you want to give a user or group access, and then click Security .
  5. In the Security dialog box, click Add .
  6. In the Select Users, Computers, or Groups dialog box, enter the name of the object (user or group) that you want to add. Click Check Names to verify your entry and then click OK . You might have to change the location or click the Advanced button to query for objects. See the dialog box Help for more details.
  7. In the Security dialog box, under Permissions , select the permissions to allow or deny the new user or group: Execute Methods (enable), Full Write (disable), Partial Write (disable), Provider Write (disable), Enable Account (enable), Remote Enable (enable), Read Security (enable), Edit Security (disable)

Note: To perform these tasks you need to be logged as a member of the local Administrators group, if it is a local computer. To access a remote computer, right-click WMI Control , click Connect to another computer , click Another computer , and then type the name of the computer to which you want to connect. If you are using WMI Control from the Computer Management console, right-click Computer Management to connect to the other computer.

It is also advisable that WinRM is enabled and remote desktop enabled in the firewall:

netsh advfirewall firewall set rule group="remote desktop" new enable=Yes  
winrm quickconfig

The second command sets up a listener on the Windows node. Answer yes to apply these changes.

Expected result:

WinRM has been updated for remote management.
WinRM service type changed to delayed auto start.
WinRM service started.
Created a WinRM listener on HTTP://* to accept WS-Man requests to any IP on this machine.

Documentation and more in [WMI for Windows Server](https://docs.microsoft.com/en-us/previous-versions/windows/it-pro/windows-server-2008-R2-and-2008/cc771551(v=ws.11))

Configuration

The following part of guide is meant for the people that have Icinga configured with the DSL, aka plain-text configuration files. The way to do this in Icinga Director module, is left for another member of the community to contribute :slight_smile:

Authentication file

Icinga needs to know with which user it can connect to the Windows machines. For this reason we create a special file for WMI, named wmi.auth or as you want, ideally under /etc/icinga2/, e.g: /etc/icinga2/wmi.auth, with the following contents:

username=myusername
password=mypassword
domain=mydomain

Take precaution and have this file owned by root and make it only readable by root aka chown root wmi.auth && chmod 0400 wmi.auth

commands.conf

Next we need to add the WMI command to Icinga. If you are using plain-text configuration files, you can populate the file where you have your custom commands or add it to commands.conf or to a separate file altogether.

object CheckCommand "check_wmi" {
    import "plugin-check-command"
    command = [ PluginDir + "/check_wmi_plus.pl" ]
    arguments = {
        "-H" = {
                value = "$host.address$"
                description = "name or IP address of host to monitor"
        }
        "-A" = {
        		value = "$wmi_authfile_path$"
        		description = "Authentication file path"
        }
        "-m" = {
         		value = "$check_mode$"
         		description = "WMI mode to use for specific check"
        }
        "-s" = {
        		value = "$wmi_submode$"
        		description = "WMI submode to be used, optional"
        }
		"-a" = {
				value = "$wmi_arg1$"
				description = "First argument to WMI"
		}
		"-o" = {
				value = "$wmi_arg2$"
				description = "Second argument to WMI"
		}
		"-3" = {
				value = "$wmi_arg3$"
				description = "Third argument to WMI"
		}
		"-4" = {
				value = "$wmi_arg4$"
				description = "Fourth argument to WMI"
		}
		"-y" = {
				value = "$wmi_delay$"
				description = "The delay between 2 consecutive WMI queries in a single call"
		}
		"-w" = {
				value = "$wmi_warn$"
				description = "Warning Thresholds"
		}
		"-c" = {
				value = "$wmi_crit$"
				description = "Critical Thresholds"
		}
		"--nodatamode" = {
			set_if = "$wmi_nodatamode$"
		}
		"--inidir" = {
				value = "$wmi_inidir$"
				description = "The full path of an ini directory to use"
		}
	}

	vars.wmi_authfile_path = "/etc/icinga2/wmi.auth"
	vars.wmi_inidir = "/opt/icinga/plugins/check_wmi_plus.d"
	vars.wmi_nodatamode = false
}

templates.conf

We also need a new template for these types of services so we can fine tune certain aspects without messing with other types. Example:

template Service "wmi-service" {
	import "generic-service"
	check_command = "check_wmi"
	check_interval = 1m
	retry_interval = 1m
}

WMI-services.conf

In a separate file or even in a pre-exisiting one, we can now add the apply rules for our new set of services. Examples:

apply Service "Free Disk Space" {
   import "generic-service"
   
   vars.check_mode = "checkvolsize"
   vars.wmi_arg1 = "."
   vars.wmi_arg2 = "1"
   vars.wmi_arg3 = "1"
   vars.wmi_warn = "88"
   vars.wmi_crit = "92"
   }
   
   check_command = "check_wmi"
   assign where host.vars.os == "Windows"
   ignore where host.vars.disable_wmi
}

apply Service "CPU Utilization" {
   import "generic-service"
   vars.check_mode = "checkeachcpu"
   vars.wmi_warn = "95"
   vars.wmi_crit = "99"
   vars.wmi_timeout = "160"
   check_command = "check_wmi"
   assign where host.vars.os == "Windows"  && host.vars.cpu_utilz
   ignore where host.vars.disable_wmi
}

apply Service "IIS: Connections" {
   import "generic-service"
   vars.check_mode = "checkiis"
   vars.wmi_submode = "connections"
   vars.wmi_arg1 = "_Total"
   vars.wmi_timeout = "190"
   check_command = "check_wmi"
   assign where host.vars.iis_server
   ignore where host.vars.disable_wmi
}

apply Service "MSSQL: General Statistics" {
   import "generic-service"
   vars.check_mode = "checksql"
   vars.wmi_submode = "general"
   if (host.vars.mssql_edition == "Express") {
      vars.wmi_arg1 = "MSSQLSQLEXPRESS_MSSQLSQLEXPRESS"
   }
   check_command = "check_wmi"
   assign where host.vars.mssql_server
   ignore where host.vars.disable_wmi
}

apply Service "Event Log: Application" {
   import "generic-service"
   vars.check_mode = "checkeventlog"
   vars.wmi_arg1 = "application"
   vars.wmi_arg2 = "2"
   vars.wmi_arg3 = "1"
   vars.wmi_warn = "50"
   vars.wmi_crit = "100"
   check_command = "check_wmi"
   assign where host.vars.os == "Windows"  && host.vars.event_log_application
   ignore where host.vars.disable_wmi
}

# Check for a specific process
/*
apply Service "Jenkins Process" {
   import "generic-service"
   vars.check_mode = "checkproc"
   vars.wmi_submode = "count"
   vars.wmi_arg1 = "jenkins"
   vars.wmi_crit = "1:1"
   check_command = "check_wmi"
   assign where host.name == "jenkins"
}

For more examples, you can search but I would suggest the following linked guide

Conclusion

Maybe not the best solution for the future in the face of PowerShell and SSH support, but actually good for monitoring legacy (aka pre-WinSrv2019 installations)

FAQ

  • Q: What types of services can I monitor through WMI?

    A: There some types but you need to check the documentation of the plugin itself. A small list includes: Uptime, Disk, CPU, AD, RDP sessions, IIS, MSSQL, EventLog and processes

  • Q: Can I write my own scripts?

    A: I assume yes and you are welcome to try. But probably it is not worth your time. PowerShell scripts through remote or SSH is more versatile and future-proof. See here

  • Q: Can you help me with my configuration files?

    A: Try asking a question in the community forum and me or someone else will reply when we have some time.

4 Likes

Should have read that earlier :slight_smile:

As the wmic connect havnt worked on my side, I used Powershell core and WinRM from Linux. Works very well, but need to recreate all checks then :slight_smile:

1 Like