Check_ssl_cert plugin to monitor ssl cert expiry

I need to monitor URL using check_ssl_cert plugin which I see is defined under /usr/share/icinga2/include/plugins-contrib.d/web.conf as follows: I am new to icinga and not sure where to add this plugin and with what parameters:

object CheckCommand “ssl_cert” {
command = [ PluginContribDir + “/check_ssl_cert” ]
value = “$ssl_cert_address$”
value = “$ssl_cert_port$”
value = “$ssl_cert_file$”
value = “$ssl_cert_warn$”
value = “$ssl_cert_critical$”
value = “$ssl_cert_cn$”
set_if = “$ssl_cert_altnames$”
value = “$ssl_cert_issuer$”
value = “$ssl_cert_org$”
value = “$ssl_cert_email$”
set_if = “$ssl_cert_match_host$”
value = “$ssl_cert_serial$”
set_if = “$ssl_cert_noauth$”
set_if = “$ssl_cert_selfsigned$”
value = “$ssl_cert_sni$”
value = “$ssl_cert_timeout$”
value = “$ssl_cert_protocol$”
value = “$ssl_cert_clientssl_cert$”
value = “$ssl_cert_clientpass$”
value = “$ssl_cert_ssllabs$”
description = “SSL Labs assestment”
“–ignore-ssl-labs-cache” = {
set_if = “$ssl_cert_ssllabs_nocache$”
description = “Forces a new check by SSL Labs”
value = “$ssl_cert_rootssl_cert$”
“–ssl2” = {
return macro("$ssl_cert_ssl_version$") == “ssl2”
“–ssl3” = {
return macro("$ssl_cert_ssl_version$") == “ssl3”
return macro("$ssl_cert_ssl_version$") == “tls1”
return macro("$ssl_cert_ssl_version$") == “tls1_1”
return macro("$ssl_cert_ssl_version$") == “tls1_2”
“–no_ssl2” = {

Hi,

when you want to use the check_ssl_cert plugin you have to install it into the PluginContribDir. You can find installation instructions in the development repository of the check plugin. You can lookup the path for PluginContribDir in the /etc/icinga2/constants.conf file.

Now you can define a service object using the ssl_cert CheckCommand.

If you only need to monitor the valid days of a certificate I always used the http CheckCommand using the http_http_certificate custom variable. The check_http plugin is part of the Monitoring Plugins and they are installed during the Icinga 2 setup process.

https://icinga.com/docs/icinga2/latest/doc/10-icinga-template-library/#http

Best regards
Michael

1 Like

Hi Michael,

Thanks for your response, if you could explain me what is check_ssl_cert defined as object CheckCommand “ssl-cert” in /usr/share/icinga2/include/plugins-contrib.d/web.conf?

what would be the syntax to use check_http and I should add it in which file?

Regards,
Anup

I have installed check_ssl_cert under PluginContribDir, where and how can I define a service object using the ssl_cert CheckCommand?

I need to check that web URL is up and its cert status (warning at 30d, critical at 15d), can I do this with check_http?

Yes, thats possible with check_http.

1 Like

Hi,

The files that are located in /usr/share/icinga2/include/*defining the Icinga Template Library or short ITL. The ITL is a collection of standard templates for object definitions like the CheckCommand definitions that are needed to define a host or service object. You need a CheckCommand definition in order to define a host or service object, the CheckCommand is than passed as check_command attribute, with this information the Icinga 2 daemon knows how to execute the check plugin i.e. which parameter should be used, some check plugins expect a given order for parameter, all this is defined in a CheckCommand object.

You should not manually edit the files that are located in /usr/share/icinga2/include/* since all these files are part of the ITL and may be overwritten when you update Icinga 2. If you want to define a CheckCommand that is currently not part of the ITL you can create a config file for it in the following path:

  • for non-distributed environments: /etc/icinga2/conf.d/*
  • for distributed environments: /etc/icinga2/zones.d/global-templates/*

In distributed environments it is necessary that the satellite and agents also get the CheckCommand definition, so the definition should be placed inside a global zone. This is a more advanced topic that is descried in our documentation.

Host object:

We define a host object using the standard generic-host template (the template is included in the default configuration in /etc/icinga2/conf.d/templates.conf). We define also some custom variables, two of them are important for our service that is using check_http.

  • vars.http_vhost defines the VHost that should be used.
  • vars.http_certificate defines that the certificate should be valid for 7 days, if the value is lower the service switches to a warning state, if the value is below the threshold of 3 days the service goes into a critical state.

A full documentation of all available attributes can be found in the documentation:

object Host "test-host-001" {
	import "generic-host"

	address = "10.1.2.3"

	vars.os = "Linux"
	vars.osversion = "Debian 10"

    vars.http_vhost = "internal-service.my.domain"
	vars.http_certificate = "7,3"
}

Service object:

We define a apply rule to assign a service object to the host object. The service object will use the http_* custom variables. We assign the service to every host that starts with test-* as name.

apply Service "certificate" {
	import "generic-service"

	check_command = "http"

	assign where match("test-*", host.name)
}

It depends on your environment, so I can’t say in which file you should add the definition. In a non-distributed environment you should be safe to put it somewhere below /etc/icinga2/conf.d/* in a distributed environment you should put it somewhere below /etc/icinga2/zones.d/*/*.

Best regards
Michael

2 Likes

Thanks for your detailed response, really appreciate that.
I have recently installed icinga2 and just configured mysql on it. so it has a basic setup, now I have a URL, example https://xyz.com/abc to monitor its up and check its ssl cert status. will this be ok to add in templates.conf, also do I need to add http_uri here with /abc in addition?

object Host “name of the host where xyz.com is hosted” {
import “generic-host”

address = "ip address of the host where xyz.com is hosted"

vars.os = "Linux"
vars.osversion = "Debian 10"

vars.http_vhost = "internal-service.my.domain"
vars.http_certificate = "7,3"

}

Hi,

So I assume you currently don’t have a distributed environment.

You can put it there but I wouldn’t recommend it. Icinga 2 reads all *.conf files in the conf.d directory in the default configuration. You can put a service definition/apply rule inside the templates.conf file since the Icinga 2 config compiler does not care about which object type is configured in which config file.

To keep the configuration clean and clear I recommend to put the service definitions/apply rules in the services.conf file. Host objects on the other hand in the hosts.conf file. :slight_smile:

This are basically two services:

  1. HTTP status check
  2. Certificate expiration check

Both can be handled with the check_http plugin which is provided by the Monitoring Plugins package and is defined in the ITL with the definition http.

Now it becomes a bit tricky since both checks would use the custom variables vars.http_*. If we create an service apply rule both services would use the same attributes and would do a certificate expiration check, we don’t want this. You could define single service objects for each check, for two services this might be a easy possibility, but think ahead and think about you have 10 HTTP status checks and 10 certificate expiration checks, this becomes a bit messy overtime and you have a lot of typing. We don’t want that too. :wink:

So let’s dive into a more advanced topic: Apply for rules.

We modify the known host and declare a set for the HTTP status check and a set for the certificate expiration check. The name inside the [ ] brackets will become the service name that you’ll later see in Icinga Web 2. Since your HTTP service provides a certificate we can turn on SSL and SNI for the HTTP status check.

# vim /etc/icinga2/conf.d/hosts.conf

object Host “test-host-001” {
    import “generic-host”

    address = "127.0.0.1"

    vars.os = "Linux"
    vars.osversion = "Debian 10"

    vars.vhosts["http internal-service.my.domain"] = {
		http_vhost = "internal-service.my.domain"
		http_ssl = true
		http_sni = true
	}
	
	vars.vhosts["certificate internal-service.my.domain"] = {
	    http_vhost = "internal-service.my.domain"
		http_ssl = true
		http_sni = true
		http_certificate = "7,3"
	}
}

Now we define the apply for rule inside the services.conf file. This will create a service for each set defined using the syntax vars.vhosts["service name"]. There are also more advanced techniques possible where you can ignore specific values using ignore where expressions. Please have a look inside the documentation for an example. We now focus on our example:

# vim /etc/icinga2/conf.d/services.conf

apply Service for (vhost => config in host.vars.vhosts) {
	import "generic-service"

	check_command = "http"

	vars += config
}

The config compiler will now create two services for your host.

  1. http internal-service.my.domain
  2. certificate internal-service.my.domain

If you need additional attributes for the services you can add them from the ITL http template to the set e.g. your URI you can be added with http_uri = /example.

https://icinga.com/docs/icinga2/latest/doc/10-icinga-template-library/#http

Best regards
Michael

Thanks for the detailed response! I am awaiting firewall request to complete to be able to hit URLs from my monitoring server. Once that is done, I will test these configs and reply with the results or questions.

Thanks again!

Hi, After adding firewall rules to allow vhost on port 443, when I try to run the check manually with check_http, I see socket timeout: What does this mean?

[icinga@pl-bnaaeraap469 ~]$ /usr/lib64/nagios/plugins/check_http -v -I riskview.changehealthcare.com
CRITICAL - Socket timeout
[icinga@pl-bnaaeraap469 ~]$ /usr/lib64/nagios/plugins/check_http -v -I 10.162.9.10
CRITICAL - Socket timeout

and even when I run it against localhost, I see:

[icinga@pl-bnaaeraap469 ~]$ /usr/lib64/nagios/plugins/check_http -I 127.0.0.1
HTTP WARNING: HTTP/1.1 403 Forbidden - 4293 bytes in 0.001 second response time |time=0.001234s;;;0.000000 size=4293B;;;0

just to add, I haven’t added monitoring configuration yet. I am checking the URL manually with check_http before adding it to monitoring.

Hi.

Honestly, I have only read this thread partially.

But some points that may help:

  1. Are you behind a proxy?
  2. Can you establish the connection to the target by other commands?
    E.g.:
nc 10.162.9.10 -v
# or:
nc <target_host> -v

# or
openssl s_client -connect <the_target_url>:443 # or the correct port
  1. What output do you get by:
/usr/lib64/nagios/plugins/check_http -C 7,4 -H <target_host> -I <target_host> -S

Greetings

Hi Michael,

This is how my services show now in web. How would I know its checking the full URL https://riskview.changehealthcare.com/medicaid or just https://riskview.changehealthcare.com.

Service: certificate riskview.changehealthcare.com

Plugin Output

SSL OK - Certificate ‘bna.dcasolutions.com’ will expire in 643 days on 2022-05-30 18:59 -0500/CDT.

Service: http riskview.changehealthcare.com

Plugin Output

HTTP OK: HTTP/1.1 302 Moved Temporarily - 309 bytes in 0.018 second response time

This is the full configuration I added in hosts.conf:

object Host “pl-bnaaeraws401.dcasolutions.com”{
import “generic-host”

address = "10.162.6.10"

vars.os = "Linux"
vars.osversion = "RHEL 7"

vars.vhosts["http riskview.changehealthcare.com"] = {
	http_vhost = "riskview.changehealthcare.com"
            http_uri = "/medicaid"
	http_ssl = true
	http_sni = true
}

vars.vhosts["certificate riskview.changehealthcare.com"] = {
    http_vhost = "riskview.changehealthcare.com"
	http_ssl = true
	http_sni = true
	http_certificate = "30,15"
}

}

and services.conf:

apply Service for (vhost => config in host.vars.vhosts) {
import “generic-service”

check_command = "http"

vars += config

}

I ran the check manually with both vhost, ipaddress and -u for uri and it returns ok but is there a way to check the output in verbose to confirm its checking the whole URL:

[icinga@pl-bnaaeraap469 ~]$ /usr/lib64/nagios/plugins/check_http -H riskview.changehealthcare.com -I 10.162.6.10 -S -u /medicaid
HTTP OK: HTTP/1.1 302 Moved Temporarily - 309 bytes in 0.018 second response time |time=0.017625s;;;0.000000 size=309B;;;0

[icinga@pl-bnaaeraap469 ~]$ /usr/lib64/nagios/plugins/check_http -C 30,15 -H riskview.changehealthcare.com -I 10.162.6.10 -S -u /medicaid
SSL OK - Certificate ‘bna.dcasolutions.com’ will expire in 643 days on 2022-05-30 18:59 -0500/CDT.

Hi,

check_http has the -v parameter for verbose output.

You can set the custom variable http_onredirect = "follow".The check plugin will then follow redirects.

Best regards
Michael

Thanks Michael for all the help so far!

-v shows me correctly that it is checking the full URL and when I add http_onredirect = “follow” like following, my check shows critical state:

vars.vhosts[“http riskview.changehealthcare.com”] = {
http_vhost = “riskview.changehealthcare.com
http_uri = “/medicaid”
http_onredirect = “follow”
http_ssl = true
http_sni = true
}

I have another question in continuation, Now I have a webhost “pl-bnaaeraws401.dcasolutions.com” with three vhosts with ipaddresses - “riskview.changehealthcare.com”-10.162.6.10
eri.changehealthcare.com”- 10.162.6.11 and
edge.changehealthcare.com/edge" - 10.162.6.12

How can I add monitoring for all three web URLs of these vhosts together under one webserver. This is how I added it with webserver pointing to only 1 ipaddress out of 3 vhosts, please check if this is the correct approach:

object Host “pl-bnaaeraws401.dcasolutions.com”{
import “generic-host”

address = "10.162.6.10"

vars.os = "Linux"
vars.osversion = "RHEL 7"

vars.vhosts["http riskview.changehealthcare.com/medicaid"] = {
            http_vhost = "riskview.changehealthcare.com"
            http_uri = "/medicaid"
            http_ssl = true
            http_sni = true
    }

    vars.vhosts["certificate riskview.changehealthcare.com/medicaid"] = {
        http_vhost = "riskview.changehealthcare.com"
        http_uri = "/medicaid"
        http_onredirect = "follow"
            http_ssl = true
            http_sni = true
            http_certificate = "30,15"
    }
vars.vhosts["http eri.changehealthcare.com/eri-uow"] = {
            http_vhost = "eri.changehealthcare.com"
            http_uri = "/eri-uow"
            http_ssl = true
            http_sni = true
    }

    vars.vhosts["certificate eri.changehealthcare.com/eri-uow"] = {
        http_vhost = "eri.changehealthcare.com"
        http_onredirect = "follow"
        http_uri = "/eri-uow"
            http_ssl = true
            http_sni = true
            http_certificate = "30,15"
    }
vars.vhosts["http edge.changehealthcare.com/edge"] = {
            http_vhost = "edge.changehealthcare.com"
            http_uri = "/edge"
            http_ssl = true
            http_sni = true
    }

    vars.vhosts["certificate edge.changehealthcare.com/edge"] = {
        http_vhost = "edge.changehealthcare.com"
        http_onredirect = "follow"
        http_uri = "/edge"

Hi,

the host object for the three vhosts looks good to me. You can add http_address to the configuration so you set the IP addresses for the vhosts. By default the check_http check plugin will use the address from the host object since it is defined in the ITL this way.

Have a look into the documentation for more possible custom variables to modify the http check:

https://icinga.com/docs/icinga2/latest/doc/10-icinga-template-library/#http

Best regards
Michael

Hi Michael, I am coming back on this thread to have a related question on URL monitoring.

If instead of just checking that the url is up or not and taking 302 redirect as ok which my monitoring setup is doing now, how can I pick a regex in icinga2 to look for on the webpage and then take the status as ok. I know this works in nagios and looking for an equivalent monitoring in icinga2.

example, I have this status page to monitor:
https://riskview.changehealthcare.com/commercial-api/healthcheck
and I want to show my monitoring ok only when the page has regex “OK” on it.

Thanks

Hello @ajabbal

Please open a new topic for each new question, even if they are related :slight_smile:
We try to keep it that way to organising them so they can be found and to avoid duplicate content.

Have a nice day,
Feu