Website and certificate monitoring with Icinga

Hi,

I have an Icinga installation running nicely on a public server and monitoring the basics and NTP synchronization.

The next step is to monitor all the websites I’m hosting. I have half a dozen public servers that are currently hosting various web applications:

  • WordPress
  • OwnCloud
  • Roundcube
  • Dolibarr
  • GEPI
  • default (static) page

On my main production server alone, I have 30 web applications hosted each one as a virtual host with Apache.

Right now I’m wondering what the smartest approach would be to monitor all that stuff with Icinga. Should I create one Icinga service per virtual host and keep track of every single HTTP(S) response? Or should it be sufficient to just track the “default” static page of the server, the idea being that if that page is unreachable, all other web hostings on this machine will be unreachable too?

Let me give you an example. Here’s the default page on my production server. It’s a modified version of the static welcome page:

Here’s my tech blog, hosted on that server:

My OwnCloud installation also on that server:

And our local climbing club’s website, also on that server:

Now I wonder if I should create a separate service in Icinga to monitor the HTTPS response for each and every single hosting, or just check a single hosting like, say, the default server page https://sd-48011.dedibox.fr?

A similar thought goes for certificates. All the certificates for all the domains and subdomains hosted on this server are crammed into one single certificate in /etc/letsencrypt/live/sd-48011.dedibox.fr.

So I think it should be enough the check the certificate validity on the default https://sd-48011.dedibox.fr page. But I’m not sure.

Last but not least: are there other sensible things I can test on a web hosting besides HTTP(S) response and certificate validity?

Cheers from the sunny South of France,

Niki

1 Like

Hello @microlinux!

I’m also running a lot of vhosts on the same IP and even multiple sites (different URIs) under some vhosts.

For me one check is not enough. I have one SSL check per vhost (actually I had to build the monitoring plugins Debian packages by myself because of this one) and one HTTP check per website root URI (e.g. https://smtp.al2klimov.de/mail/ and https://smtp.al2klimov.de/admin/).

Best,
AK

1 Like

Thanks for the heads up. I’m currently following your advice and experimenting with it. Here’s an example of what I did.

On my main server (sd-48011.dedibox.fr) I’m hosting (among many other things) my little company’s website as well as my blog. They’re on two different virtual hosts:

Here’s what I have to monitor the site:

// /etc/icinga2/zones.d/master/website-www.microlinux.fr.conf
//
// Check https://www.microlinux.fr website & certificate
object Service "Website https://www.microlinux.fr" {
  import "generic-service"
  check_command = "http"
  vars.http_address = "www.microlinux.fr"
  vars.http_vhost = "www.microlinux.fr"
  vars.http_ssl = true
  host_name = "sd-48011.dedibox.fr"
  command_endpoint = "sd-48011.dedibox.fr"
}

object Service "Certficate https://www.microlinux.fr" {
  import "generic-service"
  check_command = "http"
  vars.http_address = "www.microlinux.fr"
  vars.http_vhost = "www.microlinux.fr"
  vars.http_ssl = true
  vars.http_certificate = "20,10"
  host_name = "sd-48011.dedibox.fr"
  command_endpoint = "sd-48011.dedibox.fr"
}

And here’s the configuration stanza to monitor the blog:

// /etc/icinga2/zones.d/master/website-blog.microlinux.fr.conf
//
// Check https://blog.microlinux.fr website & certificate
object Service "Website https://blog.microlinux.fr" {
  import "generic-service"
  check_command = "http"
  vars.http_address = "blog.microlinux.fr"
  vars.http_vhost = "blog.microlinux.fr"
  vars.http_ssl = true
  host_name = "sd-48011.dedibox.fr"
  command_endpoint = "sd-48011.dedibox.fr"
}

object Service "Certficate https://blog.microlinux.fr" {
  import "generic-service"
  check_command = "http"
  vars.http_address = "blog.microlinux.fr"
  vars.http_vhost = "blog.microlinux.fr"
  vars.http_ssl = true
  vars.http_certificate = "20,10"
  host_name = "sd-48011.dedibox.fr"
  command_endpoint = "sd-48011.dedibox.fr"
}

After reloading, I get a total of four new entries. Each website has a check for “SSL OK - certificate will expire in xx days” and “HTTP OK”.

As far as I can tell, this looks like a no-nonsense webserver check for me. Please correct me if you have a better suggestion.

If it looks right, then I will edit the corresponding configurations for the remaining two dozen sites that are hosted on this machine.

Cheers,

Niki

Why don’t you just list all of your sites in a host custom var and apply two services?

Could you give me a practical example?

With this kind of answer - which seems to happen pretty often - I suggest you just replace this whole forum with a static redirection to the Icinga documentation.

I’ve read - and reread - the Icinga documentation. And I’m asking here because I don’t know how to do it.

I’m just asking for a configuration example, a snippet to grasp the - often confusing - documentation. If you consider these like your trade secrets, then please don’t participate in a public forum in the first place.

1 Like

Have you read the section my link points to? (Unfortunately not obvious due to how the link is rendered.)

Yes. I’ve read and reread the Icinga documentation, taking extensive notes. The bits I’m asking here are the bits I don’t understand in the documentation.

I feel like the guy asking how much tomato you put into the bolognese sauce, and who gets a link to a Harvard Course in Biochemistry Basics.

I’ll consider the problem as solved, since my actual configuration works, even if it’s a bit klutzy.

Hi.

(Hint: Click on the small arrows to display the details)

To put checks with similar arguments into an array within the host-definition of the host which executes the check, you could do something similar like

this
vars.http_statuscode_targets = [
  "https://www.microlinux.fr",
  "https://blog.microlinux.fr",
// ... your other great vhosts
  ]

The corresponding service could look like this (these are just dummy settings)

corresponding check
apply Service "HTTP Statuscode" for (http_target in host.vars.http_statuscode_targets) {
  import "generic-service"

  check_command = "check_http_statuscode" // i am just an example 
  check_interval = 2m

  name = http_target + "_statuscode_check"

  vars.http_statuscode_target = http_target // CHANGEME to the required ones

  assign where host.vars.http_statuscode_targets
}

If you want to use different arguments, you could put each check into
a dictionary which all the required settings

Using a dictionary
  vars.http_vhosts["https://www.microlinux.fr"] = {
     http_vhost = "https://www.microlinux.fr"
     http_address = "https://www.microlinux.fr"
     http_uri   = "/"
     http_ssl = true
     http_certificate = 30

  }

The corresponding check could look like this:

corresponding service
apply Service for (http_vhost => config in host.vars.http_vhosts) {
  import "generic-service"

  check_command = "http"
  check_interval = 1d
  name = http_vhost + "_cert_check"
  display_name = http_vhost + " cert check"

  enable_perfdata = false

  vars += config
}

Hope this helps.

Greetings

2 Likes

Homerjay, you’re a star ! Thank you very much !

I played around with what you suggested, and here’s what I have now in my host configuration:

  vars.vhosts_ssl = [
    "www.microlinux.fr",
    "blog.microlinux.fr",
    "cloud.microlinux.fr",
    "bsco.microlinux.fr"
  ]

And here’s the service definition:

// /etc/icinga2/zones.d/master/service-websites.conf
//
// Check secure virtual hosts
apply Service "Website " for (vhost in host.vars.vhosts_ssl) {
  import "generic-service"
  check_command = "http"
  vars.http_address = vhost
  vars.http_vhost = vhost
  vars.http_ssl = true
  assign where host.vars.vhosts_ssl
}

apply Service "Certificate " for (vhost in host.vars.vhosts_ssl) {
  import "generic-service"
  check_command = "http"
  vars.http_address = vhost
  vars.http_vhost = vhost
  vars.http_ssl = true
  vars.http_certificate = "20,10"
  assign where host.vars.vhosts_ssl
}

Works like a charm and is much easier to configure indeed. The official documentation would greatly benefit from hands-on examples like this.

Cheers !

Niki

1 Like

Hi again.

You are welcome.
I am very happy that it has helped.

Greetings.

2 Likes

In HTTPS, a TLS handshake takes place first, before the HTTP conversation can begin. Without SNI, then, there is no way for the client to indicate to the server which hostname they’re talking to.
https://www.cloudflare.com/learning/ssl/what-is-sni/

If you have multiple hostnames and you set up your webserver properly (which menas https://someipaddress reject ssl handshake) you have to add this line:
vars.http_sni = true;