Monitoring multiple process in a Linux host

Icinga 2
1

Hi.

I am trying to monitor multiple “procs_command” in a single host. For example, I want to monitor the processes “postgres” and “apache2”.

host.conf:

object Endpoint "status.datacentre.example.com" {
}

object Zone "status.datacentre.example.com" {
     endpoints = [ "status.datacentre.example.com" ]
     parent = "master"
}

// Host Objects
object Host "status.datacentre.example.com" {
    import "generic-host"
    check_command = "hostalive"
    address = "192.168.199.6"
    vars.kernel = "ubuntu"
    vars.os = "Linux"
    vars.ssh_port = "22"
    vars.client_endpoint = name //follows the convention that host name == endpoint name
    vars.procs_command = "apache2"
    vars.procs_command = "postgres"

However, on icinga web I can only see the “apache2” process.
How can I monitor multiple processes with Icinga?

services.conf:
    apply Service "Process" {
      import "generic-service"
      check_command = "procs"
      command_endpoint = host.vars.client_endpoint
      assign where host.vars.client_endpoint
    }

Do I need a service definition for every single process that I want to monitor?

Thanks!

2

I think I found the solution.

apply Service for (procs => config in host.vars.local_procs) {
  check_command = "procs"
  vars += config
  command_endpoint = host.vars.client_endpoint
  assign where host.vars.client_endpoint
}

    vars.local_procs["Procs"] = {
        vars.procs_command = [ "master", "redis-server", "apache2", "postgres" ]
}
3

Hi,

The inner vars is not needed here, the final key would be procs_command in each iteration. Still, if you just want to use a list of procs to monitor, I would simplify the apply for loop to use an array instead of a nested dictionary.

Host Array for generating Services

Host Object

object Host "..." {
  ...
  vars.monitored_processes = [ "master", "redis-server", "apache2", "postgres" ]
}

Service Apply For

apply Service "procs-" for (proc_name in host.vars.monitored_processes) {
  check_command = "procs"

  command_endpoint = host.vars.client_endpoint
  assign where host.vars.client_endpoint
}

Service Thresholds from the Host

If you want to specify certain thresholds for each process, change that logic into a nested dictionary like so

object Host "..." {
  ...
  vars.monitored_processes["master"] = {
    procs_warning = "..."
  },
  vars.monitored_processes["redis-server"] = {}
  vars.monitored_processes["apache2"] = {}
  vars.monitored_processes["postgres"] = {}
}

// use the "procs-" prefix for generated object names, to make them unique. The display_name is used by Icinga Web to show the short one.
apply Service "procs-" for (procs => config in host.vars.local_procs) { 
  display_name = procs
  check_command = "procs"
  vars += config //copy over the defined thresholds
  command_endpoint = host.vars.client_endpoint
  assign where host.vars.client_endpoint
}

Ensure to keep the name like procs_warning on the host exactly the same as stated in the docs for procs.

Cheers,
Michael

4

Thanks, @dnsmichi.

However, I was able to find a solution, and because of the “account on hold”, I couldn’t post it before.

services.conf:

apply Service "process-linux-" for (identifier => process in host.vars.processes) {
  import "generic-service"

  if (!process.proc_crit) { process.proc_crit = "1:" }

  vars.procs_warning = process.proc_warn
  vars.procs_critical = process.proc_crit

  if (process.argument) {
    vars.procs_argument = process.argument

    check_command = "procs"

  } else if (process.command) {
    vars.procs_command = process.command

    check_command = "procs"

  }

  display_name = identifier + " Processes"

  command_endpoint = host.vars.client_endpoint
  assign where host.vars.client_endpoint
}

host.conf:

vars.processes["Postgres"] = {
  argument = "/usr/lib/postgresql/10/bin/postgres"
}

vars.processes["Postfix"] = {
  argument = "/usr/lib/postfix/sbin/master"
}

vars.processes["Redis"] = {
  command = "redis-server"
}

Thanks!

1 Like