# Different kinds of service/http checks in multiple servers

**URL:** <https://community.icinga.com/t/different-kinds-of-service-http-checks-in-multiple-servers/1592>\
**Category:** Service Monitoring\
**Created:** [July 1, 2019, 10:54pm UTC](https://community.icinga.com/t/different-kinds-of-service-http-checks-in-multiple-servers/1592 "2019-07-01T22:54:02Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![mani.vajrala](https://community.icinga.com/letter_avatar_proxy/v4/letter/m/0ea827/32.png) [@mani.vajrala](https://community.icinga.com/u/mani.vajrala)\
**Post date:** [July 1, 2019, 10:54pm UTC](https://community.icinga.com/t/different-kinds-of-service-http-checks-in-multiple-servers/1592/1 "2019-07-01T22:54:02Z")

</div>

Hi Team,

In my environment, we have almost 200-300 servers are there. I have been struggling to monitor service & http checks in multiple servers… Currently for every host i have been adding one separate host config file… For example i have got the requirement to monitor A & B services that are exists in all servers. A & B both services exists in around 50 servers. A service exists in 70 servers & B service exists in 80 Servers.  
This is so much of work if i do update custom attribute in every host conf file… Here the configuration files looks like.

**Host.config file**

```auto
object Endpoint "<hostname>" {
}

object Zone "<hostname>" {
     endpoints = ["<hostname>"]
     parent = "master"
}

// Host Objects
object Host "<hostname>" {
    check_command = "hostalive"
    address = " *********"
    vars.os="Windows"
     vars.service1 ="A"
    vars.service2 ="B"
    vars.client_endpoint = name //follows the convention that host name == endpoint name
// Custom Optional check - START
    vars.local_disks["/app Filesystem"] = {
       disk_partitions = "/app"
  }

    vars.local_http_vhosts["http"] = {
       http_uri = "/"
  }
  vars.nscp_boot = false
    vars.notification["mail"] = {
        groups = ["icingausers"]
}
    vars.local_tcp_port["tcp"] ={
       tcp_port = "23"
       service_name = "Telnet Check"
       port_number = "Port 23"

```

**Server config file**

```auto
apply Service "A-service" {
  check_command = "service-windows"
  vars.service_win_service = "A"
  notes = "Service check for A"
  command_endpoint = host.vars.client_endpoint
  assign where host.vars.client_endpoint && host.vars.service1 == "A"
}
apply Service "B-service" {
  check_command = "service-windows"
  vars.service_win_service = "B"
  notes = "Service check for B"
  command_endpoint = host.vars.client_endpoint
  assign where host.vars.client_endpoint && host.vars.service2 == "B"
}

```

If i do this way… This is taking so much of work & this is the only way we have?

Can someone please advise me the best way of monitoring services in multiples servers? thanks in advance

---

<div class="post-metadata">

**Author:** ![dnsmichi](https://community.icinga.com/user_avatar/community.icinga.com/dnsmichi/32/7721_2.png) [@dnsmichi](https://community.icinga.com/u/dnsmichi)\
**Post date:** [July 2, 2019, 7:04am UTC](https://community.icinga.com/t/different-kinds-of-service-http-checks-in-multiple-servers/1592/2 "2019-07-02T07:04:25Z")

</div>

Hi,

AFAIK we had this discussion in another topic already, where I had suggested to hide these things in host templates. With importing the host templates into specific hosts, you can create generic apply rules.

## Templates and Apply

```auto
template Host "basic-host" {
  check_command = "hostalive"
}

template Host "windows-host-service-A" {
  vars.service = "A"
}

template Host "windows-host-service-B" {
   vars.service = "B"
}

template Host "windows-host-service-all" {
  import "windows-host-service-A"
  import "windows-host-service-B"
}

object Host "host-A" {
  import "base-host"
  import "windows-host-service-A" 
}

object Host "host-B" {
  import "base-host"
  import "windows-host-service-B" 
}

object Host "host-all" {
  import "base-host"
  import "windows-host-service-all"
}

```

Base on the template tree, you’ll automatically get certain attributes and the imported templates stored in the `templates` attribute.

This detail can be used to create generic service apply rules.

```auto
apply Service "service-A" {
  check_command = "service-windows"
  vars.service_win_service = host.vars.service //access the host custom vars

  command_endpoint = host.vars.client_endpoint

  assign where host.vars.service == "A" && host.vars.client_endpoint
}

apply Service "service-B" {
  check_command = "service-windows"
  vars.service_win_service = host.vars.service //access the host custom vars

  command_endpoint = host.vars.client_endpoint

  assign where host.vars.service == "B" && host.vars.client_endpoint
}

```

## Advanced: Apply for

Based on the example above, you can use an array with service names hidden in the host template tree.  
The idea is to only have one apply for rule, and the hosts as information provider with `host.vars.services`. Bonus: Arrays can be extended from template inheritance.

```auto
template Host "basic-host" {
  check_command = "hostalive"
}

template Host "windows-host-service-A" {
  vars.services += ["A"] //note the additive assignment operator, and also adding a new array element
}

template Host "windows-host-service-B" {
  vars.services += ["B"] //note the additive assignment operator, and also adding a new array element
}

template Host "windows-host-service-all" {
  import "windows-host-service-A"
  import "windows-host-service-B"
}

object Host "host-A" {
  import "base-host"
  import "windows-host-service-A" 
}

object Host "host-B" {
  import "base-host"
  import "windows-host-service-B" 
}

object Host "host-all" {
  import "base-host"
  import "windows-host-service-all"
}

```

This technique uses a little DSL trick: Whether `services` is initialized as array already, it adds a new array element with the additive assignment operator `+=`. If the value was previously undefined, the added array element initializes this variable as array, with one element.

The resulting host objects now either have one array element with `A` or `B`, and the `host-all` object has `["A", "B"]`.

You can test such things in the debug console:

```auto
$ icinga2 console
Icinga 2 (version: v2.10.5-854-gf72ef9360)
Type $help to view available commands.
<1> => var services
null
<2> => services
null
<3> => services += ["A"]
null
<4> => services
["A"]
<5> => services += ["B"]
null
<6> => services
["A", "B"]

```

This detail can be used to build an apply for loop over this host scoped custom variable array.

```auto
apply for "service-" (s in host.vars.services) {
  check_command = "service-windows"

  vars.service_win_service = s //take the loop iteration variable value

  command_endpoint = host.vars.client_endpoint

  assign where host.vars.client_endpoint //In addition to host.vars.services being not empty, this condition must be met as well.
}

```

Specific notes, excerpt from the [docs](https://icinga.com/docs/icinga2/latest/doc/03-monitoring-basics/#using-apply-for-rules):

- `service-` + the loop iteration variable value for `s` results in the final name
- `host.vars.services` contains the array elements as service name strings
- `s` can be used inside the apply for context, e.g. to create the display\_name or, more importantly, to assign it to a custom variable required as parameter.
- the apply for loop will only be evaluated if `host.vars.services` contains 1+ elements. In addition to creating service objects, the `client_endpoint` custom variable must also be set. This is checked again with `assign where`.

Note: I did not test the above, may contain minor typos. You’ll get the idea though.

Cheers,  
Michael

---

<div class="post-metadata">

**Author:** ![mani.vajrala](https://community.icinga.com/letter_avatar_proxy/v4/letter/m/0ea827/32.png) [@mani.vajrala](https://community.icinga.com/u/mani.vajrala)\
**Post date:** [July 2, 2019, 10:42pm UTC](https://community.icinga.com/t/different-kinds-of-service-http-checks-in-multiple-servers/1592/3 "2019-07-02T22:42:49Z")

</div>

yes… Now i understood… thanks a lot… One question as you mentioned object definitions like

object Host “host-A” {  
import “base-host”  
import “windows-host-service-A”  
}

object Host “host-B” {  
import “base-host”  
import “windows-host-service-B”  
}

object Host “host-all” {  
import “base-host”  
import “windows-host-service-all”  
}

is this object defintions are just like host confi files that made? I have to do the template import in the below host config file for all servers? I have made 200 config files for 200 servers…

object Endpoint “” {  
}

object Zone “” {  
endpoints = [“”]  
parent = “master”  
}

// Host Objects  
object Host “” {  
check\_command = “hostalive”  
address = “\*\*\*\*\*\*\*\*\*”  
vars.os=“Windows”  
vars.service1 =“A”  
vars.service2 =“B”  
vars.client\_endpoint = name //follows the convention that host name == endpoint name  
// Custom Optional check - START  
vars.local\_disks["/app Filesystem"] = {  
disk\_partitions = “/app”  
}

```
vars.local_http_vhosts["http"] = {
   http_uri = "/"

```

}  
vars.nscp\_boot = false  
vars.notification[“mail”] = {  
groups = [“icingausers”]  
}  
vars.local\_tcp\_port[“tcp”] ={  
tcp\_port = “23”  
service\_name = “Telnet Check”  
port\_number = “Port 23”

---

<div class="post-metadata">

**Author:** ![dnsmichi](https://community.icinga.com/user_avatar/community.icinga.com/dnsmichi/32/7721_2.png) [@dnsmichi](https://community.icinga.com/u/dnsmichi)\
**Post date:** [July 3, 2019, 6:21am UTC](https://community.icinga.com/t/different-kinds-of-service-http-checks-in-multiple-servers/1592/4 "2019-07-03T06:21:55Z")

</div>

> [@mani.vajrala](#):
>
> is this object defintions are just like host confi files that made? I have to do the template import in the below host config file for all servers? I have made 200 config files for 200 servers…

Put the templates into a file called `templates.conf`, not necessarily filled with Host templates but others as well. If you need that template on a satellite endpoint where the checks are executed, you might put them into a global zone as well.

Depending on your file structure, either put all hosts into a single file (decreases config compiler load with 1 vs many files), or have them separated with `FQDN.conf` to find them easier.

Typically, I would also advise to put **everything** into a template which is used multiple times in your hosts. `local_tcp_port` for instance looks like as if this would be used commonly, hide that in an imported template. Same goes for `notification` with mail, groups, icingausers thing. Or `nscp_boot`.

My host objects typically look like:

- import template tree
- check\_command, if not already imported
- address
- special custom variables which override the template imports

Short and readable. The detailed attribute view is what you get with `object list` and the REST API anyways.

Cheers,  
Michael
