# RHEL6 - Custom Commands.conf stops working

**URL:** <https://community.icinga.com/t/rhel6-custom-commands-conf-stops-working/2561>\
**Category:** Icinga 2\
**Created:** [October 22, 2019, 11:07pm UTC](https://community.icinga.com/t/rhel6-custom-commands-conf-stops-working/2561 "2019-10-22T23:07:26Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![fireheadman](https://community.icinga.com/user_avatar/community.icinga.com/fireheadman/32/1037_2.png) [@fireheadman](https://community.icinga.com/u/fireheadman)\
**Post date:** [October 22, 2019, 11:07pm UTC](https://community.icinga.com/t/rhel6-custom-commands-conf-stops-working/2561/1 "2019-10-22T23:07:26Z")

</div>

Trying to resolve strange issue…  
All my RHEL7 clients work flawlessly!  
All my RHEL6 clients work for a few min, then go to a UNKNOWN state and remain that way.

I have some custom service checks that are listed in my commands.conf (see below). The commands.conf is located on my master server in /etc/icinga2/zones.d/global-templates/commands.conf

**All clients have the api.conf set to:**

```
object ApiListener "api" {
    accept_commands = true
    accept_config = true
}

```

**I have also verified the commands.conf is being sent to the clients**

> [user@nagiostst]$ sudo grep -E ‘disk\_usage|mem\_used’ /var/lib/icinga2/api/zones/global-templates/\_etc/commands.conf  
> object CheckCommand “mem\_used” {  
> object CheckCommand “disk\_usage” {

**Example:**  
RHEL6 service checks  
 ![image](https://community.icinga.com/uploads/default/original/2X/8/89388138e5fb95b033f381f079b3a879c3c6a8a5.png)

RHEL7 service checks  
 ![image](https://community.icinga.com/uploads/default/original/2X/a/a50ff642328de3fb311fe5f11d3bc6fe93dac991.png)

**These are the custom commands in commands.conf**

```
object CheckCommand "mem_used" {
    import "plugin-check-command"
    command = [PluginDir + "/check_memory.py"]
    arguments = {
        "-W" = "$mem_warn$"
        "-C" = "$mem_crit$"
    }
}

object CheckCommand "disk_usage" {
    import "plugin-check-command"
    command = [PluginDir + "/check_disk_usage.py"]
    arguments = {
        "-D" = "$disk$"
        "-W" = "$warn$"
        "-C" = "$crit$"
    }
}

```

**The plugins are present in nagios plugins dir on each client.**

> [user@nagiostst]$ ls -la /usr/lib64/nagios/plugins/\*.py  
> -rwxr-xr-x. 1 root root 1854 Oct 9 00:06 /usr/lib64/nagios/plugins/check\_disk\_usage.py  
> -rwxr-xr-x. 1 root root 3476 Oct 9 00:06 /usr/lib64/nagios/plugins/check\_memory.py

**This is the client conf file on the master**

```
object Endpoint "nagiostst" {
	host = "1.2.3.4"
}

object Zone "nagiostst" {
	endpoints = ["nagiostst"]
	parent = "master"
}
object Host "nagiostst" {
	import "icon_rhel_vhost"
	import "unix"
	address = "1.2.3.4"
	vars.os = "linux"
	vars.disks["disk"] = { /* No parameters. */ }
	vars.filesystem["/"] = {}
	vars.filesystem["/export"] = {}
	vars.filesystem["/var"] = {}
	vars.filesystem["/var/log"] = {}
	vars.client_endpoint = name
	vars.program = "unixadm"
	vars.location = "xxxxx"
	vars.type = "virtual"
}

```

---

<div class="post-metadata">

**Author:** ![dnsmichi](https://community.icinga.com/user_avatar/community.icinga.com/dnsmichi/32/7721_2.png) [@dnsmichi](https://community.icinga.com/u/dnsmichi)\
**Post date:** [October 23, 2019, 7:02am UTC](https://community.icinga.com/t/rhel6-custom-commands-conf-stops-working/2561/2 "2019-10-23T07:02:17Z")

</div>

Verify that the daemon actually reads the commands, e.g. with `icinga2 daemon -C -x notice | grep commands.conf`. Also, please share the service apply rules in place here.

Which Icinga versions are involved?

Cheers,  
Michael

---

<div class="post-metadata">

**Author:** ![fireheadman](https://community.icinga.com/user_avatar/community.icinga.com/fireheadman/32/1037_2.png) [@fireheadman](https://community.icinga.com/u/fireheadman)\
**Post date:** [October 23, 2019, 3:40pm UTC](https://community.icinga.com/t/rhel6-custom-commands-conf-stops-working/2561/3 "2019-10-23T15:40:12Z")

</div>

Appreciate the quick response Michael!

Here’s the output:  
**Client - RHEL6** :

> [user@nagiostst]$ sudo icinga2 daemon -C -x notice | grep commands.conf  
> Empty return  
> [user@nagiostst]$ sudo icinga2 -V | head -1  
> icinga2 - The Icinga 2 network monitoring daemon (version: **2.11.0-1** )

**Master** :

> [user@icinga01]$ sudo icinga2 daemon -C -x notice | grep commands.conf  
> [2019-10-23 09:36:20 -0600] notice/ConfigCompiler: Compiling config file: /etc/icinga2/zones.d/global-templates/commands.conf  
> [user@icinga01 ~ 14272]$ sudo icinga2 -V | head -1  
> icinga2 - The Icinga 2 network monitoring daemon (version: **r2.10.5-1** )

**And a RHEL7 Client:**

> [user@unixadm]$ sudo icinga2 daemon -C -x notice | grep commands.conf  
> [2019-10-23 09:40:36 -0600] notice/ConfigCompiler: Compiling config file: /var/lib/icinga2/api/zones/global-templates/\_etc/commands.conf  
> [user@unixadm]$ sudo icinga2 -V | head -1  
> icinga2 - The Icinga 2 network monitoring daemon (version: **r2.10.5-1** )

Here are the services:  
NOTE: the services.conf is located in /etc/icinga2/zones.d/master/services.conf

```
apply Service "Disk Usage: " for (filesystem_name => config in host.vars.filesystem) {
    import "service-check-alarm-settings"
    check_command = "disk_usage"
    command_endpoint = host.vars.client_endpoint
    vars.grafana_graph_disable = true
    vars.disk = filesystem_name
    if (!vars.warn) { vars.warn = "80" }
    if (!vars.crit) { vars.crit = "90" }
    vars += config
    assign where host.address && host.vars.os == "linux"
}

apply Service "Memory Used" {
    import "service-check-alarm-settings"
    check_command = "mem_used"
    command_endpoint = host.vars.client_endpoint
    vars.mem_warn = "80"
    vars.mem_crit = "90"
    enable_notifications = false
    assign where host.address && host.vars.os == "linux"
}

```

---

<div class="post-metadata">

**Author:** ![dnsmichi](https://community.icinga.com/user_avatar/community.icinga.com/dnsmichi/32/7721_2.png) [@dnsmichi](https://community.icinga.com/u/dnsmichi)\
**Post date:** [October 23, 2019, 4:49pm UTC](https://community.icinga.com/t/rhel6-custom-commands-conf-stops-working/2561/4 "2019-10-23T16:49:25Z")

</div>

You might see a bug fixed in 2.11.1. Try upgrading the agent, and purge away `/var/lib/icinga2/api/{zones,zones-stage}/*` then restart.

---

<div class="post-metadata">

**Author:** ![fireheadman](https://community.icinga.com/user_avatar/community.icinga.com/fireheadman/32/1037_2.png) [@fireheadman](https://community.icinga.com/u/fireheadman)\
**Post date:** [October 23, 2019, 9:39pm UTC](https://community.icinga.com/t/rhel6-custom-commands-conf-stops-working/2561/5 "2019-10-23T21:39:25Z")

</div>

Gave that a try…  
Looks much better! HUGE THANKS!

> [user@nagiostst]$ sudo yum -y update  
> [user@nagiostst]$ sudo rm -rf /var/lib/icinga2/api/zones/\*  
> [user@nagiostst]$ sudo rm -rf /var/lib/icinga2/api/zones-stage/\*  
> [user@nagiostst]$ sudo ls /var/lib/icinga2/api/zones  
> [user@nagiostst]$ sudo reboot  
> [user@nagiostst]$ sudo icinga2 -V | head -1  
> icinga2 - The Icinga 2 network monitoring daemon (version: 2.11.1-1)  
> [user@nagiostst]$ sudo icinga2 daemon -C -x notice | grep commands.conf  
> [2019-10-23 15:38:40 -0600] notice/ConfigCompiler: Compiling config file: /var/lib/icinga2/api/zones/global-templates/\_etc/commands.conf

---

<div class="post-metadata">

**Author:** ![dnsmichi](https://community.icinga.com/user_avatar/community.icinga.com/dnsmichi/32/7721_2.png) [@dnsmichi](https://community.icinga.com/u/dnsmichi)\
**Post date:** [October 24, 2019, 7:22am UTC](https://community.icinga.com/t/rhel6-custom-commands-conf-stops-working/2561/6 "2019-10-24T07:22:08Z")

</div>

Cool, thanks for confirming. I’ve added these details to the upgrading docs to allow everyone to follow along.

> <https://github.com/Icinga/icinga2/pull/7588>
