General description of the problem:
I have Icinga2 2.11.3-1 server running in a FreeBSD 11.4 server. The configuration is done in the master server and distributed to the clients. We are monitoring ssh service, however the monitoring is being done to 127.0.0.1 in port 22. Our FreeBSD jails don’t use localhost address, and when it is used, it doesn’t necessary has to be 127.0.0.1. So we try to monitor the public IP in a non standard port.
We are getting false positives that say:
Preformatted text connect to address 127.0.0.1 and port 22: Connection refused
When I manually run the check from IcingaWeb, the alert dissapear and I get a:
SSH OK - OpenSSH_7.5 FreeBSD-20170903 (protocol 2.0)
If it was actually checking 127.0.0.1:22, it should be impossible to get an OK, because we don’t have the 127.0.0.1 IP and we are not using port 22 There are some BSD servers that don’t have a issue. I found a particular case where the server uses a different domain. Most servers are host1.foo.bar, host2.foo.bar… This particular server is otherdomain.com and not a subdomain of foo.bar.
Configuration
Host configuration is allocated in /usr/local/etc/icinga2/zones.d/master/hostname.com.conf
. The basic template for host is this:
object Endpoint "host.foo.bar" {
host = "10.10.10.xx"
}
object Host "host.foo.bar" {
import "generic-host"
address = "10.10.10.xx"
vars.os = "FreeBSD"
vars.ssh_port = 4545
}
object Zone "host.foo.bar" {
endpoints = [ "host.foo.bar", ]
parent = "master"
}
The ssh service check is defined in: /usr/local/etc/icinga2/conf.d/services/ssh.conf
and the original configuration was:
apply Service "ssh" {
import "generic-service"
check_command = "ssh"
vars.port = host.vars.ssh_port
assign where host.address
}
I added vars.ssh_address = host.address
to explicitly say to use the ssh IP.
apply Service "ssh" {
import "generic-service"
check_command = "ssh"
vars.port = host.vars.ssh_port
vars.ssh_address = host.address
assign where host.address
}
I decided it was possible because I see that variable defined in /usr/local/share/icinga2/include/command-plugins.conf
:
...
object CheckCommand "ssh" {
import "ipv4-or-ipv6"
command = [ PluginDir + "/check_ssh" ]
arguments = {
"-p" = {
value = "$ssh_port$"
description = "Port number (default: 22)"
}
"-t" = {
value = "$ssh_timeout$"
description = "Seconds before connection times out (default: 10)"
}
"host" = {
value = "$ssh_address$"
skip_key = true
order = 1
}
"-4" = {
set_if = "$ssh_ipv4$"
description = "Use IPv4 connection"
}
"-6" = {
set_if = "$ssh_ipv6$"
description = "Use IPv6 connection"
}
}
vars.ssh_address = "$check_address$"
vars.check_ipv4 = "$ssh_ipv4$"
vars.check_ipv6 = "$ssh_ipv6$"
}
...
Also I also that variable in the documentation.
The error persists, however the variable was defined as I can see in Icinga2 Web under Custom Variables.
From icinga2 object list
I get the definition for the ssh check command:
Object 'ssh' of type 'CheckCommand':
% declared in '/usr/local/share/icinga2/include/command-plugins.conf', lines 1302:1-1302:25
* __name = "ssh"
* arguments
% = modified in '/usr/local/share/icinga2/include/command-plugins.conf', lines 1307:2-1329:2
* -4
* description = "Use IPv4 connection"
* set_if = "$ssh_ipv4$"
* -6
* description = "Use IPv6 connection"
* set_if = "$ssh_ipv6$"
* -p
* description = "Port number (default: 22)"
* value = "$ssh_port$"
* -t
* description = "Seconds before connection times out (default: 10)"
* value = "$ssh_timeout$"
* host
* order = 1
* skip_key = true
* value = "$ssh_address$"
* command = [ "/usr/local/libexec/nagios/check_ssh" ]
% = modified in '/usr/local/share/icinga2/include/command-plugins.conf', lines 1305:2-1305:39
* env = null
* execute
% = modified in 'methods-itl.conf', lines 19:3-19:23
* arguments = [ "checkable", "cr", "resolvedMacros", "useResolvedMacros" ]
* deprecated = false
* name = "Internal#PluginCheck"
* side_effect_free = false
* type = "Function"
* name = "ssh"
* package = "_etc"
* source_location
* first_column = 1
* first_line = 1302
* last_column = 25
* last_line = 1302
* path = "/usr/local/share/icinga2/include/command-plugins.conf"
* templates = [ "ssh", "plugin-check-command", "ipv4-or-ipv6" ]
% = modified in '/usr/local/share/icinga2/include/command-plugins.conf', lines 1302:1-1302:25
% = modified in 'methods-itl.conf', lines 18:2-18:94
% = modified in '/usr/local/share/icinga2/include/command-plugins.conf', lines 3:1-3:36
* timeout = 60
* type = "CheckCommand"
* vars
* check_address
% = modified in '/usr/local/share/icinga2/include/command-plugins.conf', lines 4:2-13:3
* arguments = [ ]
* deprecated = false
* name = "<anonymous>"
* side_effect_free = false
* type = "Function"
* check_ipv4 = "$ssh_ipv4$"
% = modified in '/usr/local/share/icinga2/include/command-plugins.conf', lines 15:2-15:24
% = modified in '/usr/local/share/icinga2/include/command-plugins.conf', lines 1332:2-1332:31
* check_ipv6 = "$ssh_ipv6$"
% = modified in '/usr/local/share/icinga2/include/command-plugins.conf', lines 16:2-16:24
% = modified in '/usr/local/share/icinga2/include/command-plugins.conf', lines 1333:2-1333:31
* ssh_address = "$check_address$"
% = modified in '/usr/local/share/icinga2/include/command-plugins.conf', lines 1331:2-1331:37
* zone = ""
Here an example of a host definition, also from icinga2 object list
Object 'host.foo.bar' of type 'Host':
% declared in '/usr/local/etc/icinga2/zones.d/master/host.foo.bar.conf', lines 5:1-5:42
* __name = "host.foo.bar"
* action_url = ""
* address = "10.10.10.xx"
% = modified in '/usr/local/etc/icinga2/zones.d/master/host.foo.bar.conf', lines 8:5-8:29
* address6 = ""
* check_command = "hostalive"
% = modified in '/usr/local/etc/icinga2/conf.d/templates/generic-host.conf', lines 6:3-6:29
* check_interval = 60
% = modified in '/usr/local/etc/icinga2/conf.d/templates/generic-host.conf', lines 3:3-3:21
* check_period = ""
* check_timeout = null
* command_endpoint = ""
* display_name = "host.foo.bar"
* enable_active_checks = true
* enable_event_handler = true
* enable_flapping = false
* enable_notifications = true
* enable_passive_checks = true
* enable_perfdata = true
* event_command = ""
* flapping_threshold = 0
* flapping_threshold_high = 30
* flapping_threshold_low = 25
* groups = [ ]
* icon_image = ""
* icon_image_alt = ""
* max_check_attempts = 2
% = modified in '/usr/local/etc/icinga2/conf.d/templates/generic-host.conf', lines 2:3-2:24
* name = "host.foo.bar"
* notes = ""
* notes_url = ""
* package = "_etc"
* retry_interval = 30
% = modified in '/usr/local/etc/icinga2/conf.d/templates/generic-host.conf', lines 4:3-4:22
* source_location
* first_column = 1
* first_line = 5
* last_column = 42
* last_line = 5
* path = "/usr/local/etc/icinga2/zones.d/master/host.foo.bar.conf"
* templates = [ "host.foo.bar", "generic-host" ]
% = modified in '/usr/local/etc/icinga2/zones.d/master/host.foo.bar.conf', lines 5:1-5:42
% = modified in '/usr/local/etc/icinga2/conf.d/templates/generic-host.conf', lines 1:0-1:27
* type = "Host"
* vars
* disks
* disk /
% = modified in '/usr/local/etc/icinga2/conf.d/templates/generic-host.conf', lines 8:3-10:3
* disk_partitions = "/"
* os = "FreeBSD"
% = modified in '/usr/local/etc/icinga2/zones.d/master/host.foo.bar.conf', lines 10:5-10:23
* ssh_port = 4545
% = modified in '/usr/local/etc/icinga2/zones.d/master/host.foo.bar.conf', lines 11:5-11:24
* volatile = false
* zone = "master"
I am sure I am missing something very simple. I am still new too Icinga and trying to understund how things work.
The question would be, how to explicitly define the port and the IP for ssh checks?
Thanks a lot,
Cholan.