Default Service Checks

Icinga 2
1

Hello all

I have a Rocky Linux 8 running Icinga2 version 2.13.2-1. 
Enabled features: api checker mainlog notification

I am new to this version and finding it a little tough figuring out where things are placed. IcingaWeb2 is running on a custom port 20801 on which I am able to logon successfully as well. Since this is a fresh installation, I still haven’t configured any services or host, however I find a default service checks in place like ssh, disk, http etc to the local machine.
The problem is, I am unable to find where these are defined. From the earlier version I used 2.12, I know if there are services or hosts defined, they should be in /etc/icinga2/conf.d/hosts.conf or services.conf. To eliminate confusion, I commented all default checks in both hosts.conf, services.conf and few other configuration files, but Icinga2 continues to check 127.0.0.1 on port 80 and reports an error. Could you please help by letting me know where exactly is this defined…?

Thanks for all your time

BA

2

here you go:
this is the hosts.conf for the settings for a host after a new icinga2 roleout:

here are the services:

here it says create the service for host.vars.http_vhosts

apply Service for (http_vhost => config in host.vars.http_vhosts) {
  import "generic-service"

  check_command = "http"

  vars += config
}

so to change to port you need to look up what check is used in the service, the check command is http it is part of the itl icinga template library:

here you can find a variable setting the http port $http_port$ and you will notice this looks similar to the vars in the hosts.conf

so what your default hosts.conf needs is a change from

object Host NodeName {
....

  vars.http_vhosts["http"] = {
    http_uri = "/"
  }
```
to
```
  vars.http_vhosts["http"] = {
    http_uri = "/"
    http_port = "20801"
  }
```
3

Hi @moreamazingnick thank you for your quick reply. Yes, I did try your suggestion, removed the comments and changed the port number

 vars.http_vhosts["http"] = {
   http_uri = "/"
   http_port = "20801"
  }

However, Icinga still continues to probe port 127.0.0.1 : 80 and reports a connection refused error. Attaching a screenshot FYI

image
image835×237 13.8 KB

4

did you restart icinga2?

5

Hi @moreamazingnick yes, Icinga was restarted several times in the process

sudo systemctl restart icinga2.service

For the sake of testing, took a backup of the default hosts.conf and created a new hosts.conf file with just one single entry

object Host "Tycho" {
	check_command = "hostalive"
	check_interval = 15m
        address = "<Public IP>"
        display_name = "Sample Test"
}

I noticed that whatever changes I make is not taking effect when I login using web. There must be something much more at the fundamental level.

Thanks ,for all the help
BA

6

If you use 'icinga2 daemon -C --dump-objects && icinga2 object list --name YOURHOST you can see, line by line, where the settings or properties are coming from.

1 Like
7

Hi @rivad I took a little time to try various option before replying to you. For some reason the --dump-objects option did not work in my case, however, icinga2 object list --name <hostname> worked. I also trying icinga2 object list | grep 'port = "80"' etc to check where these are defined.
The initial suspect was /etc/icinga2/zones.conf file because the output pointed out this. I edited and double checked this file, but found nothing suspicious. The surprising this is, I had defined a test object in /etc/icinga2/conf.d/hosts.conf although this object appears in the object list output, it does not take effect, when checked from the web, there are only these pre-defined objects, some of which we can’t figure out where they are defined while others which we have defined are not shown either.

Thanks, Best wishes
BA

8

AFAIK, icinga2 daemon -C --dump-objects is a feature of newer versions.

If the objects are out of sync with the web, then I would check if the database connections are clean.

9

Hi @rivad
I gave all this a full check. When I check netstat -tnlp | grep 3306 it appeared that the DB was listening on an IPv6 address. I had this changed by editing /etc/my.cnf.d/mariadb-server.cnf and adding a line bind-address=127.0.0.1
After restarting the service, the netstat returned 127.0.0.1:3306 . Later after logging on through web, I checked the Configuration - > Application - > Resources and validated icinga_ido and icingaweb_db . Similarly had the Access Control Backends checked and it validates without any errors

image
image1688×563 67.7 KB

The icinga2 daemon -C returns

image
image1865×599 125 KB

The situation of the sample host object defined in /etc/icinga2/conf.d/hosts.conf remains unchanged. The web does not show this object and still continues to throw up error stating it is unable to connect to 127.0.0.1:80

Is there anything else that can be tried…? Also as an additional note, despite no errors in DB validation and startup, when I logon through web, it also shows an error under System - > Monitoring Health - > Backend Icinga is not running

image
image1395×599 55.6 KB

Thanks, Best wishes
BA

10

If Icingaweb2 states that the “Backend icinga is not running” it means, that it doesn’t see writes to the DB by the Icinga2 agent. So check the DB connections under /etc/icinga2/features-enabled/.

Also you should have 2 DB schemata, one for icinga (IDO or icingadb with redis) and one for icingaweb2.
The first is required to see icinga objects, checks and there states the second is for the icingaweb2 framework to function and some icingaweb2 modules also require additional schemata in the DB.

In my case it looks like this:

image
The last one is used as icinga director import source.