Hi there, I am new to the Icinga monitoring and trying to set up to monitor AWS env from cloudwatch, I have zero knowledge of this. Can someone point me to the appropriate docs or guide, please?
I get the following error while performing any action from the icinga dashboard icinga2: Can’t connect to the Icinga 2 API: 0 cURL error 7: Failed to connect to localhost port 5665 after 0 ms: Connection refused (see libcurl - Error Codes)
in icingaweb2 the module responsible for monitoring is either “monitoring” or icingadb.
in one or both of these modules there is a configuration page for a transport layer.
there you have to fill out a icinga2 api user
here are some useful links:
https://icinga.com/docs/icinga-web/latest/modules/monitoring/doc/05-Command-Transports/
I’m not sure I understand “icinga dashboard icinga2” but as @moreamazingnick mentioned, you need to setup the Icinga2 API and tell Icingaweb2 and possibly others the credentials for the error to go way.
Thanks for getting back. Unfortunately, we encountered another issue while setting up the Icinga web. while setting up with the DB resource we get this message:
The database you've configured to use for Icinga Web 2 seems to be the one of Icinga. Please be aware that this database configuration is supposed to be used for Icinga Web 2's configuration and that it is highly recommended to not mix different schemas in the same database. If this is intentional, you can skip the validation and ignore this warning. If not, please provide a different database.
We have created an entirely new DB named icingaweb but still that message pops out. not sure if we have missed anything.
To add more we get the below message after finishing up the setup:
General configuration has been successfully written to: /etc/icingaweb2/config.ini
Authentication configuration has been successfully written to: /etc/icingaweb2/authentication.ini
Account "icinga2" has been successfully defined as initial administrator.
User Group Backend configuration has been successfully written to: /etc/icingaweb2/groups.ini
Unable to create user group "Administrators". An error occured:
ERROR: Zend_Db_Statement_Exception in /usr/share/icingaweb2/library/vendor/Zend/Db/Statement/Pdo.php:225 with message: SQLSTATE[42P01]: Undefined table: 7 ERROR: relation "icingaweb_group" does not exist
LINE 1: SELECT COUNT(*) AS cnt FROM icingaweb_group AS g WHERE (g.na...
^, query was: SELECT COUNT(*) AS cnt FROM icingaweb_group AS g WHERE (g.name = 'Administrators')
Resource configuration has been successfully written to: /etc/icingaweb2/resources.ini
Monitoring backend configuration has been successfully written to: /etc/icingaweb2/modules/monitoring/backends.ini
Resource configuration has been successfully updated: /etc/icingaweb2/resources.ini
Command transport configuration has been successfully created: /etc/icingaweb2/modules/monitoring/commandtransports.ini
Monitoring security configuration has been successfully created: /etc/icingaweb2/modules/monitoring/config.ini
Module "monitoring" could not be enabled. An error occured:
ERROR: Icinga\Exception\SystemPermissionException in /usr/share/php/Icinga/Application/Modules/Manager.php:295 with message: Cannot enable module "monitoring". Check the permissions for the enabledModules directory: /etc/icingaweb2/enabledModules
What is your /etc/icingaweb2/resources.ini looking like? Mine looks like this:
[icinga_web2]
type = "db"
db = "mysql"
host = "host"
port = "3306"
dbname = "icinga_web2"
username = "icinga"
password = "PW"
charset = "utf8mb4"
[icinga_ido]
type = "db"
db = "mysql"
host = "host"
port = "3306"
dbname = "icinga_ido"
username = "icinga"
password = "PW"
use_ssl = "0"
The first one if for Icingaweb2 and the second one is the one Icinga uses.
If you fixed the config but the error still persists, maybe there are leftover tables in the icinga_ido DB.
Hi Dominik, thank you for your reply after setting up with the permissions, now we are getting the below error after logging in to icinga web
SQLSTATE[42P01]: Undefined table: 7 ERROR: relation “service” does not exist
LINE 1: SELECT COUNT(*) AS cnt FROM service INNER JOIN service_state…
#0 /usr/share/icinga-php/ipl/vendor/ipl/sql/src/Connection.php(402): PDOStatement->execute()
#1 /usr/share/icinga-php/ipl/vendor/ipl/sql/src/Connection.php(418): ipl\Sql\Connection->prepexec()
#2 /usr/share/icinga-php/ipl/vendor/ipl/orm/src/Query.php(704): ipl\Sql\Connection->select()
#3 /usr/share/icingaweb2/modules/icingadb/application/controllers/ServicesController.php(130): ipl\Orm\Query->count()
#4 /usr/share/icingaweb2/modules/icingadb/library/Icingadb/Web/Controller.php(492): Icinga\Module\Icingadb\Controllers\ServicesController->indexAction()
#5 /usr/share/php/Icinga/Web/Controller/Dispatcher.php(76): Icinga\Module\Icingadb\Web\Controller->dispatch()
#6 /usr/share/icingaweb2/library/vendor/Zend/Controller/Front.php(937): Icinga\Web\Controller\Dispatcher->dispatch()
#7 /usr/share/php/Icinga/Application/Web.php(290): Zend_Controller_Front->dispatch()
#8 /usr/share/php/Icinga/Application/webrouter.php(105): Icinga\Application\Web->dispatch()
#9 /usr/share/icingaweb2/public/index.php(4): require_once(String)
#10 {main}
Just curious about what I have missed…
Looks to me that something went wrong with the schema creation or when you added the icingaweb2 db to the same schema.
As you are still setting up and have no data to loose, I would just wipe the DB and rebuild the schema according to the step in the installation instruction.
Also, do you use the now IcingaDB or the IDO DB like I do?
Hi Dominik,
Now I am using the IcingaDB only.
is the IDO DB required?
I followed the https://host/icingaweb2/setup wizard after finishing that page is gone. if I remove the icingaweb db how can I rebuild it?
No, It’s IcingaDB and/or IDO DB.
I would still restart with clean icingaweb2 DB and IcingaDB also you need the right resources in the icingaweb2 config.
Thank you Dominik, finally a refresh install with IDO DB marks a successful deployment of the application.
Now coming back to the first question, I would like to monitor the AWS ec2 instance without installing any agents on any of the hosts, is this possible by any chance?
Icinga 2 API is powerful and the AWS API provides some metrics that can be used for monitoring.
GitHub - Icinga/icingaweb2-module-aws: Import AWS hosts into Icinga Director can be a starting point.
Thanks, Dominik,
Things have started to make sense now, thanks for all the support really appreciate it.
Regards
Shiv
Hi Dominik,
I am facing the below issues while monitoring mount points on aws asg instances.
I have created a command under - Icinga director->commands
check_disk -w 80% -c 90% -p /var/lib/prod
However when I assign it to a host is sends the below error:
DISK CRITICAL - /var/lib/prod is not accessible: No such file or directory
Would you know how can we resolve this problem?
Do you have a Icinga2 agent on your aws instances? If not, this approach can’t work!
I would try to use a check against the AWS API to get this data and put the service using the check into the template of the hosts that the Director creates from the icingaweb2 aws module import source.
alternatively you can try to use SNMP or automate the installation of the Icinga2 agent on your AWS hosts.
Hi Dominik,
We do not have the agent installed, is there any possibility to monitor it without the agents being installed in the EC2 instances?
EC2 API (?), SNMP, check by SSH and such stuff works without the Icinga2 agent.
By applying the by ssh it gives the below error 
not sure how to use the EC2 API yet =, will do more findings on that.
Plugin Output
Error: Invalid type in for expression: Empty
Location: in /usr/share/icinga2/include/command-plugins.conf: 2205:4-2207:4
/usr/share/icinga2/include/command-plugins.conf(2203):
/usr/share/icinga2/include/command-plugins.conf(2204): var escaped_args = []
/usr/share/icinga2/include/command-plugins.conf(2205): for (arg in resolve_arguments(command, arguments)) {
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
/usr/share/icinga2/include/command-plugins.conf(2206): escaped_args.add(escape_shell_arg(arg))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
/usr/share/icinga2/include/command-plugins.conf(2207): }
^^^^
/usr/share/icinga2/include/command-plugins.conf(2208): return escaped_args.join(" ")
/usr/share/icinga2/include/command-plugins.conf(2209): }}
I guess “Render as string” could fix the error.
Maybe you can find EC2 API based checks on the exchanges or GitHub.
Hi Dominik,
Thanks for that after setting the "Render as string” I get the below now:
DISK CRITICAL - /var/lib/prod is not accessible: No such file or directory
However, the disk is there.
is not accessible: No such file or directory
This could also mean that your check can’t list/traverse the path. So maybe you need a sudo in your check command or modify rights.