Can't send external Icinga command: 500 Action execution failed: 'Error: Could not create comment

Version:

icinga2 --version

icinga2 - The Icinga 2 network monitoring daemon (version: 2.11.3-1)

Copyright (c) 2012-2020 Icinga GmbH (https://icinga.com/)
License GPLv2+: GNU GPL version 2 or later <http://gnu.org/licenses/gpl2.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

System information:
  Platform: CentOS Linux
  Platform version: 7 (Core)
  Kernel: Linux
  Kernel version: 3.10.0-1062.12.1.el7.x86_64
  Architecture: x86_64

Build information:
  Compiler: GNU 4.8.5
  Build host: runner-LTrJQZ9N-project-322-concurrent-0

Application information:

General paths:
  Config directory: /etc/icinga2
  Data directory: /var/lib/icinga2
  Log directory: /var/log/icinga2
  Cache directory: /var/cache/icinga2
  Spool directory: /var/spool/icinga2
  Run directory: /run/icinga2

Old paths (deprecated):
  Installation root: /usr
  Sysconf directory: /etc
  Run directory (base): /run
  Local state directory: /var

Internal paths:
  Package data directory: /usr/share/icinga2
  State path: /var/lib/icinga2/icinga2.state
  Modified attributes path: /var/lib/icinga2/modified-attributes.conf
  Objects path: /var/cache/icinga2/icinga2.debug
  Vars path: /var/cache/icinga2/icinga2.vars
  PID path: /run/icinga2/icinga2.pid

Feature list:

# icinga2 feature list
Disabled features: command compatlog debuglog elasticsearch gelf graphite influxdb livestatus opentsdb perfdata statusdata syslog
Enabled features: api checker ido-mysql mainlog notification

Daemon:

# icinga2 daemon -C
[2020-03-18 13:19:44 +0000] information/cli: Icinga application loader (version: 2.11.3-1)
[2020-03-18 13:19:44 +0000] information/cli: Loading configuration file(s).
[2020-03-18 13:19:44 +0000] information/ConfigItem: Committing config item(s).
[2020-03-18 13:19:44 +0000] information/ApiListener: My API identity: mcic-icinga.local
[2020-03-18 13:19:44 +0000] information/ConfigItem: Instantiated 1 ScheduledDowntime.
[2020-03-18 13:19:44 +0000] information/ConfigItem: Instantiated 2 HostGroups.
[2020-03-18 13:19:44 +0000] information/ConfigItem: Instantiated 1 FileLogger.
[2020-03-18 13:19:44 +0000] information/ConfigItem: Instantiated 2 NotificationCommands.
[2020-03-18 13:19:44 +0000] information/ConfigItem: Instantiated 1 NotificationComponent.
[2020-03-18 13:19:44 +0000] information/ConfigItem: Instantiated 12 Notifications.
[2020-03-18 13:19:44 +0000] information/ConfigItem: Instantiated 1 IcingaApplication.
[2020-03-18 13:19:44 +0000] information/ConfigItem: Instantiated 3 Hosts.
[2020-03-18 13:19:44 +0000] information/ConfigItem: Instantiated 1 ApiListener.
[2020-03-18 13:19:44 +0000] information/ConfigItem: Instantiated 1 CheckerComponent.
[2020-03-18 13:19:44 +0000] information/ConfigItem: Instantiated 5 Zones.
[2020-03-18 13:19:44 +0000] information/ConfigItem: Instantiated 3 Endpoints.
[2020-03-18 13:19:44 +0000] information/ConfigItem: Instantiated 1 ApiUser.
[2020-03-18 13:19:44 +0000] information/ConfigItem: Instantiated 1 IdoMysqlConnection.
[2020-03-18 13:19:44 +0000] information/ConfigItem: Instantiated 235 CheckCommands.
[2020-03-18 13:19:44 +0000] information/ConfigItem: Instantiated 3 TimePeriods.
[2020-03-18 13:19:44 +0000] information/ConfigItem: Instantiated 1 UserGroup.
[2020-03-18 13:19:44 +0000] information/ConfigItem: Instantiated 1 User.
[2020-03-18 13:19:44 +0000] information/ConfigItem: Instantiated 19 Services.
[2020-03-18 13:19:44 +0000] information/ConfigItem: Instantiated 3 ServiceGroups.
[2020-03-18 13:19:44 +0000] information/ScriptGlobal: Dumping variables to file '/var/cache/icinga2/icinga2.vars'
[2020-03-18 13:19:44 +0000] information/cli: Finished validating the configuration file(s).

Icinga Web Version:
http://i.imgur.com/VXWTOkE.png

Greetings,

I have the setup above and currently i’m having issues when acknowledging alerts.
I have the following error:
http://i.imgur.com/nVIUpbm.png

On the /var/log/icinga2/icinga2.log i have the following text:

[2020-03-18 13:25:56 +0000] critical/Downtime: Config package broken: Error: Cannot repair package '_api', please check the troubleshooting docs.
[2020-03-18 13:25:56 +0000] critical/ThreadPool: Exception thrown in event handler:
Error: Could not create downtime.
	(0) icinga2: icinga::Downtime::AddDowntime(boost::intrusive_ptr<icinga::Checkable> const&, icinga::String const&, icinga::String const&, double, double, bool, icinga::String const&, double, icinga::String const&, icinga::String const&, icinga::String const&, boost::intrusive_ptr<icinga::MessageOrigin> const&) (+0xb47) [0xa16457]
	(1) icinga2: icinga::ScheduledDowntime::CreateNextDowntime() (+0x497) [0x9cbb97]
	(2) icinga2: icinga::ScheduledDowntime::TimerProc() (+0x223) [0x9cc723]
	(3) /usr/lib64/icinga2/sbin/icinga2() [0x7794ac]
	(4) icinga2: icinga::Timer::Call() (+0x28) [0xae00c8]
	(5) /usr/lib64/icinga2/sbin/icinga2() [0x8fefd3]
	(6) icinga2: boost::asio::detail::executor_op<boost::asio::detail::work_dispatcher<bool icinga::ThreadPool::Post<std::function<void ()> >(std::function<void ()>, icinga::SchedulerPolicy)::{lambda()#1}>, std::allocator<void>, boost::asio::detail::scheduler_operation>::do_complete(void*, std::allocator<void>*, boost::system::error_code const&, unsigned long) (+0x106) [0xbf7776]
	(7) /usr/lib64/icinga2/sbin/icinga2() [0x630e91]
	(8) /usr/lib64/icinga2/sbin/icinga2() [0x6311e2]
	(9) icinga2: boost_asio_detail_posix_thread_function (+0xf) [0x8211bf]
	(10) libpthread.so.0: <unknown function> (+0x7e65) [0x7f8f9e7f0e65]
	(11) libc.so.6: clone (+0x6d) [0x7f8f9e51988d]

I have tryed following the troubleshooting guide ( icinga . com/docs/icinga2/latest/doc/15-troubleshooting/#missing-runtime-objects-hosts-downtimes-etc ) but i do not have the id on the /var/lib/icinga2/api/packages/_api/. Here is the ls -alsh of that same directory on my server:

# ls -alsh /var/lib/icinga2/api/packages/_api/
total 8,0K
   0 drwx------ 2 icinga icinga  43 Mar 12 17:01 .
   0 drwx------ 4 icinga icinga  32 Mar 12 18:08 ..
4,0K -rw-r--r-- 1 icinga icinga 417 Mar 12 17:01 active.conf
4,0K -rw-r--r-- 1 icinga icinga  25 Mar 12 17:01 include.conf

Is there something wrong with my config? Can anyone help out?

Hello @dtferreira and thank you for reporting!

Please share the contents of the files in /var/lib/icinga2/api/packages/_api.

Best,
AK

Hello @Al2Klimov i posted the content of that folder on the original post. Here it is again:

# ls -als /var/lib/icinga2/api/packages/_api
total 8
0 drwx------ 2 icinga icinga  43 Mar 12 17:01 .
0 drwx------ 4 icinga icinga  32 Mar 12 18:08 ..
4 -rw-r--r-- 1 icinga icinga 417 Mar 12 17:01 active.conf
4 -rw-r--r-- 1 icinga icinga  25 Mar 12 17:01 include.conf

Best regards,

… but not the content of the files inside the folder.

1 Like

sorry @Al2Klimov .

active.conf:

if (!globals.contains(“ActiveStages”)) {
globals.ActiveStages = {}
}

if (globals.contains(“ActiveStageOverride”)) {
var arr = ActiveStageOverride.split(“:”)
if (arr[0] == “_api”) {
if (arr.len() < 2) {
log(LogCritical, “Config”, “Invalid value for ActiveStageOverride”)
} else {
ActiveStages[“_api”] = arr[1]
}
}
}

if (!ActiveStages.contains(“_api”)) {
ActiveStages[“_api”] = “”
}

include.conf:

include “*/include.conf”