Notification - not send out to any contact

Hello there:)

After reading lots of this friendly community I am still stuck in getting notifications automatically out of director - custom, i.e. manually sent notifications are about OK.

My setup:

  • Icinga Web 2 Version 2.7.1
  • director 1.6.2
  • monitoring 2.7.1

To be monitored are about 40 linux-containers (LXC) and their hosts (LXD) at two different locations.

User-, Host-, TimePeriod-Templates/Objects and Notification rules are defined.

When triggering an outage (say: stop container) and the opposite (start container again), the event is logged both for host- and service-events:

[2019-10-16 13:57:24 +0200] information/Checkable: Checkable 'testing_art' has 1 notification(s). Checking filters for type 'Problem', sends will be logged.
[2019-10-16 14:35:56 +0200] information/Checkable: Checkable 'testing_art' has 1 notification(s). Checking filters for type 'Recovery', sends will be logged.
[2019-10-16 08:14:40 +0200] information/Checkable: Checkable 'dev.xxx.yyy.ch!apt_sv' has 1 notification(s). Checking filters for type 'Problem', sends will be logged.
[2019-10-16 08:52:56 +0200] information/Checkable: Checkable 'dev.xxx.yyy.ch!apt_sv' has 1 notification(s). Checking filters for type 'Recovery', sends will be logged.

But no notification is sent…

My user-config:

Interestingly the DOWN-Events trigger no notification - only the UP-Events do:

Finally i noticed a missing “legacy-timeperiod” which might be related to the problem?!


icinga2 daemon looks good:

root@nuc2019:~# icinga2 daemon -C
[2019-10-16 20:58:24 +0200] information/cli: Icinga application loader (version: r2.11.0-1)
[2019-10-16 20:58:24 +0200] information/cli: Loading configuration file(s).
[2019-10-16 20:58:24 +0200] information/ConfigItem: Committing config item(s).
[2019-10-16 20:58:24 +0200] information/ApiListener: My API identity: dev.xxx.yyy
[2019-10-16 20:58:25 +0200] information/ConfigItem: Instantiated 1 FileLogger.
[2019-10-16 20:58:25 +0200] information/ConfigItem: Instantiated 2 NotificationCommands.
[2019-10-16 20:58:25 +0200] information/ConfigItem: Instantiated 1 NotificationComponent.
[2019-10-16 20:58:25 +0200] information/ConfigItem: Instantiated 21 Notifications.
[2019-10-16 20:58:25 +0200] information/ConfigItem: Instantiated 1 IcingaApplication.
[2019-10-16 20:58:25 +0200] information/ConfigItem: Instantiated 2 HostGroups.
[2019-10-16 20:58:25 +0200] information/ConfigItem: Instantiated 9 Hosts.
[2019-10-16 20:58:25 +0200] information/ConfigItem: Instantiated 1 Comment.
[2019-10-16 20:58:25 +0200] information/ConfigItem: Instantiated 1 CheckerComponent.
[2019-10-16 20:58:25 +0200] information/ConfigItem: Instantiated 5 Zones.
[2019-10-16 20:58:25 +0200] information/ConfigItem: Instantiated 2 Endpoints.
[2019-10-16 20:58:25 +0200] information/ConfigItem: Instantiated 2 ApiUsers.
[2019-10-16 20:58:25 +0200] information/ConfigItem: Instantiated 1 UserGroup.
[2019-10-16 20:58:25 +0200] information/ConfigItem: Instantiated 1 ApiListener.
[2019-10-16 20:58:25 +0200] information/ConfigItem: Instantiated 1 IdoMysqlConnection.
[2019-10-16 20:58:25 +0200] information/ConfigItem: Instantiated 237 CheckCommands.
[2019-10-16 20:58:25 +0200] information/ConfigItem: Instantiated 2 TimePeriods.
[2019-10-16 20:58:25 +0200] information/ConfigItem: Instantiated 2 Users.
[2019-10-16 20:58:25 +0200] information/ConfigItem: Instantiated 12 Services.
[2019-10-16 20:58:25 +0200] information/ConfigItem: Instantiated 1 ServiceGroup.
[2019-10-16 20:58:25 +0200] information/ScriptGlobal: Dumping variables to file '/var/cache/icinga2/icinga2.vars'
[2019-10-16 20:58:25 +0200] information/cli: Finished validating the configuration file(s).

does anyone got an idea?

Best Regards, stephan

ok -
i’ve already read other posts about similar probs (like Director Notifications.conf).

The ‘commands.conf’ is available to director: /etc/icinga2/conf.d/ contains api-users.conf and commands.conf only.

i’ve exchanged ‘hostalive’ for ‘ping4’ - since mostly i want to try to start simple (check host UP/Down and notify acc. …) - no change in the result.

After enabling ‘debug-mode’ (# icinga2 feature enable debuglog) the following is shown after ‘host-DOWN’ event:

...
[2019-10-17 07:58:00 +0200] information/Checkable: Checkable 'testing_dev' has 1 notification(s). Checking filters for type 'Problem', sends will be logged.
[2019-10-17 07:58:00 +0200] notice/Notification: Attempting to send notifications of type 'Problem' for notification object 'testing_dev!host_notification_rule'.
[2019-10-17 07:58:00 +0200] notice/ApiListener: Relaying 'event::SendNotifications' message
[2019-10-17 07:58:00 +0200] notice/Notification: Not sending notifications for notification object 'testing_dev!host_notification_rule': after specified end time (less than 1 millisecond)
[2019-10-17 07:58:00 +0200] notice/ApiListener: Relaying 'event::SetForceNextNotification' message
[2019-10-17 07:58:00 +0200] debug/IdoMysqlConnection: Query: UPDATE icinga_hoststatus SET acknowledgement_type = '0',  active_checks_enabled = '1',  check_command = 'hostalive',  check_source = 'master.xxx.yyy',  check_timeperiod_object_id = 312,  check_type = '0',  current_check_attempt = '1',  current_notification_number = '0',  current_state = '1',  endpoint_object_id = 246,  event_handler_enabled = '1',  execution_time = '3.070907',  flap_detection_enabled = '1',  has_been_checked = '1',  host_object_id = 318,  instance_id = 1,  is_flapping = '0',  is_reachable = '1',  last_check = FROM_UNIXTIME(1571291880),  last_hard_state = '1',  last_hard_state_change = FROM_UNIXTIME(1571291880),  last_notification = FROM_UNIXTIME(1571290866),  last_state_change = FROM_UNIXTIME(1571291759),  last_time_down = FROM_UNIXTIME(1571291880),  last_time_up = FROM_UNIXTIME(1571291456),  latency = '0.001303',  long_output = '',  max_check_attempts = '3',  next_check = FROM_UNIXTIME(1571292171),  next_notification = FROM_UNIXTIME(1571292341),  normal_check_interval = '5',  notifications_enabled = '1',  original_attributes = 'null',  output = 'CRITICAL - Host Unreachable (10.0.0.99)',  passive_checks_enabled = '1',  percent_state_change = '15.700000',  perfdata = '',  problem_has_been_acknowledged = '0',  process_performance_data = '1',  retry_check_interval = '1',  scheduled_downtime_depth = '0',  should_be_scheduled = '1',  state_type = '1',  status_update_time = FROM_UNIXTIME(1571291880) WHERE host_object_id = 318
...

So it looks like a timing-problem?!

Best regards, sg

Hi,

thanks for the kind words, a friendly community is all what matters :slight_smile:

The times attribute is likely wrong, end must not be set to 0. And you should try to remove begin as well, allowing the notification to fire all the time and not one-shot. You can enable this later on once the notifications work in general.

Cheers,
Michael

Thanks Michael :slight_smile:

After removing the times-attributes, notifications are generated as wanted.

Best Regards, stephan

1 Like