Strange behaviour of notifications

Icinga 2
1

Hi everyone

I want to ask for your help in order to understand a very strange behaviour of the notifications.

Checking /var/log/icinga2/debug.log I noticed that some notification are sent correctly and some aren’t, even if they are linked to the same problem, same host or same service, and nothing has changed in between!
I left the system running with default check_interval (so every half an hour) at night, so nobody changed nothing in that time, because nobody was working, and still some notifications are sent and some aren’t.

The error regarding an host is reported like this on /var/log/icinga2/debug.log (the error regarding a service is very similar):

[2020-10-06 06:29:02 +0200] warning/PluginNotificationTask: Notification command for object ‘hostname’ (PID: 9780, arguments: ‘/etc/icinga2/scripts/mail-host-notification.sh’ ‘-4’ ‘ip address’ ‘-6’ ‘’ ‘-b’ ‘’ ‘-c’ ‘’ ‘-d’ ‘2020-10-06 06:29:01 +0200’ ‘-f’ ‘sender’ ‘-l’ ‘hostname’ ‘-n’ ‘hostname’ ‘-o’ ‘output’ ‘-r’ ‘receiver’ ‘-s’ ‘state’ ‘-t’ ‘notification type’ ‘-v’ ‘false’) terminated with exit code 1, output: + …

I replaced some information with general information. The output is always different: sometimes it is part of the body of the email, sometimes it is “detecting MIME Type…”. The email doesn’t have attachments.

The exit code 1 in /etc/icinga2/scripts/mail-host-notification.sh happens only in these two situations:

if [ -z “which $MAILBIN” ] ; then
echo “$MAILBIN not found in $PATH. Consider installing it.”
exit 1
fi

Function helpers

Usage() {
cat << EOF

Required parameters:
-d LONGDATETIME ($icinga.long_date_time$)
-l HOSTNAME ($host.name$)
-n HOSTDISPLAYNAME ($host.display_name$)
-o HOSTOUTPUT ($host.output$)
-r USEREMAIL ($user.email$)
-s HOSTSTATE ($host.state$)
-t NOTIFICATIONTYPE ($notification.type$)

Optional parameters:
-4 HOSTADDRESS ($address$)
-6 HOSTADDRESS6 ($address6$)
-b NOTIFICATIONAUTHORNAME ($notification.author$)
-c NOTIFICATIONCOMMENT ($notification.comment$)
-i ICINGAWEB2URL ($notification_icingaweb2url$, Default: unset)
-f MAILFROM ($notification_mailfrom$, requires GNU mailutils (Debian/Ubuntu) or mailx (RHEL/SUSE))
-v ($notification_sendtosyslog$, Default: false)

EOF
}

Error() {
if [ “$1” ]; then
echo $1
fi
Usage;
exit 1;
}

I checked and $MAILBIN is present in $PATH.

These checks are referred to two different satellites.
Icinga2 version on master: r2.12.0-1
Icinga2 version on satellite1: r2.12.0-1
Icinga2 version on satellite2: r2.12.0-1

If anyone knows something, it would help me understand this problem so much!

2

Hi,

which server is the notifier your master or the satellites (where is the notificiation feature enabled)? Do you find something in the mail or syslog about the mail program?

3

Is this a copy&paste error?

4

The master is the notifier.
We are using mailsend-go and it doesn’t have a log, it prints on standard output and it hasn’t.
If I lauch the script manually it works and there are no errors so I don’t think I would find something in the log of the mail program anyway.

5

Yes, I’m sorry. Now it is correct.

6

My idea was following: if you have more icinga-servers than the master which is sending notification, that on one server the script is working correct and at one not. Or the notification feature isn’t correctly configured.

But @rsx found the typing mistake which I overread. So nice to read that it works now for you.

7

It does not work. I meant to say that the text of the question is correct now, because I corrected the copy and paste error.

Your solution wouldn’t work, as the only notifier is the master and sometimes notification work, sometimes don’t, even if they are associated with the same problem.

8

That’s why I asked to be sure, if this isn’t confgured wrong. :wink:

To be sure, does the hint form @rsx is working for you?

9

I am sorry for the delay in the reply.

No, the hint from @rsx is not working, because what he pointed out was just a copy and paste error from the script to this post. The script was already correct, didn’t have that error.