Ansible Icinga Director - telekom_mms.icinga_director throws error when removing checks

Icinga 2 Automation & Infrastructure
,
1

Since over two years now I used t-systems_mms.icinga_director to automatically add and remove checks, host, … to our monitoring. After an update of the OS of the monitoring host (from Ubuntu 20.04 to Ubuntu 22.04, combined with icinga update to v2.12.1, my ansible playbook creates the checks, but the removal (state: absent) fails.
Also every following manipulations throws an error (disable check instead of removal), and I don’t find any reason within my playbook. In the meantime I also checks for a newer version of t-systems_mms.icinga_director and found that it was “moved” to telekom_mms.icinga_director and installed this (in parallel, rewrote the playbook) - but no change.

Perhaps anyone can give me a hint what I need to change/correct. The ansible output is as follows:

TASK [icinga-checks_new : Remove moxis healthCheck] *************************************************************************************************************************************************
task path: /home/ansiblemaster/ansible/roles/icinga-checks_new/tasks/remove-service-checks.yml:2
<192.168.111.7> ESTABLISH SSH CONNECTION FOR USER: ansiblemaster
<192.168.111.7> SSH: EXEC ssh -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="ansiblemaster"' -o ConnectTimeout=10 -o 'ControlPath="/home/ansiblemaster/.ansible/cp/05ba4bc0aa"' 192.168.111.7 '/bin/sh -c '"'"'echo ~ansiblemaster && sleep 0'"'"''
<192.168.111.7> (0, b'/home/ansiblemaster\n', b'')
<192.168.111.7> ESTABLISH SSH CONNECTION FOR USER: ansiblemaster
<192.168.111.7> SSH: EXEC ssh -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="ansiblemaster"' -o ConnectTimeout=10 -o 'ControlPath="/home/ansiblemaster/.ansible/cp/05ba4bc0aa"' 192.168.111.7 '/bin/sh -c '"'"'( umask 77 && mkdir -p "` echo /home/ansiblemaster/.ansible/tmp `"&& mkdir "` echo /home/ansiblemaster/.ansible/tmp/ansible-tmp-1708611340.9987586-795379-60030409915690 `" && echo ansible-tmp-1708611340.9987586-795379-60030409915690="` echo /home/ansiblemaster/.ansible/tmp/ansible-tmp-1708611340.9987586-795379-60030409915690 `" ) && sleep 0'"'"''
<192.168.111.7> (0, b'ansible-tmp-1708611340.9987586-795379-60030409915690=/home/ansiblemaster/.ansible/tmp/ansible-tmp-1708611340.9987586-795379-60030409915690\n', b'')
Using module file /home/ansiblemaster/.ansible/collections/ansible_collections/telekom_mms/icinga_director/plugins/modules/icinga_service.py
<192.168.111.7> PUT /home/ansiblemaster/.ansible/tmp/ansible-local-795343nncee3ce/tmp9uddzpy3 TO /home/ansiblemaster/.ansible/tmp/ansible-tmp-1708611340.9987586-795379-60030409915690/AnsiballZ_icinga_service.py
<192.168.111.7> SSH: EXEC sftp -b - -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="ansiblemaster"' -o ConnectTimeout=10 -o 'ControlPath="/home/ansiblemaster/.ansible/cp/05ba4bc0aa"' '[192.168.111.7]'
<192.168.111.7> (0, b'sftp> put /home/ansiblemaster/.ansible/tmp/ansible-local-795343nncee3ce/tmp9uddzpy3 /home/ansiblemaster/.ansible/tmp/ansible-tmp-1708611340.9987586-795379-60030409915690/AnsiballZ_icinga_service.py\n', b'')
<192.168.111.7> ESTABLISH SSH CONNECTION FOR USER: ansiblemaster
<192.168.111.7> SSH: EXEC ssh -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="ansiblemaster"' -o ConnectTimeout=10 -o 'ControlPath="/home/ansiblemaster/.ansible/cp/05ba4bc0aa"' 192.168.111.7 '/bin/sh -c '"'"'chmod u+x /home/ansiblemaster/.ansible/tmp/ansible-tmp-1708611340.9987586-795379-60030409915690/ /home/ansiblemaster/.ansible/tmp/ansible-tmp-1708611340.9987586-795379-60030409915690/AnsiballZ_icinga_service.py && sleep 0'"'"''
<192.168.111.7> (0, b'', b'')
<192.168.111.7> ESTABLISH SSH CONNECTION FOR USER: ansiblemaster
<192.168.111.7> SSH: EXEC ssh -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="ansiblemaster"' -o ConnectTimeout=10 -o 'ControlPath="/home/ansiblemaster/.ansible/cp/05ba4bc0aa"' -tt 192.168.111.7 '/bin/sh -c '"'"'sudo -H -S -n  -u root /bin/sh -c '"'"'"'"'"'"'"'"'echo BECOME-SUCCESS-lwfehxtzwqfgpudckkbdaozibthvesdf ; /usr/bin/python3 /home/ansiblemaster/.ansible/tmp/ansible-tmp-1708611340.9987586-795379-60030409915690/AnsiballZ_icinga_service.py'"'"'"'"'"'"'"'"' && sleep 0'"'"''
Escalation succeeded
<192.168.111.7> (1, b'\r\n{"failed": true, "msg": "exception when deleting: Expecting value: line 1 column 1 (char 0)", "exception": "  File \\"/tmp/ansible_telekom_mms.icinga_director.icinga_service_payload_e_pyqobe/ansible_telekom_mms.icinga_director.icinga_service_payload.zip/ansible_collections/telekom_mms/icinga_director/plugins/module_utils/icinga.py\\", line 281, in update\\n    exists = self.exists()\\n  File \\"/tmp/ansible_telekom_mms.icinga_director.icinga_service_payload_e_pyqobe/ansible_telekom_mms.icinga_director.icinga_service_payload.zip/ansible_collections/telekom_mms/icinga_director/plugins/modules/icinga_service.py\\", line 256, in exists\\n  File \\"/tmp/ansible_telekom_mms.icinga_director.icinga_service_payload_e_pyqobe/ansible_telekom_mms.icinga_director.icinga_service_payload.zip/ansible_collections/telekom_mms/icinga_director/plugins/module_utils/icinga.py\\", line 75, in call_url\\n    content = json.loads(rsp.read().decode(\\"utf-8\\"))\\n  File \\"/usr/lib/python3.10/json/__init__.py\\", line 346, in loads\\n    return _default_decoder.decode(s)\\n  File \\"/usr/lib/python3.10/json/decoder.py\\", line 337, in decode\\n    obj, end = self.raw_decode(s, idx=_w(s, 0).end())\\n  File \\"/usr/lib/python3.10/json/decoder.py\\", line 355, in raw_decode\\n    raise JSONDecodeError(\\"Expecting value\\", s, err.value) from None\\n", "invocation": {"module_args": {"state": "present", "disabled": true, "url": "https://192.168.10.101/", "validate_certs": false, "url_username": "director", "url_password": "VALUE_SPECIFIED_IN_NO_LOG_PARAMETER", "object_name": "check.fullstacks.healthCheck", "host": "maas-test-tom01", "vars": {"customer": "fullstacks"}, "force": false, "http_agent": "ansible-httpget", "use_proxy": true, "force_basic_auth": false, "use_gssapi": false, "groups": [], "imports": [], "client_cert": null, "client_key": null, "append": null, "check_command": null, "check_interval": null, "check_period": null, "check_timeout": null, "display_name": null, "enable_active_checks": null, "enable_event_handler": null, "enable_notifications": null, "enable_passive_checks": null, "enable_perfdata": null, "max_check_attempts": null, "notes": null, "notes_url": null, "retry_interval": null, "service_set": null, "use_agent": null, "volatile": null}}}\r\n', b'Shared connection to 192.168.111.7 closed.\r\n')
<192.168.111.7> Failed to connect to the host via ssh: Shared connection to 192.168.111.7 closed.
<192.168.111.7> ESTABLISH SSH CONNECTION FOR USER: ansiblemaster
<192.168.111.7> SSH: EXEC ssh -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="ansiblemaster"' -o ConnectTimeout=10 -o 'ControlPath="/home/ansiblemaster/.ansible/cp/05ba4bc0aa"' 192.168.111.7 '/bin/sh -c '"'"'rm -f -r /home/ansiblemaster/.ansible/tmp/ansible-tmp-1708611340.9987586-795379-60030409915690/ > /dev/null 2>&1 && sleep 0'"'"''
<192.168.111.7> (0, b'', b'')
The full traceback is:
  File "/tmp/ansible_telekom_mms.icinga_director.icinga_service_payload_e_pyqobe/ansible_telekom_mms.icinga_director.icinga_service_payload.zip/ansible_collections/telekom_mms/icinga_director/plugins/module_utils/icinga.py", line 281, in update
    exists = self.exists()
  File "/tmp/ansible_telekom_mms.icinga_director.icinga_service_payload_e_pyqobe/ansible_telekom_mms.icinga_director.icinga_service_payload.zip/ansible_collections/telekom_mms/icinga_director/plugins/modules/icinga_service.py", line 256, in exists
  File "/tmp/ansible_telekom_mms.icinga_director.icinga_service_payload_e_pyqobe/ansible_telekom_mms.icinga_director.icinga_service_payload.zip/ansible_collections/telekom_mms/icinga_director/plugins/module_utils/icinga.py", line 75, in call_url
    content = json.loads(rsp.read().decode("utf-8"))
  File "/usr/lib/python3.10/json/__init__.py", line 346, in loads
    return _default_decoder.decode(s)
  File "/usr/lib/python3.10/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib/python3.10/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
fatal: [maas-test-tom01]: FAILED! => {
    "changed": false,
    "invocation": {
        "module_args": {
            "append": null,
            "check_command": null,
            "check_interval": null,
            "check_period": null,
            "check_timeout": null,
            "client_cert": null,
            "client_key": null,
            "disabled": true,
            "display_name": null,
            "enable_active_checks": null,
            "enable_event_handler": null,
            "enable_notifications": null,
            "enable_passive_checks": null,
            "enable_perfdata": null,
            "force": false,
            "force_basic_auth": false,
            "groups": [],
            "host": "maas-test-tom01",
            "http_agent": "ansible-httpget",
            "imports": [],
            "max_check_attempts": null,
            "notes": null,
            "notes_url": null,
            "object_name": "check.fullstacks.healthCheck",
            "retry_interval": null,
            "service_set": null,
            "state": "present",
            "url": "https://192.168.10.101/",
            "url_password": "VALUE_SPECIFIED_IN_NO_LOG_PARAMETER",
            "url_username": "director",
            "use_agent": null,
            "use_gssapi": false,
            "use_proxy": true,
            "validate_certs": false,
            "vars": {
                "customer": "testvolk"
            },
            "volatile": null
        }
    },
    "msg": "exception when deleting: Expecting value: line 1 column 1 (char 0)"
}

The task is really simple:

- name: Remove moxis healthCheck
  tags: remove_moxis_service
  telekom_mms.icinga_director.icinga_service:
    state: absent
    url: "{{ icinga_url }}"
    validate_certs: no
    url_username: "{{ icinga_user }}"
    url_password: "{{ icinga_pass }}"
    imports: "tpl_check_DB"
    object_name: "check.{{ customer }}.healthCheck"
    display_name: "check.{{ customer }}.healthCheck"
    use_agent: false
    host: "{{ ansible_hostname }}"
    vars:
      customer: "{{ customer }}"

I know there are some not needed parameters, but even with only the mandatory ones I get the same error.

Edit: Ansible is running on a different host as icinga

2

Hello :slight_smile:

Check if setting this to true does help.
At least it helped in our cases where the tasks would suddenly/occasionally fail.
https://github.com/telekom-mms/ansible-collection-icinga-director?tab=readme-ov-file#troubleshooting

3

Hello,

thanks a lot for this hint, unfortunately it did not change the behaviour, the error still exists; it seems there shall be a json created to transfer to the api of director, and this creation fails for some reason way before connecting to director

4

searching for this in the github issues of the module yields one issue: https://github.com/telekom-mms/ansible-collection-icinga-director/issues?q=exception+when+deleting
which should be fixed with the most recent versions of the Director and the module.

In case you haven’t updated both, try it, I’d say. Otherwise I would open a issue on the modules github.