No, I am going to now and will report back.
All, I regret to inform you that this was all due to my own stupidity. 
I never changed the url for director in the powershell script to https://
Thank you all for talking it through with me.
On a side note: Does anyone know if I place an updated version of the Windows MSI package in the share I am using and then change Director to use that one and select allow updates… will that upgrade all my Windows agents to the latest one I have? How does that behave?
Don’t mind, we all are just human beings.
For updating you could choose any kind of software distribution or even manually just run msi installation.
allow updates has a different meaning: It configures the agent to accept config updates from its parent. This does not belong to the agent (and its msi package).
I see. Thanks for clarifying. So the Director settings and the MSI package defined there (I restricted it to the hash of the MSI package I am using to deploy) is what the powershell script uses to do the initial agent deployment and config with Director.
To keep the agent up to date after that, I can just use something like SCCM to detect the originally deployed version of the agent installed on a Windows server and if detected run the latest MSI to upgrade it? This doesn’t impact the communication/config/certs etc… with the Icinga2 server/director?
Yes, and I’d try this with just one machine for test.
Will do. Thanks again.
Using the icinga2-powershell-modul it will look like this example:
...
Notice: Flushing content of "C:\ProgramData\icinga2\var\lib\icinga2\api"
Notice: Stopping the Icinga 2 Service...
Notice: Icinga 2 service has been stopped.
Notice: Using Icinga version "2.10.5", setting certificate directory to "C:\ProgramData\icinga2\var\lib\icinga2\certs"
Notice: Found Icinga 2 Agent version 2.10.5 installed at "C:\Program Files\ICINGA2\"
Notice: Current Icinga 2 Agent Version (2.10.5) is not matching intended version (2.11.2). Downloading new version...
Notice: Downloading Icinga 2 Agent Binary from "http://192.168.53.210/download/Icinga2-v2.11.2-x86_64.msi"
Notice: Icinga 2 Agent hash verification successfull.
Notice: Removing previous Icinga 2 Agent version
Notice: Icinga 2 Agent successfully removed.
Notice: No migration of Icinga 2 Agent data required.
Notice: Installing new Icinga 2 Agent version
Notice: Icinga 2 Agent successfully updated.
Notice: Using Icinga version "2.11.2", setting certificate directory to "C:\ProgramData\icinga2\var\lib\icinga2\certs"
Notice: Found Icinga 2 Agent version 2.11.2 installed at "C:\Program Files\ICINGA2\"
Notice: Removing downloaded Icinga 2 Agent installer
Notice: Host is already registered within Icinga Director.
Notice: Fetched ticket "*******************************" from Icinga Director
Notice: Icinga 2 certificates already exist. Nothing to do.
Notice: icinga2.conf did not change or required parameters not set. Nothing to do
...
Interesting. I am currently using SCCM to deploy the script(s) to install the agent. The script executes and there is a detection method that looks for the product code of the MSI existing on the system to verify that the agent is either already installed and do nothing or it is missing and run the script.
It sounds like I could change the detection method to look for the product code of the new MSI package, it will detect that it isn’t installed and execute the script again… this would see the current version of the agent is installed, contact Director and see that I have defined the new MS package/hash/version and force it to upgrade. Once installed SCCM would then detect the new product code and consider it installed and compliant.