Reviewing the output of an environement's CAS Namespaces showed that there was an unexpecte URL present for the version of Exchagne that was installed. With Exchange 2013 onwards InternalNLBBypassURL is not something that we need to set. That was an Exchange 2007 and 2010 thing.
In the environment below note that there are couple of things that pique my interest.
Any thoughts?
What is interesting is the highlighted line. If all servers were installed in order why is Exch-2019-2 dead last?
Also why is there an InternalNLBBypassURL set on onlt that vDir?
This is the URL that is unexpected:
https://exch-2019-2.tailspintoys.ca/ews/exchange.asmx
The backend URL is normal and expected to listen on TCP 444. This is how the proxy architecture works on Exchange 2013 onwards.
https://exch-2019-2.tailspintoys.ca:444/ews/exchange.asmx
Removing Erant Virtual Directory
We can fix this by running the below command to null out the virtual directory. Note that $Null must be used rather that an empty string such as "".
Get-WebServicesVirtualDirectory -Server Exch-2019-2 | Set-WebServicesVirtualDirectory -InternalNLBBypassURL $Null
But why?
Pondering PowerShell
There is a clue upstream, the ordering of the servers. AD has a habit of returning things in a date sorted manner with the olderst at the top of the list and newest at the end.
If we look at the order of the Exchange Server object they are as expected and reflect the installation order. However this is NOT the case with any virtual directory. The virtual directories belonging to server Exch-2019-2 are listed last. The WhenCreated timestamp is shown and is highlighted with the red arrow. This virtual directory was created a day after the others, i.e. it is the newest.
But again, why?
This was due to having to recover the server. Running Exchange server setup with the /RecoverServer switch means that Exchange reads the previous configuration out of AD DS and uses that information to figure out what needs to be recovered. As part of /RecoverServer the original virtual directories are deleted out of AD and then recreated. This is done to ensure that the settings in AD DS, the metabase and registry are aligned. It would be nice if /RecoverServer did things a bit more intelligently, but that has been the behaviour since Exchange 2010 introduced the feature.
This is doucmented, but not in a lot of detail. After recovering the server TLS certificates, URLs and other customisations need to be reapplied to the server. Most admins will update OWA, ECP virtual directories but the webservices can be overlooked.
In this case the server being recovered was Exchange 2019 installed onto Windows Server 2019. If we look in the ExchangeSetup.log the version of Exchange setup is recorded. Version 15.2.986.5 is Exchange 2019 CU11.
These details are shown below.
For the benefit of search engines, the relevant log content is included below.
[12/15/2021 23:17:24.0374] [0] **********************************************
[12/15/2021 23:17:24.0409] [0] Starting Microsoft Exchange Server 2019 Setup
[12/15/2021 23:17:24.0409] [0] **********************************************
[12/15/2021 23:17:24.0434] [0] Local time zone: (UTC) Coordinated Universal Time.
[12/15/2021 23:17:24.0439] [0] Operating system version: Microsoft Windows NT 6.2.9200.0.
[12/15/2021 23:17:24.0449] [0] Setup version: 15.2.986.5.
[12/15/2021 23:17:24.0474] [0] Logged on user: TAILSPINTOYS\XXXredactedXXX.
[12/15/2021 23:17:24.0489] [0] The registry key, HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\ExchangeServer\V15\Setup, wasn't found.
[12/15/2021 23:17:24.0539] [0] Command Line Parameter Name='iacceptexchangeserverlicenseterms_diagnosticdataon', Value=''.
[12/15/2021 23:17:24.0539] [0] Command Line Parameter Name='mode', Value='RecoverServer'.
[12/15/2021 23:17:24.0539] [0] Command Line Parameter Name='sourcedir', Value='E:\'.
Cheers,
Rhoderick
