tl;dr Just in case you’re not interested in the narrative and want only facts.
- vRA Software Component won’t properly install IIS
- Found older Windows version uses
Add-WindowsFeature, notInstall-WindowsFeature - Had to test if-statement for automation
- Learned that Powershell 2.0 no longer supported
- Installed WMF 5.0 to get Powershell 5.0
- Guest Customization now broken – need C++ Redistributable due to new version of VMware Tools, WMF 5.0 requires registry edit, and MSDTC needs re-install
- All done to get back to troubleshooting Software Components…
For a while now, I’ve been playing with vRealize Automation at work with the primary use-case of automating system setup in an effort to assist Support with re-creating issues that a customer might be facing. Unfortunately, vRealize Automation has seen varying levels of focus as a result of staff changes. It’s been difficult to keep up with some of the things I had hoped to do with the product.
It was brought to my attention recently that there was a need to install an older version of software that I had unsuccessfully made available in the Tenant. To fix the broken Software Component, I had to revisit Windows Server 2008 R2 SP1 which, honestly, hasn’t aged as poorly as I had imagined. To get started with the Software Component, I figured I would run it line by line in the guest OS. You might argue that I should have done that before, but I had written some code that should have sufficed for all versions of Windows… but I wrote it on Windows Server 2016.
What I quickly realized was that in Server 2008 (and R2), Powershell was not yet using the Install-WindowsFeature cmdlet. In the first iterations of Powershell, the same command was called with Add-WindowsFeature. I manually stepped through my code to identify any differences in WindowsFeature names – I don’t recall if I found any. I re-wrote my Software Component with a new if statement. I figure it might allow me to step each Server version by calling out the Minor version (if you have better ideas, reach out!).
if (([System.Environment]::OSVersion.Version.Major -eq "6") -and
([System.Environment]::OSVersion.Version.Minor -eq "1")) {
Add-WindowsFeature Webserver, Web-WebServer, etc...
}
At deployment, I found that I was receiving errors in Powershell. After going back and forth for a while, I reviewed the vRealize Automation Support Matrix just to check my sanity. I know that support for the older OSes is starting to dwindle. I verified that Windows Server 2008 R2 was supported – what I didn’t expect to find was that Powershell 2.0, which is shipped with the base OS, hasn’t been a supported Task Automation Engine since vRA 7.3 (my environment is vRA 7.5).
After some research, I converted my Template and powered my VM on. I installed .NET Framework 4.8 and immediately followed it with Windows Management Framework 5.1 which includes Powershell 5.0 (which IS a supported Task Engine!). Shut down the VM, converted it back to a template, and went to deploy again.
During this attempt, I started to receive Guest Customization timeouts in vRA. I thought it was a fluke and called it a night. Resuming the next morning, I had to dig a little deeper. vCenter was correctly showing the “Started customization of…” events that I was expecting to see. I went to log into the VM before vRA destroyed the system. I was prompted to reset my Administrator password.
I’ll be honest, I don’t even know if I have ever had to make a Server 2008 template up until this point in my career (I’m still mostly a rookie). This is where I started running down the wrong path. I learned that Server 2008/2008 R2 were the first (and only, I believe) version of Windows Server to wipe the Administrator password blank at each run of Sysprep. I thought the solution was writing a custom unattend.xml, sysprepping the system before turning it into a template, and THEN deploying it. I don’t know why I thought this – it was working before now. It was the wrong course of action anyway. My customizations were still failing.
I checked the toolsPkgDeploy.log file and found an error that eventually led me to this VMware KB article. Because of the newer version of VMware Tools, I needed a newer version of a Microsoft C++ Redistributable. I downloaded the x64 version and tried to deploy only to find the same issues. After reading a second (or third) time, I installed the x86 version that the KB article explicitly states. Now that I’m in the clear, I try to deploy again – only to run into the SAME ISSUE.
This time around toolsPkgDeploy.log didn’t have any gems for me. I had to check some of the Sysprep logs found in C:\Windows\System32\Sysprep\Panther. I found that I was receiving this error:
Sysprep_Generalize_MiStreamProv: RegDeleteValue for target uri failed with error = 2[gle=0x00000002]
Sysprep_Generalize_MiStreamProv: RegDeleteValue for full payload time failed with error = 2[gle=0x00000002]
A (very) quick Google search found me this blog post which gave me a registry key to update. This led to yet another attempted deployment… another failed attempt. This time the logs in Panther pointed to MSDTC. AHA!
I’ve installed vRealize Automation enough to know how to install/uninstall MSDTC, but not enough to remember if the command has one dash (-) or two (–). The answer is: only one. With MSDTC effectively reset, I gave it one last attempt. I was finally able to enjoy the fruits of my labor (and frustration)… the Template cloned successfully!
The good news is that this put me back into stride after a few days of not really being able to focus on the problem. I was immediately good to go… go back to troubleshooting my Software Components. It really is a never ending cycle of troubleshooting/debugging. A handful of Software Components down and I think I’m at a successful deployment of some older software. If not, I think I might just troubleshoot myself… ha!