VMM 2022 Agent Upgrade Problem and Adding New physical hosts

Hello Everyone

VMM 2022 is out and allows you to run Windows Server 2022 and Azure Stack HCI 21H2 wich you wan’t to upgrade to to get new features if you have any of those 2 OS’es. I upgraded my lab last week and had some issues. The VMM 2022 upgrade went perfect.

But upgrading my 2019 cluster agents gave an error. The same with adding new physical hosts to VMM.

Upgrade agent Problem

Upgrading an agent in VMM has been safe to do in VMM since 2012R2 came out. But this time it failed flat on it’s ass. When upgrading the process threw this error msg in VMM.

It failed on deploying the driver and installing the virtual switch extension. At the same time i lost some of my VM’s and they where running on that node.

What happend was networking dropped on that node when installing the switch extension causing the VMM upgrade to fail as it lost connectivity. And the node dropped out of the cluster for a like 1 minute. And it caused all my VM’s on that node to be moved to another node as they failed. They started up, but it’s not a good solution.

The Solution

Was to drain the node of all resources, aka drain and pause. Either via VMM or FCM, i chose FCM as it fails vm’s and storage back to the same node. And then run the upgrade. On the 3rd node it failed as well. But on the 4th node it went trough without a problem.

So be aware that the node can loose connectivity when you try to upgrade VMM agent on your hyper-v nodes.

2nd issue adding new nodes to VMM

As i upgraded to VMM 2022 i also wanted to add my Azure Stack HCI 21H2 nodes to VMM as well. This failed with a error

Error (413)
The files needed to install agent, version 10.22.1287.0, are missing from the VMM management server JTVMM.

It was failing to install the agent on the nodes. Permission is there, the files are located in the correct place on the VMM server and so on.

The Solution

The solution was to install the VMM agent files manually on the nodes first via Console access. Then add them in VMM and it worked great. This will attach the VMM agent that is installed and add all the features needed for the agent to do it’s things

The VMM teams is following this up and is aware of the issue and is trying to replicate and find a solution for this. Multiple MVP’s have reported this and it’s a top priority of the VMM team to fix this issue asap.

8 thoughts on “VMM 2022 Agent Upgrade Problem and Adding New physical hosts

  • August 8, 2022 at 5:52 pm
    Permalink

    I had a similar issue, same error and message. Mine was for upgrading existing agents on a registered cluster from SCVMM2019 to SCVMM2022. I was using an existing RunAs profile that seemingly had full admin rights to the cluster since SCVMM is already managing them. But, was still failing.

    I ran a Process Monitor against the system while the job was running and then filtered out all the unrelated noise. Finally, I saw a couple lines of interest (one pair of messages):

    Process Name = VMMService.exe
    Class: FileSystem
    Operation = CreateFile
    Result: PATH NOT FOUND
    Path: C:\Program Files\Microsoft System Center\Virtual Machine Manager\agents\I386\10.22.1287.0\msiInstaller.exe

    Process Name = VMMService.exe
    Class: FileSystem
    Operation = CreateFile
    Result: NAME NOT FOUND
    Path: \\server-name.domain.tld\ADMIN$\msiInstaller.exe

    I verified that the PID was running under the SCVMM RunAs account I was using to send the update agents. It’s interesting it’s trying to locate an i386 version of the installer files and in fact there is no such installer source path on my SCVMM server. I also saw a couple lines complaining about inability to properly talk to the HV server I was pushing the updates to.

    I switched over to directly entering creds of another administrative user that is a full domain admin and pushed the update again. This time it worked and the process looked a little different:

    Process Name = VMMService.exe
    Class: FileSystem
    Operation = CreateFile
    Result: Success
    Path: C:\Program Files\Microsoft System Center\Virtual Machine Manager\agents\amd64\10.22.1287.0\msiInstaller.exe

    Process Name = VMMService.exe
    Class: FileSystem
    Operation = CreateFile
    Result: Success
    Path: \\server-name.domain.tld\ADMIN$\msiInstaller.exe

    This time it seems to know it needs to send the 64bit version of the binary and then successfully does so.

    Maybe check this on your side to see if it’s the same issue.

    Reply
  • July 21, 2022 at 9:26 pm
    Permalink

    before installing the agent manually , check whether the time is synced between scvmm and host also active directory ? after that restart the task if it failed install agent manually

    Reply
  • June 30, 2022 at 7:57 am
    Permalink

    Hello, is there already an official solution from Microsoft for the problem “Install Agent”? I have nothing on the net.
    Greetings

    Reply
    • August 12, 2022 at 10:35 am
      Permalink

      No we think we found the issue as i was running a pre release of 2019 UR3 and that was the cause of it.

      Reply
  • May 12, 2022 at 9:39 pm
    Permalink

    also you wrote:
    “Multiple MVP’s have reported this”

    How do you know this?

    Is there some VMM Community im not aware of that i could participate in? Its a lonely problem solving with SCVMM sometimes…

    Reply
    • August 12, 2022 at 10:40 am
      Permalink

      No this was internal MVP Talks. And it’s related to the deployment of the agent to AzHCI.

      I have not recived a OK for it yet. Il try and ping the VMM team about it.

      Reply
  • May 12, 2022 at 9:35 pm
    Permalink

    i posted about these issues a couple weeks ago here, nbever got an answer so far

    https://docs.microsoft.com/en-us/answers/questions/826419/scvmm-2022-error-413-when-trying-to-add-host.html?childToView=829855#comment-829855

    unfortunately for the 413 error, your solution didnt work for me. It was the first thing i tried, but it still throws the error.

    Also, seeing my last issue with sysprep (further down the comment chain), are you willing to give this ago yourself and see if it also throws for you? I have a feeling the casue of these issues is related

    Reply
    • August 12, 2022 at 10:42 am
      Permalink

      You tried to manually install the agent on the host, then add the host to VMM? That one works for me and everyone else i have spoken with.

      Im doing a new VMM 2022 upgrade in a months time. Il update once i have some feedback on that one.

      Reply

Leave a Reply

Your email address will not be published. Required fields are marked *