RDMA issues with SMB vNIC after deploying new host with VMM 2022, MaxQueuePairCount 0

Hello all, back with another issue i stumbled upon this week. I am working on redeploying a 2019 S2D Cluster to 2022 and started on the first host to be redeployed with VMM. The deployment process worked fine, but when running my checks including Test-RDMA.ps1 script something was off.

As one of the last steps i compeer all settings set by the post-s2ddeployment script that sets, networking settings, power settings and so on on the deployed host.

When i got to checking RDMA settings with Get-NetadapterRdma it said that the SMB1 and SMB2 vnics where RDMA enabled.

Then next step had me run this command

If you notice on node FAS2DS11 the SMB1 and 2 vnic had 0 on MaxQueuePairCount. On the other nodes this was the same value as the Physical Mellanox NIC’s.

If we go a bit deeper with Get-NetadapterRdma -Name SMB1 | fl we can see that RdmaTechnology at the bottom is Unknown

I spent some time redoing the network setup, reapplying config. By accident i ran Get-Netadapter | Enable-Netadapter and noticed that the Management vNic had RDMA on

of it should not have that so i removed it from the Management vNIC. At this time it was time for dinner, after dinner i went for a walk with the puppy and it occurred to me the only difference between the Management vNic and the SMB vNic’s is that they have a different Port Profile set in VMM. The Storage vNic’s have a Storage RDMA Port Profile to enable RDMA. This has the offloading setting of enable remote direct memory access(RDMA)

On a all versions of Windows Server before this has always been there and worked great.

On 2022 this Port Profile needs to be removed. There must be a change in Windows Server 2022 where this RDMA setting from VMM causes an Unknown situation for RDMA on nic’s.

So the conclusion is if you are using VMM to deploy or manage your network and you use Port Profiles to set enable RDMA on the nic then this will cause a problem on Windows Server 2022. So remove the port profile from the nic’s its set on and RDMA will work normally.

Leave a Reply

Your email address will not be published.