SR-IOV and nested ESXi
I was always curious about some VMware options that I never had the hardware to replicate. One in particular was SR-IOV where you can expose a physical Intel X540 10Gb network card to guest VMs. I wanted to find out for curiosity if this would work with nested ESXi Hosts?!
As mentioned in a previous post you need to enable this in the BIOS before it will work in VMware.
As I don't have a 10Gb switch I've connected both of the ports to a 1Gb switch to enable me to use the feature. I wasn't allowed to use SR-IOV when I selected ESXi 6.0 as the OS, and existing nested VMs only let me switch between E1000 and E1000E as the network driver. When I created a new nested ESXi I could select VMXNET3 which appears to work ok. I really wanted to see if the SR-IOV ports would work so I cheated and created a Windows 2012 VM, then edited it to make it an ESXi one. You can see the results below. It does warn you SR-IOV is NOT compatible with ESXi.
So, sure enough when I'd finished this is the result I found (Only the VMXNET3 Driver Nic listed):
And so my curiosity was satisfied, it's not currently possible or compatible. There may be a hack but I'll stick with VMXNET3 drivers for now...!! I could always turn off SR-IOV and see what speeds I get internally from nested ESXi connected to the same port group on a 10Gb chipset.....!
Note: I did get a Windows 2012 R2 VM to use the SR-IOV ports fine and could configure the NIC advanced settings without issue, was interesting to see it all work!