ESXi USB PCIe Passthrough Failing to Connect Connected Devicesblu-ray archiving in vmware ESXi 4Disable ESXi passthrough on SATA controllerFreeBSD ixgbe driver not workingHow can I setup PCI passthrough for my HighPoint 1144A USB 3.0 card under ESXi 5.1?ESXi 5.5 crash on passthroughESXi PCI passthrough on HP DL360 G5Cannot remove passthrough setup on ESXiESXI PCIe GPU Passthrough does not allow for CUDANVIDIA QUADRO M4000 vGPU - ESXI 6.0.3 - GPU PassthroughESXi Tesla passthrough enabled but not assignable
Array initialization optimization
How large would a mega structure have to be to host 1 billion people indefinitely?
What was the Shuttle Carrier Aircraft escape tunnel?
What's currently blocking the construction of the wall between Mexico and the US?
What did River say when she woke from her proto-comatose state?
Drawing people along with x and y axis
If I wouldn't want to read the story, is writing it still a good idea?
Suggested order for Amazon Prime Doctor Who series
Can you find x?
How to draw this center trajectory of rolling ball?
Why do even high-end cameras often still include normal (non-cross-type) AF sensors?
Output of "$OSTYPE:6" on old releases of Mac OS X
Greeting with "Ho"
Can Ogre clerics use Purify Food and Drink on humanoid characters?
Why is it recommended to mix yogurt starter with a small amount of milk before adding to the entire batch?
Why do textbooks often include the solutions to odd or even numbered problems but not both?
Should I prioritize my 401k over my student loans?
Is a single radon-daughter atom in air a solid?
Find the C-factor of a vote
Why do all the teams that I have worked with always finish a sprint without completion of all the stories?
How can I politely work my way around not liking coffee or beer when it comes to professional networking?
Why do some professors with PhDs leave their professorships to teach high school?
Hot coffee brewing solutions for deep woods camping
"How can you guarantee that you won't change/quit job after just couple of months?" How to respond?
ESXi USB PCIe Passthrough Failing to Connect Connected Devices
blu-ray archiving in vmware ESXi 4Disable ESXi passthrough on SATA controllerFreeBSD ixgbe driver not workingHow can I setup PCI passthrough for my HighPoint 1144A USB 3.0 card under ESXi 5.1?ESXi 5.5 crash on passthroughESXi PCI passthrough on HP DL360 G5Cannot remove passthrough setup on ESXiESXI PCIe GPU Passthrough does not allow for CUDANVIDIA QUADRO M4000 vGPU - ESXI 6.0.3 - GPU PassthroughESXi Tesla passthrough enabled but not assignable
.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty height:90px;width:728px;box-sizing:border-box;
I have a fresh install of ESXi 6.7U2 up and running on which I can successfully start VMs. To this setup I've added a StarTech.com 4 Port USB 3.0 Card Adapter (PEXUSB3S44V). This has been configured for pass through following https://kb.vmware.com/s/article/1010789. This can be configured for passthrough to a VM:
The USB PCIe controllers are detected here as uPD720202 USB 3.0 Host Controller. A single PCIe adaptor has 4 independent controllers and I've chosen to pass then all through for simplicity while trying to debug this issue.
I have tried using both Amazon Linux 2 (on prem) and Ubuntu Server 18.04.2 for the VMs, what follows appears to be equivalent in both of them.
lspci output shows the expected pass through devices:
00:00.0 Host bridge: Intel Corporation 440BX/ZX/DX - 82443BX/ZX/DX Host bridge (rev 01)
00:01.0 PCI bridge: Intel Corporation 440BX/ZX/DX - 82443BX/ZX/DX AGP bridge (rev 01)
00:07.0 ISA bridge: Intel Corporation 82371AB/EB/MB PIIX4 ISA (rev 08)
00:07.1 IDE interface: Intel Corporation 82371AB/EB/MB PIIX4 IDE (rev 01)
00:07.3 Bridge: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 08)
00:07.7 System peripheral: VMware Virtual Machine Communication Interface (rev 10)
00:0f.0 VGA compatible controller: VMware SVGA II Adapter
00:10.0 SCSI storage controller: LSI Logic / Symbios Logic 53c1030 PCI-X Fusion-MPT Dual Ultra320 SCSI (rev 01)
00:11.0 PCI bridge: VMware PCI bridge (rev 02)
00:15.0 PCI bridge: VMware PCI Express Root Port (rev 01)
00:15.1 PCI bridge: VMware PCI Express Root Port (rev 01)
00:15.2 PCI bridge: VMware PCI Express Root Port (rev 01)
00:15.3 PCI bridge: VMware PCI Express Root Port (rev 01)
00:15.4 PCI bridge: VMware PCI Express Root Port (rev 01)
00:15.5 PCI bridge: VMware PCI Express Root Port (rev 01)
00:15.6 PCI bridge: VMware PCI Express Root Port (rev 01)
00:15.7 PCI bridge: VMware PCI Express Root Port (rev 01)
00:16.0 PCI bridge: VMware PCI Express Root Port (rev 01)
00:16.1 PCI bridge: VMware PCI Express Root Port (rev 01)
00:16.2 PCI bridge: VMware PCI Express Root Port (rev 01)
00:16.3 PCI bridge: VMware PCI Express Root Port (rev 01)
00:16.4 PCI bridge: VMware PCI Express Root Port (rev 01)
00:16.5 PCI bridge: VMware PCI Express Root Port (rev 01)
00:16.6 PCI bridge: VMware PCI Express Root Port (rev 01)
00:16.7 PCI bridge: VMware PCI Express Root Port (rev 01)
00:17.0 PCI bridge: VMware PCI Express Root Port (rev 01)
00:17.1 PCI bridge: VMware PCI Express Root Port (rev 01)
00:17.2 PCI bridge: VMware PCI Express Root Port (rev 01)
00:17.3 PCI bridge: VMware PCI Express Root Port (rev 01)
00:17.4 PCI bridge: VMware PCI Express Root Port (rev 01)
00:17.5 PCI bridge: VMware PCI Express Root Port (rev 01)
00:17.6 PCI bridge: VMware PCI Express Root Port (rev 01)
00:17.7 PCI bridge: VMware PCI Express Root Port (rev 01)
00:18.0 PCI bridge: VMware PCI Express Root Port (rev 01)
00:18.1 PCI bridge: VMware PCI Express Root Port (rev 01)
00:18.2 PCI bridge: VMware PCI Express Root Port (rev 01)
00:18.3 PCI bridge: VMware PCI Express Root Port (rev 01)
00:18.4 PCI bridge: VMware PCI Express Root Port (rev 01)
00:18.5 PCI bridge: VMware PCI Express Root Port (rev 01)
00:18.6 PCI bridge: VMware PCI Express Root Port (rev 01)
00:18.7 PCI bridge: VMware PCI Express Root Port (rev 01)
02:01.0 SATA controller: VMware SATA AHCI controller
03:00.0 USB controller: Renesas Technology Corp. uPD720202 USB 3.0 Host Controller (rev 02)
04:00.0 USB controller: Renesas Technology Corp. uPD720202 USB 3.0 Host Controller (rev 02)
0b:00.0 Ethernet controller: VMware VMXNET3 Ethernet Controller (rev 01)
13:00.0 USB controller: Renesas Technology Corp. uPD720202 USB 3.0 Host Controller (rev 02)
1b:00.0 USB controller: Renesas Technology Corp. uPD720202 USB 3.0 Host Controller (rev 02)
lsusb output before connecting a device to the adaptor:
Bus 008 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 007 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 006 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 005 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 004 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 003 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Viewing dmesg output when trying to connect a device, depending on device (USB hub, USB data drive etc.), I either get no additional output or:
[ 818.213414] xhci_hcd 0000:04:00.0: Abort failed to stop command ring: -110
[ 818.213435] xhci_hcd 0000:04:00.0: xHCI host controller not responding, assume dead
[ 818.213453] xhci_hcd 0000:04:00.0: HC died; cleaning up
[ 818.213467] xhci_hcd 0000:04:00.0: WARNING: Host System Error
[ 818.213497] xhci_hcd 0000:04:00.0: Error while assigning device slot ID
[ 818.213511] xhci_hcd 0000:04:00.0: Max number of devices this xHCI host supports is 32.
[ 818.213526] usb usb4-port1: couldn't allocate usb_device
[ 818.213538] xhci_hcd 0000:04:00.0: Error while assigning device slot ID
[ 818.213551] xhci_hcd 0000:04:00.0: Max number of devices this xHCI host supports is 32.
[ 818.213565] usb usb3-port1: couldn't allocate usb_device
Clearly a device is being detected in the VM but something is preventing it from connecting successfully.
Things I tried (Ref https://bbs.archlinux.org/viewtopic.php?id=236536):
- Setting the kernel parameters iommu=soft pci=nomsi
- Setting the kernel parameter intel_iommu=off
- Different VMs (as mentioned above)
- Ensuring the USB devices are connected on VM boot
- Setting the ESXi host power management to 'High performance'
To confirm I wasn't dealing with a hardware/power issue I ditched the hypervisor on the same hardware by performing a boot from USB (LinuxMint). Here the PCIe adaptor worked correctly with all connected devices so I think that can be ruled out.
At the moment my suspicions are:
- This uPD720202 chipset isn't compatible with ESXi, even though many posts exist online suggesting it should work fine (which is why I bought it).
- I'm missing some critical configuration within the ESIx host.
If anyone spots anything obvious that I could try, or would like additional output that would be greatly appreciated.
vmware-esxi passthrough
asked Jun 5 at 18:00
MarcFMarcF
15819
add a comment |
I have a fresh install of ESXi 6.7U2 up and running on which I can successfully start VMs. To this setup I've added a StarTech.com 4 Port USB 3.0 Card Adapter (PEXUSB3S44V). This has been configured for pass through following https://kb.vmware.com/s/article/1010789. This can be configured for passthrough to a VM:
The USB PCIe controllers are detected here as uPD720202 USB 3.0 Host Controller. A single PCIe adaptor has 4 independent controllers and I've chosen to pass then all through for simplicity while trying to debug this issue.
I have tried using both Amazon Linux 2 (on prem) and Ubuntu Server 18.04.2 for the VMs, what follows appears to be equivalent in both of them.
lspci output shows the expected pass through devices:
00:00.0 Host bridge: Intel Corporation 440BX/ZX/DX - 82443BX/ZX/DX Host bridge (rev 01)
00:01.0 PCI bridge: Intel Corporation 440BX/ZX/DX - 82443BX/ZX/DX AGP bridge (rev 01)
00:07.0 ISA bridge: Intel Corporation 82371AB/EB/MB PIIX4 ISA (rev 08)
00:07.1 IDE interface: Intel Corporation 82371AB/EB/MB PIIX4 IDE (rev 01)
00:07.3 Bridge: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 08)
00:07.7 System peripheral: VMware Virtual Machine Communication Interface (rev 10)
00:0f.0 VGA compatible controller: VMware SVGA II Adapter
00:10.0 SCSI storage controller: LSI Logic / Symbios Logic 53c1030 PCI-X Fusion-MPT Dual Ultra320 SCSI (rev 01)
00:11.0 PCI bridge: VMware PCI bridge (rev 02)
00:15.0 PCI bridge: VMware PCI Express Root Port (rev 01)
00:15.1 PCI bridge: VMware PCI Express Root Port (rev 01)
00:15.2 PCI bridge: VMware PCI Express Root Port (rev 01)
00:15.3 PCI bridge: VMware PCI Express Root Port (rev 01)
00:15.4 PCI bridge: VMware PCI Express Root Port (rev 01)
00:15.5 PCI bridge: VMware PCI Express Root Port (rev 01)
00:15.6 PCI bridge: VMware PCI Express Root Port (rev 01)
00:15.7 PCI bridge: VMware PCI Express Root Port (rev 01)
00:16.0 PCI bridge: VMware PCI Express Root Port (rev 01)
00:16.1 PCI bridge: VMware PCI Express Root Port (rev 01)
00:16.2 PCI bridge: VMware PCI Express Root Port (rev 01)
00:16.3 PCI bridge: VMware PCI Express Root Port (rev 01)
00:16.4 PCI bridge: VMware PCI Express Root Port (rev 01)
00:16.5 PCI bridge: VMware PCI Express Root Port (rev 01)
00:16.6 PCI bridge: VMware PCI Express Root Port (rev 01)
00:16.7 PCI bridge: VMware PCI Express Root Port (rev 01)
00:17.0 PCI bridge: VMware PCI Express Root Port (rev 01)
00:17.1 PCI bridge: VMware PCI Express Root Port (rev 01)
00:17.2 PCI bridge: VMware PCI Express Root Port (rev 01)
00:17.3 PCI bridge: VMware PCI Express Root Port (rev 01)
00:17.4 PCI bridge: VMware PCI Express Root Port (rev 01)
00:17.5 PCI bridge: VMware PCI Express Root Port (rev 01)
00:17.6 PCI bridge: VMware PCI Express Root Port (rev 01)
00:17.7 PCI bridge: VMware PCI Express Root Port (rev 01)
00:18.0 PCI bridge: VMware PCI Express Root Port (rev 01)
00:18.1 PCI bridge: VMware PCI Express Root Port (rev 01)
00:18.2 PCI bridge: VMware PCI Express Root Port (rev 01)
00:18.3 PCI bridge: VMware PCI Express Root Port (rev 01)
00:18.4 PCI bridge: VMware PCI Express Root Port (rev 01)
00:18.5 PCI bridge: VMware PCI Express Root Port (rev 01)
00:18.6 PCI bridge: VMware PCI Express Root Port (rev 01)
00:18.7 PCI bridge: VMware PCI Express Root Port (rev 01)
02:01.0 SATA controller: VMware SATA AHCI controller
03:00.0 USB controller: Renesas Technology Corp. uPD720202 USB 3.0 Host Controller (rev 02)
04:00.0 USB controller: Renesas Technology Corp. uPD720202 USB 3.0 Host Controller (rev 02)
0b:00.0 Ethernet controller: VMware VMXNET3 Ethernet Controller (rev 01)
13:00.0 USB controller: Renesas Technology Corp. uPD720202 USB 3.0 Host Controller (rev 02)
1b:00.0 USB controller: Renesas Technology Corp. uPD720202 USB 3.0 Host Controller (rev 02)
lsusb output before connecting a device to the adaptor:
Bus 008 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 007 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 006 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 005 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 004 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 003 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Viewing dmesg output when trying to connect a device, depending on device (USB hub, USB data drive etc.), I either get no additional output or:
[ 818.213414] xhci_hcd 0000:04:00.0: Abort failed to stop command ring: -110
[ 818.213435] xhci_hcd 0000:04:00.0: xHCI host controller not responding, assume dead
[ 818.213453] xhci_hcd 0000:04:00.0: HC died; cleaning up
[ 818.213467] xhci_hcd 0000:04:00.0: WARNING: Host System Error
[ 818.213497] xhci_hcd 0000:04:00.0: Error while assigning device slot ID
[ 818.213511] xhci_hcd 0000:04:00.0: Max number of devices this xHCI host supports is 32.
[ 818.213526] usb usb4-port1: couldn't allocate usb_device
[ 818.213538] xhci_hcd 0000:04:00.0: Error while assigning device slot ID
[ 818.213551] xhci_hcd 0000:04:00.0: Max number of devices this xHCI host supports is 32.
[ 818.213565] usb usb3-port1: couldn't allocate usb_device
Clearly a device is being detected in the VM but something is preventing it from connecting successfully.
Things I tried (Ref https://bbs.archlinux.org/viewtopic.php?id=236536):
- Setting the kernel parameters iommu=soft pci=nomsi
- Setting the kernel parameter intel_iommu=off
- Different VMs (as mentioned above)
- Ensuring the USB devices are connected on VM boot
- Setting the ESXi host power management to 'High performance'
To confirm I wasn't dealing with a hardware/power issue I ditched the hypervisor on the same hardware by performing a boot from USB (LinuxMint). Here the PCIe adaptor worked correctly with all connected devices so I think that can be ruled out.
At the moment my suspicions are:
- This uPD720202 chipset isn't compatible with ESXi, even though many posts exist online suggesting it should work fine (which is why I bought it).
- I'm missing some critical configuration within the ESIx host.
If anyone spots anything obvious that I could try, or would like additional output that would be greatly appreciated.
vmware-esxi passthrough
asked Jun 5 at 18:00
MarcFMarcF
15819
add a comment |
I have a fresh install of ESXi 6.7U2 up and running on which I can successfully start VMs. To this setup I've added a StarTech.com 4 Port USB 3.0 Card Adapter (PEXUSB3S44V). This has been configured for pass through following https://kb.vmware.com/s/article/1010789. This can be configured for passthrough to a VM:
The USB PCIe controllers are detected here as uPD720202 USB 3.0 Host Controller. A single PCIe adaptor has 4 independent controllers and I've chosen to pass then all through for simplicity while trying to debug this issue.
I have tried using both Amazon Linux 2 (on prem) and Ubuntu Server 18.04.2 for the VMs, what follows appears to be equivalent in both of them.
lspci output shows the expected pass through devices:
00:00.0 Host bridge: Intel Corporation 440BX/ZX/DX - 82443BX/ZX/DX Host bridge (rev 01)
00:01.0 PCI bridge: Intel Corporation 440BX/ZX/DX - 82443BX/ZX/DX AGP bridge (rev 01)
00:07.0 ISA bridge: Intel Corporation 82371AB/EB/MB PIIX4 ISA (rev 08)
00:07.1 IDE interface: Intel Corporation 82371AB/EB/MB PIIX4 IDE (rev 01)
00:07.3 Bridge: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 08)
00:07.7 System peripheral: VMware Virtual Machine Communication Interface (rev 10)
00:0f.0 VGA compatible controller: VMware SVGA II Adapter
00:10.0 SCSI storage controller: LSI Logic / Symbios Logic 53c1030 PCI-X Fusion-MPT Dual Ultra320 SCSI (rev 01)
00:11.0 PCI bridge: VMware PCI bridge (rev 02)
00:15.0 PCI bridge: VMware PCI Express Root Port (rev 01)
00:15.1 PCI bridge: VMware PCI Express Root Port (rev 01)
00:15.2 PCI bridge: VMware PCI Express Root Port (rev 01)
00:15.3 PCI bridge: VMware PCI Express Root Port (rev 01)
00:15.4 PCI bridge: VMware PCI Express Root Port (rev 01)
00:15.5 PCI bridge: VMware PCI Express Root Port (rev 01)
00:15.6 PCI bridge: VMware PCI Express Root Port (rev 01)
00:15.7 PCI bridge: VMware PCI Express Root Port (rev 01)
00:16.0 PCI bridge: VMware PCI Express Root Port (rev 01)
00:16.1 PCI bridge: VMware PCI Express Root Port (rev 01)
00:16.2 PCI bridge: VMware PCI Express Root Port (rev 01)
00:16.3 PCI bridge: VMware PCI Express Root Port (rev 01)
00:16.4 PCI bridge: VMware PCI Express Root Port (rev 01)
00:16.5 PCI bridge: VMware PCI Express Root Port (rev 01)
00:16.6 PCI bridge: VMware PCI Express Root Port (rev 01)
00:16.7 PCI bridge: VMware PCI Express Root Port (rev 01)
00:17.0 PCI bridge: VMware PCI Express Root Port (rev 01)
00:17.1 PCI bridge: VMware PCI Express Root Port (rev 01)
00:17.2 PCI bridge: VMware PCI Express Root Port (rev 01)
00:17.3 PCI bridge: VMware PCI Express Root Port (rev 01)
00:17.4 PCI bridge: VMware PCI Express Root Port (rev 01)
00:17.5 PCI bridge: VMware PCI Express Root Port (rev 01)
00:17.6 PCI bridge: VMware PCI Express Root Port (rev 01)
00:17.7 PCI bridge: VMware PCI Express Root Port (rev 01)
00:18.0 PCI bridge: VMware PCI Express Root Port (rev 01)
00:18.1 PCI bridge: VMware PCI Express Root Port (rev 01)
00:18.2 PCI bridge: VMware PCI Express Root Port (rev 01)
00:18.3 PCI bridge: VMware PCI Express Root Port (rev 01)
00:18.4 PCI bridge: VMware PCI Express Root Port (rev 01)
00:18.5 PCI bridge: VMware PCI Express Root Port (rev 01)
00:18.6 PCI bridge: VMware PCI Express Root Port (rev 01)
00:18.7 PCI bridge: VMware PCI Express Root Port (rev 01)
02:01.0 SATA controller: VMware SATA AHCI controller
03:00.0 USB controller: Renesas Technology Corp. uPD720202 USB 3.0 Host Controller (rev 02)
04:00.0 USB controller: Renesas Technology Corp. uPD720202 USB 3.0 Host Controller (rev 02)
0b:00.0 Ethernet controller: VMware VMXNET3 Ethernet Controller (rev 01)
13:00.0 USB controller: Renesas Technology Corp. uPD720202 USB 3.0 Host Controller (rev 02)
1b:00.0 USB controller: Renesas Technology Corp. uPD720202 USB 3.0 Host Controller (rev 02)
lsusb output before connecting a device to the adaptor:
Bus 008 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 007 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 006 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 005 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 004 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 003 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Viewing dmesg output when trying to connect a device, depending on device (USB hub, USB data drive etc.), I either get no additional output or:
[ 818.213414] xhci_hcd 0000:04:00.0: Abort failed to stop command ring: -110
[ 818.213435] xhci_hcd 0000:04:00.0: xHCI host controller not responding, assume dead
[ 818.213453] xhci_hcd 0000:04:00.0: HC died; cleaning up
[ 818.213467] xhci_hcd 0000:04:00.0: WARNING: Host System Error
[ 818.213497] xhci_hcd 0000:04:00.0: Error while assigning device slot ID
[ 818.213511] xhci_hcd 0000:04:00.0: Max number of devices this xHCI host supports is 32.
[ 818.213526] usb usb4-port1: couldn't allocate usb_device
[ 818.213538] xhci_hcd 0000:04:00.0: Error while assigning device slot ID
[ 818.213551] xhci_hcd 0000:04:00.0: Max number of devices this xHCI host supports is 32.
[ 818.213565] usb usb3-port1: couldn't allocate usb_device
Clearly a device is being detected in the VM but something is preventing it from connecting successfully.
Things I tried (Ref https://bbs.archlinux.org/viewtopic.php?id=236536):
- Setting the kernel parameters iommu=soft pci=nomsi
- Setting the kernel parameter intel_iommu=off
- Different VMs (as mentioned above)
- Ensuring the USB devices are connected on VM boot
- Setting the ESXi host power management to 'High performance'
To confirm I wasn't dealing with a hardware/power issue I ditched the hypervisor on the same hardware by performing a boot from USB (LinuxMint). Here the PCIe adaptor worked correctly with all connected devices so I think that can be ruled out.
At the moment my suspicions are:
- This uPD720202 chipset isn't compatible with ESXi, even though many posts exist online suggesting it should work fine (which is why I bought it).
- I'm missing some critical configuration within the ESIx host.
If anyone spots anything obvious that I could try, or would like additional output that would be greatly appreciated.
vmware-esxi passthrough
asked Jun 5 at 18:00
MarcFMarcF
15819
I have a fresh install of ESXi 6.7U2 up and running on which I can successfully start VMs. To this setup I've added a StarTech.com 4 Port USB 3.0 Card Adapter (PEXUSB3S44V). This has been configured for pass through following https://kb.vmware.com/s/article/1010789. This can be configured for passthrough to a VM:
The USB PCIe controllers are detected here as uPD720202 USB 3.0 Host Controller. A single PCIe adaptor has 4 independent controllers and I've chosen to pass then all through for simplicity while trying to debug this issue.
I have tried using both Amazon Linux 2 (on prem) and Ubuntu Server 18.04.2 for the VMs, what follows appears to be equivalent in both of them.
lspci output shows the expected pass through devices:
00:00.0 Host bridge: Intel Corporation 440BX/ZX/DX - 82443BX/ZX/DX Host bridge (rev 01)
00:01.0 PCI bridge: Intel Corporation 440BX/ZX/DX - 82443BX/ZX/DX AGP bridge (rev 01)
00:07.0 ISA bridge: Intel Corporation 82371AB/EB/MB PIIX4 ISA (rev 08)
00:07.1 IDE interface: Intel Corporation 82371AB/EB/MB PIIX4 IDE (rev 01)
00:07.3 Bridge: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 08)
00:07.7 System peripheral: VMware Virtual Machine Communication Interface (rev 10)
00:0f.0 VGA compatible controller: VMware SVGA II Adapter
00:10.0 SCSI storage controller: LSI Logic / Symbios Logic 53c1030 PCI-X Fusion-MPT Dual Ultra320 SCSI (rev 01)
00:11.0 PCI bridge: VMware PCI bridge (rev 02)
00:15.0 PCI bridge: VMware PCI Express Root Port (rev 01)
00:15.1 PCI bridge: VMware PCI Express Root Port (rev 01)
00:15.2 PCI bridge: VMware PCI Express Root Port (rev 01)
00:15.3 PCI bridge: VMware PCI Express Root Port (rev 01)
00:15.4 PCI bridge: VMware PCI Express Root Port (rev 01)
00:15.5 PCI bridge: VMware PCI Express Root Port (rev 01)
00:15.6 PCI bridge: VMware PCI Express Root Port (rev 01)
00:15.7 PCI bridge: VMware PCI Express Root Port (rev 01)
00:16.0 PCI bridge: VMware PCI Express Root Port (rev 01)
00:16.1 PCI bridge: VMware PCI Express Root Port (rev 01)
00:16.2 PCI bridge: VMware PCI Express Root Port (rev 01)
00:16.3 PCI bridge: VMware PCI Express Root Port (rev 01)
00:16.4 PCI bridge: VMware PCI Express Root Port (rev 01)
00:16.5 PCI bridge: VMware PCI Express Root Port (rev 01)
00:16.6 PCI bridge: VMware PCI Express Root Port (rev 01)
00:16.7 PCI bridge: VMware PCI Express Root Port (rev 01)
00:17.0 PCI bridge: VMware PCI Express Root Port (rev 01)
00:17.1 PCI bridge: VMware PCI Express Root Port (rev 01)
00:17.2 PCI bridge: VMware PCI Express Root Port (rev 01)
00:17.3 PCI bridge: VMware PCI Express Root Port (rev 01)
00:17.4 PCI bridge: VMware PCI Express Root Port (rev 01)
00:17.5 PCI bridge: VMware PCI Express Root Port (rev 01)
00:17.6 PCI bridge: VMware PCI Express Root Port (rev 01)
00:17.7 PCI bridge: VMware PCI Express Root Port (rev 01)
00:18.0 PCI bridge: VMware PCI Express Root Port (rev 01)
00:18.1 PCI bridge: VMware PCI Express Root Port (rev 01)
00:18.2 PCI bridge: VMware PCI Express Root Port (rev 01)
00:18.3 PCI bridge: VMware PCI Express Root Port (rev 01)
00:18.4 PCI bridge: VMware PCI Express Root Port (rev 01)
00:18.5 PCI bridge: VMware PCI Express Root Port (rev 01)
00:18.6 PCI bridge: VMware PCI Express Root Port (rev 01)
00:18.7 PCI bridge: VMware PCI Express Root Port (rev 01)
02:01.0 SATA controller: VMware SATA AHCI controller
03:00.0 USB controller: Renesas Technology Corp. uPD720202 USB 3.0 Host Controller (rev 02)
04:00.0 USB controller: Renesas Technology Corp. uPD720202 USB 3.0 Host Controller (rev 02)
0b:00.0 Ethernet controller: VMware VMXNET3 Ethernet Controller (rev 01)
13:00.0 USB controller: Renesas Technology Corp. uPD720202 USB 3.0 Host Controller (rev 02)
1b:00.0 USB controller: Renesas Technology Corp. uPD720202 USB 3.0 Host Controller (rev 02)
lsusb output before connecting a device to the adaptor:
Bus 008 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 007 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 006 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 005 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 004 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 003 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Viewing dmesg output when trying to connect a device, depending on device (USB hub, USB data drive etc.), I either get no additional output or:
[ 818.213414] xhci_hcd 0000:04:00.0: Abort failed to stop command ring: -110
[ 818.213435] xhci_hcd 0000:04:00.0: xHCI host controller not responding, assume dead
[ 818.213453] xhci_hcd 0000:04:00.0: HC died; cleaning up
[ 818.213467] xhci_hcd 0000:04:00.0: WARNING: Host System Error
[ 818.213497] xhci_hcd 0000:04:00.0: Error while assigning device slot ID
[ 818.213511] xhci_hcd 0000:04:00.0: Max number of devices this xHCI host supports is 32.
[ 818.213526] usb usb4-port1: couldn't allocate usb_device
[ 818.213538] xhci_hcd 0000:04:00.0: Error while assigning device slot ID
[ 818.213551] xhci_hcd 0000:04:00.0: Max number of devices this xHCI host supports is 32.
[ 818.213565] usb usb3-port1: couldn't allocate usb_device
Clearly a device is being detected in the VM but something is preventing it from connecting successfully.
Things I tried (Ref https://bbs.archlinux.org/viewtopic.php?id=236536):
- Setting the kernel parameters iommu=soft pci=nomsi
- Setting the kernel parameter intel_iommu=off
- Different VMs (as mentioned above)
- Ensuring the USB devices are connected on VM boot
- Setting the ESXi host power management to 'High performance'
To confirm I wasn't dealing with a hardware/power issue I ditched the hypervisor on the same hardware by performing a boot from USB (LinuxMint). Here the PCIe adaptor worked correctly with all connected devices so I think that can be ruled out.
At the moment my suspicions are:
- This uPD720202 chipset isn't compatible with ESXi, even though many posts exist online suggesting it should work fine (which is why I bought it).
- I'm missing some critical configuration within the ESIx host.
If anyone spots anything obvious that I could try, or would like additional output that would be greatly appreciated.
vmware-esxi passthrough
vmware-esxi passthrough
asked Jun 5 at 18:00
MarcFMarcF
15819
asked Jun 5 at 18:00
MarcFMarcF
15819
asked Jun 5 at 18:00
MarcFMarcF
15819
asked Jun 5 at 18:00
MarcFMarcF
15819
asked Jun 5 at 18:00
MarcFMarcF
15819
15819
add a comment |
add a comment |
1 Answer
1
active
oldest
votes
Looks like it was option 1, hardware compatibility.
I swapped the StarTech.com 4 Port USB 3.0 Card Adapter (PEXUSB3S44V) for a 4 Port HighPoint RocketU RU1144D which now works as expected.
answered Jun 7 at 14:49
MarcFMarcF
15819
MANY PCIe devices aren't compatible, in fact there's a HCL for just that reason - that said we had similar needs for USB and chose to use a USB-over-IP server which worked great and also allowed for vMotions/DRS.
– Chopper3
Jun 7 at 15:00
The HCL is TERRRIBBLEE! I tried to find a section in it for USB PCIe cards and couldn't. If you can highlight how to locate this it will most likely help others in future.
– MarcF
Jun 7 at 15:05
vmw keep changing the urls or I would have.
– Chopper3
Jun 9 at 8:35
add a comment |
Your Answer
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "2"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fserverfault.com%2fquestions%2f970264%2fesxi-usb-pcie-passthrough-failing-to-connect-connected-devices%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
Looks like it was option 1, hardware compatibility.
I swapped the StarTech.com 4 Port USB 3.0 Card Adapter (PEXUSB3S44V) for a 4 Port HighPoint RocketU RU1144D which now works as expected.
answered Jun 7 at 14:49
MarcFMarcF
15819
MANY PCIe devices aren't compatible, in fact there's a HCL for just that reason - that said we had similar needs for USB and chose to use a USB-over-IP server which worked great and also allowed for vMotions/DRS.
– Chopper3
Jun 7 at 15:00
The HCL is TERRRIBBLEE! I tried to find a section in it for USB PCIe cards and couldn't. If you can highlight how to locate this it will most likely help others in future.
– MarcF
Jun 7 at 15:05
vmw keep changing the urls or I would have.
– Chopper3
Jun 9 at 8:35
add a comment |
Looks like it was option 1, hardware compatibility.
I swapped the StarTech.com 4 Port USB 3.0 Card Adapter (PEXUSB3S44V) for a 4 Port HighPoint RocketU RU1144D which now works as expected.
answered Jun 7 at 14:49
MarcFMarcF
15819
MANY PCIe devices aren't compatible, in fact there's a HCL for just that reason - that said we had similar needs for USB and chose to use a USB-over-IP server which worked great and also allowed for vMotions/DRS.
– Chopper3
Jun 7 at 15:00
The HCL is TERRRIBBLEE! I tried to find a section in it for USB PCIe cards and couldn't. If you can highlight how to locate this it will most likely help others in future.
– MarcF
Jun 7 at 15:05
vmw keep changing the urls or I would have.
– Chopper3
Jun 9 at 8:35
add a comment |
Looks like it was option 1, hardware compatibility.
I swapped the StarTech.com 4 Port USB 3.0 Card Adapter (PEXUSB3S44V) for a 4 Port HighPoint RocketU RU1144D which now works as expected.
answered Jun 7 at 14:49
MarcFMarcF
15819
Looks like it was option 1, hardware compatibility.
I swapped the StarTech.com 4 Port USB 3.0 Card Adapter (PEXUSB3S44V) for a 4 Port HighPoint RocketU RU1144D which now works as expected.
answered Jun 7 at 14:49
MarcFMarcF
15819
answered Jun 7 at 14:49
MarcFMarcF
15819
answered Jun 7 at 14:49
MarcFMarcF
15819
answered Jun 7 at 14:49
MarcFMarcF
15819
15819
MANY PCIe devices aren't compatible, in fact there's a HCL for just that reason - that said we had similar needs for USB and chose to use a USB-over-IP server which worked great and also allowed for vMotions/DRS.
– Chopper3
Jun 7 at 15:00
The HCL is TERRRIBBLEE! I tried to find a section in it for USB PCIe cards and couldn't. If you can highlight how to locate this it will most likely help others in future.
– MarcF
Jun 7 at 15:05
vmw keep changing the urls or I would have.
– Chopper3
Jun 9 at 8:35
add a comment |
MANY PCIe devices aren't compatible, in fact there's a HCL for just that reason - that said we had similar needs for USB and chose to use a USB-over-IP server which worked great and also allowed for vMotions/DRS.
– Chopper3
Jun 7 at 15:00
The HCL is TERRRIBBLEE! I tried to find a section in it for USB PCIe cards and couldn't. If you can highlight how to locate this it will most likely help others in future.
– MarcF
Jun 7 at 15:05
vmw keep changing the urls or I would have.
– Chopper3
Jun 9 at 8:35
MANY PCIe devices aren't compatible, in fact there's a HCL for just that reason - that said we had similar needs for USB and chose to use a USB-over-IP server which worked great and also allowed for vMotions/DRS.
– Chopper3
Jun 7 at 15:00
MANY PCIe devices aren't compatible, in fact there's a HCL for just that reason - that said we had similar needs for USB and chose to use a USB-over-IP server which worked great and also allowed for vMotions/DRS.
– Chopper3
Jun 7 at 15:00
The HCL is TERRRIBBLEE! I tried to find a section in it for USB PCIe cards and couldn't. If you can highlight how to locate this it will most likely help others in future.
– MarcF
Jun 7 at 15:05
The HCL is TERRRIBBLEE! I tried to find a section in it for USB PCIe cards and couldn't. If you can highlight how to locate this it will most likely help others in future.
– MarcF
Jun 7 at 15:05
vmw keep changing the urls or I would have.
– Chopper3
Jun 9 at 8:35
vmw keep changing the urls or I would have.
– Chopper3
Jun 9 at 8:35
add a comment |
Thanks for contributing an answer to Server Fault!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fserverfault.com%2fquestions%2f970264%2fesxi-usb-pcie-passthrough-failing-to-connect-connected-devices%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
