Bug #1956849 “Almost all USB ports suddenly stopped working; unb...” : Bugs : linux package : Ubuntu

Revision history for this message

dgatwood (dgatwood) wrote on 2022-01-09:

#1

Reported by another person yesterday:

https://askubuntu.com/questions/1385823/some-usb-ports-suddenly-stop-working-not-only-mouse-keyboard

Revision history for this message

Aaron (aklinker1) wrote on 2022-01-09:

#2

What kernel version did you have to revert to? I'm also experiencing this bug

Revision history for this message

Jazz (jazzzz) wrote on 2022-01-09:

#3

I had the same issue, also with 5.11.0-44-generic, reverting to kernel 5.11.0-41-generic fixes it. Prior to this I upgraded from Ubuntu 21.04 to 21.10 and no USB port worked, with kernel 5.13.0-23-generic.

Revision history for this message

dgatwood (dgatwood) wrote on 2022-01-09:

#4

-38 was fine.

Revision history for this message

Heitor Alves de Siqueira (halves) wrote on 2022-01-10:

#5

@dgatwood would you be able to upload kernel logs and lsusb output from the -44 kernel? From what you're describing it seems like a possible issue with host controllers, but it's hard to tell without looking at what is going in the kernel.

You can use `apport-collect 1956849` to upload relevant data directly to this LP bug.

Revision history for this message

Matthew Ruffell (mruffell) wrote on 2022-01-10:

#6

Hi @dgatwood, @Jazz. I took a look at the USB patches that landed between 5.11.0-41-generic and 5.11.0-44-generic in the Focal HWE tree:

https://paste.ubuntu.com/p/mfJCw5XmBv/

One commit which I thought was interesting was:

commit 5caa90d5fedfcdf57a42c272896cfd0ef6b5c667
commit-upstream: 5255660b208aebfdb71d574f3952cf48392f4306
Author: Jonathan Bell <email address hidden>
Date: Fri Oct 8 12:25:44 2021 +0300
Subject: xhci: add quirk for host controllers that don't update endpoint DCS
Link: https://github.com/torvalds/linux/commit/5255660b208aebfdb71d574f3952cf48392f4306

I took a look at the Amazon B08PF8XR73 card you have:

https://www.amazon.com/YEELIYA-PCI-7-Port-USB-C-USB/dp/B08PF8XR73

The page mentions "Advanced Dual Chip Uses Fresco Logic 1100 +VL820 dual chips".

The above commit is a quirk for VIA VL8xx chipsets, and looking at the code:

if (pdev->vendor == PCI_VENDOR_ID_VIA && pdev->device == 0x3483) {
xhci->quirks |= XHCI_LPM_SUPPORT;
xhci->quirks |= XHCI_EP_CTX_BROKEN_DCS;
}

It was applied to PCI_VENDOR_ID_VIA && pdev->device == 0x3483. Looking these up:

https://devicehunt.com/view/type/pci/vendor/1106/device/3483

We see that the VL820 is listed:

VX800/820-Series PCI-Express Root Port 1

So this new piece of code would apply to your PCIe card.

There was a similar bug filed a few days ago:

https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1956542

Looking at the lsusb output, they are also using a VIA Labs VL813 Hub, and USB 3 no longer works.

Bus 003 Device 056: ID 2109:2813 VIA Labs, Inc. VL813 Hub

In this case though, the VL813 is under a different vendor and device ID, and the code in that commit would not apply:

https://devicehunt.com/view/type/usb/vendor/2109/device/2813

Since its 0x2813 and not 0x3483.

Still, this is very interesting.

@Jazz, does your system have any "VIA Labs, Inc" USB host devices? Check "lsusb -v".

Note that the commit was also present in 5.13.0-23-generic on Impish, but is not in 5.13.0-22-generic.

@dgatwood, could you please collect "lsusb -v" as well, and maybe try 5.11.0-41-generic?

Thanks,
Matthew

Hi @dgatwood, @Jazz. I took a look at the USB patches that landed between 5.11.0-41-generic and 5.11.0-44-generic in the Focal HWE tree:

https://paste.ubuntu.com/p/mfJCw5XmBv/

One commit which I thought was interesting was:

commit 5caa90d5fedfcdf57a42c272896cfd0ef6b5c667
commit-upstream: 5255660b208aebfdb71d574f3952cf48392f4306
Author: Jonathan Bell <jonathan@raspberrypi.org>
Date:   Fri Oct 8 12:25:44 2021 +0300
Subject: xhci: add quirk for host controllers that don't update endpoint DCS
Link: https://github.com/torvalds/linux/commit/5255660b208aebfdb71d574f3952cf48392f4306

I took a look at the Amazon B08PF8XR73 card you have:

https://www.amazon.com/YEELIYA-PCI-7-Port-USB-C-USB/dp/B08PF8XR73

The page mentions "Advanced Dual Chip Uses Fresco Logic 1100 +VL820 dual chips".

The above commit is a quirk for VIA VL8xx chipsets, and looking at the code:

if (pdev->vendor == PCI_VENDOR_ID_VIA && pdev->device == 0x3483) {
		xhci->quirks |= XHCI_LPM_SUPPORT;
		xhci->quirks |= XHCI_EP_CTX_BROKEN_DCS;
	}

It was applied to PCI_VENDOR_ID_VIA && pdev->device == 0x3483. Looking these up:

https://devicehunt.com/view/type/pci/vendor/1106/device/3483

We see that the VL820 is listed:

VX800/820-Series PCI-Express Root Port 1

So this new piece of code would apply to your PCIe card.

There was a similar bug filed a few days ago:

https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1956542

Looking at the lsusb output, they are also using a VIA Labs VL813 Hub, and USB 3 no longer works.

Bus 003 Device 056: ID 2109:2813 VIA Labs, Inc. VL813 Hub

In this case though, the VL813 is under a different vendor and device ID, and the code in that commit would not apply:

https://devicehunt.com/view/type/usb/vendor/2109/device/2813

Since its 0x2813 and not 0x3483.

Still, this is very interesting.

@Jazz, does your system have any "VIA Labs, Inc" USB host devices? Check "lsusb -v".

Note that the commit was also present in 5.13.0-23-generic on Impish, but is not in 5.13.0-22-generic.

@dgatwood, could you please collect "lsusb -v" as well, and maybe try 5.11.0-41-generic?

Thanks,
Matthew

Revision history for this message

dgatwood (dgatwood) wrote on 2022-01-10:

#7

lsusb -v -v -v Edit (25.8 KiB, text/plain)

Revision history for this message

dgatwood (dgatwood) wrote on 2022-01-10:

#8

lsusb -v from working kernel.txt Edit (35.1 KiB, text/plain)

Also attached lsusb output from a working kernel for comparison. And yes, it looks like this is a VIA-chipset-specific regression.

I stand corrected about the Mac's built-in USB-C ports. Those don't work even after rolling back the kernel to the previous version. I'll file a separate bug for that.

Revision history for this message

Jazz (jazzzz) wrote on 2022-01-10:

#9

Yes, I have "VIA Labs, Inc. Hub" device:

Bus 002 Device 010: ID 2109:2811 VIA Labs, Inc. Hub
Device Descriptor:
  bLength 18
  bDescriptorType 1
  bcdUSB 2.10
  bDeviceClass 9 Hub
  bDeviceSubClass 0
  bDeviceProtocol 1 Single TT
  bMaxPacketSize0 64
  idVendor 0x2109 VIA Labs, Inc.
  idProduct 0x2811 Hub
  bcdDevice 90.90
  iManufacturer 1 VIA Labs, Inc.
  iProduct 2 USB2.0 Hub
  iSerial 0
  bNumConfigurations 1
  Configuration Descriptor:
    bLength 9
    bDescriptorType 2
    wTotalLength 0x0019
    bNumInterfaces 1
    bConfigurationValue 1
    iConfiguration 0
    bmAttributes 0xe0
      Self Powered
      Remote Wakeup
    MaxPower 0mA
    Interface Descriptor:
      bLength 9
      bDescriptorType 4
      bInterfaceNumber 0
      bAlternateSetting 0
      bNumEndpoints 1
      bInterfaceClass 9 Hub
      bInterfaceSubClass 0
      bInterfaceProtocol 0 Full speed (or root) hub
      iInterface 0
      Endpoint Descriptor:
        bLength 7
        bDescriptorType 5
        bEndpointAddress 0x81 EP 1 IN
        bmAttributes 3
          Transfer Type Interrupt
          Synch Type None
          Usage Type Data
        wMaxPacketSize 0x0001 1x 1 bytes
        bInterval 12

Revision history for this message

Matthew Ruffell (mruffell) wrote on 2022-01-12:

#10

Hi @dgatwood, @jazzzz,

I think we should try a test kernel with the VIA chipset changes reverted, to see if it is the commit causing you issues.

I have a couple of test kernels based on 5.11.0-44-generic for Focal HWE and Hirsute, and 5.13.0-23-generic for Impish currently building, and they have the below commit reverted:

commit 5caa90d5fedfcdf57a42c272896cfd0ef6b5c667
commit-upstream: 5255660b208aebfdb71d574f3952cf48392f4306
Author: Jonathan Bell <email address hidden>
Date: Fri Oct 8 12:25:44 2021 +0300
Subject: xhci: add quirk for host controllers that don't update endpoint DCS
Link: https://github.com/torvalds/linux/commit/5255660b208aebfdb71d574f3952cf48392f4306

Its building in https://launchpad.net/~mruffell/+archive/ubuntu/lp1956849-test and will be ready in a few hours, although I will post proper instructions on how to install the test kernel tomorrow.

I will write back once the kernels have finished building, and I have had a chance to boot test them.

Thanks,
Matthew

Revision history for this message

Matthew Ruffell (mruffell) wrote on 2022-01-12:

#11

Hi @dgatwood, @jazzzz,

The test kernels have finished building and are ready for you to try.

Like I said before, they are 5.11.0-44-generic for Focal HWE and Hirsute, and 5.13.0-23-generic for Impish, with just the one commit reverted:

commit 5caa90d5fedfcdf57a42c272896cfd0ef6b5c667
commit-upstream: 5255660b208aebfdb71d574f3952cf48392f4306
Author: Jonathan Bell <email address hidden>
Date: Fri Oct 8 12:25:44 2021 +0300
Subject: xhci: add quirk for host controllers that don't update endpoint DCS
Link: https://github.com/torvalds/linux/commit/5255660b208aebfdb71d574f3952cf48392f4306

Let's see if this is the commit which is causing you issues. Could you please install a test kernel and let me know if your USB ports work again?

Please note, these test packages are NOT SUPPORTED by Canonical and are for TEST PURPOSES ONLY. ONLY install in a dedicated test environment.

Instructions to install:
1) sudo add-apt-repository ppa:mruffell/lp1956849-test
2) sudo apt update

For Focal HWE and Hirsute users:
3) sudo apt install linux-image-unsigned-5.11.0-44-generic linux-modules-5.11.0-44-generic linux-modules-extra-5.11.0-44-generic linux-headers-5.11.0-44-generic
For Impish users:
3) sudo apt install linux-image-unsigned-5.13.0-23-generic linux-modules-5.13.0-23-generic linux-modules-extra-5.13.0-23-generic linux-headers-5.13.0-23-generic

4) sudo reboot
5) uname -rv
Focal:
5.11.0-44-generic #48~20.04.2+TEST1956849v20220112b1-Ubuntu SMP Wed Jan 12 02:26:0
Hirsute:
5.11.0-44-generic #48+TEST1956849v20220112b1-Ubuntu SMP Wed Jan 12 02:31:07 UTC 20
Impish:
5.13.0-23-generic #23+TEST1956849v20220112b1-Ubuntu SMP Wed Jan 12 02:37:08 UTC 20

If you are asked to abort the current kernel removal, say no.

You may need to change your grub config to boot the correct kernel. You can follow these instructions to do that: https://paste.ubuntu.com/p/WGpCWTPyTj/

Please make sure the uname is correct on boot. Sometimes newer kernels get pulled in due to metapackage dependencies not liking the linux-image-unsigned package.

Let me know if your USB ports start to work again. If they do, we can revert the commit in the Ubuntu kernels, and go and report the issue upstream and get it fixed. If it doesn't, then we will need to do some more tests to try find the commit which introduced the regression.

Thanks,
Matthew

Hi @dgatwood, @jazzzz,

The test kernels have finished building and are ready for you to try.

Like I said before, they are 5.11.0-44-generic for Focal HWE and Hirsute, and 5.13.0-23-generic for Impish, with just the one commit reverted:

commit 5caa90d5fedfcdf57a42c272896cfd0ef6b5c667
commit-upstream: 5255660b208aebfdb71d574f3952cf48392f4306
Author: Jonathan Bell <jonathan@raspberrypi.org>
Date: Fri Oct 8 12:25:44 2021 +0300
Subject: xhci: add quirk for host controllers that don't update endpoint DCS
Link: https://github.com/torvalds/linux/commit/5255660b208aebfdb71d574f3952cf48392f4306