Slow flood of do_IRQ: No irq for vector

This change was made by a bot.

Did this issue start happening after an update/upgrade? Was there a prior kernel version where you were not having this particular problem?

Would it be possible for you to test the latest upstream kernel? Refer to https://wiki.ubuntu.com/KernelMainlineBuilds . Please test the latest v4.18 kernel[0].

If this bug is fixed in the mainline kernel, please add the following tag 'kernel-fixed-upstream'.

If the mainline kernel does not fix this bug, please add the tag: 'kernel-bug-exists-upstream'.

Once testing of the upstream kernel is complete, please mark this bug as "Confirmed".

Thanks in advance.

[0] http://kernel.ubuntu.com/~kernel-ppa/mainline/v4.18-rc5

Exactly. It never happened in ubuntu 17.10. Started happening after installing ubuntu 18.04. Can't say I know exactly what kernel version 17.10 was up to, sorry. It was the standard one, whatever dist-upgrade would have gave me.

I'll try the kernel version you proposed and update this bug after I try it.

Thanks!

Still happening:

doug@doug-dt:~$ uname -a
Linux doug-dt 4.18.0-041800rc5-generic #201807152130 SMP Sun Jul 15 21:32:10 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
doug@doug-dt:~$ dmesg | tail
[ 169.701341] usb 1-4: Warning! Unlikely big volume range (=4096), cval->res is probably wrong.
[ 169.701344] usb 1-4: [11] FU [Headset Playback Volume] ch = 1, val = 0/4096/1
[ 169.755604] usbcore: registered new interface driver snd-usb-audio
[ 2603.508893] do_IRQ: 15.38 No irq handler for vector
[ 2611.510716] do_IRQ: 15.38 No irq handler for vector
[ 2624.514513] do_IRQ: 15.38 No irq handler for vector
[ 2627.515471] do_IRQ: 15.38 No irq handler for vector
[ 2642.520161] do_IRQ: 15.38 No irq handler for vector
[ 2644.520907] do_IRQ: 15.38 No irq handler for vector
[ 2647.522088] do_IRQ: 15.38 No irq handler for vector
doug@doug-dt:~$

Same here. Not sure when this started:

~$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 18.04 LTS
Release: 18.04
Codename: bionic

~$ uname -a
Linux w00t 4.15.0-29-generic #31-Ubuntu SMP Tue Jul 17 15:39:52 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

~$ dmesg

[ 2864.829849] do_IRQ: 2.33 No irq handler for vector
[ 2865.830173] do_IRQ: 2.33 No irq handler for vector
[ 2866.830508] do_IRQ: 2.33 No irq handler for vector
[ 2867.830819] do_IRQ: 2.33 No irq handler for vector
[ 2868.831143] do_IRQ: 2.33 No irq handler for vector
[ 2874.833340] do_IRQ: 2.33 No irq handler for vector
[ 2875.833649] do_IRQ: 2.33 No irq handler for vector
[ 2876.833973] do_IRQ: 2.33 No irq handler for vector
[ 2877.834289] do_IRQ: 2.33 No irq handler for vector
[ 2878.834606] do_IRQ: 2.33 No irq handler for vector
[ 2879.834868] do_IRQ: 2.33 No irq handler for vector
[ 2880.835212] do_IRQ: 2.33 No irq handler for vector
[ 2881.835559] do_IRQ: 2.33 No irq handler for vector
[ 2882.835855] do_IRQ: 2.33 No irq handler for vector
[ 2883.836186] do_IRQ: 2.33 No irq handler for vector
[ 2889.838551] do_IRQ: 2.33 No irq handler for vector
[ 2890.838868] do_IRQ: 2.33 No irq handler for vector
[ 2892.839543] do_IRQ: 2.33 No irq handler for vector

Ubuntu does not appear to be the only one with this issue on the 4.15 kernel:
https://bugzilla.redhat.com/show_bug.cgi?id=1551605

I'd also like to mention that my system was a clean 18.04 install. Also, would it be beneficial if I added my system info somehow? If so, is there a way to add the same type of info Doug added without generating a whole new bug report with say "ubuntu-bug linux"??

Do you see this after S3?

Mine is a desktop, and no suspend occurs, ever. Are you (Kai-Heng Feng)
asking me to suspend it and wake it up and see if it goes away?

On Mon, Sep 24, 2018 at 10:45 AM Kai-Heng Feng <email address hidden>
wrote:

> Do you see this after S3?
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1781016
>
> Title:
> Slow flood of do_IRQ: No irq for vector
>
> Status in linux package in Ubuntu:
> Incomplete
>
> Bug description:
> Every few seconds I get something like this in my dmesg:
>
> [44984.933221] do_IRQ: 1.34 No irq handler for vector
> [44986.933976] do_IRQ: 1.34 No irq handler for vector
> [44991.935614] do_IRQ: 1.34 No irq handler for vector
> [45004.938590] do_IRQ: 1.34 No irq handler for vector
> [45005.939002] do_IRQ: 4.34 No irq handler for vector
> [45034.949161] do_IRQ: 4.34 No irq handler for vector
> [45051.954569] do_IRQ: 4.34 No irq handler for vector
> [45058.956019] do_IRQ: 4.34 No irq handler for vector
> [45062.957727] do_IRQ: 4.34 No irq handler for vector
> [45063.958204] do_IRQ: 4.34 No irq handler for vector
> [45073.961103] do_IRQ: 4.34 No irq handler for vector
>
> The numbers vary. Some examples are 4.34, 4.36, 1.34, 0.33, 6.34,
> 7.33, 5.37. Eventually dmesg's ring buffer is full and my entire log
> is just messages like above with varying #.## numbers.
>
> I researched it and tried disabling irqbalance. It had no effect, the
> messages continued.
>
> The system is a Ryzen 2700X with B350 chipset on Asus Prime B350-PLUS.
> I have 64GB of Kingston DDR4-2400 ECC memory and Samsung 960 PRO M.2
> NVMe SSD, and an Intel I350-T4 quad port gigabit NIC. This issue never
> occurred before installing 18.03 on the same hardware. No overclocking
> whatsoever has ever been done to this machine.
>
> Also, I had to disable selective suspend on USB in this version to
> prevent major USB issues, which was not required in 17.10, that might
> be related.
>
> My system seems stable, but spurious IRQs can't be good. I'd like to
> resolve this if possible.
>
> ProblemType: Bug
> DistroRelease: Ubuntu 18.04
> Package: linux-image-4.15.0-26-generic 4.15.0-26.28
> ProcVersionSignature: Ubuntu 4.15.0-26.28-generic 4.15.18
> Uname: Linux 4.15.0-26-generic x86_64
> NonfreeKernelModules: nvidia_modeset nvidia
> ApportVersion: 2.20.9-0ubuntu7.2
> Architecture: amd64
> AudioDevicesInUse:
> USER PID ACCESS COMMAND
> /dev/snd/controlC0: gdm 2145 F.... pulseaudio
> doug 3736 F.... pulseaudio
> /dev/snd/controlC1: gdm 2145 F.... pulseaudio
> doug 3736 F.... pulseaudio
> Date: Tue Jul 10 12:49:00 2018
> HibernationDevice:
>
> InstallationDate: Installed on 2017-06-26 (379 days ago)
> InstallationMedia: Ubuntu 16.04 LTS "Xenial Xerus" - Release amd64
> (20160420.1)
> MachineType: System manufacturer System Product Name
> ProcEnviron:
> LANGUAGE=en_CA:en
> TERM=xterm-256color
> PATH=(custom, no user)
> LANG=en_CA.UTF-8
> SHELL=/bin/bash
> ProcFB:
>
> ProcKernelCmdLine: BOOT_I...

I am also affected by this issue, however on a wholly different platform - an intel H370 based motherboard with a Core i3-8100 CPU. It was suggested that I attempt to supress logging output via a sysrq keyboard combo, so I pressed alt+sysrq+0 but this appears to be disabled by default as I saw the following printed to the TTY along with the IRQ handler messages:

sysrq: SysRq : This sysrq operation is disabled.

It would be great if some proper attention would be paid to this issue as it makes the use of such systems via TTY extremely frustrating.

Should I perhaps try the most up to date mainline kernel I can find?

Thanks

Please install and boot kernel [1] which enables IRQ debugfs.

Then you can find the "do_IRQ: X.Y" means. On my system:

# ls /sys/kernel/debug/irq/irqs
0 1 10 11 12 120 121 122 123 124 125 126 127 128 129 13 130 131 132 133 14 15 16 17 18 19 2 3 4 5 51 6 67 7 8 9

# cat /sys/kernel/debug/irq/irqs/129
handler: handle_edge_irq
device: 0000:01:00.0
status: 0x00004000
istate: 0x00000000
ddepth: 0
wdepth: 0
dstate: 0x01409200
            IRQD_ACTIVATED
            IRQD_IRQ_STARTED
            IRQD_SINGLE_TARGET
            IRQD_MOVE_PCNTXT
            IRQD_AFFINITY_SET
node: -1
affinity: 4
effectiv: 4
pending:
domain: INTEL-IR-MSI-1-3
 hwirq: 0x80000
 chip: IR-PCI-MSI
  flags: 0x30
             IRQCHIP_SKIP_SET_WAKE
             IRQCHIP_ONESHOT_SAFE
 parent:
    domain: INTEL-IR-1
     hwirq: 0x1c0000
     chip: INTEL-IR
      flags: 0x0
     parent:
        domain: VECTOR
         hwirq: 0x81
         chip: APIC
          flags: 0x0
         Vector: 34
         Target: 4
         move_in_progress: 0
         is_managed: 0
         can_reserve: 0
         has_reserved: 0
         cleanup_pending: 0

Target and Vector are the values in "do_IRQ: Target.Vector".

The affected device can be found by finding the matched Target and Vector.

[1] https://people.canonical.com/~khfeng/lp1781016-irq-debug/

Many thanks, I have given this a try today but unfortunately can't locate the specific Target & Vector combination. Here's what I saw:

$ uname -r
4.20.0-1-generic

$ dmesg | tail
[ 167.279809] do_IRQ: 1.35 No irq handler for vector

# grep 35 /sys/kernel/debug/irq/irqs/*
/sys/kernel/debug/irq/irqs/125: Vector: 35
/sys/kernel/debug/irq/irqs/126: Vector: 35

# cat /sys/kernel/debug/irq/irqs/125
handler: handle_edge_irq
device: 0000:00:02.0
status: 0x00000000
istate: 0x00000000
ddepth: 0
wdepth: 0
dstate: 0x01401300
            IRQD_ACTIVATED
            IRQD_IRQ_STARTED
            IRQD_SINGLE_TARGET
            IRQD_AFFINITY_SET
            IRQD_SETAFFINITY_PENDING
node: -1
affinity: 0-3
effectiv: 2
pending: 1
domain: PCI-MSI-3
 hwirq: 0x8000
 chip: PCI-MSI
  flags: 0x30
             IRQCHIP_SKIP_SET_WAKE
             IRQCHIP_ONESHOT_SAFE
 parent:
    domain: VECTOR
     hwirq: 0x7d
     chip: APIC
      flags: 0x0
     Vector: 35
     Target: 2
     move_in_progress: 0
     is_managed: 0
     can_reserve: 0
     has_reserved: 0
     cleanup_pending: 0

# cat /sys/kernel/debug/irq/irqs/126
handler: handle_edge_irq
device: 0000:00:16.0
status: 0x00000000
istate: 0x00000000
ddepth: 0
wdepth: 0
dstate: 0x01400200
            IRQD_ACTIVATED
            IRQD_IRQ_STARTED
            IRQD_SINGLE_TARGET
node: -1
affinity: 0-3
effectiv: 3
pending:
domain: PCI-MSI-3
 hwirq: 0x58000
 chip: PCI-MSI
  flags: 0x30
             IRQCHIP_SKIP_SET_WAKE
             IRQCHIP_ONESHOT_SAFE
 parent:
    domain: VECTOR
     hwirq: 0x7e
     chip: APIC
      flags: 0x0
     Vector: 35
     Target: 3
     move_in_progress: 0
     is_managed: 0
     can_reserve: 0
     has_reserved: 0
     cleanup_pending: 0

Some suggestions from friends with experience in this matter run along the lines of attempting to boot with an ACPI_OSI kernel parameter given the bugs reported within my dmesg, shall I give that a try? Alternatively, I am game for any further suggestions! :)

daftykins:

Your issue is there is no handler so you won't find anything in debug/irg for it:

[ 167.279809] do_IRQ: 1.35 No irq handler for vector

HOWEVER, you should be able to find which target device is responsible by searching for matching Target: value, e.g. something like:

$ grep -l 'Target: 1' /sys/kernel/debug/irq/irqs/*

Examine/report the matching targets which should help identify the PCI bus *device* id (e.g. "device: 0000:00:16.0" in the report for irqs/126). Correlate that to the output of:

$ lspci -nnk

From our previous IRC discussions you already know I believe it is the NIC since these reports only start once the NIC interface moves to UP state.

TJ,

I think Target is the CPU#, not a device identity.

Running an Intel S2600ST motherboard with dual Gold 6148 I get this error regularly after the last kernel update I did.

The number before the dot seems to be a CPU number since I see many varied numbers and there's 40 cores/80 threads on my motherboard
e.g.
[Wed Dec 26 14:20:25 2018] do_IRQ: 14.216 No irq handler for vector
[Wed Dec 26 18:35:32 2018] do_IRQ: 46.40 No irq handler for vector
[Thu Dec 27 00:56:21 2018] do_IRQ: 15.55 No irq handler for vector
(there are 37 of these over the last 2 weeks)

There's no /sys/kernel/debug/irq/ folder

The current kernel on "Ubuntu 16.04.5 LTS" I am using is "vmlinuz-4.4.0-140-generic"

I updated from "vmlinuz-4.4.0-119-generic" (which was also 16.04 LTS)

I've had this server for over a year and with the 4.4.0-140 update is the first time I've seen this.

A,

The default kernel doesn't enable IRQ debugfs so there's no "/sys/kernel/debug/irq".

HP Z820 Workstation machine
HP 158B motherboard
4.15.0-47-generic kernel
same error
...
[ 821.269834] do_IRQ: 4.38 No irq handler for vector
[ 823.270705] do_IRQ: 4.38 No irq handler for vector
[ 824.271303] do_IRQ: 4.38 No irq handler for vector
[ 826.272120] do_IRQ: 4.38 No irq handler for vector
[ 833.340194] do_IRQ: 4.38 No irq handler for vector
[ 838.342193] do_IRQ: 4.38 No irq handler for vector
...

On my system, I found that the immediate cause of those messages was running apcupsd, with the default configuration of trying to talk over a serial cable, but without a UPS connected to it. Even if it was otherwise configured for USB, I also had to remove the "DEVICE /dev/ttyS0" line to fix the IRQ errors.

Kyle - excellent find, I have confirmed that this was the cause of my issue, too. Thank you!

I had taken a quick glance at /etc/apcupsd/apcupsd.conf when I was failing to see communication working with my UPS, as I saw the expected lines to configure apcupsd for USB mode I had assumed there was a deeper problem and never got around to looking at it again. To confirm, the config contains these lines by default:

UPSTYPE usb
DEVICE /dev/ttyS0

I commented out the 'DEVICE' line like so:

UPSTYPE usb
#DEVICE /dev/ttyS0

Then I restarted the service with:

# sudo systemctl restart apcupsd

I no longer see the IRQ messages in dmesg output. Great! Does this apply to everyone else that has encountered this, I wonder?

Confirmed on my system as well. . .

Even though /etc/default/apcupsd  ISCONFIGURED=no  I was getting the
do_IRQ: 6.34 No irq handler for vector message continuously.

After commenting out the "DEVICE /dev/ttyS0" and restarting the apcupsd
service the no irq error has stopped.

Thankyou for this find

Rob A

On 2019-04-12 11:46 a.m., Kyle Brenneman wrote:
> On my system, I found that the immediate cause of those messages was
> running apcupsd, with the default configuration of trying to talk over a
> serial cable, but without a UPS connected to it. Even if it was
> otherwise configured for USB, I also had to remove the "DEVICE
> /dev/ttyS0" line to fix the IRQ errors.
>
--
Chief SW Engineering
Andrews Consulting
Ottawa, ON, CA
+1 613.435.7489 (Office)
+1 613.447.7154 (Cell)

+1 Kyle, configuring apcupsd properly solved the problem for me too.

Thanks for the info. Have anyone filed a bug against upstream apcupsd?

I have an APC UPS too. Great find. Who would have guessed an IRQ handling
problem message from the kernel was due to a serial or USB device. Wow.

On Thu, Apr 18, 2019 at 7:16 AM Kai-Heng Feng <email address hidden>
wrote:

> Thanks for the info. Have anyone filed a bug against upstream apcupsd?
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1781016
>
> Title:
> Slow flood of do_IRQ: No irq for vector
>
> Status in linux package in Ubuntu:
> Incomplete
>
> Bug description:
> Every few seconds I get something like this in my dmesg:
>
> [44984.933221] do_IRQ: 1.34 No irq handler for vector
> [44986.933976] do_IRQ: 1.34 No irq handler for vector
> [44991.935614] do_IRQ: 1.34 No irq handler for vector
> [45004.938590] do_IRQ: 1.34 No irq handler for vector
> [45005.939002] do_IRQ: 4.34 No irq handler for vector
> [45034.949161] do_IRQ: 4.34 No irq handler for vector
> [45051.954569] do_IRQ: 4.34 No irq handler for vector
> [45058.956019] do_IRQ: 4.34 No irq handler for vector
> [45062.957727] do_IRQ: 4.34 No irq handler for vector
> [45063.958204] do_IRQ: 4.34 No irq handler for vector
> [45073.961103] do_IRQ: 4.34 No irq handler for vector
>
> The numbers vary. Some examples are 4.34, 4.36, 1.34, 0.33, 6.34,
> 7.33, 5.37. Eventually dmesg's ring buffer is full and my entire log
> is just messages like above with varying #.## numbers.
>
> I researched it and tried disabling irqbalance. It had no effect, the
> messages continued.
>
> The system is a Ryzen 2700X with B350 chipset on Asus Prime B350-PLUS.
> I have 64GB of Kingston DDR4-2400 ECC memory and Samsung 960 PRO M.2
> NVMe SSD, and an Intel I350-T4 quad port gigabit NIC. This issue never
> occurred before installing 18.03 on the same hardware. No overclocking
> whatsoever has ever been done to this machine.
>
> Also, I had to disable selective suspend on USB in this version to
> prevent major USB issues, which was not required in 17.10, that might
> be related.
>
> My system seems stable, but spurious IRQs can't be good. I'd like to
> resolve this if possible.
>
> ProblemType: Bug
> DistroRelease: Ubuntu 18.04
> Package: linux-image-4.15.0-26-generic 4.15.0-26.28
> ProcVersionSignature: Ubuntu 4.15.0-26.28-generic 4.15.18
> Uname: Linux 4.15.0-26-generic x86_64
> NonfreeKernelModules: nvidia_modeset nvidia
> ApportVersion: 2.20.9-0ubuntu7.2
> Architecture: amd64
> AudioDevicesInUse:
> USER PID ACCESS COMMAND
> /dev/snd/controlC0: gdm 2145 F.... pulseaudio
> doug 3736 F.... pulseaudio
> /dev/snd/controlC1: gdm 2145 F.... pulseaudio
> doug 3736 F.... pulseaudio
> Date: Tue Jul 10 12:49:00 2018
> HibernationDevice:
>
> InstallationDate: Installed on 2017-06-26 (379 days ago)
> InstallationMedia: Ubuntu 16.04 LTS "Xenial Xerus" - Release amd64
> (20160420.1)
> MachineType: System manufacturer System Product Name
> ProcEnviron:
> LANGUAGE=en_CA:en
> TERM=xterm-256color
> PATH=(custom, no user)
> LANG=en_CA.UTF-8
> SHELL...