Undefined Behavior Sanitizer (UBSAN) causes failure to match symbols

Bug #2003374 reported by John Cabaj
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Ubuntu on IBM z Systems
Fix Committed
Undecided
Skipper Bug Screeners
dh-kpatches
Unknown
Unknown
kpatch (Ubuntu)
In Progress
Undecided
Unassigned
Jammy
In Progress
Undecided
Unassigned
Kinetic
Won't Fix
Undecided
Unassigned
linux (Ubuntu)
Incomplete
Undecided
Unassigned
Jammy
Fix Released
Undecided
Unassigned

Bug Description

[ Impact ]

 * When UBSAN is enabled in an s390x kernel configuration, kpatch-build can fail to find matching symbols in the vmlinux symbol table (see attached example_livepatch.patch). This was discovered in both Jammy 5.15 and Kinetic 5.19 kernels, where UBSAN was first enabled (releases up to Focal did not enable UBSAN). See attached kpatch-build console output (output.log) and kpatch-build log (build.log).

* Disabling UBAAN in s390x kernel configurations resolved the issue for both Jammy 5.15 and Kinetic 5.19. Possibly this could be fixed in kpatch/kpatch-build to continue to enable UBSAN while still allowing Livepatch functionality.

[ Test Plan ]

 * Use kpatch-build testcases to build and load a fs/proc/meminfo.c Livepatch on s390x kernel (see attached example_livepatch.patch). This should be successful.

[ Where problems could occur ]

 * A fix in kpatch/kpatch-build to properly handle UBSAN objects shouldn't yield any regressions. If UBSAN is disabled to ultimately get past this issue, it could lead to undefined behavior not being caught.

Revision history for this message
John Cabaj (john-cabaj) wrote :
Revision history for this message
John Cabaj (john-cabaj) wrote :
Revision history for this message
John Cabaj (john-cabaj) wrote :
John Cabaj (john-cabaj)
description: updated
description: updated
Revision history for this message
John Cabaj (john-cabaj) wrote :

Currently attempting to make changes to kpatch/kpatch-build

Changed in kpatch (Ubuntu Jammy):
assignee: nobody → John Cabaj (john-cabaj)
Frank Heimes (fheimes)
tags: added: s390x
bugproxy (bugproxy)
tags: added: architecture-s39064 bugnameltc-201635 severity-high targetmilestone-inin---
Revision history for this message
Frank Heimes (fheimes) wrote :

Please see the referenced upstream issue:
https://github.com/dynup/kpatch/issues/1328

John Cabaj (john-cabaj)
Changed in kpatch (Ubuntu Jammy):
assignee: John Cabaj (john-cabaj) → nobody
Frank Heimes (fheimes)
Changed in ubuntu-z-systems:
assignee: nobody → Skipper Bug Screeners (skipper-screen-team)
Revision history for this message
bugproxy (bugproxy) wrote : Comment bridged from LTC Bugzilla

------- Comment From <email address hidden> 2023-03-06 08:46 EDT-------
Thanks @Sumanth for your proposed patch which is now in approved state: https://github.com/dynup/kpatch/pull/1332

Frank Heimes (fheimes)
Changed in ubuntu-z-systems:
status: New → In Progress
Changed in kpatch (Ubuntu Jammy):
status: New → In Progress
Changed in kpatch (Ubuntu):
status: New → In Progress
Revision history for this message
John Cabaj (john-cabaj) wrote :

Adding Jammy package to affected so UBSAN can be re-enabled once kpatch is patched.

no longer affects: linux (Ubuntu Kinetic)
Revision history for this message
Ubuntu Kernel Bot (ubuntu-kernel-bot) wrote : Missing required logs.

This bug is missing log files that will aid in diagnosing the problem. While running an Ubuntu kernel (not a mainline or third-party kernel) please enter the following command in a terminal window:

apport-collect 2003374

and then change the status of the bug to 'Confirmed'.

If, due to the nature of the issue you have encountered, you are unable to run this command, please add a comment stating that fact and change the bug status to 'Confirmed'.

This change has been made by an automated script, maintained by the Ubuntu Kernel Team.

Changed in linux (Ubuntu):
status: New → Incomplete
Changed in linux (Ubuntu Jammy):
status: New → Incomplete
Changed in linux (Ubuntu Jammy):
status: Incomplete → Fix Committed
Revision history for this message
Ubuntu Kernel Bot (ubuntu-kernel-bot) wrote :

This bug is awaiting verification that the linux/5.15.0-79.86 kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-jammy' to 'verification-done-jammy'. If the problem still exists, change the tag 'verification-needed-jammy' to 'verification-failed-jammy'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

tags: added: kernel-spammed-jammy-linux verification-needed-jammy
Revision history for this message
John Cabaj (john-cabaj) wrote :

Successfully created Livepatch for s390x with UBSAN enabled.

Changed in linux (Ubuntu Jammy):
status: Fix Committed → Fix Released
status: Fix Released → Fix Committed
tags: added: verification-done-jammy
tags: removed: verification-needed-jammy
Frank Heimes (fheimes)
Changed in ubuntu-z-systems:
status: In Progress → Fix Committed
Revision history for this message
Ubuntu Kernel Bot (ubuntu-kernel-bot) wrote :

This bug is awaiting verification that the linux-nvidia-tegra-igx/5.15.0-1002.2 kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-jammy' to 'verification-done-jammy'. If the problem still exists, change the tag 'verification-needed-jammy' to 'verification-failed-jammy'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

tags: added: kernel-spammed-jammy-linux-nvidia-tegra-igx verification-needed-jammy
removed: verification-done-jammy
Revision history for this message
Ubuntu Kernel Bot (ubuntu-kernel-bot) wrote :

This bug is awaiting verification that the linux-nvidia-tegra-5.15/5.15.0-1016.16~20.04.1 kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-focal' to 'verification-done-focal'. If the problem still exists, change the tag 'verification-needed-focal' to 'verification-failed-focal'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

tags: added: kernel-spammed-focal-linux-nvidia-tegra-5.15 verification-needed-focal
Revision history for this message
Ubuntu Kernel Bot (ubuntu-kernel-bot) wrote :

This bug is awaiting verification that the linux-nvidia-tegra/5.15.0-1016.16 kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-jammy' to 'verification-done-jammy'. If the problem still exists, change the tag 'verification-needed-jammy' to 'verification-failed-jammy'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

tags: added: kernel-spammed-jammy-linux-nvidia-tegra
Revision history for this message
Utkarsh Gupta (utkarsh) wrote :

Ubuntu 22.10 (Kinetic Kudu) has reached end of life, so this bug will not be fixed for that specific release.

Changed in kpatch (Ubuntu Kinetic):
status: New → Won't Fix
Revision history for this message
Launchpad Janitor (janitor) wrote :
Download full text (83.7 KiB)

This bug was fixed in the package linux - 5.15.0-79.86

---------------
linux (5.15.0-79.86) jammy; urgency=medium

  * jammy/linux: 5.15.0-79.86 -proposed tracker (LP: #2026531)

  * Jammy update: v5.15.111 upstream stable release (LP: #2025095)
    - ASOC: Intel: sof_sdw: add quirk for Intel 'Rooks County' NUC M15
    - ASoC: soc-pcm: fix hw->formats cleared by soc_pcm_hw_init() for dpcm
    - x86/hyperv: Block root partition functionality in a Confidential VM
    - iio: adc: palmas_gpadc: fix NULL dereference on rmmod
    - ASoC: Intel: bytcr_rt5640: Add quirk for the Acer Iconia One 7 B1-750
    - selftests mount: Fix mount_setattr_test builds failed
    - asm-generic/io.h: suppress endianness warnings for readq() and writeq()
    - x86/cpu: Add model number for Intel Arrow Lake processor
    - wireguard: timers: cast enum limits members to int in prints
    - wifi: mt76: mt7921e: Set memory space enable in PCI_COMMAND if unset
    - arm64: Always load shadow stack pointer directly from the task struct
    - arm64: Stash shadow stack pointer in the task struct on interrupt
    - PCI: pciehp: Fix AB-BA deadlock between reset_lock and device_lock
    - PCI: qcom: Fix the incorrect register usage in v2.7.0 config
    - IMA: allow/fix UML builds
    - USB: dwc3: fix runtime pm imbalance on probe errors
    - USB: dwc3: fix runtime pm imbalance on unbind
    - hwmon: (k10temp) Check range scale when CUR_TEMP register is read-write
    - hwmon: (adt7475) Use device_property APIs when configuring polarity
    - posix-cpu-timers: Implement the missing timer_wait_running callback
    - blk-mq: release crypto keyslot before reporting I/O complete
    - blk-crypto: make blk_crypto_evict_key() return void
    - blk-crypto: make blk_crypto_evict_key() more robust
    - ext4: use ext4_journal_start/stop for fast commit transactions
    - staging: iio: resolver: ads1210: fix config mode
    - tty: Prevent writing chars during tcsetattr TCSADRAIN/FLUSH
    - xhci: fix debugfs register accesses while suspended
    - tick/nohz: Fix cpu_is_hotpluggable() by checking with nohz subsystem
    - MIPS: fw: Allow firmware to pass a empty env
    - ipmi:ssif: Add send_retries increment
    - ipmi: fix SSIF not responding under certain cond.
    - kheaders: Use array declaration instead of char
    - wifi: mt76: add missing locking to protect against concurrent rx/status
      calls
    - pwm: meson: Fix axg ao mux parents
    - pwm: meson: Fix g12a ao clk81 name
    - soundwire: qcom: correct setting ignore bit on v1.5.1
    - pinctrl: qcom: lpass-lpi: set output value before enabling output
    - ring-buffer: Sync IRQ works before buffer destruction
    - crypto: api - Demote BUG_ON() in crypto_unregister_alg() to a WARN_ON()
    - crypto: safexcel - Cleanup ring IRQ workqueues on load failure
    - rcu: Avoid stack overflow due to __rcu_irq_enter_check_tick() being kprobe-
      ed
    - reiserfs: Add security prefix to xattr name in reiserfs_security_write()
    - KVM: nVMX: Emulate NOPs in L2, and PAUSE if it's not intercepted
    - relayfs: fix out-of-bounds access in relay_file_read
    - writeback, cgroup: fix null-ptr-deref write in bdi_split_work_to_wbs
 ...

Changed in linux (Ubuntu Jammy):
status: Fix Committed → Fix Released
Revision history for this message
Ubuntu Kernel Bot (ubuntu-kernel-bot) wrote :

This bug is awaiting verification that the linux-aws/5.15.0-1044.49 kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-jammy-linux-aws' to 'verification-done-jammy-linux-aws'. If the problem still exists, change the tag 'verification-needed-jammy-linux-aws' to 'verification-failed-jammy-linux-aws'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

tags: added: kernel-spammed-jammy-linux-aws-v2 verification-needed-jammy-linux-aws
Revision history for this message
Ubuntu Kernel Bot (ubuntu-kernel-bot) wrote :

This bug is awaiting verification that the linux-azure/5.15.0-1046.53 kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-jammy-linux-azure' to 'verification-done-jammy-linux-azure'. If the problem still exists, change the tag 'verification-needed-jammy-linux-azure' to 'verification-failed-jammy-linux-azure'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

tags: added: kernel-spammed-jammy-linux-azure-v2 verification-needed-jammy-linux-azure
Revision history for this message
Ubuntu Kernel Bot (ubuntu-kernel-bot) wrote :

This bug is awaiting verification that the linux-aws-5.15/5.15.0-1046.51~20.04.1 kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-focal-linux-aws-5.15' to 'verification-done-focal-linux-aws-5.15'. If the problem still exists, change the tag 'verification-needed-focal-linux-aws-5.15' to 'verification-failed-focal-linux-aws-5.15'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

tags: added: kernel-spammed-focal-linux-aws-5.15-v2 verification-needed-focal-linux-aws-5.15
Revision history for this message
Ubuntu Kernel Bot (ubuntu-kernel-bot) wrote :

This bug is awaiting verification that the linux-xilinx-zynqmp/5.15.0-1024.28 kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-jammy-linux-xilinx-zynqmp' to 'verification-done-jammy-linux-xilinx-zynqmp'. If the problem still exists, change the tag 'verification-needed-jammy-linux-xilinx-zynqmp' to 'verification-failed-jammy-linux-xilinx-zynqmp'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

tags: added: kernel-spammed-jammy-linux-xilinx-zynqmp-v2 verification-needed-jammy-linux-xilinx-zynqmp
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.