Jammy builds of xen segfault, but only on launchpad x86 builders
| Affects | Status | Importance | Assigned to | Milestone | |
|---|---|---|---|---|---|
| binutils |
Fix Released
|
Medium
|
|||
| launchpad-buildd |
New
|
Undecided
|
Unassigned | ||
| binutils (Debian) |
Fix Released
|
Unknown
|
|||
| binutils (Ubuntu) |
Fix Released
|
Critical
|
Unassigned | ||
| Jammy |
Fix Released
|
Critical
|
Unassigned | ||
| xen (Ubuntu) |
Invalid
|
Undecided
|
Unassigned | ||
| Jammy |
Invalid
|
Undecided
|
Unassigned | ||
Bug Description
FTBFS in Jammy on LP infra:
https:/
https:/
Related PPA:
https:/
Summary:
- Build reliably fails on LP
- Build in local sbuild works reliably on my Laptop
- Build in local VM (sizing like LP builders) works (other crashes but works)
- Build on AMD server (chip more similar to LP) works reliably
Failing step:
On Launchpad build infrastructure it breaks on ld:
$ x86_64-linux-gnu-ld -mi386pep --subsystem=10 --image-
Segmentation fault (core dumped
---
Steps to recreate (result depends on platform)
# you can grab the package from https:/
sudo vim /etc/apt/
sudo apt update
sudo apt dist-upgrade -y
sudo apt build-dep xen
sudo apt install flex bison python3-dev libpython3-dev dpkg-dev devscripts apport-retrace
sudo mkdir /mnt/build
sudo chmod go+w /mnt/build
cd /mnt/build
# copy in things from host
scp xen_4.16.
dpkg-source -x xen_4.16.
cd xen_4.16.0
dpkg-buildpackage -i -us -uc -b
---
In a jammy VM 4cpu/8G I get some avx2 crashes but the build works:
Jan 19 07:41:27 j kernel: x86_64-
Jan 19 07:41:27 j kernel: Code: f8 77 c3 66 2e 0f 1f 84 00 00 00 00 00 f3 0f 1e fa 89 f8 48 89 fa c5 f9 ef c0 25 ff 0f 00 00 3d e0 0f 00 00 0f 87 33 01 00 00 <c5> fd 74 0f c5 fd d7 c1 85 c0 74 57 f3 0f bc c0 c5 f8 77 c3 66 66
#0 __strlen_avx2 () at ../sysdeps/
74 ../sysdeps/
(gdb) bt
#0 __strlen_avx2 () at ../sysdeps/
#1 0x00007fa98d63c2d0 in ?? () from /lib/x86_
#2 0x00007fa98d6021e8 in ?? () from /lib/x86_
#3 0x00007fa98d602509 in coff_write_
#4 0x00007fa98d6033bd in _bfd_coff_
#5 0x0000562bdaaae3bf in ?? ()
#6 0x00007fa98d2e8fd0 in __libc_
#7 0x00007fa98d2e907d in __libc_
stack_
#8 0x0000562bdaaad515 in ?? ()
^^ that is a different crash than on th LP builders
! And despite those crashes the build does appear to work oO?!
The same crashes I see on my local sbuild runs, the full set of one build is
Jan 19 07:39:02 Keschdeichel kernel: x86_64-
Jan 19 07:39:03 Keschdeichel kernel: x86_64-
Jan 19 07:39:03 Keschdeichel kernel: x86_64-
Jan 19 07:39:42 Keschdeichel kernel: x86_64-
Jan 19 07:44:57 Keschdeichel kernel: x86_64-
Jan 19 07:44:57 Keschdeichel kernel: x86_64-
Jan 19 07:44:58 Keschdeichel kernel: x86_64-
Jan 19 07:45:05 Keschdeichel kernel: x86_64-
I checked, this is not in configure stage where such things sometimes are intentional.
Running in local VM with reduced cpu features (e.g. no avx2) still triggers
the same bfd issues, but still works to build.
---
The LP run is on a Rome chip, from the build env:
Model name: AMD EPYC-Rome Processor
Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm rep_good nopl xtopology cpuid extd_apicid tsc_known_freq pni pclmulqdq ssse3 fma cx16 sse4_1 sse4_2 x2apic movbe popcnt aes xsave avx f16c rdrand hypervisor lahf_lm svm cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw topoext perfctr_core ibpb stibp vmmcall fsgsbase bmi1 avx2 smep bmi2 rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 xsaves clzero xsaveerptr arat npt nrip_save umip rdpid
So I thought I might need to re-create the build on such a chip to check
if it fails there too.
Running on Riccioli from the kernel Team as similar HW (AMD EPYC 7713)
works fine (like my local build does, this time without any crashes)
---
I do not know how to continue, repro on laptop, repro in VM guests, repro on
AMD servers similar to the build farm, ... they all build the package.
But on launchpad it crashes with the reported error.
Is it the toolchain that needs a fix, is it the launchpad builder setup, both?
I do not know ... :-/
Filing this against xen+binutils+
| Christian Ehrhardt (paelzer) wrote | #1 |
| Colin Watson (cjwatson) wrote | #2 |
| Christian Ehrhardt (paelzer) wrote | #3 |
| Colin Watson (cjwatson) wrote | #4 |
| Colin Watson (cjwatson) wrote | #5 |
| Christian Ehrhardt (paelzer) wrote | #6 |
| Christian Ehrhardt (paelzer) wrote | #7 |
| Changed in xen (Ubuntu): | |
| status: | New → Invalid |
| Christian Ehrhardt (paelzer) wrote | #8 |
| Christian Ehrhardt (paelzer) wrote | #9 |
| Christian Ehrhardt (paelzer) wrote | #10 |
| Changed in binutils (Ubuntu): | |
| importance: | Undecided → Critical |
| Christian Ehrhardt (paelzer) wrote | #11 |
| tags: | added: rls-jj-incoming |
|
|
#12 |
| Changed in binutils: | |
| importance: | Unknown → Medium |
| status: | Unknown → Confirmed |
| Changed in binutils (Debian): | |
| status: | Unknown → New |
|
|
#13 |
| tags: | added: fr-2001 |
| tags: | removed: rls-jj-incoming |
|
|
#14 |
|
|
#15 |
|
|
#16 |
|
|
#17 |
| Changed in binutils: | |
| status: | Confirmed → In Progress |
| Changed in binutils (Debian): | |
| status: | New → Confirmed |
| Launchpad Janitor (janitor) wrote | #18 |
| Changed in binutils (Ubuntu Jammy): | |
| status: | New → Fix Released |
| Changed in binutils (Debian): | |
| status: | Confirmed → Fix Released |
|
|
#19 |
|
|
#20 |
| Changed in binutils: | |
| status: | In Progress → Fix Released |
FYI: I've seen some shortening of the reported crashes going on.
For example in journal it was "x86_64-linux-gn" and in the build log it was after "x86_64-
This might or might not be the same crash on LP and locally, but as I mentioned above one fails critically the other one is ignored.
Could someone run that build manually on LP and/or stop it before cleanup to gather the crash from there for comparison?