Ubuntu
valgrind package

[SRU] valgrind fails to use debug symbols from glib/gtk

  1. Bionic (18.04)
  2. Bug #1848211
Bug #1848211 reported by Sebastien Bacher
10
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Valgrind
Fix Released
Medium
valgrind (Debian)
Fix Released
Unknown
valgrind (Ubuntu)
Fix Released
High
Balint Reczey
Bionic
Triaged
Low
Unassigned
Disco
Won't Fix
Low
Unassigned
Eoan
Fix Released
High
Balint Reczey
Focal
Fix Released
High
Balint Reczey

Bug Description

[Impact]

 * With dwz the .gnu_debuglink section may appear duplicated in the debug file referenced originally in the .gnu_debuglink section. In that case Valgrind fails to load the debug symbols.

[Test Case]

 * In Ubuntu releases earlier than 19.10 use gio from a package rebuilt with dwz.
 * Install debug symbols for gio following the guide at https://wiki.ubuntu.com/Debug%20Symbol%20Packages
 * Observe no relevant error when running gio with valgrind:
   $ valgrind /usr/bin/gio ~

[Regression Potential]

 * The fix is very simple, just ignoring the second .gnu_debuglink found. There are no likely regressions.

[Original Bug Text]

The debug symbols from libglib2.0-0-dbgsym and libgtk-3-0-dbgsym don't get properly picked up by valgrind in eoan for some reason. The symbols correctly work under gdb.

There is an warning printed from valgrind
'debuginfo section duplicates a section in the main ELF file'

Laney tried to rebuild without dh_dwz and it makes the warning go away and the symbol work.
Downgrading valgrind to the disco version doesn't fix the issue.

See original description
Revision history for this message
In , Karthik-datt (karthik-datt) wrote :

Created attachment 114009
Callgrind.out file is also attached

==967== Callgrind, a call-graph generating cache profiler

==967== Copyright (C) 2002-2015, and GNU GPL'd, by Josef Weidendorfer et al.

==967== Using Valgrind-3.11.0 and LibVEX; rerun with -h for copyright info

==967== Command: /usr/share/WashingMachine/WashingMachineApp --platform eglfs

==967==

==967== For interactive control, run 'callgrind_control -h'.

-967- WARNING: Serious error when reading debug info

-967- When reading debug info from /lib/ld-2.24.so:

-967- debuginfo section duplicates a section in the main ELF file

-967- WARNING: Serious error when reading debug info

-967- When reading debug info from /lib/libgcc_s.so.1:

-967- debuginfo section duplicates a section in the main ELF file

-967- WARNING: Serious error when reading debug info

-967- When reading debug info from /lib/libc-2.24.so:

-967- debuginfo section duplicates a section in the main ELF file

-967- WARNING: Serious error when reading debug info

-967- When reading debug info from /lib/libpthread-2.24.so:

-967- debuginfo section duplicates a section in the main ELF file

-967- WARNING: Serious error when reading debug info

-967- When reading debug info from /lib/libm-2.24.so:

-967- debuginfo section duplicates a section in the main ELF file

-967- WARNING: Serious error when reading debug info

-967- When reading debug info from /lib/libdl-2.24.so:

-967- debuginfo section duplicates a section in the main ELF file

-967- WARNING: Serious error when reading debug info

-967- When reading debug info from /lib/librt-2.24.so:

-967- debuginfo section duplicates a section in the main ELF file

^C==967==

==967== Process terminating with default action of signal 2 (SIGINT)

==967== at 0x4019678: ??? (in /lib/ld-2.24.so)

==967==

==967== Events : Ir

==967== Collected : 12295033

==967==

==967== I refs: 12,295,033

We are trying it on a Linux Arm platform , Imx6 Boundary devices Boards.

Tries it with 3.11 and also 3.13 version. Same warnings are thrown

Iain Lane (laney)
tags: added: rls-ee-incoming rls-ff-incoming
Changed in valgrind (Ubuntu):
status: New → Triaged
importance: Undecided → High
Revision history for this message
Iain Lane (laney) wrote :
Download full text (24.1 KiB)

Here's how I reproduce (I just picked a simple glib-using program, could be anything AFAICT):

laney@raleigh> valgrind /usr/bin/gio ~
==2653== Memcheck, a memory error detector
==2653== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==2653== Using Valgrind-3.15.0 and LibVEX; rerun with -h for copyright info
==2653== Command: /usr/bin/gio
==2653==
--2653-- WARNING: Serious error when reading debug info
--2653-- When reading debug info from /usr/lib/x86_64-linux-gnu/libgmodule-2.0.so.0.6200.1:
--2653-- debuginfo section duplicates a section in the main ELF file
--2653-- WARNING: Serious error when reading debug info
--2653-- When reading debug info from /usr/lib/x86_64-linux-gnu/libglib-2.0.so.0.6200.1:
--2653-- debuginfo section duplicates a section in the main ELF file
--2653-- WARNING: Serious error when reading debug info
--2653-- When reading debug info from /usr/lib/x86_64-linux-gnu/libgobject-2.0.so.0.6200.1:
--2653-- debuginfo section duplicates a section in the main ELF file
[...]
==3711==
==3711== HEAP SUMMARY:
==3711== in use at exit: 36,900 bytes in 236 blocks
==3711== total heap usage: 469 allocs, 233 frees, 68,995 bytes allocated
==3711==
==3711== 16 bytes in 1 blocks are possibly lost in loss record 10 of 85
==3711== at 0x483A723: malloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==3711== by 0x483D017: realloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==3711== by 0x4ABC58F: ??? (in /usr/lib/x86_64-linux-gnu/libglib-2.0.so.0.6200.1)
==3711==
==3711== 16 bytes in 1 blocks are possibly lost in loss record 11 of 85
==3711== at 0x483CD99: calloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==3711== by 0x4ABC540: ??? (in /usr/lib/x86_64-linux-gnu/libglib-2.0.so.0.6200.1)
==3711== by 0xF: ???
==3711== by 0x4BC0ECE: ??? (in /usr/lib/x86_64-linux-gnu/libgobject-2.0.so.0.6200.1)
==3711==
==3711== 16 bytes in 1 blocks are possibly lost in loss record 12 of 85
==3711== at 0x483A723: malloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==3711== by 0x483D017: realloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==3711== by 0x4ABC58F: ??? (in /usr/lib/x86_64-linux-gnu/libglib-2.0.so.0.6200.1)
==3711== by 0xF: ???
==3711== by 0x4BC0E47: ??? (in /usr/lib/x86_64-linux-gnu/libgobject-2.0.so.0.6200.1)
==3711==
==3711== 32 bytes in 2 blocks are possibly lost in loss record 16 of 85
==3711== at 0x483CD99: calloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==3711== by 0x4ABC540: ??? (in /usr/lib/x86_64-linux-gnu/libglib-2.0.so.0.6200.1)
==3711== by 0x2F: ???
==3711== by 0x4BC0ECE: ??? (in /usr/lib/x86_64-linux-gnu/libgobject-2.0.so.0.6200.1)
==3711==
==3711== 32 bytes in 2 blocks are possibly lost in loss record 17 of 85
==3711== at 0x483A723: malloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memc...

Brian Murray (brian-murray)
tags: removed: rls-ff-incoming
Bug Watch Updater (bug-watch-updater)
Changed in valgrind:
status: Unknown → Confirmed
Francis Ginther (fginther)
tags: added: id-5db1d566ba640b107bc522cd
Revision history for this message
Balint Reczey (rbalint) wrote :

This seems similar to LP: #1843479 , maybe strip is broken, will check.

Brian Murray (brian-murray)
tags: removed: rls-ee-incoming
Changed in valgrind (Ubuntu Eoan):
status: New → Triaged
importance: Undecided → High
Revision history for this message
Balint Reczey (rbalint) wrote :

The problematic debug symbol files triggering the valgrind issue.

root@ee:~# readelf --debug-dump=links /usr/lib/x86_64-linux-gnu/libgmodule-2.0.so.0.6200.1
Contents of the .gnu_debugaltlink section:

  Separate debug info file: /usr/lib/debug/.dwz/x86_64-linux-gnu/libglib2.0-0.debug
  Build-ID (0x14 bytes):
 cb 51 16 ac 07 e2 cb 3d 10 5e 39 f1 63 ba b7 b1 45 94 a1 49

Contents of the .gnu_debuglink section:

  Separate debug info file: 73a7789b7bf820c8546a35c13938e61fac4de0.debug
  CRC value: 0xc19e667a

root@ee:~# readelf --debug-dump=links /usr/lib/debug/.build-id/0f/73a7789b7bf820c8546a35c13938e61fac4de0.debug
readelf: Warning: Unsupported form (DW_FORM_GNU_strp_alt) for attribute DW_AT_comp_dir
Contents of the .gnu_debugaltlink section:

  Separate debug info file: /usr/lib/debug/.dwz/x86_64-linux-gnu/libglib2.0-0.debug
  Build-ID (0x14 bytes):
 cb 51 16 ac 07 e2 cb 3d 10 5e 39 f1 63 ba b7 b1 45 94 a1 49

root@ee:~# readelf --debug-dump=links /usr/lib/debug/.dwz/x86_64-linux-gnu/libglib2.0-0.debug

.gnu_debuglink should not be present in 73a7789b7bf820c8546a35c13938e61fac4de0.debug.

description: updated
summary: - valgrind fails to use debug symbols from glib/gtk
+ [SRU] valgrind fails to use debug symbols from glib/gtk
Changed in valgrind (Ubuntu Bionic):
importance: Undecided → Low
Changed in valgrind (Ubuntu Disco):
importance: Undecided → Low
Balint Reczey (rbalint)
Changed in valgrind (Ubuntu Bionic):
status: New → Triaged
Changed in valgrind (Ubuntu Disco):
status: New → Triaged
Revision history for this message
Balint Reczey (rbalint) wrote :

Valgrind is typically used with binaries from the same Ubuntu release or locally built ones.

Since dwz was enabled by default for Debhelper compat level 12 in 19.04 I suggest not backporting the fix to 18.04. Since the issue was not reported against 19.04 and 19.04 goes EOL soon I suggest not backporting the fix to Disco either.

Revision history for this message
In , Balint Reczey (rbalint) wrote :

Created attachment 124162
Don't look for debug alt file in debug image if it is already found

In a case observed in Ubuntu the duplicated section was .gnu_debuglink and it is safe to ignore it like it is done for similar sections.

Revision history for this message
Balint Reczey (rbalint) wrote :

I've sent the fix upstream at: https://bugs.kde.org/show_bug.cgi?id=396656

Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package valgrind - 1:3.15.0-1ubuntu5

---------------
valgrind (1:3.15.0-1ubuntu5) focal; urgency=medium

  * Drop MPI 1 support to fix FTBFS with openmpi 4.0

valgrind (1:3.15.0-1ubuntu4) focal; urgency=medium

  * Don't look for debug alt file in debug image if it is already found
    (LP: #1848211)

 -- Balint Reczey <email address hidden> Thu, 28 Nov 2019 19:35:25 +0100

Changed in valgrind (Ubuntu Focal):
status: Triaged → Fix Released
Revision history for this message
Balint Reczey (rbalint) wrote :

I have uploaded the SRU for 19.10.

Changed in valgrind (Ubuntu Eoan):
status: Triaged → In Progress
assignee: nobody → Balint Reczey (rbalint)
Changed in valgrind (Ubuntu Focal):
assignee: nobody → Balint Reczey (rbalint)
Revision history for this message
Timo Aaltonen (tjaalton) wrote : Please test proposed package

Hello Sebastien, or anyone else affected,

Accepted valgrind into eoan-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/valgrind/1:3.15.0-1ubuntu3.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-eoan to verification-done-eoan. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-eoan. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in valgrind (Ubuntu Eoan):
status: In Progress → Fix Committed
tags: added: verification-needed verification-needed-eoan
Revision history for this message
Ubuntu SRU Bot (ubuntu-sru-bot) wrote : Autopkgtest regression report (valgrind/1:3.15.0-1ubuntu3.1)

All autopkgtests for the newly accepted valgrind (1:3.15.0-1ubuntu3.1) for eoan have finished running.
The following regressions have been reported in tests triggered by the package:

apport/2.20.11-0ubuntu8.2 (amd64)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/eoan/update_excuses.html#valgrind

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

Revision history for this message
Balint Reczey (rbalint) wrote :

The apport failure is unrelated and is caused and tracked by LP: #1854237.

Mathew Hodson (mhodson)
Changed in valgrind:
status: Confirmed → Unknown
Bug Watch Updater (bug-watch-updater)
Changed in valgrind:
importance: Unknown → Medium
status: Unknown → New
Bug Watch Updater (bug-watch-updater)
Changed in valgrind (Debian):
status: Unknown → Confirmed
Revision history for this message
Balint Reczey (rbalint) wrote :
Download full text (7.4 KiB)

Verified with 1:3.15.0-1ubuntu3.1 on Eoan:

root@ee-valgrind:~# echo "deb http://ddebs.ubuntu.com $(lsb_release -cs) main restricted universe multiverse
> deb http://ddebs.ubuntu.com $(lsb_release -cs)-updates main restricted universe multiverse
> deb http://ddebs.ubuntu.com $(lsb_release -cs)-proposed main restricted universe multiverse" | \
> sudo tee -a /etc/apt/sources.list.d/ddebs.list
 deb http://ddebs.ubuntu.com eoan main restricted universe multiverse
deb http://ddebs.ubuntu.com eoan-updates main restricted universe multiverse
deb http://ddebs.ubuntu.com eoan-proposed main restricted universe multiverse
root@ee-valgrind:~# sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys F2EDC64DC5AEE1F6B9C621F0C8CAB6595FDFF622
Executing: /tmp/apt-key-gpghome.p3fDDHpLDX/gpg.1.sh --keyserver keyserver.ubuntu.com --recv-keys F2EDC64DC5AEE1F6B9C621F0C8CAB6595FDFF622
gpg: key C8CAB6595FDFF622: 4 signatures not checked due to missing keys
gpg: key C8CAB6595FDFF622: public key "Ubuntu Debug Symbol Archive Automatic Signing Key (2016) <email address hidden>" imported
gpg: Total number processed: 1
gpg: imported: 1
root@ee-valgrind:~# apt update -qq
28 packages can be upgraded. Run 'apt list --upgradable' to see them.
root@ee-valgrind:~# apt install libglib2.0-0-dbgsym=2.62.1-1
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following package was automatically installed and is no longer required:
  libfreetype6
Use 'apt autoremove' to remove it.
The following NEW packages will be installed:
  libglib2.0-0-dbgsym
0 upgraded, 1 newly installed, 0 to remove and 28 not upgraded.
Need to get 3738 kB of archives.
After this operation, 4418 kB of additional disk space will be used.
Get:1 http://ddebs.ubuntu.com eoan/main amd64 libglib2.0-0-dbgsym amd64 2.62.1-1 [3738 kB]
Fetched 3738 kB in 0s (60.2 MB/s)
Selecting previously unselected package libglib2.0-0-dbgsym:amd64.
(Reading database ... 30827 files and directories currently installed.)
Preparing to unpack .../libglib2.0-0-dbgsym_2.62.1-1_amd64.ddeb ...
Unpacking libglib2.0-0-dbgsym:amd64 (2.62.1-1) ...
Setting up libglib2.0-0-dbgsym:amd64 (2.62.1-1) ...
root@ee-valgrind:~# valgrind gio ~
==2560== Memcheck, a memory error detector
==2560== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==2560== Using Valgrind-3.15.0 and LibVEX; rerun with -h for copyright info
==2560== Command: gio /root
==2560==
--2560-- WARNING: Serious error when reading debug info
--2560-- When reading debug info from /usr/lib/x86_64-linux-gnu/libgmodule-2.0.so.0.6200.1:
--2560-- debuginfo section duplicates a section in the main ELF file
--2560-- WARNING: Serious error when reading debug info
--2560-- When reading debug info from /usr/lib/x86_64-linux-gnu/libglib-2.0.so.0.6200.1:
--2560-- debuginfo section duplicates a section in the main ELF file
--2560-- WARNING: Serious error when reading debug info
--2560-- When reading debug info from /usr/lib/x86_64-linux-gnu/libgobject-2.0.so.0.6200.1:
--2560-- debuginfo section duplicates a section in the main ELF file
Usage:
  gio COMMAND [ARGS…]

Commands:
  help Pri...

Read more...

description: updated
Balint Reczey (rbalint)
tags: added: verification-done verification-done-eoan
removed: verification-needed verification-needed-eoan
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package valgrind - 1:3.15.0-1ubuntu3.1

---------------
valgrind (1:3.15.0-1ubuntu3.1) eoan; urgency=medium

  * Don't look for debug alt file in debug image if it is already found
    (LP: #1848211)

 -- Balint Reczey <email address hidden> Wed, 27 Nov 2019 13:52:54 +0100

Changed in valgrind (Ubuntu Eoan):
status: Fix Committed → Fix Released
Revision history for this message
Łukasz Zemczak (sil2100) wrote : Update Released

The verification of the Stable Release Update for valgrind has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

Bug Watch Updater (bug-watch-updater)
Changed in valgrind (Debian):
status: Confirmed → Fix Released
Steve Langasek (vorlon)
Changed in valgrind (Ubuntu Disco):
status: Triaged → Won't Fix
Revision history for this message
In , Funous (funous) wrote :

This issue still exists in Valgrind 3.16.1 compiled from source on Ubuntu 20.04. The attached patch still applies and fixes the issue.

Revision history for this message
In , Mark J. Wielaard (3y9m2vcw-ll9d-fkzsxrqg) wrote :

Is this still a real problem?

I am not sure I understand how this can happen:

    With dwz the .gnu_debuglink section may appear duplicated in the
    debug file referenced originally in the .gnu_debuglink section.

I assume that is meant to say .gnu_debugaltlink. But if you somehow got a .gnu_debugaltlink in both the main ELF file and the separate .debug image then something must have gone wrong in either the dwz step or the strip step.

Does this mean the main ELF file also still has some .debug_info section? Could you show the section headers of the main ELF file, the .debug file and the alt file?

Revision history for this message
In , Balint Reczey (rbalint) wrote :

Please see the linked original bug for the reproducer.

Revision history for this message
In , Mark J. Wielaard (3y9m2vcw-ll9d-fkzsxrqg) wrote :

(In reply to Balint Reczey from comment #4)
> Please see the linked original bug for the reproducer.

I looked at https://bugs.launchpad.net/ubuntu/+source/valgrind/+bug/1848211 but I still don't fully understand how/why/when this happens.

There is some confusion because the patch is for .gnu_debugaltlink but the bug talks about .gnu_debuglink (but I assume it really is the altlink).

I think what would be helpful was an explanation of how dh_dwz works. It looks like it runs dwz before the debuginfo is stripped into separate files. Which might explain why things look garbled (it should be run on the split .debug files of the package).

It looks like the original bug was on arm32 with ld.so. But later in the bug report a x86_64 issue with glib2 library.

Revision history for this message
In , Mark J. Wielaard (3y9m2vcw-ll9d-fkzsxrqg) wrote :

It was a bit of a struggle to get this reproduced. Installing debuginfo (dbgsym) packages is not the simplest thing. But once you finally manage then you can indeed replicate it simply with: valgrind/vg-in-place -v /usr/bin/gio ~

--3475-- Reading syms from /usr/bin/gio
--3475-- Considering /usr/lib/debug/.build-id/50/d6f5b1c2450bf79554ee66724c911a73f1f635.debug ..
--3475-- .. build-id is valid
--3475-- WARNING: Serious error when reading debug info
--3475-- When reading debug info from /usr/bin/gio:
--3475-- debuginfo section duplicates a section in the main ELF file

And indeed /usr/bin/gio has both a .gnu_debugaltlink and .gnu_debuglink section.

The .gnu_debugaltlink points to /usr/lib/debug/.dwz/x86_64-linux-gnu/libglib2.0-bin.debug

The .gnu_debuglink points to "d6f5b1c2450bf79554ee66724c911a73f1f635.debug" (odd imho that is named after part of the build-id), luckily we look it up through the build-id and end up at /usr/lib/debug/.build-id/50/d6f5b1c2450bf79554ee66724c911a73f1f635.debug (which turns out to be a real file, not a symlink?)

That .debug file also has a .debug_gnualtlink section which points to the same /usr/lib/debug/.dwz/x86_64-linux-gnu/libglib2.0-bin.debug (side-note, given that this is an absolute path instead of a relative path, this makes things even harder to resolve unless everything is installed in the correct place, or you resolve everything through ids).

Since they are the same we can indeed just pick one. I believe this is a really odd thing to do, but we shouldn't really crash and burn.

The proposed patch seems to be a simplest way to do that.

Revision history for this message
In , Mark J. Wielaard (3y9m2vcw-ll9d-fkzsxrqg) wrote :

commit 8b1961511c93962ea2a9b918af8e9c32e3c24d71
Author: Balint Reczey <email address hidden>
Date: Thu Nov 28 13:34:21 2019 +0100

    Don't look for debug alt file in debug image if it is already found

    With dwz the .gnu_debuglink section may appear duplicated in the
    debug file referenced originally in the .gnu_debuglink section.

    https://bugs.launchpad.net/ubuntu/+source/valgrind/+bug/1848211

    https://bugs.kde.org/show_bug.cgi?id=396656
    https://bugs.kde.org/show_bug.cgi?id=427969

    Signed-off-by: Balint Reczey <email address hidden>

Bug Watch Updater (bug-watch-updater)
Changed in valgrind:
status: New → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.