"ValueError: ['separator'] has no binary content" when running `apport-unpack /var/crash/...`

Bug #1889443 reported by Dan Watkins
90
This bug affects 14 people
Affects Status Importance Assigned to Milestone
Apport
Fix Released
High
Unassigned
apport (Ubuntu)
Fix Released
High
Unassigned
Bionic
Triaged
High
Unassigned
Focal
Fix Released
High
Unassigned
Jammy
Fix Released
High
Unassigned

Bug Description

[ Impact ]

Extracting a crash file will always fail in case the crash report was processed.

[ Test plan ]

1. Create crash (e.g. with divide-by-zero):
```
sudo apt install chaos-marmosets
divide-by-zero
```
2. Click on "show details" in Apport crash UI, then close it
3. exract crash report:
```
tmpdir=$(mktemp -d)
apport-unpack /var/crash/_usr_bin_divide-by-zero.1000.crash "$tmpdir"
rm -rf "$tmpdir"
```

The extraction should work and not print a stack trace.

[ Where problems could occur ]

The change touches the problem report reading code which is used in several places. This could cause regressions in reading the problem report.

[ Original report ]

The crash file I'm using is as-produced by the system (on groovy), and appears to have an empty value for separator:

$ grep separator /var/crash/_usr_bin_neomutt.1000.crash
separator:

Traceback (most recent call last):
  File "/usr/bin/apport-unpack", line 77, in <module>
    pr.extract_keys(f, bin_keys, dir)
  File "/usr/lib/python3/dist-packages/problem_report.py", line 269, in extract_keys
    raise ValueError('%s has no binary content' %
ValueError: ['separator'] has no binary content

ProblemType: Bug
DistroRelease: Ubuntu 20.10
Package: apport 2.20.11-0ubuntu42
ProcVersionSignature: Ubuntu 5.4.0-37.41-generic 5.4.41
Uname: Linux 5.4.0-37-generic x86_64
NonfreeKernelModules: zfs zunicode zavl icp zcommon znvpair nvidia_modeset nvidia
ApportVersion: 2.20.11-0ubuntu42
Architecture: amd64
CasperMD5CheckResult: skip
CrashReports:
 640:1000:117:1620030:2020-07-29 12:16:57.045274496 -0400:2020-07-29 12:19:15.559243636 -0400:/var/crash/_usr_bin_neomutt.1000.crash
 640:1000:117:4399480:2020-07-23 10:30:21.433552906 -0400:2020-07-23 10:30:18.921508067 -0400:/var/crash/_usr_bin_pavucontrol.1000.crash
 640:1000:117:29909:2020-07-29 12:19:15.603244279 -0400:2020-07-29 12:19:15.603244279 -0400:/var/crash/_usr_bin_apport-unpack.1000.crash
CurrentDesktop: i3
Date: Wed Jul 29 12:22:54 2020
InstallationDate: Installed on 2019-05-07 (448 days ago)
InstallationMedia: Ubuntu 18.04.2 LTS "Bionic Beaver" - Release amd64 (20190210)
PackageArchitecture: all
SourcePackage: apport
UpgradeStatus: Upgraded to groovy on 2020-06-22 (37 days ago)

Revision history for this message
Dan Watkins (oddbloke) wrote :
Revision history for this message
Launchpad Janitor (janitor) wrote :

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in apport (Ubuntu):
status: New → Confirmed
Benjamin Drung (bdrung)
description: updated
Benjamin Drung (bdrung)
Changed in apport:
milestone: none → 2.23.0
importance: Undecided → Medium
importance: Medium → High
Benjamin Drung (bdrung)
Changed in apport (Ubuntu):
importance: Undecided → High
Revision history for this message
Benjamin Drung (bdrung) wrote :
Changed in apport:
status: New → Fix Committed
Benjamin Drung (bdrung)
Changed in apport:
status: Fix Committed → Fix Released
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package apport - 2.23.0-0ubuntu1

---------------
apport (2.23.0-0ubuntu1) kinetic; urgency=medium

  * New upstream release:
    - apport-gtk:
      - Gracefully handle import failure of gi (LP: #1980561)
      - Catch AssertionError when importing Gdk (LP: #1980238)
    - data/apport:
      - Fix PermissionError for setuid programs inside container (LP: #1982487)
      - Fix reading from stdin inside containers (LP: #1982555)
    - apport-kde:
      - Fix inverse order of choices (LP: #1967965)
      - Import apport before usage (LP: #1980553)
      - Drop old workaround for bug in SIP destructor (LP: #1980553)
    - apport-unpack: Fix ValueError: ['separator'] has no binary content
      (LP: #1889443)
    - Fix _run_hook getting called with ui=None (LP: #1983481)
  * Refresh patches and drop backported patches
  * Fix pydocstyle and pylint complains in patches

 -- Benjamin Drung <email address hidden> Mon, 22 Aug 2022 22:31:55 +0200

Changed in apport (Ubuntu):
status: Confirmed → Fix Released
Revision history for this message
Luís Infante da Câmara (luis220413) wrote :

I believe this fix should be backported into Ubuntu 18.04, 20.04 and 22.04 through an SRU.

Revision history for this message
thedoctar (thedoctar) wrote :

I second that!

Revision history for this message
Benjamin Drung (bdrung) wrote :

Yes, this fix should be at least SRUed to Ubuntu 22.04 (jammy) and 20.04 (focal). I will prepare the SRU once the current pending one went through.

Revision history for this message
Launchpad Janitor (janitor) wrote :

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in apport (Ubuntu Bionic):
status: New → Confirmed
Changed in apport (Ubuntu Focal):
status: New → Confirmed
Changed in apport (Ubuntu Jammy):
status: New → Confirmed
Changed in apport (Ubuntu Bionic):
status: Confirmed → Triaged
Changed in apport (Ubuntu Focal):
status: Confirmed → Triaged
Changed in apport (Ubuntu Jammy):
status: Confirmed → Triaged
Changed in apport (Ubuntu Bionic):
importance: Undecided → High
Changed in apport (Ubuntu Focal):
importance: Undecided → High
Changed in apport (Ubuntu Jammy):
importance: Undecided → High
Benjamin Drung (bdrung)
description: updated
description: updated
Revision history for this message
Brian Murray (brian-murray) wrote : Please test proposed package

Hello Dan, or anyone else affected,

Accepted apport into jammy-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/apport/2.20.11-0ubuntu82.3 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-jammy to verification-done-jammy. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-jammy. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in apport (Ubuntu Jammy):
status: Triaged → Fix Committed
tags: added: verification-needed verification-needed-jammy
Revision history for this message
Brian Murray (brian-murray) wrote :

The previous SRU of apport for Focal has had the phasing of its update stop due to potential regressions. It would be good to get those addressed before accepting a new version of apport on top of that.

Changed in apport (Ubuntu Focal):
status: Triaged → Incomplete
Revision history for this message
Benjamin Drung (bdrung) wrote :

Successfully verified apport 2.20.11-0ubuntu82.3 in an Ubuntu 22.04 (jammy) virtual machine.

tags: added: verification-done verification-done-jammy
removed: verification-needed verification-needed-jammy
Revision history for this message
Benjamin Drung (bdrung) wrote :

https://people.canonical.com/~ubuntu-archive/phased-updates.html lists three error:

* 27e989: existing bug #1999034 (but bucket changed in the update)
* d33ca0: bug #1947800 which will be fixed by 2.20.11-0ubuntu27.26
* b5217d: existing bug #1553416 (but bucket changed in the update)

Summary: All known bugs, but their bucket changed. So the phasing could continue.

Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package apport - 2.20.11-0ubuntu82.3

---------------
apport (2.20.11-0ubuntu82.3) jammy; urgency=medium

  * Replace deprecated 'imp' module (LP: #1947425)
  * Fix KeyError: 'CasperMD5json' (LP: #1964828)
  * apport-kde: Fix inverse order of choices (LP: #1967965)
  * apport-unpack: Fix ValueError: ['separator'] has no binary content
    (LP: #1889443)
  * test:
    - Mock add_gdb_info calls in KDE UI tests
    - Fix KDE UI tests if whoopsie.path is disabled
    - Fix race with progress dialog in KDE UI tests
    - Run UI KDE tests again
    - Determine source package dynamically in test_run_crash_kernel
      (LP: #1992172)

 -- Benjamin Drung <email address hidden> Thu, 24 Nov 2022 12:47:19 +0100

Changed in apport (Ubuntu Jammy):
status: Fix Committed → Fix Released
Revision history for this message
Andreas Hasenack (ahasenack) wrote : Update Released

The verification of the Stable Release Update for apport has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

Revision history for this message
Steve Langasek (vorlon) wrote : Please test proposed package

Hello Dan, or anyone else affected,

Accepted apport into focal-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/apport/2.20.11-0ubuntu27.27 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-focal to verification-done-focal. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-focal. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in apport (Ubuntu Focal):
status: Incomplete → Fix Committed
tags: added: verification-needed verification-needed-focal
removed: verification-done
Revision history for this message
Benjamin Drung (bdrung) wrote :

Verified in a focal VM (using sleep since focal does not have chaos-marmosets):

```
$ sleep 3600 &
$ sudo rm /var/crash/_usr_bin_*
$ killall -11 sleep
$ tmpdir=$(mktemp -d)
$ apport-unpack /var/crash/_usr_bin_sleep.1000.crash "$tmpdir"
$
```

tags: added: verification-done verification-done-focal
removed: verification-needed verification-needed-focal
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package apport - 2.20.11-0ubuntu27.27

---------------
apport (2.20.11-0ubuntu27.27) focal; urgency=medium

  * whoopsie-upload-all: Catch zlib.error when decoding CoreDump from
    crash file (LP: #1947800)
  * Fix KeyError: 'CasperMD5json' (LP: #1964828)
  * apport-kde: Fix inverse order of choices (LP: #1967965)
  * apport-unpack: Fix ValueError: ['separator'] has no binary content
    (LP: #1889443)
  * test:
    - Clear environment for test_run_as_real_user_no_sudo
    - Mock add_gdb_info calls in KDE UI tests
    - Fix KDE UI tests if whoopsie.path is disabled
    - Fix race with progress dialog in KDE UI tests
    - Run UI KDE tests again
    - Determine source package dynamically in test_run_crash_kernel
      (LP: #1992172)

 -- Benjamin Drung <email address hidden> Fri, 14 Apr 2023 01:17:21 +0200

Changed in apport (Ubuntu Focal):
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.