On Ubuntu 18.04 virt-copy-out fails with "tar: This does not look like a tar archive"

I've been playing with this on an Ubuntu 18.04 machine this morning. Firstly I can confirm the bug with libguestfs0 1:1.36.13-1ubuntu3.1. Secondly I can confirm this does NOT affect libguestfs 1.39.9 built from source (from git).

I bisected this to:

commit 0cca7f9061455d29a7f893e715028cd3e75c19fd
Author: Pino Toscano <email address hidden>
Date: Mon Feb 19 14:46:32 2018 +0100

    appliance: include dash for Debian distros

(https://github.com/libguestfs/libguestfs/commit/0cca7f9061455d29a7f893e715028cd3e75c19fd)

This commit cherry picks cleanly on top of 1.36. I have verified that cherry picking this commit on top of 1.36 fixes the bug on Ubuntu 18.04.

I should add that there is a simple end user workaround:

echo dash > /usr/lib/guestfs/supermin.d/zz-lp-1793056

Hi,
could you check the version 1:1.38.4-1ubuntu1 that is in Ubuntu Bionic if it is affected as well?

For the fix we'd have to find the actual upstream change that fixed it to backport that, as the SRU policy [1] doesn't allow to push all of the new version.

With some luck later versions of the package are already fixed in Ubuntu.
That would be nice to know as we likely easier backport there and it makes the window of commits to search for smaller.

Note: This package is "only" in universe, so it depends on the community to fix it in cases like such. I'm willing to help as I can, but I can't promise anything.

[1]: https://wiki.ubuntu.com/StableReleaseUpdates

hah, due to being on business travel I needed so long to race your update :-)
Thanks a lot for the bisect, I'll take a look when I can to package that up.

Richard:

Thanks for the help in figuring out this easy workaround.

Just a note, for me the solution was:

# echo dash > /usr/lib/x86_64-linux-gnu/guestfs/supermin.d/packages

That probably deletes the original packages file, which isn't good. Would
definitely suggest creating a new file in the supermin.d directory.

Sorry, had a typo in my post above...I appended (>>), I didn't overwrite (>).

However, point taken about creating a new file. I've done that and is working. Thanks.

For anyone coming from google, the temp solution on Ubuntu 18.04 is:

# echo dash > /usr/lib/x86_64-linux-gnu/guestfs/supermin.d/zz-lp-1793056

(Just remember to clean up zz-lp-1793056 once the problem is fixed in future package release).

I took an older .img file I had available (any should do) and was able to confirm with just guestfish.

From there I tried to simplify the case and reached this:

$ qemu-img create -f qcow2 test.img 10M
$ sudo guestfish -a test.img
(guestfish) run
(guestfish) tar-out / - | tar tf -

I was able to confirm the case and the workaround helped.
I should soon be able to build a version which has the addition of dash in the package, but not sure if it will be today.

Confirmed to be an issue on Bionic

The upstream fix is in v1.39.1 originally, but was also picked for 1.38.1.
Due to that Cosmic already has the fix that is required.

Note: I verified that it works on cosmic and it does work

Test PPA at https://launchpad.net/~ci-train-ppa-service/+archive/ubuntu/3439
MP at: https://code.launchpad.net/~paelzer/ubuntu/+source/libguestfs/+git/libguestfs/+merge/355588

Waiting for MP review now, feel free to test from the PPA as well if time permits.

Uploaded, and waiting in Bionic-unapproved for the SRU team review

Hello Barry, or anyone else affected,

Accepted libguestfs into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/libguestfs/1:1.36.13-1ubuntu3.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Pre-Upgrade to proposed:

sudo guestfish -a test.img

Welcome to guestfish, the guest filesystem shell for
editing virtual machine filesystems and disk images.

Type: 'help' for help on commands
      'man' to read the manual
      'quit' to quit the shell

><fs> run
 100% ⟦▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒⟧ 00:00
><fs> tar-out / - | tar tf -
libguestfs: error: file receive cancelled by daemon
tar: This does not look like a tar archive
tar: Exiting with failure status due to previous errors
><fs>

---
Upgrade

$ sudo apt install libguestfs0 libguestfs-tools
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following additional packages will be installed:
  libguestfs-hfsplus libguestfs-perl libguestfs-reiserfs libguestfs-xfs
Suggested packages:
  libguestfs-gfs2 libguestfs-jfs libguestfs-nilfs libguestfs-rescue libguestfs-rsync libguestfs-zfs
The following packages will be upgraded:
  libguestfs-hfsplus libguestfs-perl libguestfs-reiserfs libguestfs-tools libguestfs-xfs libguestfs0
6 upgraded, 0 newly installed, 0 to remove and 36 not upgraded.
Need to get 7727 kB of archives.
After this operation, 0 B of additional disk space will be used.
Do you want to continue? [Y/n] Y
Get:1 http://ports.ubuntu.com/ubuntu-ports bionic-proposed/universe ppc64el libguestfs-xfs ppc64el 1:1.36.13-1ubuntu3.2 [1784 B]
Get:2 http://ports.ubuntu.com/ubuntu-ports bionic-proposed/universe ppc64el libguestfs-tools ppc64el 1:1.36.13-1ubuntu3.2 [5089 kB]
Get:3 http://ports.ubuntu.com/ubuntu-ports bionic-proposed/universe ppc64el libguestfs-perl ppc64el 1:1.36.13-1ubuntu3.2 [301 kB]
Get:4 http://ports.ubuntu.com/ubuntu-ports bionic-proposed/universe ppc64el libguestfs-reiserfs ppc64el 1:1.36.13-1ubuntu3.2 [1784 B]
Get:5 http://ports.ubuntu.com/ubuntu-ports bionic-proposed/universe ppc64el libguestfs-hfsplus ppc64el 1:1.36.13-1ubuntu3.2 [1780 B]
Get:6 http://ports.ubuntu.com/ubuntu-ports bionic-proposed/universe ppc64el libguestfs0 ppc64el 1:1.36.13-1ubuntu3.2 [2332 kB]
Fetched 7727 kB in 3s (2396 kB/s)
(Reading database ... 73754 files and directories currently installed.)
Preparing to unpack .../0-libguestfs-xfs_1%3a1.36.13-1ubuntu3.2_ppc64el.deb ...
Unpacking libguestfs-xfs:ppc64el (1:1.36.13-1ubuntu3.2) over (1:1.36.13-1ubuntu3.1) ...
Preparing to unpack .../1-libguestfs-tools_1%3a1.36.13-1ubuntu3.2_ppc64el.deb ...
Unpacking libguestfs-tools (1:1.36.13-1ubuntu3.2) over (1:1.36.13-1ubuntu3.1) ...
Preparing to unpack .../2-libguestfs-perl_1%3a1.36.13-1ubuntu3.2_ppc64el.deb ...
Unpacking libguestfs-perl (1:1.36.13-1ubuntu3.2) over (1:1.36.13-1ubuntu3.1) ...
Preparing to unpack .../3-libguestfs-reiserfs_1%3a1.36.13-1ubuntu3.2_ppc64el.deb ...
Unpacking libguestfs-reiserfs:ppc64el (1:1.36.13-1ubuntu3.2) over (1:1.36.13-1ubuntu3.1) ...
Preparing to unpack .../4-libguestfs-hfsplus_1%3a1.36.13-1ubuntu3.2_ppc64el.deb ...
Unpacking libguestfs-hfsplus:ppc64el (1:1.36.13-1ubuntu3.2) over (1:1.36.13-1ubuntu3.1) ...
Preparing to unpack .../5-libguestfs0_1%3a1.36.13-1ubuntu3.2_ppc64el.deb ...
Unpacking libguestfs0:ppc64el...

The verification of the Stable Release Update for libguestfs has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

This bug was fixed in the package libguestfs - 1:1.36.13-1ubuntu3.2

---------------
libguestfs (1:1.36.13-1ubuntu3.2) bionic; urgency=medium

  * d/p/lp-1793056-fix-internal-bin-sh.patch: fix guestfish tasks that
    required virtual environment and missed /bin/sh (LP: #1793056)

 -- Christian Ehrhardt <email address hidden> Tue, 25 Sep 2018 09:52:15 +0200