Cannot build VM

This looks like an instance of our old favourite:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/759725

@Richard Jones: I don't quite agree. It is not reported on the same package. This bug is not about requesting read-permission on the kernel (that may be one solution though). This bug is about virt-builder not working on ubuntu (where kernels are not readable by non-root users).

I agree though that this bug has emerged as a consequence of the sane decision that kernels should not be readable.
I also agree that a temporary workaround (until this bug is fixed) is that root user could make the kernels readable by others.

The problem is that the kernel isn't readable by non-root for
voodoo reasons. (I don't agree this is a sane configuration)

This affects several programs and needs to be fixed either by
making the kernels readable, or by having Ubuntu developers
change the libguestfs packaging so it triggers chmod +r on the
kernel every time the kernel is upgraded.

I tend to agree with you and the discussion on the other bug that I don't see that preventing read-access to the running kernel gives any security at all.

On the other hand I don't see any reason that specifically virt-bulider need that read-access and should/could be designed to work without that read-access.

I don't know much of the design of virt-builder and wonder why does virt-builder need read-access to the running kernel at all? At first glance this seems like a bad software design. I don't think that all the other tools for building VMs (vagrant vmware, virtualbox, obsolete vmbuilder) need that.

Anyway, if the kernel team insists on blocking read-access, which it seems like they do, I think a more proper solution design would be to make a small helper program (shipped with this package) that can read the running kernel via SUID on that small helper program that does only one thing (read the kernel).

It's how it works: http://libguestfs.org/guestfs-internals.1.html#architecture

It doesn't need access to the running kernel (nor to live kernel memory),
but to a kernel image, and the kernel image in /boot is convenient for
that purpose. Anyway this works fine on every other Linux distro except
Ubuntu (even Debian) so IMO it's still an Ubuntu problem that can only be
fixed by Ubuntu developers.

OK. Thanks for the link and explanation. So it doesn't have to be the exact image of the kernel that is running? just a reasonable image?

If it is not strictly necessary ot be the image of the running kernel, I see a better solution to this bug, which I also consider a better design of virt-builder:
Put a/the kernel image into the file that is downloaded, i.e. http://libguestfs.org/download/builder/ubuntu-18.04.xz, and use that image file when building an OS/VM because apparently virt-builder cannot count on kernel images being available on all supported operating systems (hereamong Ubuntu/Debian).

You can choose arbitrary kernels by setting the SUPERMIN_* environment
variables. See:
http://libguestfs.org/supermin.1.html#ENVIRONMENT-VARIABLES
http://libguestfs.org/guestfs-faq.1.html#how-can-i-compile-and-install-libguestfs-if-my-distro-doesnt-have-new-enough-qemu-supermin-kernel

There's already a kernel inside http://libguestfs.org/download/builder/ubuntu-18.04.xz
but how would you access it without libguestfs? And it wouldn't work
for non-Linux cases (we support Windows too).

* Jarl:

> @Richard Jones: I don't quite agree. It is not reported on the same
> package. This bug is not about requesting read-permission on the
> kernel (that may be one solution though). This bug is about
> virt-builder not working on ubuntu (where kernels are not readable by
> non-root users).

It does not really matter what you would like the bug or its solution to
be about:

Any user who wants to run anything that uses libguestfs (such as
virt-builder) needs to be able to read the kernel image which has not
been possible in the default Ubuntu installation for some time.

The Ubuntu community puts little effort into libguestfs package
maintenance except for copying whatever version from Debian and appliyng
what looks like mechanically applied patches. Which is fine, but I don't
expect anything to change.

That is unless somebody:

(1) comes up with a change to the libguestfs package that changes
    kernel image permissions as outlined by Richard and
(2) gets me to include said change in the Debian package (which I'll
    probably be willing to do.)

I have been maintaining the libguestfs package in Debian since 2011 and
I don't remember how often I have deleted notifications about these
issues in the last few years. Judging from how annoyed I have become at
this issue, there seems to be a real opportunity for anybody reading
this to make a real difference by making lots of Ubuntu users happier.

Cheers,
-Hilko

@Hilko. First of all thank you very much for putting your time into maintaining packages. This is highly appreciated.
Secondly, I don't consider this issue a packaging issue (neither on the linux nor the libguestfs package). Even though it is an issue that has emerged from the ubuntu/debian decision that system kernel images shall not be readable by non-root users, I consider it an issue of the libguestfs software. Therefore I have opened a bug report on the libguestfs bug-system: https://bugzilla.redhat.com/show_bug.cgi?id=1670790

* Jarl:

> @Hilko. First of all thank you very much for putting your time into
> maintaining packages. This is highly appreciated.

I was not fishing for thank-yous, I'm offering to get an Ubuntu-specific
problem worked around. Take it or leave it.

Instead of insisting that you know all there is to know about the issue,
how about providing that patch with the workaround we suggested? Or, how
about finding somebody to work on such a patch?

> I consider it an issue of the libguestfs software.

I suppose that you were not aware that you were exchanging messages with
the main upstream author then?

Cheers,
-Hilko

@Hilko:

> > @Hilko. First of all thank you very much for putting your time into
> > maintaining packages. This is highly appreciated.

> I was not fishing for thank-yous, I'm offering to get an Ubuntu-specific
> problem worked around. Take it or leave it.

I am sorry. I didn't mean to offend you. Yes, there are several workarounds. It's not clear to me exactly which one you mean. For now I just run virt-builder with sudo.

> how about providing that patch with the workaround we suggested?

I may take a look at working on a patch. I have cloned libguestfs, but can you point me in the right direction libguestfs package? When I open https://packages.ubuntu.com/source/trusty/libguestfs and follow the links under "Debian Source Repository", the page shows that "The alioth.debian.org service is discontinued".

> I suppose that you were not aware that you were exchanging messages with
the main upstream author then?

You are absolutely right, I was not aware of that. That is good news to me, my ideas have been reviewed and considered by the right people. My intention of opening https://bugzilla.redhat.com/show_bug.cgi?id=1670790 was exactly to reach the upstream developers, I was not aware that I've reached them already :-)

* Jarl:

>> how about providing that patch with the workaround we suggested?
>
> I may take a look at working on a patch. I have cloned libguestfs, but
> can you point me in the right direction libguestfs package? When I open
> https://packages.ubuntu.com/source/trusty/libguestfs
> and follow the links under "Debian Source Repository", the page shows
> that "The alioth.debian.org service is discontinued".

If you look at the next stable Ubuntu release "disco"
(<https://packages.ubuntu.com/source/disco/libguestfs>), you'll be
directed to <https://salsa.debian.org/libvirt-team/libguestfs>.

Here's a bit of unsolicited advice regarding a possible patch: You'll
need to add a mechanism to make kernel images readable.

This means doing more than a "chmod 644 /boot/vmlinuz*" at libguestfs0
post-installation time because any kernel image installed after the
libguestfs0 package would not be affected. Instead of chmod, please use
dpkg-statoverride(1). Using a dpkg-trigger(1) that fires on kernel
installation or removal seems like a good idea. Also you should add code
to ask users whether they want this mechanism to be activated, using
debconf(1). After all, read-only kernel images are percieved as a
security measure by the Ubuntu project.

Cheers,
-Hilko

Status changed to 'Confirmed' because the bug affects multiple users.

Hilko, you probably know this but it looks like in the original thread here:

https://bugs.launchpad.net/ubuntu/+source/linux/+bug/759725

...that this could be fixed thusly:

To have this automatically happen with each new kernel, create /etc/kernel/postinst.d/statoverride:

  #!/bin/sh
  version="$1"
  # passing the kernel version is required
  [ -z "${version}" ] && exit 0
  dpkg-statoverride --add root root 0644 /boot/vmlinux-${version} --update

...can libguestfs package ship that script? Or would that be a policy violation of some sort?

It looks like if upon installation short script could be placed in /etc/kernel/postinst.d/statoverride, along with a "chmod 0644 /boot/vmlinuz*", then all those annoying bugreports from ubuntu users should go away.

BUG https://bugs.launchpad.net/ubuntu/+source/linux/+bug/759725 is super, ultra annoying and useless!

:-@

I'm creating a file:

/etc/kernel/postinst.d/fix-damn-bug-759725 with:

---
#/bin/sh

# https://bugs.launchpad.net/ubuntu/+source/linux/+bug/759725

set -e
version="$1"
if [ -z "$version" ]; then
    exit 0
fi
exec dpkg-statoverride --update --add root root 0644 "/boot/vmlinuz-${version}"
---

As a workaround to BUG 759725 that MUST be fixed by Ubuntu.

Hiya, I was kindly wondering if there is an update on this?
I have naive suggestion:

what about creating a small optional package that has the dpkg-statoverride script and let users be responsible to install it or not install it?

That way, Ubuntu kernel maintainers wishes are respected, and users can have it their way too.

And everyone may be satisfied this way?

Quoting my message from more than 2 years ago:

> The Ubuntu community puts little effort into libguestfs package
> maintenance except for copying whatever version from Debian and appliyng
> what looks like mechanically applied patches. Which is fine, but I don't
> expect anything to change.
>
> That is unless somebody:
>
> (1) comes up with a change to the libguestfs package that changes
> kernel image permissions as outlined by Richard and
> (2) gets me to include said change in the Debian package (which I'll
> probably be willing to do.)

Since I posted this, I'm not aware to have received bug reports via Debian's BTS (https://bugs.debian.org/cgi-bin/pkgreport.cgi?src=libguestfs) containing such a patch or merge requests on Salsa (https://salsa.debian.org/libvirt-team/libguestfs).

FYI, this is the kind of wart this issue is forcing down on me and all my users:
https://github.com/nexB/extractcode/blob/aa5da29014ce4fbffca53c09689a2623e2b78196/src/extractcode/vmimage.py#L82

> def check_linux_kernel_is_readable():
> """
> Return True if the kernel executable file can be read. This is required by
> guestfish and libguestfs and this is an oddity mostly on Ubuntu.
> See:
> - https://bugs.launchpad.net/ubuntu/+source/linux/+bug/759725
> - https://bugzilla.redhat.com/show_bug.cgi?id=1670790
> - https://bugs.launchpad.net/ubuntu/+source/libguestfs/+bug/1813662
> """

I would like to create a package that fix this issue.

Is this the correct way:

1. As sudo, create the file /etc/kernel/postinst.d/statoverride with this
content, devised by Kees Cook (@kees) in
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/759725/comments/3 ::

    #!/bin/sh
    version="$1"
    # passing the kernel version is required
    [ -z "${version}" ] && exit 0
    dpkg-statoverride --update --add root root 0644 /boot/vmlinuz-${version}

2. Setup the exxecutable permissions::

    sudo chmod +x /etc/kernel/postinst.d/statoverride

And if yes, is this enough to create a package that adds this file?

I created this set of instructions in https://raw.githubusercontent.com/nexB/extractcode/main/README.rst

I would appreciate if someone could minimally validate if this is the correct way:

Adding support for VM images extraction
---------------------------------------

Adding support for VM images requires the manual installation of the
libguestfs-tools system package. This is suported only on Linux.
On Debian and Ubuntu you can use this command::

    sudo apt-get install libguestfs-tools

On Ubuntu only, an additional manual step is required as the kernel executable
file cannot be read by users as required by libguestfish.

Run this command as a temporary and immediate fix::

    sudo chmod 0644 /boot/vmlinuz-*
    for k in /boot/vmlinuz-*
        do sudo dpkg-statoverride --add --update root root 0644 /boot/vmlinuz-$k
    done

You likely want both this temporary fix and a more permanent fix; otherwise each
kernel update will revert to the default permissions and ExtractCode will stop
working for VM images extraction.

Therefore follow these instructions:

1. As sudo, create the file /etc/kernel/postinst.d/statoverride with this
content, devised by Kees Cook (@kees) in
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/759725/comments/3 ::

    #!/bin/sh
    version="$1"
    # passing the kernel version is required
    [ -z "${version}" ] && exit 0
    dpkg-statoverride --update --add root root 0644 /boot/vmlinuz-${version}

2. Set executable permissions::

    sudo chmod +x /etc/kernel/postinst.d/statoverride

See also these links for a complete discussion:

    - https://bugs.launchpad.net/ubuntu/+source/linux/+bug/759725
    - https://bugzilla.redhat.com/show_bug.cgi?id=1670790
    - https://bugs.launchpad.net/ubuntu/+source/libguestfs/+bug/1813662/comments/24

Hi Philippe,

sorry for the late reply.

> FYI, this is the kind of wart this issue is forcing down on me and all my users:
> https://github.com/nexB/extractcode/blob/aa5da29014ce4fbffca53c09689a2623e2b78196/src/extractcode/vmimage.py#L82
>
>> def check_linux_kernel_is_readable():
>> """
>> Return True if the kernel executable file can be read. This is required by
>> guestfish and libguestfs and this is an oddity mostly on Ubuntu.
>> See:
>> - https://bugs.launchpad.net/ubuntu/+source/linux/+bug/759725
>> - https://bugzilla.redhat.com/show_bug.cgi?id=1670790
>> - https://bugs.launchpad.net/ubuntu/+source/libguestfs/+bug/1813662
>> """

Yeah, I am well aware… As far as I can recall, you may well be the first
person to come up with somewhat sensible/constructive approach to this
mess and I am sorry that I let your mail sit in my inbox for so long.

There are just a few implementation details that should be done
differntly, I'll comment inline:

> I would like to create a package that fix this issue.

Why not add the hack in one of the libguestfs packages themselves? I
think that libguestfs0 would be the right choice. I'd be happy to
include this into the libguestfs package under two conditions:

- The changes are Ubuntu-specific.
- The changes have been tested.

> Is this the correct way:
>
> 1. As sudo, create the file /etc/kernel/postinst.d/statoverride with this
> content, devised by Kees Cook (@kees) in
> https://bugs.launchpad.net/ubuntu/+source/linux/+bug/759725/comments/3 ::
>
> #!/bin/sh
> version="$1"
> # passing the kernel version is required
> [ -z "${version}" ] && exit 0
> dpkg-statoverride --update --add root root 0644 /boot/vmlinuz-${version}

This looks reasonable. /etc/kernel/postinst.d/statoverride can simply be
added to a package (your package or libguestfs0), there's no need for
creating a script that does this at install time.

And instead of calling the script "statoverride", you should use
something that hints at the package name.

> 2. Setup the exxecutable permissions::
>
> sudo chmod +x /etc/kernel/postinst.d/statoverride

This is not necessary if you install that file into the package with the
right permissions in the first place.

> And if yes, is this enough to create a package that adds this file?

There are a few other issues to consider:

When a kernel image is removed, you should also remove the corresponding
statoverride entry (dpkg-statoverride remove $file), via a prerm or
postrm script.

The kernel postinst script does not help with kernel images that are
already installed when libguestfs0 (or your custom package) is added to
the system, so you'll need to call dpkg-statoverride for all kernel
images.

Similarly, users will expect that removing the package will remove the
statoverride entries, so you'll need a postrm script that does that for
all kernel images.

Cheers,
-Hilko