[OSSA 2013-029] Unchecked qcow2 root disk sizes DoS

Russell, Mikal: could you confirm the vulnerability ?

Added Pádraig Brady to help review this.

Refreshing my memory for the flow of these images ... http://www.pixelbeat.org/docs/openstack_libvirt_images/

Yes this does seem like a valid issue, though not as serious as the related bug 1177830
as there is only an allocation per image rather than per instance.

Also any sizes over the quote size etc. for an instance wouldn't be thus referenced and so
could be auto deleted at some stage.

But yes this is a valid DOS vector given the difference between input and output sizes.

Now if we did limit sizes, I suppose we could limit to the max flavor size?
But that doesn't help much. One could still upload many small images up to the max flavor size to fill up the _base/

Maybe we need quotas glance side for max actual disk usage per user for all their images?

Looks like we'll need an OSSA on this then.

Anyone on the current subscription list up for a patch ?

Padraig, Mikal: any chance you could produce a patch to fix this ?

Ok I'll take this (was away for a couple of weeks). I might get to it in the next couple of days

@Pádraig: any progress ?

Lets summarize _this_ bug as "per _image_ qcow2 root disk sizes DoS"
and bug 1177830 as "per _instance_ qcow2 root disk sizes DoS".

Unfortunately I think that even bug 1177830 mightn't be addressed in the case
where use_cow_images=False, i.e. in nova.virt.libvirt.imagebackend.Raw.create_image().
On initial inspection it seems like the large qcow2 files could be copied to each instance here too.

So thoughts on addressing this.

1. Always set min_disk = virt_size for an image in glance.
This is already checked in nova before we download anything.
That has the advantage of saving on network bandwidth and is
honored by cinder too for volumes.

2. Pass/Inspect size into nova.virt.images.fetch{,_to_raw}()
and disallow disk.get_disk_size('tmp_download') > size

Doing either should address both bugs I think,
but I'm leaning towards 1 as a more general solution.

Actually we probably should do both.
Better to have nova not rely on a particular glance implementation or version or glance at all for that matter.
I'll do the nova change now...

@Padraig: we'll take the Nova patch as the vulnerability fix. We'll consider the Glance as a hardening improvement.

First attempt at an impact description, not sure I got this right:

----------------------
Title: Potential Nova denial of service through compressed disk images
Reporter: Bernhard M. Wiedemann (SUSE)
Products: Nova
Affects: All versions

Description:
Bernhard M. Wiedemann from SUSE reported a vulnerability in Nova's control of the size of disk images. By using malicious compressed qcow2 disk images, an authenticated user may consume large amounts of disk space, potentially resulting in a Denial of Service attack on Nova compute nodes. This issue is slightly different from CVE-2013-2096 which was addressed in OSSA 2013-012.
---------------------------

Re impact description, I'd mention that CVE-2013-2096 wasn't fully addressed either
in the non default case where use_cow_images=False, and malicious qcow images
are being transferred from glance. If mentioning both it's probably worth mentioning
the original CVE-2013-2096 was a per instance issue, while this new one is per image.

RC1 will probably be out before we can properly embargo and release this as an OSSA. This should stay as havana-rc-potential just in case it can make it prerelease.

Would be good to get some core pre-approvals on the proposed patches. Feel free to subscribe fellow core developers to accelerate that.

Added a few nova developers that could help review these patches

Hey guys, please review proposed patches !

+2 on the patch

respinning with just improved commit messages

Nova core: Please review proposed patches

New attempt at impact description:

----------------------
Title: Potential Nova denial of service through compressed disk images
Reporter: Bernhard M. Wiedemann (SUSE)
Products: Nova
Affects: All versions

Description:
Bernhard M. Wiedemann from SUSE reported a vulnerability in Nova's control of the size of disk images. By using malicious compressed qcow2 disk images, an authenticated user may consume large amounts of disk space for each image, potentially resulting in a Denial of Service attack on Nova compute nodes. While fixing this issue, Pádraig Brady from Red Hat additionally discovered that OSSA 2013-012 did not fully address CVE-2013-2096 in the non-default case where use_cow_images=False, and malicious qcow images are being transferred from Glance. In that specific case, an authenticated user could still consume large amounts of disk space for each instance using the malicious image, potentially also resulting in a Denial of Service attack on Nova compute nodes. The provided fixes address both issues.
---------------------------

+2 on patches

+1 on impact description

CVE requested

CVE-2013-4463

Proposed public disclosure date/time: Thursday, October 31, 1500UTC.

CVE SPLIT w/ CVE-2013-4469:

Title: Potential Nova denial of service through compressed disk images
Reporter: Bernhard M. Wiedemann (SUSE) & Pádraig Brady (Red Hat)
Products: Nova
Affects: All versions

Description:
Bernhard M. Wiedemann from SUSE reported a vulnerability in Nova's
control of the size of disk images. By using malicious compressed qcow2
disk images, an authenticated user may consume large amounts of disk
space for each image, potentially resulting in a Denial of Service
attack on Nova compute nodes (CVE-2013-4463). While fixing this issue, Pádraig Brady from Red Hat additionally discovered that OSSA 2013-012 did not fully address CVE-2013-2096 in the non-default case where use_cow_images=False, and malicious qcow images are being transferred from Glance. In that specific case, an authenticated user could still consume large amounts of disk space for each instance using the malicious image, potentially also resulting in a Denial of Service attack on Nova compute nodes (CVE-2013-4469). The provided fixes address both issues.

Patches test runs:

master: tox, smoke PASS
havana: tox, neutron PASS
grizzly: tox, full PASS

Fix proposed to branch: stable/havana
Review: https://review.openstack.org/54767

master: https://review.openstack.org/54765
stable/havana: https://review.openstack.org/54767
stable/grizzly: https://review.openstack.org/54768

[OSSA 2013-029]

Reviewed: https://review.openstack.org/54765
Committed: http://github.com/openstack/nova/commit/f6810be4ae1a6c93e7d8017ee67d5344dfdf4a30
Submitter: Jenkins
Branch: master

commit f6810be4ae1a6c93e7d8017ee67d5344dfdf4a30
Author: Pádraig Brady <email address hidden>
Date: Fri Sep 27 04:07:14 2013 +0100

    ensure we don't boot oversized images

    Since we can't generally shrink incoming images, add extra checks
    to ensure oversized images are not allowed through.
    All cases when populating the libvirt image cache are now handled,
    including the initial download from glance, where we avoid
    converting to raw, as that could generate non sparse images
    much larger than the downloaded image.

    * nova/virt/libvirt/utils.py (fetch_image): Allow passing through
    of the max_size parameter.
    * nova/virt/images.py (fetch_to_raw): Accept the max_size parameter,
    and use it to discard images with larger (virtual) sizes.
    * nova/virt/libvirt/imagebackend.py (verify_base_size): A new
    refactored function to identify and raise exception to oversized images.
    (Raw.create_image): Pass the max_size to the fetch function.
    Also enforce virtual image size checking for already fetched images,
    as this class (despite the name) can be handling qcow files.
    (Qcow2.create_image): Pass the max_size to the fetch function,
    or verify the virtual size for the instance as done previously.
    (Lvm.create_image): Pass the max_size to the fetch function.
    Also check the size before transferring to the volume to improve
    efficiency by not even attempting the transfer of oversized images.
    (Rbd.create_image): Likewise.
    * nova/tests/virt/libvirt/fake_libvirt_utils.py: Support max_size arg.
    * nova/tests/virt/libvirt/test_libvirt.py (test_fetch_raw_image):
    Add a case to check oversized images are discarded.
    * nova/tests/virt/libvirt/test_imagebackend.py
    (test_create_image_too_small): Adjust to avoid the fetch size check.

    Fixes bug: 1177830
    Fixes bug: 1206081
    Change-Id: I3d47adaa2ad07434853f447feb27d7aae0e2e717

Reviewed: https://review.openstack.org/54767
Committed: http://github.com/openstack/nova/commit/3cdfe894ab58f7b91bf7fb690fc5bc724e44066f
Submitter: Jenkins
Branch: stable/havana

commit 3cdfe894ab58f7b91bf7fb690fc5bc724e44066f
Author: Pádraig Brady <email address hidden>
Date: Fri Sep 27 04:07:14 2013 +0100

    ensure we don't boot oversized images

    Since we can't generally shrink incoming images, add extra checks
    to ensure oversized images are not allowed through.
    All cases when populating the libvirt image cache are now handled,
    including the initial download from glance, where we avoid
    converting to raw, as that could generate non sparse images
    much larger than the downloaded image.

    * nova/virt/libvirt/utils.py (fetch_image): Allow passing through
    of the max_size parameter.
    * nova/virt/images.py (fetch_to_raw): Accept the max_size parameter,
    and use it to discard images with larger (virtual) sizes.
    * nova/virt/libvirt/imagebackend.py (verify_base_size): A new
    refactored function to identify and raise exception to oversized images.
    (Raw.create_image): Pass the max_size to the fetch function.
    Also enforce virtual image size checking for already fetched images,
    as this class (despite the name) can be handling qcow files.
    (Qcow2.create_image): Pass the max_size to the fetch function,
    or verify the virtual size for the instance as done previously.
    (Lvm.create_image): Pass the max_size to the fetch function.
    Also check the size before transferring to the volume to improve
    efficiency by not even attempting the transfer of oversized images.
    (Rbd.create_image): Likewise.
    * nova/tests/virt/libvirt/fake_libvirt_utils.py: Support max_size arg.
    * nova/tests/virt/libvirt/test_libvirt.py (test_fetch_raw_image):
    Add a case to check oversized images are discarded.
    * nova/tests/virt/libvirt/test_imagebackend.py
    (test_create_image_too_small): Adjust to avoid the fetch size check.

    Fixes bug: 1177830
    Fixes bug: 1206081
    Change-Id: I3d47adaa2ad07434853f447feb27d7aae0e2e717

Reviewed: https://review.openstack.org/54768
Committed: http://github.com/openstack/nova/commit/135faa7b5d9855312bedc19e5e1ecebae34d3d18
Submitter: Jenkins
Branch: stable/grizzly

commit 135faa7b5d9855312bedc19e5e1ecebae34d3d18
Author: Pádraig Brady <email address hidden>
Date: Fri Sep 27 04:07:14 2013 +0100

    ensure we don't boot oversized images

    Since we can't generally shrink incoming images, add extra checks
    to ensure oversized images are not allowed through.
    All cases when populating the libvirt image cache are now handled,
    including the initial download from glance, where we avoid
    converting to raw, as that could generate non sparse images
    much larger than the downloaded image.

    * nova/virt/libvirt/utils.py (fetch_image): Allow passing through
    of the max_size parameter.
    * nova/virt/images.py (fetch_to_raw): Accept the max_size parameter,
    and use it to discard images with larger (virtual) sizes.
    * nova/virt/libvirt/imagebackend.py (verify_base_size): A new
    refactored function to identify and raise exception to oversized images.
    (Raw.create_image): Pass the max_size to the fetch function.
    Also enforce virtual image size checking for already fetched images,
    as this class (despite the name) can be handling qcow files.
    (Qcow2.create_image): Pass the max_size to the fetch function,
    or verify the virtual size for the instance as done previously.
    (Lvm.create_image): Pass the max_size to the fetch function.
    Also check the size before transferring to the volume to improve
    efficiency by not even attempting the transfer of oversized images.
    (Rbd.create_image): Likewise.
    * nova/tests/fake_libvirt_utils.py: Support max_size arg.
    * nova/tests/test_libvirt.py (test_fetch_raw_image):
    Add a case to check oversized images are discarded.
    * nova/tests/test_imagebackend.py (test_create_image_too_small):
    Adjust to avoid the fetch size check.

    Fixes bug: 1177830
    Fixes bug: 1206081

    Conflicts:

     nova/tests/test_imagebackend.py
     nova/virt/libvirt/imagebackend.py

    Change-Id: Idc35fce580be4f74e23883d1b4bea6475c3f6e30