With libvirt/images_type = rbd, ephemeral instances silently ignore hw_qemu_guest_agent=yes

Boot from volume isn't affected since it doesn't go through the libvirt Rbd image backend class, the compute API makes a direct RPC call to the compute to quiesce the instance (if it's ACTIVE and configured to support that):

https://opendev.org/openstack/nova/src/commit/7bf75976016aae5d458eca9f6ddac92bfe75dc59/nova/compute/api.py#L3075

The direct snapshot functionality introduced in that referenced patch is optional - it was added to make snapshots with rbd-backed images significantly faster. You can disable the fast snapshot support by disabling direct URLs in glance as a workaround.

Obviously it would be nice to have the best of both worlds though, but I'm not sure what the fix for that looks like at this time.

Thanks for looking into this!

Disabling direct URLs in Glance for this reason would be a bit like throwing the baby out with the bathwater, as you'd also lose the ability to use RBD clones for ephemeral storage and compute nodes would have to go back to streaming images down from Glance, if I'm not mistaken. (There would also be no rbd clone followed by rbd flatten on openstack volume create --image=<image> anymore, so that process would also take a whole lot longer, particularly for large images).

Jason and I came up with a different workaround involving yanking write permissions on the Glance pool from the CephX identity that Nova uses, see this message from the ceph-users ML:
https://<email address hidden>/message/KYFAF6BPRSI3XWANHKPIS656QX545LRD/

.. but that's basically just forcing the snapshot code into an error path while trying to create an RBD clone, so that's not very pretty either.

What bugs me is really primarily that os_require_quiesce is being silently ignored. So maybe the proper fix would be to have direct_snapshot() raise exception.ImageUnacceptable if the image meta includes os_require_quiesce?

> Obviously it would be nice to have the best of both worlds though, but I'm not sure what the fix for that looks like at this time.

Wouldn't it just be as simple as wrapping the 'direct_snapshot' call [1] in similar quiesce/unquiesce calls like '_live_snapshot' already does [2][3]?

[1] https://opendev.org/openstack/nova/src/branch/stable/stein/nova/virt/libvirt/driver.py#L2031
[2] https://opendev.org/openstack/nova/src/branch/stable/stein/nova/virt/libvirt/driver.py#L2256
[3] https://opendev.org/openstack/nova/src/branch/stable/stein/nova/virt/libvirt/driver.py#L2286

> Wouldn't it just be as simple as wrapping the 'direct_snapshot' call [1] in similar quiesce/unquiesce calls like '_live_snapshot' already does [2][3]?

Yeah maybe, I think I'd want to see that code moved into a context manager or something though so we don't duplicate it since the snapshot and _live_snapshot methods in the libvirt driver are already very big and complicated.

Possibly related: if os_require_quiesce is set, wouldn't it make sense to also freeze/thaw on suspend/resume, as opposed to just on snapshots?

Sorry, I meant to put "would it make sense". Came out with a rather different connotation than intended; apologies.

Based on the above discussion I mark the bug Confirmed.