[Enhancement] Image conversion with RBD is not efficient

Bug #1970114 reported by Mohammed Naser
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Cinder
New
Wishlist
Unassigned

Bug Description

At the moment, if you're using the RBD driver and a user uploads a `qcow2` image, there is a lot of things that happen that can cause a huge resource shortage on the control plane.

1. No limits on the number of conversions that can happen at once, allowing potentially many multiples of volumes being converted at once (rather than sitting/waiting in a queue to be converted, with a set maximum at a time).
2. No "lock" on image conversion when image cache is being used. Therefore, its' possible that a volume being launched 20 times in parallel, with image cache, results in 20 conversions at once on the system.
3. High (unnecessary) I/O on the controllers: We currently download the image into a local file, and then run qemu-img convert to a local file, and then rbd import the target file. The OPTIMIUM setup would be to open a process to `qemu-img`, have it read from STDIN (which will be fed directly from the download) and then make it output DIRECTLY to the Ceph cluster: https://docs.ceph.com/en/latest/rbd/qemu-rbd/#usage. -- This would eliminate ALL IO on the controller since it will only make it a network operation.

I think #3 could be the biggest win in terms of heavily getting rid of the performance hit on I/O since that is one of the most impactful things in control plane:

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=513451#17

With that little 'workaround', we can simply download straight into qemu-rbd, and adding a lock so it's done 1 at a time would provide a huge performance benefit.

Tags: image rbd
Changed in cinder:
importance: Undecided → Wishlist
summary: - Image conversion with RBD is not efficient
+ [Enhancement] Image conversion with RBD is not efficient
tags: added: image rbd
Revision history for this message
Rico Lin (rico-lin) wrote :

here is another related bug report: https://bugs.launchpad.net/cinder/+bug/1970114

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.