If we do a BFV nova boot from a UEC image per the docs, the command looks something like this:
nova boot --flavor 1 --block-device source=image,id=72dbd780-8fa2-4fb5-8dbe-94c32cd0a511,dest=volume,shutdo
wn=preserve,bootindex=0,size=1 foo
That creates a volume from an image in cinder during the boot process. In the nova code, it uses the image id on the BDM to get the image metadata from glance. When it does, it sees that it needs direct kernel boot and does so. Thus, with ceph and a UEC image, this type of BFV succeeds.
The other way to do this is to pre-create the volume from a glance image with cinder, and then call nova-boot on that volume once it's done. In that case, nova relies on cinder's volume_image_metadata to be a copy of the glance image metadata. This is where the (important part of the) breakage is. The cinder metadata is not complete and is missing the kernel and ramdisk info when using ceph as the backend:
In this case, nova can't see that this guest needs direct kernel boot and just assumes the disk is bootable. It looks like it *is* complete when doing the same with the LVM driver, which is why it works there.
Just an update on what seems to be happening:
If we do a BFV nova boot from a UEC image per the docs, the command looks something like this:
nova boot --flavor 1 --block-device source= image,id= 72dbd780- 8fa2-4fb5- 8dbe-94c32cd0a5 11,dest= volume, shutdo bootindex= 0,size= 1 foo
wn=preserve,
That creates a volume from an image in cinder during the boot process. In the nova code, it uses the image id on the BDM to get the image metadata from glance. When it does, it sees that it needs direct kernel boot and does so. Thus, with ceph and a UEC image, this type of BFV succeeds.
The other way to do this is to pre-create the volume from a glance image with cinder, and then call nova-boot on that volume once it's done. In that case, nova relies on cinder's volume_ image_metadata to be a copy of the glance image metadata. This is where the (important part of the) breakage is. The cinder metadata is not complete and is missing the kernel and ramdisk info when using ceph as the backend:
| volume_ image_metadata | {u'container_ format' : u'ami', u'min_ram': u'0', u'disk_format': u'ami', u'image_name': u'cirros- 0.3.2-x86_ 64-uec' , u'image_id': u'72dbd780- 8fa2-4fb5- 8dbe-94c32cd0a5 11', u'checksum': u'4eada48c2843d 2a262c814ddc92e cf2c', u'min_disk': u'0', u'size': u'25165824'} |
In this case, nova can't see that this guest needs direct kernel boot and just assumes the disk is bootable. It looks like it *is* complete when doing the same with the LVM driver, which is why it works there.