[Nova]Live migration isn't really made, instance stays on same compute

Rodion, could you please elaborate on what specific error you see?

We are taking a closer look at the environment right now.

Ok. So this is a specific case of live migration (block_migrate=True), when VMs ephemeral disks *are not* shared, but stored on local disks instead, which means they have to be transferred over the network, when a VM is live migrated. This *can* be done, but generally not recommended, as disks may be big, so it's simply inefficient and puts network under high load.

Depending on the root disk size and network bandwidth, it may take a long time for a VM disk to be transferred. In Liberty a special option was introduced to the libvirt driver to abort stuck live migrations:

[libvirt] live_migration_progress_timeout = 150 (IntOpt) Time to wait, in seconds, for migration to make forward progress in transferring data before aborting the operation. Set to 0 to disable timeouts.

And this is exactly what we see on the Rodion's environment:

http://paste.openstack.org/show/486721/

nova-compute aborted the live migration because there was no progress during 150s.

It's unclear at this point, why qemu/libvirt failed to report progress of a block device migration, as we can see from tcpdump logs, that the disk was actually in the middle of migration when we stopped it. We'll take a closer look at this.

User impact is moderate: if block migration fails, an instance will continue to run on the source host. The workaround is to increase live_migration_progress_timeout value in nova.conf or set it to 0 to disable timeouts completely.

I suggest we downgrade this to High and increase the timeout value in 8.0-mu1. So this should go to release notes in 8.0.

For 9.0, we'll need to take a look, if block migration progress report can be improved from qemu/libvirt side.

I actually think this should be Medium for 9.0 as block-migration is the use case we'd like to avoid and this is mostly mitigated by increasing the timeout value anyway.

Still, if we can improve qemu/libvirt that would be even better.

Oops, forgot to update the importance for 8.0-updates.

Added move-to-10.0 tag due to the fact bug was transferred from 9.0 to 10.0

Won't Fix for 8.0-updates because of Medium importance

Guys, it is not a medium importance, we experience this issue in the customer deployments

Yuriy,

1) as we discussed, you were actually seeing a different issue, not this one

2) the fact that it affects any customer deployments does not directly affect the bug importance, which essentially denotes the users impact and whether the problem can easily be avoided by using workarounds