live block migration results in loss of console log

What kind of hypervisor are you using? libvirt?

I'm using libvirt with libvirt-type = qemu. There's no VMX support on the host hypervisor.

Is not it reasonable for zero console log after live-migration? After live-migration, we do not restart the VM, so it is correct there is no console log.

The original contents of the console log is lost after a migration and that's the issue here.

Why should a VM user lose the ability to inspect the console log when a service provider decided to migrate the VM for maintenance when everything else is torn and setup at the destination host correctly?

After reading live-migration related codes in openstack side just now, I guess most likely it's a bug of kvm.
openstack only checks the configuration and configures network for the vm by those two methods (pre_live_migration & post_live_migration),
the rest thing is left to kvm to do by invoking it's virDomainMigrateToURI method of libvirtmod module which is realized using C language.

I don't see the code of virDomainMigrateToURI method, but I guess it's possible steps are as bellow:
1, share pubkey between the two hosts with ssh-copy id root@targethost from "sourcehost"
2, copy the vm's disk from the source host to the target host by: dd if=/dev/<vgname>/<lvname> | ssh root@targethost 'dd of=/dev/<vgname>/<lvname>'
3, dump the xml definitions virsh dumpxml <vm> dump.xml
4, transfer xml-file scp dump.xml root@targethost:/root/
5, define the machine on target virsh define dump.xml the machine is then created from the xml.
6, virsh start vm

so parthipan, could you help confirm following two things:
1) are you configuring the share ssh pubkey for root account ?
2) have you tried block live migration by libvirt's CLI directly, rather than openstack's CLI. like:
   virsh migrate --live --copy-storage-all --verbose <vm-name> qemu+ssh://<target-host>/system

actually it is a bug of libvirt, we already opened a issue in libvirt community to tracking that issue. will post the url to here later.

https://bugzilla.redhat.com/show_bug.cgi?id=994882

I have registered a blueprint to slightly alter the way console logs are read in the libvirt driver. Please comment on this design. Thanks.

https://blueprints.launchpad.net/nova/+spec/support-rotated-console-logs-in-libvirt-driver

I have a feeling that this bug is not related to nova. It is libvirt's/hypervisor's job to copy every needed file to destination host, nova only triggers live migration operation. I believe that it should be solved as part of bug that GuoHui mentioned: https://bugzilla.redhat.com/show_bug.cgi?id=994882

No action on this one in a very long time, so unassigning.

Fix proposed to branch: master
Review: https://review.openstack.org/284674

[In reply to comment #9]

Pawel, seems like there's general agreement from libvirt upstream that this needs to be handled at a higher-layer than libvirt.

[Below is a comment from IRC, by Dan Berrangé, on this topic, from here: https://bugzilla.redhat.com/show_bug.cgi?id=994882#c3 ]
-------
"This bug is basically asking for the block migration functionality to extend to other qemu devices backed by files. This is pretty tricky in general, because with latest libvirt & qemu, qemu doesn't even get acess to the log files. It just gets given an anonymous pipe file descriptor, so it has no ability to migrate the log file even if it wanted to.

libvirt tries to only concern itself with what happens on a single-node
so moving stuff between nodes is left to the app using libvirt which knows better what to do".
-------

And, Pawel just agreed on #openstack-nova that it "sounds fair that higher-layer should handle this, yes".

Change abandoned by Michael Still (<email address hidden>) on branch: master
Review: https://review.openstack.org/284674
Reason: This code hasn't been updated in a long time, and is in merge conflict. I am going to abandon this review, but feel free to restore it if you're still working on this.

@stgleb: Are you still working on this bug, if not please unassign yourself and set it to 'confirmed'.

@siva-radhakrishnan: Are you working on this bug still?

This bug report has an assignee for a while now but there is no patch
for that. It looks like that the chance of getting a patch is low.
I'm going to remove the assignee to signal to others that they can take
over if they like.
If you want to work on this, please:
* add yourself as assignee AND
* set the status to "In Progress" AND
* provide a (WIP) patch within the next 2 weeks after that.
If you need assistance, reach out on the IRC channel #openstack-nova or
use the mailing list.

I can reproduce this issue without block migration. Setup with /var/lib/nova/instances/ on shared NFS, using openstack server migrate --live ... --shared-migration ...

Would it be the same root cause?

Looks like the correct link for the related spec is https://blueprints.launchpad.net/nova/+spec/support-console-log-migration
Details https://specs.openstack.org/openstack/nova-specs/specs/juno/approved/support-console-log-migration.html