Live migration fails

Bug #820339 reported by Tim Bordemann
28
This bug affects 5 people
Affects Status Importance Assigned to Milestone
libvirt (Ubuntu)
Triaged
Low
Unassigned

Bug Description

I have two identically configured hosts, based on Natty. When trying to do a live migration with following command
virsh migrate --live --copy-storage-inc admin qemu+ssh://root@remus/system --persistent --verbose
it stops with the error message "error: operation failed: migration job: unexpectedly failed".
On the second host libvirt reports

Aug 3 14:29:22 remus libvirtd: 14:29:22.451: 1223: info : virSecurityDACSetOwnership:99 : Setting DAC user and group on '/kvm-images/admin.img' to '0:0'
Aug 3 14:29:22 remus kernel: [ 1821.968789] audit_printk_skb: 30 callbacks suppressed
Aug 3 14:29:22 remus kernel: [ 1821.968791] type=1400 audit(1312374562.648:20): apparmor="STATUS" operation="profile_load" name="libvirt-f675d59e-6f84-baa1-8f20-5df1e410fb21" pid=1928 comm="apparmor_parser"
Aug 3 14:29:23 remus libvirtd: 14:29:23.153: 1223: info : brProbeVnetHdr:454 : Enabling IFF_VNET_HDR
Aug 3 14:29:23 remus kernel: [ 1822.469389] device vnet0 entered promiscuous mode
Aug 3 14:29:23 remus kernel: [ 1822.471155] br100: port 2(vnet0) entering forwarding state
Aug 3 14:29:23 remus kernel: [ 1822.471159] br100: port 2(vnet0) entering forwarding state
Aug 3 14:29:26 remus libvirtd: 14:29:26.538: 1224: info : virSecurityDACRestoreSecurityFileLabel:139 : Restoring DAC user and group on '/kvm-images/admin.img'
Aug 3 14:29:26 remus libvirtd: 14:29:26.538: 1224: info : virSecurityDACSetOwnership:99 : Setting DAC user and group on '/kvm-images/admin.img' to '0:0'
Aug 3 14:29:26 remus kernel: [ 1826.021058] type=1400 audit(1312374566.698:21): apparmor="STATUS" operation="profile_remove" name="libvirt-f675d59e-6f84-baa1-8f20-5df1e410fb21" pid=1952 comm="apparmor_parser"
Aug 3 14:29:26 remus kernel: [ 1826.258856] br100: port 2(vnet0) entering forwarding state
Aug 3 14:29:27 remus kernel: [ 1826.344260] device vnet0 left promiscuous mode
Aug 3 14:29:27 remus kernel: [ 1826.344265] br100: port 2(vnet0) entering disabled state

I can't find any error messages, does anyone have any suggestions?

Revision history for this message
Serge Hallyn (serge-hallyn) wrote :

Thanks for reporting this bug. I can't see any error messages either. To give us some more data so I can try to reproduce, can you

1. run apport-collect 820339
2. Post the .xml for the vm you are trying to migrate?

Changed in libvirt (Ubuntu):
status: New → Incomplete
importance: Undecided → Medium
Revision history for this message
Tim Bordemann (timbo) wrote :

1: apport couldn't find package libvirt?

=====================================================
root@romulus:~# apport-collect 820339
The authorization page:
[...]
Waiting to hear from Launchpad about your decision...
Package libvirt not installed and no hook available, ignoring

*** Updating problem report

No additional information collected.

Press any key to continue...

No pending crash reports. Try --help for more information.
root@romulus:~#
=====================================================

2: (You can find the xml here too: http://pastebin.com/KsVR0tjY)

<domain type='kvm'>
  <name>admin</name>
  <uuid>f675c59e-6f74-bab1-8e20-5af1e210fb21</uuid>
  <memory>2097152</memory>
  <currentMemory>1048576</currentMemory>
  <vcpu>1</vcpu>
  <os>
    <type arch='x86_64' machine='pc-0.14'>hvm</type>
    <boot dev='hd'/>
    <bootmenu enable='no'/>
  </os>
  <features>
    <acpi/>
    <pae/>
  </features>
  <clock offset='utc'/>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>restart</on_crash>
  <devices>
    <emulator>/usr/bin/kvm</emulator>
    <disk type='file' device='disk'>
      <driver name='qemu' type='raw'/>
      <source file='/kvm-images/admin.img'/>
      <target dev='vda' bus='virtio'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
    </disk>
    <interface type='bridge'>
      <mac address='54:52:00:2b:14:32'/>
      <source bridge='br100'/>
      <model type='virtio'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </interface>
    <serial type='pty'>
      <target port='0'/>
    </serial>
    <console type='pty'>
      <target type='serial' port='0'/>
    </console>
    <input type='mouse' bus='ps2'/>
    <graphics type='vnc' port='-1' autoport='yes'/>
    <video>
      <model type='cirrus' vram='9216' heads='1'/>
     <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </video>
    <memballoon model='virtio'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
    </memballoon>
  </devices>
</domain>

Revision history for this message
Serge Hallyn (serge-hallyn) wrote : Re: [Bug 820339] Re: Live migration fails

Quoting Tim Bordemann (<email address hidden>):
> 1: apport couldn't find package libvirt?

How did you install libvirt? What do

  which libvirtd
  libvirtd --version
  dpkg -l | grep libvirt

show?

Revision history for this message
Tim Bordemann (timbo) wrote :

I installed libvirt from the default repositories (libvirt-bin).

root@romulus:~# which libvirtd
/usr/sbin/libvirtd
root@romulus:~# libvirtd --version
libvirtd (libvirt) 0.8.8
root@romulus:~# dpkg -l | grep libvirt
ii libvirt-bin 0.8.8-1ubuntu6.5 the programs for the libvirt library
ii libvirt0 0.8.8-1ubuntu6.5 library for interfacing with different virtualization systems
ii python-libvirt 0.8.8-1ubuntu6.5 libvirt Python bindings
root@romulus:~#

Revision history for this message
Tom Vijlbrief (tvijlbrief) wrote :

In my situation I found the cause:

https://bugzilla.redhat.com/show_bug.cgi?id=499750

My hostname was different from my (reverse) DNS mapping on the destination host.

Changing my host name on the destination host fixed the problem for me.

Revision history for this message
Serge Hallyn (serge-hallyn) wrote :

Tim,

can you see whether Tom's fix also works for you?

(If it does, I think I still consider that a bug, but apparently upstream does not...)

We also need to figure out why apport-collect wouldn't work for you.

Revision history for this message
Tim Bordemann (timbo) wrote :

Yes, Tom's fix worked. Thank you.

Revision history for this message
Serge Hallyn (serge-hallyn) wrote :

Thanks, Tim. Until I understand better why the behavior must be so, I"m going to mark this bug as Triaged.

Changed in libvirt (Ubuntu):
status: Incomplete → Triaged
importance: Medium → Low
Revision history for this message
Serge Hallyn (serge-hallyn) wrote :

(Marking low priority as it does have a 'workaround'.)

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.