kernel stacktrace on volume detach in kvm guest

For the record, IRC snippit:

<smoser> i think i might have been missing some step to explicitly release /dev/sdb from the guest (maybe 'eject /dev/sdb' or something)
<kirkland> smoser: the stack trace on detach?
<kirkland> smoser: i've seen that, clearly wrong, but I haven't noticed any mal-effects

Subsequently, I did:
cc$ euca-attach-volume -i i-4AA8096D -d sdb vol-32F804B0
instance$ sudo mount /dev/sdb /mnt
instance$ # verifiy file is there
instance$ sudo umount /mnt
cc$ euca-detach-volume vol-32F804B0
instance$ grep sdb /proc/partitions || echo "IT IS STILL THERE"
# it wasn't still there.

So, this does indeed not seem to affect re-attaching the volume with the same device name.

Can you please reproduce outside of a eucalyptus environment

We need to test this on lucid Eucalyptus.

Scott- Can you please confirm this as present or fixed in Lucid?

Ok, I've verified this is still present in lucid.
On lucid beta1 kvm, with lucid beta-1 server install

# create a disk image
truncate 10M /tmp/my.img

# launch kvm, watch it boot
kvm -drive file=sda.img,if=scsi,index=0,boot=on -drive file=sdb.img,if=scsi,index=1 -m 512

# in kvm guest, do
$ sudo modprobe acpi_php
# dmesg | tail will now show evidence of something (slot...)

# in kvm monitor, do
% pci_add auto storage file=/tmp/my.img,if=scsi
OK domain 0, bus 0, slot 5, function 0

# in kvm guest, /proc/partitions should now have a new entity (/dev/sdc)
# make a filesystem there, mount, write, unmount
$ sudo mke2fs /dev/sdc
$ sudo mount /dev/sdc /mnt
$ sudo sh -c 'echo hi mom > /mnt/foo'
$ sudo umount /mnt

# in the kvm monitor, detach the device
% pci_del 0:5

# in the kvm guest, now dmesg will show traceback failure

It seems this is a problem in the symbios scsi driver which looks like being present in the same form upstream.

[ 271.732918] [<ffffffff8139a109>] __sym_mfree_dma+0x69/0x100

This function disables interrupts to take a spinlock. Then calls functions to free resources which will emit the warning when called with interrupts disabled.

void __sym_mfree_dma(m_pool_ident_t dev_dmat, void *m, int size, char *name)
{
        unsigned long flags;
        m_pool_p mp;

        spin_lock_irqsave(&sym53c8xx_lock, flags);
        mp = ___get_dma_pool(dev_dmat);
        if (!mp)
                goto out;
        __sym_mfree(mp, m, size, name);
#ifdef SYM_MEM_FREE_UNUSED
        if (!mp->nump)
                ___del_dma_pool(mp);
#endif
 out:
        spin_unlock_irqrestore(&sym53c8xx_lock, flags);
}

Not sure whether the spinunlock might be simply moved up a bit. That might need some discussion with upstream and or testing. But the bug task could be moved to kernel. I guess the QEMU task can only be set to invalid.

SILOGIC/SYMBIOS/NCR 53C8XX and 53C1010 PCI-SCSI drivers
M: Matthew Wilcox <email address hidden>
L: <email address hidden>
S: Maintained
F: drivers/scsi/sym53c8xx_2/

I'm moving this to 'Importance' medium, but might want it to be 'high'. This is the code path that Eucalyptus uses for EBS. EBS is suposed to be reliable storage for cloud instances. With that dmesg output you don't get warm fuzzy feelings about how reliable your data is.

Would one of the kernels at http://people.canonical.com/~smb/lp458201/ make things better or worse?

I tested the linux-image-2.6.32-17-virtual_2.6.32-17.26+lp458201v1_amd64.deb . It appears to fix the problem. Thanks.

Update to this. I sent the quick patch for upstream review and it seems this needs a more complex approach. Regarding the warning the response was that it is bogus for any platform beside ARM. But is there because ARM might sleep in there.
I will try to get a better approach together but that needs a bit of thinking.

This bug was fixed in the package linux - 2.6.32-21.31

---------------
linux (2.6.32-21.31) lucid; urgency=low

  [ Andy Whitcroft ]

  * allow modules.builtin to be optional
  * d-i: add mpt2sas to the message-modules udeb
    - LP: #530361

  [ Christopher James Halse Rogers ]

  * SAUCE: Nouveau: Add quirk framework to disable acceleration
    - LP: #544088, #546393
  * SAUCE: Nouveau: Disable acceleration on MacBook Pros
    - LP: #546393
  * SAUCE: Nouveau: Disable acceleration on GeForce3 cards
    - LP: #544088
  * SAUCE: Nouveau: Disable acceleration on 6100 cards
    - LP: #542950

  [ Stefan Bader ]

  * SAUCE: dma-mapping: Remove WARN_ON in dma_free_coherent
    - LP: #458201

  [ Surbhi Palande ]

  * SAUCE: sync before umount to reduce time taken by ext4 umount
    - LP: #543617

  [ Upstream Kernel Changes ]

  * tipc: Fix oops on send prior to entering networked mode (v3)
    - CVE-2010-1187
  * KVM: x86 emulator: Add Virtual-8086 mode of emulation
    - LP: #561425
  * KVM: x86 emulator: fix memory access during x86 emulation
    - LP: #561425
  * KVM: x86 emulator: Check IOPL level during io instruction emulation
    - LP: #561425
  * KVM: x86 emulator: Fix popf emulation
    - LP: #561425
  * KVM: Fix segment descriptor loading
    - LP: #561425
  * KVM: VMX: Update instruction length on intercepted BP
    - LP: #561425
  * KVM: VMX: Use macros instead of hex value on cr0 initialization
    - LP: #561425
  * KVM: SVM: Reset cr0 properly on vcpu reset
    - LP: #561425
  * KVM: VMX: Disable unrestricted guest when EPT disabled
    - LP: #561425
  * KVM: x86: disable paravirt mmu reporting
    - LP: #561425
  * AppArmor: Fix put of unassigned ns if aa_unpack fails
  * AppArmor: Fix refcount bug when exec fails
    - LP: #562063
  * AppArmor: Take refcount on cxt->profile to ensure it remains a valid
    reference
    - LP: #367499
  * AppArmor: fix typo in scrubbing environment variable warning
    - LP: #562060
  * AppArmor: fix regression by setting default to mediate deleted files
    - LP: #562056
  * AppArmor: fix refcount order bug that can trigger during replacement
    - LP: #367499
  * AppArmor: Make sure to unmap aliases for vmalloced dfas before they are
    live
    - LP: #529288
  * AppArmor: address performance regression of replaced profile
    - LP: #549428
  * AppArmor: make the global side the correct type
    - LP: #562047
  * AppArmor: use the kernel shared workqueue to free vmalloc'ed dfas
  * sky2: add register definitions for new chips
    - LP: #537168
  * sky2: 88E8059 support
    - LP: #537168
  * net: Fix Yukon-2 Optima TCP offload setup
    - LP: #537168
  * net: Add missing TST_CFG_WRITE bits around sky2_pci_write
    - LP: #537168
  * sky2: print Optima chip name
    - LP: #537168
  * (Upstream) dell-laptop: defer dell_rfkill_update to worker thread
    - LP: #555261
  * drm/nv40: add LVDS table quirk for Dell Latitude D620
    - LP: #539730
 -- Andy Whitcroft <email address hidden> Tue, 13 Apr 2010 18:50:58 +0100