XenServer - Nova-Compute StorageError Waiting for device

Bug #1318544 reported by Thijs van Leeuwen
12
This bug affects 2 people
Affects Status Importance Assigned to Milestone
OpenStack Compute (nova)
Confirmed
Medium
Unassigned

Bug Description

Hi All,

I started building a Openstack cloud based on the new LTS version of Ubuntu (14.04). I installed both Control and Compute nodes as VM's on a XenServer. I selected the option 'Other Operating System' so this is running as HVM.

The cluster is up and running: i can start instances, create storage and allocate IP addresses. But the error i got is popping up when the instance is in spawning state. The error i attached is from the nova-compute node.

...

2014-05-12 09:00:30.918 6660 TRACE nova.compute.manager [instance: 00e2c60e-372b-43d2-a2fd-bcc3628c24a9] StorageError: Timeout waiting for device hdb to be created
2014-05-12 09:00:30.918 6660 TRACE nova.compute.manager [instance: 00e2c60e-372b-43d2-a2fd-bcc3628c24a9]

Tags: xenserver
Revision history for this message
Thijs van Leeuwen (n-thijs) wrote :
Tracy Jones (tjones-i)
tags: added: xenserver
Revision history for this message
Bob Ball (bob-ball) wrote :

Please attach a longer output from the log file; there are debug messages immediately before the TRACE that could be useful in identifying the problem.

I suspect the issue is that the Nova Compute domain must be running as PV, not HVM with PV tools. You can convert the domain using a guide such as http://wiki.univention.com/index.php?title=Citrix_Xen_Server#Convert_the_UCS_system_to_paravirtualization

Revision history for this message
John Garbutt (johngarbutt) wrote :

Was this boot from volume? there is a bug that has been recently fixed in that area.

What version of Nova is this please?

Changed in nova:
status: New → Incomplete
importance: Undecided → Medium
Revision history for this message
Sean Dague (sdague) wrote :

Long incomplete bug

Changed in nova:
status: Incomplete → Invalid
Bob Ball (bob-ball)
description: updated
Changed in nova:
status: Invalid → Confirmed
Revision history for this message
Bob Ball (bob-ball) wrote :

Identified + confirmed this issue.

vdi_attached_here retrieves a "guess" of the device name from XAPI. When using HVM guests, XAPI assumes that the guest is Windows and therefore returns "hdb". This is clearly wrong.

We need to change the _wait_for_dev function to iterate over sysfs to find the correct disk.
e.g. assuming the UUID of the disk plugged was 8bf0e9c2-b636-4e43-a1a4-e02a339d7d82:
{quote}
root@DevStackOSDomU:~# xenstore-read domid
35
root@DevStackOSDomU:~# ls -d /sys/devices/vbd-*
/sys/devices/vbd-51712 /sys/devices/vbd-51728

root@DevStackOSDomU:~# xenstore-read /local/domain/0/backend/vbd/35/51712/vdi
["VDI", "1a6bad09-c13f-c7dd-b0af-cfb7e8de518c\\/9dcb54df-71f0-46eb-8db5-400f62e68629"]

root@DevStackOSDomU:~# xenstore-read /local/domain/0/backend/vbd/35/51728/vdi
["VDI", "1a6bad09-c13f-c7dd-b0af-cfb7e8de518c\\/8bf0e9c2-b636-4e43-a1a4-e02a339d7d82"]

root@DevStackOSDomU:~# ls /sys/devices/vbd-51728/block
xvdb
{/quote}

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (master)

Fix proposed to branch: master
Review: https://review.openstack.org/213112

Changed in nova:
assignee: nobody → huan (huan-xie)
status: Confirmed → In Progress
Changed in nova:
assignee: huan (huan-xie) → Bob Ball (bob-ball)
Changed in nova:
assignee: Bob Ball (bob-ball) → huan (huan-xie)
Changed in nova:
assignee: huan (huan-xie) → nobody
status: In Progress → Confirmed
Bob Ball (bob-ball)
Changed in nova:
assignee: nobody → huan (huan-xie)
status: Confirmed → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on nova (master)

Change abandoned by Michael Still (<email address hidden>) on branch: master
Review: https://review.openstack.org/213112
Reason: This patch has been sitting unchanged for more than 12 weeks. I am therefore going to abandon it to keep the nova review queue sane. Please feel free to restore the change if you're still working on it.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote :

Change abandoned by Huan Xie (<email address hidden>) on branch: master
Review: https://review.openstack.org/213112
Reason: Will move the fix in os-xenapi

Revision history for this message
Sean Dague (sdague) wrote :

There are no currently open reviews on this bug, changing the status back to the previous state and unassigning. If there are active reviews related to this bug, please include links in comments.

Changed in nova:
status: In Progress → Confirmed
assignee: huan (huan-xie) → nobody
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Bug attachments

Remote bug watches

Bug watches keep track of this bug in other bug trackers.