xend fails to connect guest to dom0 block device or loopback file

Did you try tap:aio instead of file: in the guest config file?

Also, there have been a number of problems fixed, but a couple lingering issues with the latest packages.

see:
https://bugs.launchpad.net/ubuntu/+source/xen-3.2

also, make sure that the modules (loaded by /etc/init.d/xend are being loaded properly)
for some reference see:
https://bugs.launchpad.net/ubuntu/+source/xen-3.2/+bug/199533

I can confirm this behavior.

I think that it is *also* a bug with xen-tools since it should use tap:aio by default instead of file:/

changing to tap:aio lets the guest boot, but brings me back to this bug:

https://bugs.launchpad.net/ubuntu/+source/xen-3.2/+bug/144631

Additional information:

Attaching bootup process that dies at:
   0.242383] /build/buildd/linux-2.6.24/debian/build/custom-source-xen/drivers/rtc/hctosys.c: unable to open rtc device (rtc0)
[ 0.242392] Freeing unused kernel memory: 216k freed
Loading, please wait...
Begin: Loading essential drivers... ...
[ 0.459390] fuse init (API version 7.9)
[ 0.478378] thermal: Unknown symbol acpi_processor_set_thermal_limit
Done.
Begin: Running /scripts/init-premount ...
Done.
Begin: Mounting root file system... ...
Begin: Running /scripts/local-top ...
Done.
Begin: Waiting for root file system... ...
Done.
 Check root= bootarg cat /proc/cmdline
 or missing modules, devices: cat /proc/modules ls /dev
ALERT! /dev/sda2 does not exist. Dropping to a shell!

BusyBox v1.1.3 (Debian 1:1.1.3-5ubuntu12) Built-in shell (ash)
Enter 'help' for a list of built-in commands.

(initramfs)

complete boot process and output of the following atached:

cat /proc/cmdline, cat /proc/modules and ls /dev

attaching xenguest1.cfg

My best guess is that it is a problem with the xen initrd (/boot/initrd.img-2.6.24-12-xen).

It seems to be only loading the fuse module and not the SCSI modules or any of the xen modules.

The following are loaded by xend:
blktap
blkbk
xenblk
netloop
netbk

Do any of these and possible also SCSI modules need to be explicitly in the initrd too?

I tried adding the following to /etc/initramfs-tools/modules and then running update-initramfs -u

sd_mod
scsi_mod
blktap
blkbk
xenblk
netloop
netbk

The following similar thing happens:

[ 0.215498] Freeing unused kernel memory: 216k freed
Loading, please wait...
Begin: Loading essential drivers... ...
[ 0.433778] SCSI subsystem initialized
[ 0.434788] Driver 'sd' needs updating - please use bus_type methods
[ 0.458819] register_blkdev: cannot get major 8 for sd
[ 0.458836] vbd vbd-2049: 19 xlvbd_add at /local/domain/0/backend/tap/3/2049
[ 0.459117] register_blkdev: cannot get major 8 for sd
[ 0.459128] vbd vbd-2050: 19 xlvbd_add at /local/domain/0/backend/tap/3/2050
[ 5.547967] XENBUS: Waiting for devices to initialise: 295s...290s...285s...280s...275s...270s...265s...260s...255s...250s...245s...240s...235s...230s...225s...220s...215s...210s...205s...200s...195s...190s...185s...180s...175s...170s...165s...160s...155s...150s...145s...140s...135s...130s...125s...120s...115s...110s...105s...100s...95s...90s...85s...80s...75s...70s...65s...60s...55s...50s...45s...40s...35s...30s...25s...20s...15s...10s...5s...0s...
[ 298.329595] XENBUS: Device not ready: device/vbd/2049
[ 298.329599] XENBUS: Device not ready: device/vbd/2050
[ 298.348441] fuse init (API version 7.9)
[ 298.373876] thermal: Unknown symbol acpi_processor_set_thermal_limit
Done.
Begin: Running /scripts/init-premount ...
Done.
Begin: Mounting root file system... ...
Begin: Running /scripts/local-top ...
Done.
Begin: Waiting for root file system... ...
Done.
 Check root= bootarg cat /proc/cmdline
 or missing modules, devices: cat /proc/modules ls /dev
ALERT! /dev/sda2 does not exist. Dropping to a shell!

BusyBox v1.1.3 (Debian 1:1.1.3-5ubuntu12) Built-in shell (ash)
Enter 'help' for a list of built-in commands.

(initramfs) cat /proc/modules
fuse 56368 1 - Live 0xffffffff8808c000
netbk 111600 0 [permanent], Live 0xffffffff8806f000
netloop 8960 0 - Live 0xffffffff8806b000
xenblk 22496 0 - Live 0xffffffff88064000
blkbk 26936 0 [permanent], Live 0xffffffff8805c000
blktap 123908 0 [permanent], Live 0xffffffff8803c000
xenbus_be 5888 3 netbk,blkbk,blktap, Live 0xffffffff88039000
sd_mod 33280 0 - Live 0xffffffff8802f000
scsi_mod 179000 1 sd_mod, Live 0xffffffff88002000
(initramfs)

I also noticed that /var/log/xen/xen-hotplug.log has:
Nothing to flush.
xenstore-read: couldn't read path backend/vbd/1/2049/node
xenstore-read: couldn't read path backend/vbd/1/2050/node
Nothing to flush.
xenstore-read: couldn't read path /local/domain/2/vm
xenstore-read: couldn't read path /local/domain/2/vm
Nothing to flush.

I still think it might have to do with the initrd, but not sure what exactly. Building from source did work, but it would be better to be able to use the ubuntu packages.

UPDATE:

changing sda to hda in the guest config

gets a little farther into the boot:

[ 1.252536] thermal: Unknown symbol acpi_processor_set_thermal_limit
Done.
Begin: Running /scripts/init-premount ...
Done.
Begin: Mounting root file system... ...
Begin: Running /scripts/local-top ...
Done.
Begin: Waiting for root file system... ...
Done.
Begin: Running /scripts/local-premount ...
/scripts/local-premount/resume: /scripts/local-premount/resume: 57: log_begin_msg: not found
/scripts/local-premount/resume: /scripts/local-premount/resume: 57: log_end_msg: not found
Done.
[ 6.782336] kjournald starting. Commit interval 5 seconds
[ 6.782349] EXT3-fs: mounted filesystem with ordered data mode.
Begin: Running /scripts/local-bottom ...
Done.
Done.
Begin: Running /scripts/init-bottom ...
Done.
INIT: version 2.86 booting
* Mount point '/dev/shm' does not exist. Skipping mount.
Activating swap...done.
Checking root file system...fsck 1.40-WIP (14-Nov-2006)
/lib/init/rw/rootdev: Superblock last mount time is in the future. FIXED.
/lib/init/rw/rootdev: Superblock last write time is in the future. FIXED.
/lib/init/rw/rootdev: clean, 10896/524288 files, 118298/1048576 blocks
done.
[ 8.028390] EXT3 FS on hda2, internal journal
Setting the system clock..
Cannot access the Hardware Clock via any known method.
Use the --debug option to see the details of our search for an access method.
Cleaning up ifupdown....
Loading kernel modules...done.
Loading device-mapper support[ 8.393666] device-mapper: uevent: version 1.0.3
[ 8.393690] device-mapper: ioctl: 4.12.0-ioctl (2007-10-02) initialised: <email address hidden>
.
Checking file systems...fsck 1.40-WIP (14-Nov-2006)
done.
Setting kernel variables...done.
Mounting local filesystems...done.
Activating swapfile swap...done.
Setting up networking....
Configuring network interfaces...Internet Systems Consortium DHCP Client V3.0.4
Copyright 2004-2006 Internet Systems Consortium.
All rights reserved.
For info, please visit http://www.isc.org/sw/dhcp/

SIOCSIFADDR: No such device
eth0: ERROR while getting interface flags: No such device
eth0: ERROR while getting interface flags: No such device
Bind socket to interface: No such device
Failed to bring up eth0.
done.
INIT: Entering runlevel: 2
Starting system log daemon: syslogd.
Starting kernel log daemon: klogd.
* Not starting internet superserver: no services enabled.
Starting OpenBSD Secure Shell server: sshd.
Starting periodic command scheduler: crond.
<freezes here>

adding the extra='xencons=tty' gives a login prompt.

This leads to the networking bug(s)...

I don't (yet) have an eth0 like those in this bug have:

https://bugs.launchpad.net/ubuntu/+source/xen-3.2/+bug/204010

ok so a modprobe xennet gives me an eth0...

no connectivity yet though.

There are several strange bugs documented here, but I will move to

bug # 204010 to see if we can figure out the network stuff

The new kernel package 2.6.24-14 fixed this problem for me. Can anyone confirm this?
The networking bug #204010 seems not to be fixed by now.

deti said:
> The new kernel package 2.6.24-14 fixed this problem for me. Can anyone confirm this?
> The networking bug #204010 seems not to be fixed by now.

Not fixed for me by this package, no. =/

Andrew.

--
Andrew Shugg <email address hidden> http://www.neep.com.au/

"Just remember, Mr Fawlty, there's always someone worse off than yourself."
"Is there? Well I'd like to meet him. I could do with a good laugh."

It is actually fixed for me, but with the workarounds listed above:

sda --> hda

file:/ --> tap:aio/

a way to get the console:

      adding extra='xencons=tty' to domU config

      OR

      adding xvc0 to /etc/inittab in domU file system

The bugs above could be fixed by fixing xen-tools and/or are different separate bugs.

So, the bug as filed has basically been fixed and has workarounds for things that are broken otherwise

Todd Deshane said:
> It is actually fixed for me, but with the workarounds listed above:
>
> sda --> hda
>
> file:/ --> tap:aio/

You're right, with the following config I have a working domU - as far
as the block devices are concerned anyway. (I did not have the hda/sda
problem: strange.)

kernel = '/boot/vmlinuz-2.6.24-15-xen'
ramdisk = '/boot/initrd.img-2.6.24-15-xen'
memory = '256'
root = '/dev/xvda2 ro'
disk = [
                  'tap:aio:/xen/domains/domU/swap.img,xvda1,w',
                  'tap:aio:/xen/domains/domU/disk.img,xvda2,w',
              ]
name = 'domU'
dhcp = 'dhcp'
vif = [ 'mac=00:16:3E:6C:07:90' ]
on_poweroff = 'destroy'
on_reboot = 'restart'
on_crash = 'restart'
extra = '2 console=xvc0'

Now I have to look at what you and other people have done to get a
functional console (got boot output but no getty) and networking.

> The bugs above could be fixed by fixing xen-tools and/or are different
> separate bugs.

And indeed xen-tools (3.8-4ubuntu4) has slipped in quietly to fix the
xvda problem and to add support for hardy guests. It hasn't fixed the
tap:aio: instead of file: yet.

> So, the bug as filed has basically been fixed and has workarounds for
> things that are broken otherwise

Agreed, I suppose.

Andrew.

--
Andrew Shugg <email address hidden> http://www.neep.com.au/

"Just remember, Mr Fawlty, there's always someone worse off than yourself."
"Is there? Well I'd like to meet him. I could do with a good laugh."

I've a problem similar to what Todd found in:
https://bugs.launchpad.net/ubuntu/+source/xen-3.2/+bug/205450/comments/3

I've filed a separate bug report here:
https://bugs.launchpad.net/ubuntu/+source/xen-3.2/+bug/214821

I'm using lvm instead of loopback image, and I've tried to change to hda without success.

Todd says:
"I still think it might have to do with the initrd, but not sure what exactly. Building from source did work, but it would be better to be able to use the ubuntu packages."
what package should I build from source? with specific options?
thanks

I can confirm this.

I added this to my initramfs modules:

sd_mod
scsi_mod
blkbk
xenblk
netloop
netbk
xennet

After i build by initramfs and uses hda1 in my dom0.cfg the guest starts and the network works

In linux-image-2.6.24-16-xen (2.6.24-16.30) I am back to the old problem:

sysadmin@dom0:~$ sudo xm create -c /etc/xen/gutsy.cfg
Using config file "/etc/xen/gutsy.cfg".
Error: Device 51713 (vbd) could not be connected. losetup /dev/loop4 /xen/domains/gutsy/swap.img failed
sysadmin@dom0:~$

Possibly this is because all the /lib/modules/`uname -r`/kernel/drivers/xen modules are missing... more detail here:

https://bugs.launchpad.net/ubuntu/+source/xen-3.2/+bug/199533/comments/41

Same here. Is there any known fix or workaround?

The "tap:aio:" and "xvda" workaround still seems to work - but should this be the solution for upcoming 8.04 LTS version? Still networking does not work either... this is depressing!

I don't the file:/ is a high priority. I am surprised that it doesn't work, I bet it is just a module not loading or something similar.

deti: Did you notice that in the network bug that someone posted a patch and some links to custom packages, you might want to try those.

Todd Deshane wrote:
> I don't the file:/ is a high priority. I am surprised that it doesn't
> work, I bet it is just a module not loading or something similar.
>
> deti: Did you notice that in the network bug that someone posted a patch
> and some links to custom packages, you might want to try those.
>
I know and I can confirm the packages are working well. It's just little
annoying that we'll have to get packages from somewhere else than from
the official repository.
Now after the code freeze it's very likely that we'll have to update
8.04 first before using XEN. That would be no real 'out of the box'
experience... - but I understand well that it's a hard job bringing out
a new release of a distribution where everything is working at the first
glance.

I can confirm the bug and I've tried every single workaround/fix/variant of domU config files with no success (waiting to mount root filesystem)... is there a unified/unique way to deal with this issue ?

another workaround:

//understand losetup -f and -a
$ sudo losetup -f
$ sudo losetup -a

//actually do the mount and check that it worked and which device it picked
$ sudo losetup `sudo losetup -f` <iso_file>
$ sudo losetup -a

//then use phy:/dev/loop<num> in your xen config file

I wouldn't recommend this for disk partition, use tap:aio: instead.

But for cdrom image files, it is a good workaround.

Also, remember that file: is deprecated due to the problems with loopback devices
which is what this workaround uses, which is why you should only use it for iso (read-only) files.

Thanks, fortunately it was just a "kept back" packages issue:

1) I installed hardy heron when it was devel
2) During subsequent upgrades, the 2.6...*-15* kernel was kept not upgrading to *-16*
3) apt-get dist-upgrade did the trick
4) tap:aio fixed the remaining vbd issue :)

Thanks !

I have had this problem for a while too. (swap.img failed error)?
I looked where Andrew Shugg said in here;
https://bugs.launchpad.net/ubuntu/+source/xen-3.2/+bug/199533/comments/41
and sure enough none of those files exist. the whole directory of xen is missing indeed?!
So whats the cure?
where can I get those files:
ls: cannot access /lib/modules/2.6.24-21-xen/kernel/drivers/xen/: No such file or directory
(But there are a lot of other folders under "drivers" directory from acpi to watchdog)!

BTW, I did apt-get dist-upgrade to no avail?!

Thanks!
mehdi

Confirming because it happens to several users.