VMBuilder

Bug #576501
Comment #17

Comment 17 for bug 576501

Revision history for this message

Henrik Bach (bach-henrik) wrote on 2011-05-03: Re: [Bug 576501] Re: vmbuilder doesn't recognize --ec2 flag

#17

On Mon, Apr 18, 2011 at 10:32 PM, Serge Hallyn
<email address hidden> wrote:
> Quoting Henrik Bach (<email address hidden>):
>> 2011-04-16 04:36:12,380 INFO : Creating config file
>> /etc/default/grub with new version
>> 2011-04-16 04:36:12,970 INFO : /proc/devices: fopen failed: No such
>> file or directory
>> 2011-04-16 04:36:12,971 INFO : Is device-mapper driver missing from kernel?
>
> Hm. Actually all the problems I've had I've traced to environments (bad
> network setup for one, device permissions for another). I finally got a
> maverick xen host in a working environment - and vmbuilder -ec2 worked
> there.

Ok.

>
> What does /proc/devices on your host show?

$ cat /proc/devices
Character devices:
  1 mem
  4 /dev/vc/0
  4 tty
  4 ttyS
  5 /dev/tty
  5 /dev/console
  5 /dev/ptmx
  6 lp
  7 vcs
10 misc
13 input
21 sg
29 fb
99 ppdev
108 ppp
116 alsa
128 ptm
136 pts
180 usb
189 usb_device
226 drm
252 usbmon
253 bsg
254 rtc

Block devices:
  1 ramdisk
  2 fd
259 blkext
  7 loop
  8 sd
  9 md
11 sr
65 sd
66 sd
67 sd
68 sd
69 sd
70 sd
71 sd
128 sd
129 sd
130 sd
131 sd
132 sd
133 sd
134 sd
135 sd
252 device-mapper
253 pktcdvd
254 mdp

> What sort of storage setup
> do you have?

I'm building the image on an ordinary harddisk with my home folder encrypted.

> The 'Is device-mapper driver missing from kernel?' is
> odd, and presumably comes from grub in the chroot. It almost sounds
> like your host is multipathed.
> What does 'df -h' and 'mount' on the host show?

$ df -h
Filesystem Size Used Avail Use% Mounted on
/dev/sda1 36G 7,5G 26G 23% /
none 491M 228K 491M 1% /dev
none 497M 1,4M 496M 1% /dev/shm
none 497M 140K 497M 1% /var/run
none 497M 0 497M 0% /var/lock
none 36G 7,5G 26G 23% /var/lib/ureadahead/debugfs
$ mount
/dev/sda1 on / type ext4 (rw,errors=remount-ro,commit=0)
proc on /proc type proc (rw,noexec,nosuid,nodev)
none on /sys type sysfs (rw,noexec,nosuid,nodev)
fusectl on /sys/fs/fuse/connections type fusectl (rw)
none on /sys/kernel/debug type debugfs (rw)
none on /sys/kernel/security type securityfs (rw)
none on /dev type devtmpfs (rw,mode=0755)
none on /dev/pts type devpts (rw,noexec,nosuid,gid=5,mode=0620)
none on /dev/shm type tmpfs (rw,nosuid,nodev)
none on /var/run type tmpfs (rw,nosuid,mode=0755)
none on /var/lock type tmpfs (rw,noexec,nosuid,nodev)
none on /var/lib/ureadahead/debugfs type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc
(rw,noexec,nosuid,nodev)
gvfs-fuse-daemon on /home/sad/.gvfs type fuse.gvfs-fuse-daemon
(rw,nosuid,nodev,user=X)

>
> Does a regular 'vmbuilder xen ubuntu' work for you?
>

First try was:

$ sudo vmbuilder xen ubuntu
[sudo] password for X:
2011-05-03 19:50:53,587 INFO : Calling hook: preflight_check
2011-05-03 19:50:53,641 INFO : Calling hook: set_defaults
2011-05-03 19:50:53,647 INFO : Calling hook: bootstrap
2011-05-03 20:02:47,612 INFO : Calling hook: configure_os
2011-05-03 20:03:23,304 INFO :
2011-05-03 20:03:23,305 INFO : Current default time zone: 'Etc/UTC'
2011-05-03 20:03:23,339 INFO : Local time is now: Tue May 3
18:03:23 UTC 2011.
2011-05-03 20:03:23,339 INFO : Universal Time is now: Tue May 3
18:03:23 UTC 2011.
2011-05-03 20:03:23,340 INFO :
Extracting templates from packages: 100%
2011-05-03 20:05:00,351 INFO :
2011-05-03 20:05:00,352 INFO : Current default time zone: 'Etc/UTC'
2011-05-03 20:05:00,384 INFO : Local time is now: Tue May 3
18:05:00 UTC 2011.
2011-05-03 20:05:00,384 INFO : Universal Time is now: Tue May 3
18:05:00 UTC 2011.
2011-05-03 20:05:00,385 INFO : Run 'dpkg-reconfigure tzdata' if you
wish to change it.
2011-05-03 20:05:00,385 INFO :
2011-05-03 20:06:36,230 INFO : start: Unknown parameter: JOB
2011-05-03 20:06:38,805 INFO : Calling hook: post_install
2011-05-03 20:06:38,809 INFO : Calling hook: preflight_check
2011-05-03 20:06:38,813 INFO : Calling hook: configure_networking
2011-05-03 20:06:38,858 INFO : Calling hook: configure_mounting
2011-05-03 20:06:38,899 INFO : Calling hook: mount_partitions
2011-05-03 20:06:38,901 INFO : Mounting target filesystems
2011-05-03 20:06:38,902 INFO : Creating filesystem: /, size: 4096,
dummy: False
2011-05-03 20:06:38,902 INFO : Not preallocated, so we create it.
2011-05-03 20:06:39,847 INFO : mke2fs 1.41.12 (17-May-2010)
2011-05-03 20:06:39,848 INFO : warning: Unable to get device
geometry for /tmp/tmpNkdP5q
2011-05-03 20:06:46,295 INFO : mke2fs 1.41.12 (17-May-2010)
2011-05-03 20:06:46,298 INFO : warning: Unable to get device
geometry for /tmp/tmpNkdP5q
2011-05-03 20:06:51,951 INFO : Creating filesystem: None, size:
1024, dummy: False
2011-05-03 20:06:51,951 INFO : Not preallocated, so we create it.
2011-05-03 20:06:52,608 INFO : mkswap: /tmp/tmpqMGgbS: warning:
don't erase bootbits sectors
2011-05-03 20:06:52,610 INFO : on whole disk. Use -f to force.
2011-05-03 20:06:52,744 INFO : mkswap: /tmp/tmpqMGgbS: warning:
don't erase bootbits sectors
2011-05-03 20:06:52,745 INFO : on whole disk. Use -f to force.
2011-05-03 20:08:23,651 INFO : Calling hook: install_kernel
2011-05-03 20:09:18,413 INFO : Done.
2011-05-03 20:09:29,506 INFO : Running depmod.
2011-05-03 20:09:29,971 INFO : update-initramfs: Generating
/boot/initrd.img-2.6.32-31-generic-pae
2011-05-03 20:09:45,695 INFO :
2011-05-03 20:09:45,695 INFO : Creating config file
/etc/default/grub with new version
2011-05-03 20:09:46,126 INFO : /proc/devices: fopen failed: No such
file or directory
2011-05-03 20:09:46,127 INFO : Is device-mapper driver missing from kernel?
2011-05-03 20:09:46,128 INFO : Failure to communicate with kernel
device-mapper driver.
2011-05-03 20:09:47,991 INFO : Calling hook: unmount_partitions
2011-05-03 20:09:48,027 INFO : Unmounting target filesystem
2011-05-03 20:09:48,599 INFO : Calling hook: convert
2011-05-03 20:09:48,606 INFO : Moving /tmp/tmpNkdP5q to ubuntu-xen/tmpNkdP5q
2011-05-03 20:10:28,628 INFO : Calling hook: fix_ownership
2011-05-03 20:10:28,891 INFO : Moving /tmp/tmpqMGgbS to ubuntu-xen/tmpqMGgbS
2011-05-03 20:10:33,374 INFO : Calling hook: fix_ownership
2011-05-03 20:10:37,543 INFO : Calling hook: fix_ownership
2011-05-03 20:10:37,548 INFO : Calling hook: deploy

-Henrik

> --
> You received this bug notification because you are a direct subscriber
> of the bug.
> https://bugs.launchpad.net/bugs/576501
>
> Title:
> vmbuilder doesn't recognize --ec2 flag
>
> Status in Virtual Machine Builder:
> New
> Status in “vm-builder” package in Ubuntu:
> In Progress
>
> Bug description:
> Description: Ubuntu 10.04 LTS
> Release: 10.04
>
> python-vm-builder-ec2:
> Installed: 0.12.3-0ubuntu1
> Candidate: 0.12.3-0ubuntu1
> Version table:
> *** 0.12.3-0ubuntu1 0
> 500 http://us-east-1.ec2.archive.ubuntu.com/ubuntu/ lucid/universe Packages
> 100 /var/lib/dpkg/status
>
> Hi,
>
> I can't generate EC2 images with vmbuilder. I'm using a command like
> this:
>
> sudo vmbuilder xen ubuntu --suite=lucid --ec2 \
> --ec2-cert=.ec2/cert-XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.pem \
> --ec2-key=.ec2/pk-XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.pem \
> --ec2-access-key=WHATEVER \
> --ec2-secret-key=WHATEVER \
> --ec2-user=424242424242 --ec2-bucket=anyrandombucket --ec2-prefix=test \
> --ec2-version='first' \
> --firstboot=/usr/share/doc/python-vm-builder-ec2/examples/ec2-firstboot.sh \
> --part=/usr/share/doc/python-vm-builder-ec2/examples/ec2-i386-part-file.txt \
> --debug
>
> Actually, I was following this howto:
> https://wiki.ubuntu.com/EC2Vmbuilder
>
> When I run vmbuilder with those flags, I just get this message:
>
> Usage: vmbuilder hypervisor distro [options]
>
> vmbuilder: error: no such option: --ec2
>
> So, I can not build images for Amazon EC2.
>
> I inspected /usr/share/pyshared/VMBuilder/plugins/ec2/__init__.py and I found
> at the last line of that file this:
>
> #register_plugin(EC2)
>
> I uncommented that line and it didn't work.
> Also I tried using register_hypervisor_plugin(EC2), but it didn't work

On Mon, Apr 18, 2011 at 10:32 PM, Serge Hallyn
<576501@bugs.launchpad.net> wrote:
> Quoting Henrik Bach (576501@bugs.launchpad.net):
>> 2011-04-16 04:36:12,380 INFO    : Creating config file
>> /etc/default/grub with new version
>> 2011-04-16 04:36:12,970 INFO    : /proc/devices: fopen failed: No such
>> file or directory
>> 2011-04-16 04:36:12,971 INFO    : Is device-mapper driver missing from kernel?
>
> Hm.  Actually all the problems I've had I've traced to environments (bad
> network setup for one, device permissions for another).  I finally got a
> maverick xen host in a working environment - and vmbuilder -ec2 worked
> there.

Ok.

>
> What does /proc/devices on your host show?

$ cat /proc/devices
Character devices:
  1 mem
  4 /dev/vc/0
  4 tty
  4 ttyS
  5 /dev/tty
  5 /dev/console
  5 /dev/ptmx
  6 lp
  7 vcs
 10 misc
 13 input
 21 sg
 29 fb
 99 ppdev
108 ppp
116 alsa
128 ptm
136 pts
180 usb
189 usb_device
226 drm
252 usbmon
253 bsg
254 rtc

Block devices:
  1 ramdisk
  2 fd
259 blkext
  7 loop
  8 sd
  9 md
 11 sr
 65 sd
 66 sd
 67 sd
 68 sd
 69 sd
 70 sd
 71 sd
128 sd
129 sd
130 sd
131 sd
132 sd
133 sd
134 sd
135 sd
252 device-mapper
253 pktcdvd
254 mdp

> What sort of storage setup
> do you have?

I'm building the image on an ordinary harddisk with my home folder encrypted.

> The 'Is device-mapper driver missing from kernel?' is
> odd, and presumably comes from grub in the chroot.  It almost sounds
> like your host is multipathed.
> What does 'df -h' and 'mount' on the host show?

$ df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/sda1              36G  7,5G   26G  23% /
none                  491M  228K  491M   1% /dev
none                  497M  1,4M  496M   1% /dev/shm
none                  497M  140K  497M   1% /var/run
none                  497M     0  497M   0% /var/lock
none                   36G  7,5G   26G  23% /var/lib/ureadahead/debugfs
$ mount
/dev/sda1 on / type ext4 (rw,errors=remount-ro,commit=0)
proc on /proc type proc (rw,noexec,nosuid,nodev)
none on /sys type sysfs (rw,noexec,nosuid,nodev)
fusectl on /sys/fs/fuse/connections type fusectl (rw)
none on /sys/kernel/debug type debugfs (rw)
none on /sys/kernel/security type securityfs (rw)
none on /dev type devtmpfs (rw,mode=0755)
none on /dev/pts type devpts (rw,noexec,nosuid,gid=5,mode=0620)
none on /dev/shm type tmpfs (rw,nosuid,nodev)
none on /var/run type tmpfs (rw,nosuid,mode=0755)
none on /var/lock type tmpfs (rw,noexec,nosuid,nodev)
none on /var/lib/ureadahead/debugfs type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc
(rw,noexec,nosuid,nodev)
gvfs-fuse-daemon on /home/sad/.gvfs type fuse.gvfs-fuse-daemon
(rw,nosuid,nodev,user=X)

>
> Does a regular 'vmbuilder xen ubuntu' work for you?
>

First try was:

$ sudo vmbuilder xen ubuntu
[sudo] password for X:
2011-05-03 19:50:53,587 INFO    : Calling hook: preflight_check
2011-05-03 19:50:53,641 INFO    : Calling hook: set_defaults
2011-05-03 19:50:53,647 INFO    : Calling hook: bootstrap
2011-05-03 20:02:47,612 INFO    : Calling hook: configure_os
2011-05-03 20:03:23,304 INFO    :
2011-05-03 20:03:23,305 INFO    : Current default time zone: 'Etc/UTC'
2011-05-03 20:03:23,339 INFO    : Local time is now:      Tue May  3
18:03:23 UTC 2011.
2011-05-03 20:03:23,339 INFO    : Universal Time is now:  Tue May  3
18:03:23 UTC 2011.
2011-05-03 20:03:23,340 INFO    :
Extracting templates from packages: 100%
2011-05-03 20:05:00,351 INFO    :
2011-05-03 20:05:00,352 INFO    : Current default time zone: 'Etc/UTC'
2011-05-03 20:05:00,384 INFO    : Local time is now:      Tue May  3
18:05:00 UTC 2011.
2011-05-03 20:05:00,384 INFO    : Universal Time is now:  Tue May  3
18:05:00 UTC 2011.
2011-05-03 20:05:00,385 INFO    : Run 'dpkg-reconfigure tzdata' if you
wish to change it.
2011-05-03 20:05:00,385 INFO    :
2011-05-03 20:06:36,230 INFO    : start: Unknown parameter: JOB
2011-05-03 20:06:38,805 INFO    : Calling hook: post_install
2011-05-03 20:06:38,809 INFO    : Calling hook: preflight_check
2011-05-03 20:06:38,813 INFO    : Calling hook: configure_networking
2011-05-03 20:06:38,858 INFO    : Calling hook: configure_mounting
2011-05-03 20:06:38,899 INFO    : Calling hook: mount_partitions
2011-05-03 20:06:38,901 INFO    : Mounting target filesystems
2011-05-03 20:06:38,902 INFO    : Creating filesystem: /, size: 4096,
dummy: False
2011-05-03 20:06:38,902 INFO    : Not preallocated, so we create it.
2011-05-03 20:06:39,847 INFO    : mke2fs 1.41.12 (17-May-2010)
2011-05-03 20:06:39,848 INFO    : warning: Unable to get device
geometry for /tmp/tmpNkdP5q
2011-05-03 20:06:46,295 INFO    : mke2fs 1.41.12 (17-May-2010)
2011-05-03 20:06:46,298 INFO    : warning: Unable to get device
geometry for /tmp/tmpNkdP5q
2011-05-03 20:06:51,951 INFO    : Creating filesystem: None, size:
1024, dummy: False
2011-05-03 20:06:51,951 INFO    : Not preallocated, so we create it.
2011-05-03 20:06:52,608 INFO    : mkswap: /tmp/tmpqMGgbS: warning:
don't erase bootbits sectors
2011-05-03 20:06:52,610 INFO    :         on whole disk. Use -f to force.
2011-05-03 20:06:52,744 INFO    : mkswap: /tmp/tmpqMGgbS: warning:
don't erase bootbits sectors
2011-05-03 20:06:52,745 INFO    :         on whole disk. Use -f to force.
2011-05-03 20:08:23,651 INFO    : Calling hook: install_kernel
2011-05-03 20:09:18,413 INFO    : Done.
2011-05-03 20:09:29,506 INFO    : Running depmod.
2011-05-03 20:09:29,971 INFO    : update-initramfs: Generating
/boot/initrd.img-2.6.32-31-generic-pae
2011-05-03 20:09:45,695 INFO    :
2011-05-03 20:09:45,695 INFO    : Creating config file
/etc/default/grub with new version
2011-05-03 20:09:46,126 INFO    : /proc/devices: fopen failed: No such
file or directory
2011-05-03 20:09:46,127 INFO    : Is device-mapper driver missing from kernel?
2011-05-03 20:09:46,128 INFO    : Failure to communicate with kernel
device-mapper driver.
2011-05-03 20:09:47,991 INFO    : Calling hook: unmount_partitions
2011-05-03 20:09:48,027 INFO    : Unmounting target filesystem
2011-05-03 20:09:48,599 INFO    : Calling hook: convert
2011-05-03 20:09:48,606 INFO    : Moving /tmp/tmpNkdP5q to ubuntu-xen/tmpNkdP5q
2011-05-03 20:10:28,628 INFO    : Calling hook: fix_ownership
2011-05-03 20:10:28,891 INFO    : Moving /tmp/tmpqMGgbS to ubuntu-xen/tmpqMGgbS
2011-05-03 20:10:33,374 INFO    : Calling hook: fix_ownership
2011-05-03 20:10:37,543 INFO    : Calling hook: fix_ownership
2011-05-03 20:10:37,548 INFO    : Calling hook: deploy

-Henrik

> --
> You received this bug notification because you are a direct subscriber
> of the bug.
> https://bugs.launchpad.net/bugs/576501
>
> Title:
>  vmbuilder doesn't recognize --ec2 flag
>
> Status in Virtual Machine Builder:
>  New
> Status in “vm-builder” package in Ubuntu:
>  In Progress
>
> Bug description:
>  Description:    Ubuntu 10.04 LTS
>  Release:        10.04
>
>  python-vm-builder-ec2:
>    Installed: 0.12.3-0ubuntu1
>    Candidate: 0.12.3-0ubuntu1
>    Version table:
>   *** 0.12.3-0ubuntu1 0
>          500 http://us-east-1.ec2.archive.ubuntu.com/ubuntu/ lucid/universe Packages
>          100 /var/lib/dpkg/status
>
>  Hi,
>
>  I can't generate EC2 images with vmbuilder. I'm using a command like
>  this:
>
>  sudo vmbuilder xen ubuntu --suite=lucid --ec2 \
>  --ec2-cert=.ec2/cert-XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.pem \
>  --ec2-key=.ec2/pk-XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.pem \
>  --ec2-access-key=WHATEVER \
>  --ec2-secret-key=WHATEVER \
>  --ec2-user=424242424242 --ec2-bucket=anyrandombucket --ec2-prefix=test \
>  --ec2-version='first' \
>  --firstboot=/usr/share/doc/python-vm-builder-ec2/examples/ec2-firstboot.sh \
>  --part=/usr/share/doc/python-vm-builder-ec2/examples/ec2-i386-part-file.txt \
>  --debug
>
>  Actually, I was following this howto:
>  https://wiki.ubuntu.com/EC2Vmbuilder
>
>  When I run vmbuilder with those flags, I just get this message:
>
>  Usage: vmbuilder hypervisor distro [options]
>
>  vmbuilder: error: no such option: --ec2
>
>  So, I can not build images for Amazon EC2.
>
>  I inspected /usr/share/pyshared/VMBuilder/plugins/ec2/__init__.py and I found
>  at the last line of that file this:
>
>  #register_plugin(EC2)
>
>  I uncommented that line and it didn't work.
>  Also I tried using register_hypervisor_plugin(EC2), but it didn't work