PPC64 version of CirrOS
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
CirrOS |
Fix Committed
|
Low
|
Unassigned |
Bug Description
This is not exactly a bug report, but I didn't find any other place to report this:
Are you guys planning to have a CirrOS image build to PPC64? If so, when?
Scott Moser (smoser) wrote : | #1 |
Changed in cirros: | |
importance: | Undecided → Low |
status: | New → Confirmed |
Arx Cruz (arxcruz) wrote : | #2 |
I can confirm that Buildroot have support, at least the latest versions.
Rafael Folco (rafaelfolco) wrote : | #3 |
Hello Scott.
We have built CirrOS on ppc64 but we're struggling with /dev/vda which is not being mount on /.
Basically the high level steps were:
1) Increase /run partition size to 2mb (800kb would be enough) on src/etc/fstab
2) Add virtio support to src/etc/modules: virtio_net, virtio_blk, and virtio_balloon
3) Small hacks to the bundle and prepare-grub scripts (rpm2cpio)
4) Download buildroot: http://
5) Create a symbolic link: buildroot --> buildroot-2013.11
6) make menuconfig
target options -> target architecture -> powerpc
Toolchain -> C Library -> uClib
Toolchain -> uClib C library version -> a última versão que tiver
Toolchain -> Enable large files
Toolchain -> enable ipv6 support
Toolchain -> enable RPC support
Toolchain -> Enable WCHAR support
Toolchain -> Compile and install uClibc utilities
Toolchain -> Enable compiler tls support
System Configuration -> remount root filesystem read-write during boot
Target Packages -> Networking applications -> dropbear
Target Packages -> Networking applications -> dnsmasq
Target Packages -> Networking applications -> iputils
Target Packages -> Networking applications -> iptables
*do not enable openssh
Target Packages -> Shell and utilities -> sudo
Target Packages -> Shell and utilities -> screen
7) make ARCH=powerpc OUT_D=$
8) ./bin/bundle -v output/
x86:
$ df -h
Filesystem Size Used Available Use% Mounted on
/dev 998.2M 0 998.2M 0% /dev
/dev/vda 193.7M 9.9M 173.8M 5% /
tmpfs 1001.8M 0 1001.8M 0% /dev/shm
tmpfs 200.0K 96.0K 104.0K 48% /run
ppc64:
$ df -h
Filesystem Size Used Available Use% Mounted on
/dev 905.6M 0 905.6M 0% /dev
tmpfs 1007.8M 0 1007.8M 0% /dev/shm
tmpfs 2.0M 1.3M 704.0K 66% /run
I'll take a look at this issue.
Please let me know if you see any problems with the steps I described above.
Thanks.
Rafael Folco (rafaelfolco) wrote : | #4 |
More differences to identify the root cause:
x86_64:
EXT3-fs (vda): mounted filesystem with ordered data mode
ppc64:
EXT4-fs (vda): mounting ext3 file system using the ext4 subsystem
x86_64:
$ lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
vda 253:0 0 20G 0 disk /
sr0 11:0 1 410K 0 rom
ppc64:
N/A
Scott Moser (smoser) wrote : | #5 |
believe that this is fix-committed in trunk.
we now have
powerpc (32 bit userspace, 64bit kernel)
ppc64: 64 bit big endian
ppc64le: 64 bit little endian
I'd appreciate feedback on a daily biuld at http://
Changed in cirros: | |
status: | Confirmed → Fix Committed |
Rafael Folco (rafaelfolco) wrote : | #6 |
- powerpc: network works only if I change driver to ibmveth (spapr-vlan)
- ppc64le: kernel panic
Haven't tried ppc64 version yet.
separate bug report for each ?
Rafael Folco (rafaelfolco) wrote : | #7 |
ppc64 also producing kernel panic, similar to LE:
http://
# nova console-log test
SLOF[0m[?25l *******
[1mQEMU Starting
[0m Build Date = Jul 4 2014 19:52:11
FW Version = mockbuild@ release 20140630
Press "s" to enter Open Firmware.
[0m[?25hC0000C0
Populating /vdevice/
Populating /vdevice/
Populating /vdevice/
C0580C05A0Popul
[root@devstack-
SLOF[0m[?25l *******
[1mQEMU Starting
[0m Build Date = Jul 4 2014 19:52:11
FW Version = mockbuild@ release 20140630
Press "s" to enter Open Firmware.
[0m[?25hC0000C0
Populating /vdevice/
Populating /vdevice/
Populating /vdevice/
C0580C05A0Popul
Adapters on 0800000020000000
C0600C0640C0690
C0890C08A0C08A8
OHCI: initializing
C08C0C08D0Using default console: /vdevice/
C08E0C08E8Detected RAM kernel at 400000 (1a62994 bytes) C08FF
Welcome to Open Firmware
Copyright (c) 2004, 2011 IBM Corporation All rights reserved.
This program and the accompanying materials are made available
under the terms of the BSD License available at
http://
Booting from memory...
OF stdout device is: /vdevice/
Preparing to boot Linux version 3.19.0-
Detected machine type: 0000000000000101
Max number of cores passed to firmware: 1024 (NR_CPUS = 1024)
Calling ibm,client-
command line: root=/dev/vda console=tty0 console=ttyS0
memory layout at init:
memory_limit : 0000000000000000 (16 MB aligned)
alloc_bottom : 0000000001e67000
alloc_top : 0000000010000000
alloc_top_hi : 0000000040000000
rmo_top : 0000000010000000
ram_top : 0000000040000000
instantiating rtas at 0x000000000ffff
prom_hold_cpus: skipped
copying OF device tree...
Building dt strings...
Building dt structure...
Device tree strings 0x0000000002768000 -> 0x00000000027687f5
Device tree struct 0x0000000002769000 -> 0x000000000276c000
Calling quiesce...
returning from prom_init
[ 0.000000] Using pSeries machine descr...
Scott Moser (smoser) wrote : | #8 |
Hi, heres what I tested just now as functional for the power builds:
$ for arch in powerpc ppc64 ppc64le;
do burl=http://
for f in kernel initramfs; do [ -f "${burl##*/}-$f" ] || wget $burl-$f; done; done
$ ls -l cirros-d150604-p*
-rw-rw-r-- 1 ubuntu ubuntu 4362299 Jun 4 04:53 cirros-
-rw-rw-r-- 1 ubuntu ubuntu 25280072 Jun 4 04:53 cirros-
-rw-rw-r-- 1 ubuntu ubuntu 8702396 Jun 4 04:54 cirros-
-rw-rw-r-- 1 ubuntu ubuntu 25280072 Jun 4 04:54 cirros-
-rw-rw-r-- 1 ubuntu ubuntu 8461981 Jun 4 04:54 cirros-
-rw-rw-r-- 1 ubuntu ubuntu 21761512 Jun 4 04:54 cirros-
On the Ubuntu host, I did:
sudo apt-get install -qy qemu-system-ppc
sudo ppc64_cpu --smt=off
sudo modprobe kvm_hv
sudo modprobe kvm_pv
sudo chmod 666 /dev/kvm
Then for each of the kernel/initramfs pairs, i ran 'go' (see below).
$ ./go cirros-
For each kernel, it booted to finding the root device, the seed image, copied itself
over to the disk image and brought up networking via dhcp.
$ cat go
#!/bin/sh
kernel="$1"
initrd="$2"
shift 2
cmdline="$1"
disk="disk.img"
fail() { echo "$@" 1>&2; exit 1; }
rm -f "$disk"
qemu-img create -f raw "$disk" 1G || fail
out=$(mkfs.ext2 -F -L cirros-rootfs "$disk" >/dev/null 2>&1) ||
fail "failed mkfs $disk: $out"
if [ ! -f seed.img ]; then
echo '{"instance-id": "9068aef2-
{ echo '#!/bin/sh'; echo 'poweroff'; } > user-data
cloud-localds seed.img user-data meta-data
fi
qemu-system-ppc64 -enable-kvm -machine pseries,usb=off -device spapr-vscsi \
-device spapr-vlan,
-drive if=virtio,
-m 1G -echr 0x05 -nographic -vga none \
-kernel "$kernel" -initrd "$initrd" -append "$cmdline"
Scott Moser (smoser) wrote : | #9 |
It also seems that using
-device virtio-
for my network drivers at least seems to work.
Rafael Folco (rafaelfolco) wrote : | #10 |
Ok, apparently my kernel has issues. I believe I am hitting a bug related to HTM. I've tested the images and confirm that ppc64, powerpc and ppc64le versions work well in 1st level for me. So the images seem to be good.
Rafael Folco (rafaelfolco) wrote : | #11 |
Upgraded kernel in 1st level guest and resolved my HTM issues.
Results running in OpenStack as nested VMs:
- powerpc (32-bit): works as long as you change hw_vif_model from virtio to spapr-vlan (ibmveth)
- ppc64 (64-bit): works good
- ppc64le: works but produces a call trace while booting. See below.
ppc64le boot call trace:
[ 1.776496] irq 18: nobody cared (try booting with the "irqpoll" option)
[ 1.776501] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 3.19.0-20-generic #20~14.04.1-Ubuntu
[ 1.776501] Call Trace:
[ 1.776507] [c00000000fffbcb0] [c000000000a26030] dump_stack+
[ 1.776510] [c00000000fffbce0] [c00000000012a670] __report_
[ 1.776512] [c00000000fffbd70] [c00000000012aeb0] note_interrupt+
[ 1.776513] [c00000000fffbe10] [c000000000127124] handle_
[ 1.776514] [c00000000fffbed0] [c000000000127318] handle_
[ 1.776516] [c00000000fffbf00] [c00000000012b7b4] handle_
[ 1.776517] [c00000000fffbf30] [c000000000126288] generic_
[ 1.776519] [c00000000fffbf60] [c000000000010f10] __do_irq+0x80/0x190
[ 1.776521] [c00000000fffbf90] [c00000000002476c] call_do_
[ 1.776522] [c00000002e642af0] [c0000000000110c0] do_IRQ+0xa0/0x120
[ 1.776524] [c00000002e642b50] [c0000000000099cc] restore_
[ 1.776526] --- interrupt: 501 at arch_local_
[ 1.776526] LR = arch_local_
[ 1.776527] [c00000002e642e40] [c00000002e630000] 0xc00000002e630000 (unreliable)
[ 1.776529] [c00000002e642e60] [c0000000000b4e8c] __do_softirq+
[ 1.776530] [c00000002e642f60] [c0000000000b54a8] irq_exit+0xc8/0x100
[ 1.776531] [c00000002e642f80] [c00000000001fa54] timer_interrupt
[ 1.776533] [c00000002e642fb0] [c0000000000099f4] restore_
[ 1.776534] --- interrupt: 901 at arch_local_
[ 1.776534] LR = arch_local_
[ 1.776537] [c00000002e6432a0] [c0000000016027b8] logbuf_lock+0x0/0x8 (unreliable)
[ 1.776538] [c00000002e6432c0] [c000000000124a48] console_
[ 1.776540] [c00000002e643390] [c000000000124d94] vprintk_
[ 1.776542] [c00000002e643410] [c00000000064556c] dev_vprintk_
[ 1.776543] [c00000002e643530] [c0000000006456f0] dev_printk_
[ 1.776544] [c00000002e643570] [c00000000064577c] __dev_printk+
[ 1.776545] [c00000002e6435f0] [c000000000645cdc] _dev_info+0x6c/0x90
[ 1.776547] [c00000002e643640] [c00000000076e0f8] usb_new_
[ 1.776549] [c00000002e6436f0] [c000000000774c94] usb_add_
[ 1.776550] [c00000002e6437b0] [c00000000078dfcc] usb_hcd_
[ 1.776553] [c00000002e643850] [c00000000057270c] local_pci_
[ 1.776554] [c00000002e6438e0] [c000000000572938] pci_device_
[ 1.776556] [c00000002e643940] [c00000000064b3ec] driver_
[ 1.776557] [c00000002e6439d0] [c00000000064b92c] __driver_
[ 1.7...
Rafael Folco (rafaelfolco) wrote : | #12 |
smoser,
Any plans to add ppc64 to 0.3.4 download dir ?
It's working fine from what I tested.
Thanks.
Rafael Folco (rafaelfolco) wrote : | #13 |
@smoser
We're using ppc64le cirros for months.
http://
I confirmed that it works fine with virtio-blk, virtio-scsi, spapr-vscsi, virtio-net and spapr-vlan.
Would you consider including ppc64le cirros to the latest Cirros version dir?
http://
Thanks in advance.
--rfolco
I've not looked at it at all, but if buildroot supports ppc64, then we can try it.