| 2018-08-07 17:43:11 |
Andres Rodriguez |
bug |
|
|
added bug |
| 2018-08-07 17:43:43 |
Andres Rodriguez |
bug task added |
|
maas-images |
|
| 2018-08-07 17:44:02 |
Andres Rodriguez |
summary |
ppc64el grub may be auto-generating MAC address instead of using that of the VM |
ppc64el grub may be auto-generating MAC address instead of using the assigned to the VM. |
|
| 2018-08-07 18:47:43 |
Steve Langasek |
tags |
|
regression-release |
|
| 2018-08-07 18:47:58 |
Steve Langasek |
summary |
ppc64el grub may be auto-generating MAC address instead of using the assigned to the VM. |
[REGRESSION] ppc64el grub in bionic may be auto-generating MAC address instead of using the assigned to the VM. |
|
| 2018-08-07 22:21:38 |
Lee Trager |
branch linked |
|
lp:~ltrager/maas-images/xenial_ppc_bootloader |
|
| 2018-08-10 12:23:10 |
Francis Ginther |
tags |
regression-release |
id-5b69f1b756c6033c5589ff34 regression-release |
|
| 2018-08-17 17:33:53 |
Andres Rodriguez |
maas-images: status |
New |
Fix Released |
|
| 2018-08-23 09:36:06 |
Julian Andres Klode |
grub2 (Ubuntu): status |
New |
Triaged |
|
| 2018-08-23 11:17:51 |
Julian Andres Klode |
grub2 (Ubuntu): status |
Triaged |
In Progress |
|
| 2018-08-23 11:17:58 |
Julian Andres Klode |
grub2 (Ubuntu): assignee |
|
Julian Andres Klode (juliank) |
|
| 2018-08-23 19:35:46 |
Julian Andres Klode |
description |
In MAAS, we use grub-mkimage to create a bootloader for VM's to PXE boot of MAAS by a grub bootloader that we provide over the network. (we call this bootppc64.bin).
This bootloader is created with the following grub configuration:
grub_config: |
# MAAS GRUB2 pre-loader configuration file
# Load based on MAC address first.
configfile (pxe)/grub/grub.cfg-${net_default_mac}
# Failed to load based on MAC address.
# Load arm64 by default, UEFI only supported by 64-bit
configfile (pxe)/grub/grub.cfg-default-arm64
When we use bionic bootloader, we see the following output in the console:
"Booting under MAAS direction... [ grub.cfg-default-ppc 642B 100% 57.16B/s ]"
And in rackd.log we see:
2018-08-07 17:37:21 provisioningserver.rackdservices.tftp: [info] bootppc64.bin requested by 10.245.136.136
2018-08-07 17:37:24 provisioningserver.rackdservices.tftp: [info] /grub/grub.cfg-80:1c:b0:c0:7d:c5 requested by 10.245.136.136
2018-08-07 17:37:24 provisioningserver.rackdservices.tftp: [info] /grub/grub.cfg-default-ppc64el requested by 10.245.136.136
2018-08-07 17:37:24 provisioningserver.rackdservices.tftp: [info] ubuntu/ppc64el/ga-18.04/bionic/daily/boot-kernel requested by 10.245.136.136
This means that the VM attempted to PXE boot with MAC address "80:1c:b0:c0:7d:c5", but the VM itself has a different MAC address "52:54:00:7f:83:62"
<interface type='network'>
<mac address='52:54:00:7f:83:62'/>
<source network='maas'/>
<model type='virtio'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
</interface>
If we are to switch the bootloader to the one created with xenial as a base, we see correct behavior.
"Booting under MAAS direction... [ grub.cfg-52:54:00:7f 640B 100% 0.62B/s ] "
And the result is that grub has used to correct MAC address for the PXE process.
2018-08-07 17:07:45 provisioningserver.rackdservices.tftp: [info] bootppc64.bin requested by 10.245.136.136
2018-08-07 17:07:46 provisioningserver.rackdservices.tftp: [info] /grub/grub.cfg-52:54:00:7f:83:62 requested by 10.245.136.136
2018-08-07 17:07:47 provisioningserver.rackdservices.tftp: [info] ubuntu/ppc64el/generic/bionic/daily/boot-kernel requested by 10.245.136.136
2018-08-07 17:08:17 provisioningserver.rackdservices.tftp: [info] ubuntu/ppc64el/generic/bionic/daily/boot-initrd requested by 10.245.136.136
It would seem as the firmware in bionic is automatically generating a new MAC address.
Bionic console log
-------------------------------------------------------------------------------
Using default console: /vdevice/vty@30000000
Welcome to Open Firmware
Copyright (c) 2004, 2017 IBM Corporation All rights reserved.
This program and the accompanying materials are made available
under the terms of the BSD License available at
http://www.opensource.org/licenses/bsd-license.php
Trying to load: from: /pci@800000020000000/ethernet@4 ...
Initializing NIC
Reading MAC address from device: 52:54:00:7f:83:62
Requesting information via DHCP: done
Using IPv4 address: 10.245.136.136
Requesting file "bootppc64.bin" via TFTP from 10.245.136.6
Receiving data: 1665 KBytes
TFTP: Received bootppc64.bin (1665 KBytes)
Successfully loaded
Booting under MAAS direction... [ grub.cfg-default-ppc 642B 100% 57.16B/s ]
OF stdout device is: /vdevice/vty@30000000initrd 55.89MiB 100% 647.54KiB/s ]
Preparing to boot Linux version 4.15.0-30-generic (buildd@bos02-ppc64el-011) (gcc version 7.3.0 (Ubuntu 7
.3.0-16ubuntu3)) #32-Ubuntu SMP Thu Jul 26 17:43:11 UTC 2018 (Ubuntu 4.15.0-30.32-generic 4.15.18)
Detected machine type: 0000000000000101
command line: BOOT_IMAGE=ubuntu/ppc64el/ga-18.04/bionic/daily/boot-kernel nomodeset ro root=squash:http:/
/10.245.136.6:5248/images/ubuntu/ppc64el/ga-18.04/bionic/daily/squashfs ip=::::maas-enlist:BOOTIF ip6=off
overlayroot=tmpfs overlayroot_cfgdisk=disabled cc:{datasource_list: [MAAS]}end_cc cloud-config-url=http:
//10-245-136-0--21.maas-internal:5248/MAAS/metadata/latest/enlist-preseed/?op=get_enlist_preseed apparmor
=0 log_host=10.245.136.6 log_port=514 BOOTIF=01-80:1c:b0:c0:7d:c5
Xenial console log
--------------------------------------------------------------------
Using default console: /vdevice/vty@30000000 [1032/2975]
Welcome to Open Firmware
Copyright (c) 2004, 2017 IBM Corporation All rights reserved.
This program and the accompanying materials are made available
under the terms of the BSD License available at
http://www.opensource.org/licenses/bsd-license.php
Trying to load: from: /pci@800000020000000/ethernet@4 ...
Initializing NIC
Reading MAC address from device: 52:54:00:7f:83:62
Requesting information via DHCP: done
Using IPv4 address: 10.245.136.136
Requesting file "bootppc64.bin" via TFTP from 10.245.136.6
Receiving data: 1714 KBytes
TFTP: Received bootppc64.bin (1714 KBytes)
Successfully loaded
Booting under MAAS direction... [ grub.cfg-52:54:00:7f 640B 100% 0.62B/s ]
OF stdout device is: /vdevice/vty@30000000initrd 55.89MiB 100% 958.95KiB/s ]
Preparing to boot Linux version 4.15.0-30-generic (buildd@bos02-ppc64el-011) (gcc version 7.3.0 (Ubuntu $
.3.0-16ubuntu3)) #32-Ubuntu SMP Thu Jul 26 17:43:11 UTC 2018 (Ubuntu 4.15.0-30.32-generic 4.15.18)
Detected machine type: 0000000000000101
command line: BOOT_IMAGE=ubuntu/ppc64el/generic/bionic/daily/boot-kernel nomodeset ro root=squash:http:/$
10.245.136.6:5248/images/ubuntu/ppc64el/generic/bionic/daily/squashfs ip=::::fast-cow:BOOTIF ip6=off ove$
layroot=tmpfs overlayroot_cfgdisk=disabled cc:{datasource_list: [MAAS]}end_cc cloud-config-url=http://10$
245-136-0--21.maas-internal:5248/MAAS/metadata/latest/by-id/4ysp63/?op=get_preseed apparmor=0 log_host=1$
.245.136.6 log_port=514 rootdelay=60 BOOTIF=01-52:54:00:7f:83:62
Max number of cores passed to firmware: 2048 (NR_CPUS = 2048) |
[Impact]
grub2 on ppc64el wrongfully configures two interfaces for the same card when running in qemu with a tftp boot. This causes net_default_mac to point to the wrong mac address, amongst other things.
[Test case]
(1) grub-mknetdir --net-directory=$dir
(2) qemu-system-ppc64le -device virtio-net-pci,netdev=mynet,mac=$mac -netdev user,id=mynet,tftp=$dir,bootfile=boot/grub/powerpc-ieee1275/core.elf
grub loads successfully via tftp.
Verify that
(1) net_ls_cards shows the card with the correct mac addr $mac
(2) net_ls_addr shows one interface with mac $mac
(3) echo $net_default_mac shows the mac $mac
[Regression potential]
grub now correctly stores the mac address in variables, meaning that code now might not fallback to a default.
Apart from that/On the other hand, the fix is a simple case of initializing variables, therefore that was possible before too, as it was basically random anyway.
[Original bug report]
In MAAS, we use grub-mkimage to create a bootloader for VM's to PXE boot of MAAS by a grub bootloader that we provide over the network. (we call this bootppc64.bin).
This bootloader is created with the following grub configuration:
grub_config: |
# MAAS GRUB2 pre-loader configuration file
# Load based on MAC address first.
configfile (pxe)/grub/grub.cfg-${net_default_mac}
# Failed to load based on MAC address.
# Load arm64 by default, UEFI only supported by 64-bit
configfile (pxe)/grub/grub.cfg-default-arm64
When we use bionic bootloader, we see the following output in the console:
"Booting under MAAS direction... [ grub.cfg-default-ppc 642B 100% 57.16B/s ]"
And in rackd.log we see:
2018-08-07 17:37:21 provisioningserver.rackdservices.tftp: [info] bootppc64.bin requested by 10.245.136.136
2018-08-07 17:37:24 provisioningserver.rackdservices.tftp: [info] /grub/grub.cfg-80:1c:b0:c0:7d:c5 requested by 10.245.136.136
2018-08-07 17:37:24 provisioningserver.rackdservices.tftp: [info] /grub/grub.cfg-default-ppc64el requested by 10.245.136.136
2018-08-07 17:37:24 provisioningserver.rackdservices.tftp: [info] ubuntu/ppc64el/ga-18.04/bionic/daily/boot-kernel requested by 10.245.136.136
This means that the VM attempted to PXE boot with MAC address "80:1c:b0:c0:7d:c5", but the VM itself has a different MAC address "52:54:00:7f:83:62"
<interface type='network'>
<mac address='52:54:00:7f:83:62'/>
<source network='maas'/>
<model type='virtio'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
</interface>
If we are to switch the bootloader to the one created with xenial as a base, we see correct behavior.
"Booting under MAAS direction... [ grub.cfg-52:54:00:7f 640B 100% 0.62B/s ] "
And the result is that grub has used to correct MAC address for the PXE process.
2018-08-07 17:07:45 provisioningserver.rackdservices.tftp: [info] bootppc64.bin requested by 10.245.136.136
2018-08-07 17:07:46 provisioningserver.rackdservices.tftp: [info] /grub/grub.cfg-52:54:00:7f:83:62 requested by 10.245.136.136
2018-08-07 17:07:47 provisioningserver.rackdservices.tftp: [info] ubuntu/ppc64el/generic/bionic/daily/boot-kernel requested by 10.245.136.136
2018-08-07 17:08:17 provisioningserver.rackdservices.tftp: [info] ubuntu/ppc64el/generic/bionic/daily/boot-initrd requested by 10.245.136.136
It would seem as the firmware in bionic is automatically generating a new MAC address.
Bionic console log
-------------------------------------------------------------------------------
Using default console: /vdevice/vty@30000000
Welcome to Open Firmware
Copyright (c) 2004, 2017 IBM Corporation All rights reserved.
This program and the accompanying materials are made available
under the terms of the BSD License available at
http://www.opensource.org/licenses/bsd-license.php
Trying to load: from: /pci@800000020000000/ethernet@4 ...
Initializing NIC
Reading MAC address from device: 52:54:00:7f:83:62
Requesting information via DHCP: done
Using IPv4 address: 10.245.136.136
Requesting file "bootppc64.bin" via TFTP from 10.245.136.6
Receiving data: 1665 KBytes
TFTP: Received bootppc64.bin (1665 KBytes)
Successfully loaded
Booting under MAAS direction... [ grub.cfg-default-ppc 642B 100% 57.16B/s ]
OF stdout device is: /vdevice/vty@30000000initrd 55.89MiB 100% 647.54KiB/s ]
Preparing to boot Linux version 4.15.0-30-generic (buildd@bos02-ppc64el-011) (gcc version 7.3.0 (Ubuntu 7
.3.0-16ubuntu3)) #32-Ubuntu SMP Thu Jul 26 17:43:11 UTC 2018 (Ubuntu 4.15.0-30.32-generic 4.15.18)
Detected machine type: 0000000000000101
command line: BOOT_IMAGE=ubuntu/ppc64el/ga-18.04/bionic/daily/boot-kernel nomodeset ro root=squash:http:/
/10.245.136.6:5248/images/ubuntu/ppc64el/ga-18.04/bionic/daily/squashfs ip=::::maas-enlist:BOOTIF ip6=off
overlayroot=tmpfs overlayroot_cfgdisk=disabled cc:{datasource_list: [MAAS]}end_cc cloud-config-url=http:
//10-245-136-0--21.maas-internal:5248/MAAS/metadata/latest/enlist-preseed/?op=get_enlist_preseed apparmor
=0 log_host=10.245.136.6 log_port=514 BOOTIF=01-80:1c:b0:c0:7d:c5
Xenial console log
--------------------------------------------------------------------
Using default console: /vdevice/vty@30000000 [1032/2975]
Welcome to Open Firmware
Copyright (c) 2004, 2017 IBM Corporation All rights reserved.
This program and the accompanying materials are made available
under the terms of the BSD License available at
http://www.opensource.org/licenses/bsd-license.php
Trying to load: from: /pci@800000020000000/ethernet@4 ...
Initializing NIC
Reading MAC address from device: 52:54:00:7f:83:62
Requesting information via DHCP: done
Using IPv4 address: 10.245.136.136
Requesting file "bootppc64.bin" via TFTP from 10.245.136.6
Receiving data: 1714 KBytes
TFTP: Received bootppc64.bin (1714 KBytes)
Successfully loaded
Booting under MAAS direction... [ grub.cfg-52:54:00:7f 640B 100% 0.62B/s ]
OF stdout device is: /vdevice/vty@30000000initrd 55.89MiB 100% 958.95KiB/s ]
Preparing to boot Linux version 4.15.0-30-generic (buildd@bos02-ppc64el-011) (gcc version 7.3.0 (Ubuntu $
.3.0-16ubuntu3)) #32-Ubuntu SMP Thu Jul 26 17:43:11 UTC 2018 (Ubuntu 4.15.0-30.32-generic 4.15.18)
Detected machine type: 0000000000000101
command line: BOOT_IMAGE=ubuntu/ppc64el/generic/bionic/daily/boot-kernel nomodeset ro root=squash:http:/$
10.245.136.6:5248/images/ubuntu/ppc64el/generic/bionic/daily/squashfs ip=::::fast-cow:BOOTIF ip6=off ove$
layroot=tmpfs overlayroot_cfgdisk=disabled cc:{datasource_list: [MAAS]}end_cc cloud-config-url=http://10$
245-136-0--21.maas-internal:5248/MAAS/metadata/latest/by-id/4ysp63/?op=get_preseed apparmor=0 log_host=1$
.245.136.6 log_port=514 rootdelay=60 BOOTIF=01-52:54:00:7f:83:62
Max number of cores passed to firmware: 2048 (NR_CPUS = 2048) |
|
| 2018-08-23 19:35:52 |
Julian Andres Klode |
nominated for series |
|
Ubuntu Bionic |
|
| 2018-08-23 19:35:52 |
Julian Andres Klode |
bug task added |
|
grub2 (Ubuntu Bionic) |
|
| 2018-08-23 19:35:56 |
Julian Andres Klode |
grub2 (Ubuntu Bionic): status |
New |
In Progress |
|
| 2018-08-23 19:35:58 |
Julian Andres Klode |
grub2 (Ubuntu): status |
In Progress |
Fix Committed |
|
| 2018-08-26 14:13:49 |
Launchpad Janitor |
grub2 (Ubuntu): status |
Fix Committed |
Fix Released |
|
| 2018-08-27 15:04:45 |
Łukasz Zemczak |
grub2 (Ubuntu Bionic): status |
In Progress |
Fix Committed |
|
| 2018-08-27 15:04:47 |
Łukasz Zemczak |
bug |
|
|
added subscriber Ubuntu Stable Release Updates Team |
| 2018-08-27 15:04:48 |
Łukasz Zemczak |
bug |
|
|
added subscriber SRU Verification |
| 2018-08-27 15:04:51 |
Łukasz Zemczak |
tags |
id-5b69f1b756c6033c5589ff34 regression-release |
id-5b69f1b756c6033c5589ff34 regression-release verification-needed verification-needed-bionic |
|
| 2018-08-28 08:39:32 |
Julian Andres Klode |
tags |
id-5b69f1b756c6033c5589ff34 regression-release verification-needed verification-needed-bionic |
id-5b69f1b756c6033c5589ff34 regression-release verification-done verification-done-bionic |
|
| 2018-08-30 14:21:48 |
Julian Andres Klode |
bug task added |
|
grub2-signed (Ubuntu) |
|
| 2018-08-30 14:23:31 |
Julian Andres Klode |
grub2-signed (Ubuntu): status |
New |
Fix Released |
|
| 2018-08-30 14:23:36 |
Julian Andres Klode |
grub2-signed (Ubuntu Bionic): status |
New |
Triaged |
|
| 2018-08-30 14:23:58 |
Julian Andres Klode |
grub2-signed (Ubuntu Bionic): status |
Triaged |
Fix Committed |
|
| 2018-08-30 14:24:00 |
Julian Andres Klode |
grub2-signed (Ubuntu Bionic): status |
Fix Committed |
In Progress |
|
| 2018-09-03 07:42:39 |
Łukasz Zemczak |
grub2-signed (Ubuntu Bionic): status |
In Progress |
Fix Committed |
|
| 2018-09-03 07:42:45 |
Łukasz Zemczak |
tags |
id-5b69f1b756c6033c5589ff34 regression-release verification-done verification-done-bionic |
id-5b69f1b756c6033c5589ff34 regression-release verification-needed verification-needed-bionic |
|
| 2018-09-03 07:52:04 |
Julian Andres Klode |
tags |
id-5b69f1b756c6033c5589ff34 regression-release verification-needed verification-needed-bionic |
id-5b69f1b756c6033c5589ff34 regression-release verification-done verification-done-bionic |
|
| 2018-09-05 20:51:59 |
Steve Langasek |
removed subscriber Ubuntu Stable Release Updates Team |
|
|
|
| 2018-09-05 20:52:09 |
Launchpad Janitor |
grub2-signed (Ubuntu Bionic): status |
Fix Committed |
Fix Released |
|
| 2018-09-05 21:02:04 |
Launchpad Janitor |
grub2 (Ubuntu Bionic): status |
Fix Committed |
Fix Released |
|
