curtin

curtin fails to partition software raid md0

Bug #1641661 reported by Ruslan Lutsenko
12
This bug affects 1 person
Affects Status Importance Assigned to Milestone
curtin
Fix Released
Medium
Unassigned
curtin (Ubuntu)
Fix Released
Medium
Unassigned
Xenial
Fix Released
Medium
Unassigned
Yakkety
Fix Released
Medium
Unassigned
Zesty
Fix Released
Medium
Unassigned

Bug Description

----- Begin SRU Template -----
[Impact]
Curtin is unable to partition md (raid) devices.
A system configured to put partitions on a raided device will fail to
install.

[Test Case]
We added 2 test paths for this code to curtin
a.) unit tests - these will run in the build process, so successful build
    verifies that this has passed.
b.) vmtests named: *MirrorbootPartitions

    vmtests are currently only run on trunk, but we can modify source
    to run with the packaged version. So, then to test the md path
    with a vm install, we will:
    i) run a xenial system
    ii) enable proposed
    iii) install curtin
    iv) run vmtests of the Mirrorboot tests using curtin from the sru.

[Regression Potential]
Quite low, the path for failure would be if there were some cases
when the device name for a mdadm partition was /dev/mdNM rather than
/dev/mdNpM.

[Other Info]
The upstream merge proposal is at
 https://code.launchpad.net/~raharper/curtin/trunk.lp1641661/+merge/311275
and shows the code that went in to fix this.

----- End SRU Template -----

It seem curtin has problem with software RAID partitioning: md*
Trying to deploy node in MAAS Version 2.0.0+bzr5189-0ubuntu1 (16.04.1).
During partitioning stage curtin returns following error:

An error occured handling 'md0-part1': OSError - could not get path to dev from kname: md01
finish: cmd-install/stage-partitioning/builtin/cmd-block-meta: FAIL: failed: configuring partition: md0-part1
finish: cmd-install/stage-partitioning/builtin/cmd-block-meta: FAIL: failed: curtin command block-meta
Traceback (most recent call last):
  File "/curtin/curtin/commands/main.py", line 211, in main
    ret = args.func(args)
  File "/curtin/curtin/commands/block_meta.py", line 62, in block_meta
    meta_custom(args)
  File "/curtin/curtin/commands/block_meta.py", line 1041, in meta_custom
    handler(command, storage_config_dict)
  File "/curtin/curtin/commands/block_meta.py", line 550, in partition_handler
    get_path_to_storage_volume(info.get('id'), storage_config),
  File "/curtin/curtin/commands/block_meta.py", line 267, in get_path_to_storage_volume
    volume_path = block.kname_to_path(partition_kname)
  File "/curtin/curtin/block/__init__.py", line 114, in kname_to_path
    raise OSError('could not get path to dev from kname: {}'.format(kname))
OSError: could not get path to dev from kname: md01
could not get path to dev from kname: md01
builtin command failed
finish: cmd-install/stage-partitioning/builtin: FAIL: failed: running 'curtin block-meta custom'
builtin took 8.861 seconds

I've added commit which fixes mentioned error: http://bazaar.launchpad.net/~ruslan-lutcenko/curtin/curtin/revision/431

See original description

Related branches

lp://staging/~ruslan-lutcenko/curtin/curtin
lp://staging/~raharper/curtin/trunk.lp1641661
Scott Moser (community): Approve
Server Team CI bot: Approve (continuous-integration)
Diff: 169 lines (+124/-1)
4 files modified
curtin/block/__init__.py (+1/-1)
examples/tests/mirrorboot-msdos-partition.yaml (+82/-0)
tests/unittests/test_block.py (+2/-0)
tests/vmtests/test_mdadm_bcache.py (+39/-0)
lp://staging/~curtin-dev/curtin/trunk
lp://staging/~smoser/ubuntu/zesty/curtin/pkg
lp://staging/~smoser/ubuntu/xenial/curtin/pkg-sru-r437
Ryan Harper (community): Approve
Scott Moser: Pending requested
Diff: 1365 lines (+661/-168)
24 files modified
curtin/block/__init__.py (+1/-1)
curtin/commands/apt_config.py (+16/-6)
curtin/commands/block_meta.py (+1/-2)
curtin/commands/curthooks.py (+28/-1)
curtin/deps/__init__.py (+11/-2)
curtin/util.py (+15/-6)
debian/changelog (+20/-0)
debian/new-upstream-snapshot (+47/-3)
examples/tests/centos_basic.yaml (+13/-0)
examples/tests/mirrorboot-msdos-partition.yaml (+82/-0)
helpers/list-flash-kernel-packages (+13/-0)
tests/unittests/test_apt_source.py (+1/-0)
tests/unittests/test_block.py (+2/-0)
tests/unittests/test_curthooks.py (+134/-0)
tests/vmtests/__init__.py (+83/-102)
tests/vmtests/helpers.py (+20/-8)
tests/vmtests/image_sync.py (+31/-17)
tests/vmtests/releases.py (+38/-10)
tests/vmtests/test_apt_config_cmd.py (+4/-0)
tests/vmtests/test_centos_basic.py (+42/-0)
tests/vmtests/test_mdadm_bcache.py (+39/-0)
tests/vmtests/test_raid5_bcache.py (+2/-1)
tools/vmtest-sync-images (+15/-6)
tools/xkvm (+3/-3)
lp://staging/~smoser/ubuntu/xenial/curtin/pkg
lp://staging/~curtin-dev/curtin/zesty
lp://staging/~curtin-dev/curtin/yakkety
Revision history for this message
Ruslan Lutsenko (ruslan-lutcenko) wrote :
Ryan Harper (raharper)
Changed in curtin:
status: New → Confirmed
importance: Undecided → Medium
Scott Moser (smoser)
Changed in curtin (Ubuntu Zesty):
status: New → Fix Released
importance: Undecided → Medium
Changed in curtin (Ubuntu Yakkety):
status: New → Triaged
Changed in curtin (Ubuntu Xenial):
status: New → Triaged
importance: Undecided → Medium
Changed in curtin (Ubuntu Yakkety):
importance: Undecided → Medium
Scott Moser (smoser)
description: updated
Revision history for this message
Łukasz Zemczak (sil2100) wrote : Please test proposed package

Hello Ruslan, or anyone else affected,

Accepted curtin into yakkety-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/curtin/0.1.0~bzr437-0ubuntu1~16.10.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in curtin (Ubuntu Yakkety):
status: Triaged → Fix Committed
tags: added: verification-needed
Scott Moser (smoser)
tags: added: verification-needed-xenial verification-needed-yakkety
removed: verification-needed
Scott Moser (smoser)
Changed in curtin (Ubuntu Xenial):
status: Triaged → Fix Committed
Revision history for this message
Scott Moser (smoser) wrote :

I've tested yakkety using curtin vmtest, which includes several installations
including one that has been made to show this failure.

I'm attaching the vmtest output directory for full logs.

$ git clone https://gist.github.com/4e25b5a9e6245692be90098464990d00.git vtools
$ export PATH=$PWD/vtools:$PATH

$ bzr branch curtin trunk
$ cd trunk
$ bzr revno
443

$ rel=yakkety
$ cname=test-$rel;
$ setup-curtin-container --proposed ubuntu-daily:$rel $cname
Creating test-yakkety
...

$ lxc exec $cname -- apt-cache policy curtin
curtin:
  Installed: 0.1.0~bzr437-0ubuntu1~16.10.1
  Candidate: 0.1.0~bzr437-0ubuntu1~16.10.1
  Version table:
 *** 0.1.0~bzr437-0ubuntu1~16.10.1 500
        500 http://archive.ubuntu.com/ubuntu yakkety-proposed/universe amd64 Packages
        100 /var/lib/dpkg/status
     0.1.0~bzr425-0ubuntu1 500
        500 http://archive.ubuntu.com/ubuntu yakkety/universe amd64 Packages

$ PATH=$PWD/../vtools:$PATH \
    CURTIN_VMTEST_CURTIN_EXE="curtin-from-container $cname curtin" \
    CURTIN_VMTEST_TOPDIR=$PWD/$rel-1641661 \
    ./tools/jenkins-runner \
     tests/vmtests/test_mdadm_bcache.py
........
----------------------------------------------------------------------
Ran 196 tests in 3942.426s
OK
Tue, 31 Jan 2017 22:27:21 +0000: vmtest end [0] in 3944s

Revision history for this message
Scott Moser (smoser) wrote :

I've tested xenial using curtin vmtest, which includes several installations
including one that has been made to show this failure.

I'm attaching the vmtest output directory for full logs.

$ git clone https://gist.github.com/4e25b5a9e6245692be90098464990d00.git vtools
$ export PATH=$PWD/vtools:$PATH

$ bzr branch curtin trunk
$ cd trunk
$ bzr revno
443

$ rel=xenial
$ cname=test-$rel;
$ setup-curtin-container --proposed ubuntu-daily:$rel $cname
Creating test-xenial
...

$ lxc exec $cname apt-cache policy curtin
curtin:
  Installed: 0.1.0~bzr437-0ubuntu1~16.04.1
  Candidate: 0.1.0~bzr437-0ubuntu1~16.04.1
  Version table:
 *** 0.1.0~bzr437-0ubuntu1~16.04.1 500
        500 http://archive.ubuntu.com/ubuntu xenial-proposed/universe amd64 Packages
        100 /var/lib/dpkg/status
     0.1.0~bzr425-0ubuntu1~16.04.1 500
        500 http://archive.ubuntu.com/ubuntu xenial-updates/universe amd64 Packages
     0.1.0~bzr365-0ubuntu1 500
        500 http://archive.ubuntu.com/ubuntu xenial/universe amd64 Packages

$ PATH=$PWD/../vtools:$PATH \
    CURTIN_VMTEST_CURTIN_EXE="curtin-from-container $cname curtin" \
    CURTIN_VMTEST_TOPDIR=$PWD/$rel-1641661 \
    ./tools/jenkins-runner \
     tests/vmtests/test_mdadm_bcache.py
...........
----------------------------------------------------------------------
Ran 196 tests in 4012.451s
OK
Tue, 31 Jan 2017 22:14:21 +0000: vmtest end [0] in 4014s

tags: added: verification-done-xenial verification-done-yakkety
removed: verification-needed-xenial verification-needed-yakkety
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package curtin - 0.1.0~bzr437-0ubuntu1~16.04.1

---------------
curtin (0.1.0~bzr437-0ubuntu1~16.04.1) xenial-proposed; urgency=medium

  * debian/new-upstream-snapshot: change to not use bzr merge-upstream.
  * New upstream snapshot.
    - pep8: fix pep8 errors found with 'make pep8' on zesty.
    - Workaround failures caused by gpg2 daemons left running in chroot.
      (LP: #1645680)
    - Install u-boot-tools when running on a system with u-boot. (LP: #1640519)
    - block: fix partition kname for raid devices (LP: #1641661)
    - Fix up tox errors that slipped in and new pycodestyle 2.1.0 complaints.
    - vmtests: adjust vmtest image sync metadata filenames
    - vmtests: Add centos support
    - Disable WilyTestRaid5Bcache vmtest
    - tools/xkvm: fix --netdev=<bridge>
    - bytes2human: fix for values larger than 32 bit int on 32 bit python2.

 -- Scott Moser <email address hidden> Wed, 18 Jan 2017 12:39:01 -0500

Changed in curtin (Ubuntu Xenial):
status: Fix Committed → Fix Released
Revision history for this message
Robie Basak (racb) wrote : Update Released

The verification of the Stable Release Update for curtin has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package curtin - 0.1.0~bzr437-0ubuntu1~16.10.1

---------------
curtin (0.1.0~bzr437-0ubuntu1~16.10.1) yakkety-proposed; urgency=medium

  * debian/new-upstream-snapshot: change to not use bzr merge-upstream.
  * New upstream snapshot.
    - revert: Test Workaround: skip XenialTestNvme for a short time.
    - Test Workaround: skip XenialTestNvme for a short time.
    - pep8: fix pep8 errors found with 'make pep8' on zesty.
    - Workaround failures caused by gpg2 daemons left running in chroot.
      (LP: #1645680)
    - Install u-boot-tools when running on a system with u-boot. (LP: #1640519)
    - block: fix partition kname for raid devices (LP: #1641661)
    - Fix up tox errors that slipped in and new pycodestyle 2.1.0 complaints.
    - vmtests: adjust vmtest image sync metadata filenames
    - vmtests: Add centos support
    - Disable WilyTestRaid5Bcache vmtest
    - tools/xkvm: fix --netdev=<bridge>
    - bytes2human: fix for values larger than 32 bit int on 32 bit python2.

 -- Scott Moser <email address hidden> Mon, 23 Jan 2017 14:30:25 -0500

Changed in curtin (Ubuntu Yakkety):
status: Fix Committed → Fix Released
Revision history for this message
Scott Moser (smoser) wrote : Fixed in Curtin 17.1

This bug is believed to be fixed in curtin in 17.1. If this is still a problem for you, please make a comment and set the state back to New

Thank you.

Changed in curtin:
status: Confirmed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.