Fuel for OpenStack

Provision failed in multipath configuration

  1. Series newton
  2. Bug #1652788
Bug #1652788 reported by Nastya Urlapova
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Fuel for OpenStack
Fix Committed
High
Vladimir Kozhukalov
Fuel for OpenStack 11.0
Mitaka
Fix Released
High
Vladimir Kozhukalov
Fuel for OpenStack 9.2
Newton
Fix Committed
High
Vladimir Kozhukalov
Fuel for OpenStack 10.1

Bug Description

Provision failed with error in astute.log:
"Unexpected error\nUnexpected error while running command.\nCommand: pvcreate --metadatacopies 2 --metadatasize 28m /dev/sde3\nExit code: 5\nStdout: ''\nStderr: 'File descriptor 3 (/run/lock/provision.lock) leaked on pvcreate invocation. Parent PID 7122: /usr/bin/python2.7\\nFile descriptor 4 (/var/log/fuel-agent.log) leaked on pvcreate invocation. Parent PID 7122: /usr/bin/python2.7\\n Device /dev/sde3 not found (or ignored by filtering).\\n'\n",
   :exit_code=>255}}

Scenario:
        1. Create cluster with 1 controller, 1 compute and 1 cinder roles
        2. Run network verification
>>>> 3. Provision the cluster <<< failed here
        4. Verify multipath devices on nodes
        5. Deploy the cluster
        6. Run OSTF

Version:
9.2 snapshot #684

Revision history for this message
Nastya Urlapova (aurlapova) wrote :

https://product-ci.infra.mirantis.net/job/9.x.system_test.ubuntu.deploy_multipath/157/testReport/(root)/deploy_multipath/deploy_multipath/

Changed in fuel:
milestone: none → 9.2
assignee: nobody → Fuel Sustaining (fuel-sustaining-team)
tags: added: swarm-fail
summary: - Deployment failed in multipath configuration
+ Provision failed in multipath configuration
Oleksiy Molchanov (omolchanov)
Changed in fuel:
importance: Undecided → High
status: New → Confirmed
tags: added: area-python
Vladimir Kozhukalov (kozhukalov)
Changed in fuel:
assignee: Fuel Sustaining (fuel-sustaining-team) → Vladimir Kozhukalov (kozhukalov)
Revision history for this message
Vladimir Kozhukalov (kozhukalov) wrote :

This issue appears when nailgun-agent starts before device mapper assembles multipath devices and reports them as usual block devices. As a workaround for this issue I suggest to introduce 30 sec delay before starting nailgun-agent first time.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to fuel-nailgun-agent (stable/mitaka)

Fix proposed to branch: stable/mitaka
Review: https://review.openstack.org/416661

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to fuel-nailgun-agent (stable/mitaka)

Reviewed: https://review.openstack.org/416661
Committed: https://git.openstack.org/cgit/openstack/fuel-nailgun-agent/commit/?id=8fbc4d640535387b402aa38cb6227df11c010ff0
Submitter: Jenkins
Branch: stable/mitaka

commit 8fbc4d640535387b402aa38cb6227df11c010ff0
Author: Vladimir Kozhukalov <email address hidden>
Date: Wed Jan 4 19:37:10 2017 +0300

    Fix multipath device search

    When nailgun-agent starts before device mapper
    assembles multipath devices it reports physical disks
    that are used for multipath devices as usual disks.

    This patch does the following

    * filter out devices that are DM_MULTIPATH_DEVICE_PATH = 1
    * run 'udevadm settle' before scanning multipath devices
    * delays scanning multipath devices until building block device info

    Change-Id: I088aede0cf3bd1d16a57e7cdec4e50cab2c19175
    Closes-Bug: #1652788

tags: added: in-stable-mitaka
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to fuel-nailgun-agent (stable/newton)

Fix proposed to branch: stable/newton
Review: https://review.openstack.org/418429

Changed in fuel:
status: Confirmed → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to fuel-nailgun-agent (master)

Fix proposed to branch: master
Review: https://review.openstack.org/418430

Dmitry Pyzhov (dpyzhov)
Changed in fuel:
milestone: 9.2 → 11.0
Dmitry Belyaninov (dbelyaninov)
tags: added: on-verification
Revision history for this message
Dmitry Belyaninov (dbelyaninov) wrote :

waiting https://bugs.launchpad.net/fuel/+bug/1656872

Revision history for this message
Dmitry Belyaninov (dbelyaninov) wrote :

Appropriate 9.X SWARM test passed since #178 run
https://product-ci.infra.mirantis.net/job/9.x.system_test.ubuntu.deploy_multipath/

Dmitry Belyaninov (dbelyaninov)
tags: removed: on-verification
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to fuel-nailgun-agent (stable/newton)

Reviewed: https://review.openstack.org/418429
Committed: https://git.openstack.org/cgit/openstack/fuel-nailgun-agent/commit/?id=6b864564bc1a24782343b365de3bc52f7822520c
Submitter: Jenkins
Branch: stable/newton

commit 6b864564bc1a24782343b365de3bc52f7822520c
Author: Vladimir Kozhukalov <email address hidden>
Date: Wed Jan 4 19:37:10 2017 +0300

    Fix multipath device search

    When nailgun-agent starts before device mapper
    assembles multipath devices it reports physical disks
    that are used for multipath devices as usual disks.

    This patch does the following

    * filter out devices that are DM_MULTIPATH_DEVICE_PATH = 1
    * run 'udevadm settle' before scanning multipath devices
    * delays scanning multipath devices until building block device info

    Change-Id: I088aede0cf3bd1d16a57e7cdec4e50cab2c19175
    Closes-Bug: #1652788
    (cherry picked from commit 8fbc4d640535387b402aa38cb6227df11c010ff0)

Changed in fuel:
status: In Progress → Fix Committed
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to fuel-nailgun-agent (master)

Reviewed: https://review.openstack.org/418430
Committed: https://git.openstack.org/cgit/openstack/fuel-nailgun-agent/commit/?id=fd0f4664b381067d04c7c26f3e634bc51ab145fe
Submitter: Jenkins
Branch: master

commit fd0f4664b381067d04c7c26f3e634bc51ab145fe
Author: Vladimir Kozhukalov <email address hidden>
Date: Wed Jan 4 19:37:10 2017 +0300

    Fix multipath device search

    When nailgun-agent starts before device mapper
    assembles multipath devices it reports physical disks
    that are used for multipath devices as usual disks.

    This patch does the following

    * filter out devices that are DM_MULTIPATH_DEVICE_PATH = 1
    * run 'udevadm settle' before scanning multipath devices
    * delays scanning multipath devices until building block device info

    Change-Id: I088aede0cf3bd1d16a57e7cdec4e50cab2c19175
    Closes-Bug: #1652788
    (cherry picked from commit 8fbc4d640535387b402aa38cb6227df11c010ff0)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/fuel-nailgun-agent 11.0.0.0rc1

This issue was fixed in the openstack/fuel-nailgun-agent 11.0.0.0rc1 release candidate.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.