Juju Charms Collection
cinder package

cinder-volume fails to start in Ocata when no backend is connected

Bug #1665272 reported by Liam Young
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Cinder Charm
Fix Released
Critical
Liam Young
OpenStack Cinder Charm 17.02
cinder (Juju Charms Collection)
Invalid
Critical
Liam Young
Juju Charms Collection 17.01

Bug Description

When deploying the cinder charm using Ocata and no backend charm connected, the cinder-volume service fails to start. Error from cinder-volume.log:

Configuration for cinder-volume does not specify "enabled_backends". Using DEFAULT section to configure drivers is not supported since Ocata.
No volume service(s) started successfully, terminating.

The charm eventually reports the dead services via juju status:

Services not running that should be: cinder-volume

The easiest way to see this is to run the amulet test tests/gate-basic-xenial-ocata. It fails with

2017-02-16 09:17:07,356 create_cinder_volume DEBUG: Creating cinder volume...
2017-02-16 09:17:14,114 resource_reaches_status DEBUG: Volume status wait status check: 0 [creating:available] 0974c72f-e88d-40c2-a00a-effa49abee9f
2017-02-16 09:17:18,304 resource_reaches_status DEBUG: Volume status wait status check: 1 [error:available] 0974c72f-e88d-40c2-a00a-effa49abee9f
....
2017-02-16 09:19:19,590 resource_reaches_status DEBUG: Volume status wait: expected, actual status = error, available
2017-02-16 09:19:19,590 resource_reaches_status DEBUG: 0974c72f-e88d-40c2-a00a-effa49abee9f never reached expected status: available

Liam Young (gnuoy)
Changed in cinder (Juju Charms Collection):
status: New → Confirmed
importance: Undecided → Critical
assignee: nobody → Liam Young (gnuoy)
milestone: none → 17.01
Revision history for this message
Liam Young (gnuoy) wrote :

I've looked into this and it seems more involved than I had first thought. The requirement to use enabled_backends causes volumes to be created with a different os-vol-host-attr:host. If a cloud is upgraded to Ocata any existing volumes will have an invalid os-vol-host-attr:host. The existing volumes need to have that attribute updated via cinder-manage *1

*1 https://docs.openstack.org/admin-guide/blockstorage-multi-backend.html#enable-multiple-storage-back-ends

OpenStack Infra (hudson-openstack)
Changed in cinder (Juju Charms Collection):
status: Confirmed → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to charm-cinder (master)

Reviewed: https://review.openstack.org/435459
Committed: https://git.openstack.org/cgit/openstack/charm-cinder/commit/?id=8641e81498dce5a7fa2b42bb48aa9655c5df7d04
Submitter: Jenkins
Branch: master

commit 8641e81498dce5a7fa2b42bb48aa9655c5df7d04
Author: Liam Young <email address hidden>
Date: Thu Feb 16 12:13:03 2017 +0000

    Move backends into their own sections

    Using DEFAULT section to configure drivers is not supported since
    Ocata. This change lists backends in enabled_backends.

    Note: Using sectional config causes the os-vol-host-attr:host
    volume attribute to change which invalidates existing volumes.
    A subsequant change is needed to add an action for renaming
    the host attribute of existing volumes *1

    *1 https://docs.openstack.org/admin-guide/blockstorage-multi-backend.html#enable-multiple-storage-back-ends

    Partial-Bug: #1665272
    Change-Id: I22c3e74b0874c051bee89e4609088facf95b4664

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to charm-cinder (master)

Fix proposed to branch: master
Review: https://review.openstack.org/436848

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to charm-cinder (master)

Reviewed: https://review.openstack.org/436848
Committed: https://git.openstack.org/cgit/openstack/charm-cinder/commit/?id=d21160a0d354a815b563d55a33d204c210e815cf
Submitter: Jenkins
Branch: master

commit d21160a0d354a815b563d55a33d204c210e815cf
Author: Liam Young <email address hidden>
Date: Wed Feb 22 09:48:08 2017 +0000

    Add actions for renaming volume host attr

    Add two new actions which are essentially wrappers around

    cinder-manage volume update_host \
        --currenthost CURRENTHOST \
        --newhost NEWHOST

    In previous versions of the charm if block-device is set or the
    legacy ceph relation is used (as opposed to storage-backend relation)
    then the configuration of those backends is done in the [DEFAULT]
    section of the cinder.conf. As of Ocata that is no longer supported
    and backends need to be listed in their own sections and referenced
    via enable_backends parameter. This change in config results in a
    change of host name and existing volumes need to have their metadata
    updated to point at the new hostname.

    Old Hostname: <unit-name>
    New Hostname: <unit-name>@<backend section name>#<volume-backend-name>

    New Action: volume-backend-name
    Used for updating the host attribute of volumes to add the driver
    name. This is needed after an upgrade to Ocata if there are existing
    volumes which have been configured prior to multi-backends
    being enabled.

    New Action: rename-volume-host
    Used for updating the host attribute of volumes. This action is
    a lower level action then volume-host-add-driver and simply passes
    the old and new hosts verbatim to cinder-manage.

    Change-Id: I989074a3f41126aa57c514f7e18b887733bc18fe
    Partial-Bug: #1665272

James Page (james-page)
Changed in charm-cinder:
assignee: nobody → Liam Young (gnuoy)
importance: Undecided → Critical
status: New → In Progress
Changed in cinder (Juju Charms Collection):
status: In Progress → Invalid
James Page (james-page)
Changed in charm-cinder:
status: In Progress → Fix Released
milestone: none → 17.02
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.