Cinder

Volume Manager failed to start when some volumes are not found on the backend

Series newton
Bug #1628340

Bug #1628340 reported by Huang Zhiteng on 2016-09-28

This bug affects 1 person

	Status	Importance	Assigned to	Milestone
Cinder	Fix Released	High	Huang Zhiteng
Mitaka	Fix Released	Undecided	Huang Zhiteng
Newton	Fix Released	High	Huang Zhiteng
Ocata	Fix Released	High	Huang Zhiteng	Cinder ocata-1 "o1"

Bug Description

When volume manager initializes a backend, one of the steps is to let backend driver to update provider info for list of volumes in Cinder DB. Current code assumed that every volume (no matter what status it is in) can be found in backend, which in some cases are not true. In those cases, Cinder volume service would fail to start due to a unhandled IndexError.

====================
    def _sync_provider_info(self, ctxt, volumes, snapshots):
         updates, snapshot_updates = self.driver.update_provider_info(
             volumes, snapshots)

         if updates:
             for volume in volumes:
                 # NOTE(JDG): Make sure returned item is in this hosts volumes
                 update = (
                     [updt for updt in updates if updt['id'] ==
                         volume['id']][0]) <--
                 if update:
                     self.db.volume_update(
                         ctxt,
                         update['id'],
                         {'provider_id': update['provider_id']})
----------------------

File "cinder/volume/manager.py", line 428, in init_host
        self._sync_provider_info(ctxt, volumes, snapshots)
      File "cinder/volume/manager.py", line 349, in _sync_provider_info
        volume['id']][0])
    IndexError: list index out of range

Tags:

Huang Zhiteng (zhiteng-huang) on 2016-09-28

Changed in cinder:
assignee:	nobody → Huang Zhiteng (zhiteng-huang)
importance:	Undecided → High

Sean McGinnis (sean-mcginnis) on 2016-09-28

Changed in cinder:
milestone:	none → ocata-1

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2016-09-28: Fix proposed to cinder (master)

Fix proposed to branch: master
Review: https://review.openstack.org/378175

Changed in cinder:
status:	New → In Progress

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2016-09-29: Fix merged to cinder (master)

Reviewed: https://review.openstack.org/378175
Committed: https://git.openstack.org/cgit/openstack/cinder/commit/?id=d96fb9a10135df062732279bac2d7eb6073872b0
Submitter: Jenkins
Branch: master

commit d96fb9a10135df062732279bac2d7eb6073872b0
Author: Zhiteng Huang <email address hidden>
Date: Tue Sep 27 19:31:59 2016 -0700

Fix IndexError in volume manager

    A list comprehension in volume manager can raise IndexError due
    to unsafe assumption of input. This change fix it by not assuming
    list comprehension always return none empty list. This also fixes
    a snapshot DB call by using OVO.

Change-Id: I83e8ac8dcf9e84b37847d6eb09065d55e843f82e
Closes-bug: #1628340

Changed in cinder:
status:	In Progress → Fix Released

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2016-09-29: Fix proposed to cinder (stable/newton)

Fix proposed to branch: stable/newton
Review: https://review.openstack.org/379775

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2016-09-29: Fix proposed to cinder (stable/mitaka)

Fix proposed to branch: stable/mitaka
Review: https://review.openstack.org/379776

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2016-10-10: Fix merged to cinder (stable/mitaka)

Reviewed: https://review.openstack.org/379776
Committed: https://git.openstack.org/cgit/openstack/cinder/commit/?id=5b22d174d6feed70c92556af48546b22aec1e86d
Submitter: Jenkins
Branch: stable/mitaka

commit 5b22d174d6feed70c92556af48546b22aec1e86d
Author: Zhiteng Huang <email address hidden>
Date: Tue Sep 27 19:31:59 2016 -0700

Fix IndexError in volume manager

    Change-Id: I83e8ac8dcf9e84b37847d6eb09065d55e843f82e
    Closes-bug: #1628340
    (cherry picked from commit d96fb9a10135df062732279bac2d7eb6073872b0)

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2016-10-25: Fix merged to cinder (stable/newton)

Reviewed: https://review.openstack.org/379775
Committed: https://git.openstack.org/cgit/openstack/cinder/commit/?id=6079b7b0b9098e96b903d39e5d27803c9c97a5a0
Submitter: Jenkins
Branch: stable/newton

commit 6079b7b0b9098e96b903d39e5d27803c9c97a5a0
Author: Zhiteng Huang <email address hidden>
Date: Tue Sep 27 19:31:59 2016 -0700

Fix IndexError in volume manager

    Change-Id: I83e8ac8dcf9e84b37847d6eb09065d55e843f82e
    Closes-bug: #1628340
    (cherry picked from commit d96fb9a10135df062732279bac2d7eb6073872b0)