neutron

Race condition with failover logic in BigSwitch plugin

Bug #1200022 reported by Kevin Benton
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
neutron
Fix Released
Medium
Kevin Benton
neutron 2013.2 "havana"
Grizzly
Fix Released
Medium
Gary Kotton
neutron 2013.1.3

Bug Description

The failover logic modifies a list of servers as it iterates through them.

https://github.com/openstack/neutron/blob/ee3fe4e836ca1c81e50a8324a9b5f982de4fa97f/neutron/plugins/bigswitch/plugin.py#L300

If a thread fails for an unexpected exception, a server could get permanently removed from the failover pool.

Tags: bigswitch
Kevin Benton (kevinbenton)
Changed in neutron:
assignee: nobody → Kevin Benton (blak111)
Sumit Naiksatam (snaiksat)
Changed in neutron:
importance: Undecided → Medium
milestone: none → havana-2
tags: added: bigswitch
Changed in neutron:
status: New → Confirmed
OpenStack Infra (hudson-openstack)
Changed in neutron:
status: Confirmed → In Progress
Sumit Naiksatam (snaiksat)
tags: added: grizzly-backport-potential
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (master)

Reviewed: https://review.openstack.org/36581
Committed: http://github.com/openstack/neutron/commit/3732f022778a7c4a47c58029ae77e32a0a4a6ee9
Submitter: Jenkins
Branch: master

commit 3732f022778a7c4a47c58029ae77e32a0a4a6ee9
Author: Kevin Benton <email address hidden>
Date: Wed Jul 10 16:28:12 2013 -0700

    Prevent possible server list damage in BigSwitch plugin

    The old failover logic for the BigSwitch plugin modified a
    list of controllers as it tested them. If the code for one
    thread unexpectedly raised an exception or died, the global
    server list could lose a server permanently.
    This patch addresses that by flagging servers as failed instead
    so the global server list is never modified.

    Fixes: bug #1200022
    Change-Id: Id2dcb820ef9f62fd03e3215bff3345e56c78afe2

Changed in neutron:
status: In Progress → Fix Committed
Thierry Carrez (ttx)
Changed in neutron:
status: Fix Committed → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (stable/grizzly)

Fix proposed to branch: stable/grizzly
Review: https://review.openstack.org/38491

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (stable/grizzly)

Reviewed: https://review.openstack.org/38491
Committed: http://github.com/openstack/neutron/commit/49278f22b64cf8eb1b99b12e55ff33d88b62a9fc
Submitter: Jenkins
Branch: stable/grizzly

commit 49278f22b64cf8eb1b99b12e55ff33d88b62a9fc
Author: Kevin Benton <email address hidden>
Date: Wed Jul 10 16:28:12 2013 -0700

    Prevent possible server list damage in BigSwitch plugin

    The old failover logic for the BigSwitch plugin modified a
    list of controllers as it tested them. If the code for one
    thread unexpectedly raised an exception or died, the global
    server list could lose a server permanently.
    This patch addresses that by flagging servers as failed instead
    so the global server list is never modified.

    Fixes: bug #1200022

    (cherry picked from commit 3732f022778a7c4a47c58029ae77e32a0a4a6ee9)

    Conflicts:

     quantum/plugins/bigswitch/plugin.py

    Change-Id: Id2dcb820ef9f62fd03e3215bff3345e56c78afe2

tags: added: in-stable-grizzly
Alan Pevec (apevec)
tags: removed: grizzly-backport-potential in-stable-grizzly
Thierry Carrez (ttx)
Changed in neutron:
milestone: havana-2 → 2013.2
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.