OpenStack Identity (keystone)

Eliminate use of with_lockmode('update')

Bug #1325143 reported by Morgan Fainberg
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Identity (keystone)
Fix Released
Medium
Morgan Fainberg
OpenStack Identity (keystone) 2014.2 "juno"
Icehouse
Fix Released
Medium
Morgan Fainberg
OpenStack Identity (keystone) 2014.1.3

Bug Description

As discussed here: http://lists.openstack.org/pipermail/openstack-dev/2014-May/035264.html
 the use of "with_lockmode('update')" can cause a number of issues when run on top of MySQL+Galera because galera does not support the 'SELECT ... FOR UPDATE' SQL call.

We currently only use with_lockmode('update') for coordinating consuming trusts (limited use trusts).

We should eliminate this and handle the coordination of consumption to ensure only the specified number of tokens can be issued from a trust. Unfortunately, this is not as straightforward as it could be, we need to handle the following deployment scenarios:

* Eventlet
* Multiple Keystone Processes (same physical server) [same issue as mod_wsgi]
* Multiple Keystone Processes (different physical servers)

The first and second ones could be handled with the lockutils (external file-based) locking decorator. The last scenario will take more thought.

See original description
Morgan Fainberg (mdrnstm)
Changed in keystone:
importance: Undecided → High
status: New → Triaged
tags: added: icehouse-backport-potential
Changed in keystone:
importance: High → Medium
Adam Young (ayoung)
description: updated
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to keystone (master)

Fix proposed to branch: master
Review: https://review.openstack.org/97059

Changed in keystone:
assignee: nobody → Morgan Fainberg (mdrnstm)
status: Triaged → In Progress
Morgan Fainberg (mdrnstm)
Changed in keystone:
milestone: none → juno-3
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to keystone (master)

Reviewed: https://review.openstack.org/97059
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=0724fc4d27c90461de3310b3b2cb7387e6c92bba
Submitter: Jenkins
Branch: master

commit 0724fc4d27c90461de3310b3b2cb7387e6c92bba
Author: Morgan Fainberg <email address hidden>
Date: Sat Jun 28 00:07:45 2014 -0700

    Remove `with_lockmode` use from Trust SQL backend.

    Due to the lack of support of `SELECT .. FOR UPDATE` call in
    MySQL + Galera, the use of `with_lockmode('update')` needs to be
    removed from the Trust SQL backend when utilizing the MySQL
    dialect.

    Instead of the pessimistic locking method (row lock, table lock),
    use optimistic locking. The consume_use method now attempts to
    update the remaining_uses column for the trust in question only
    if the remaining_uses equal the same number as on the initial
    query. This is done in a loop with a 10-iteration failsafe to
    prevent endless looping.

    Change-Id: I1b8af6ce5709f829f345cd351ec9242d0217e743
    Closes-Bug: #1325143

Changed in keystone:
status: In Progress → Fix Committed
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to keystone (master)

Fix proposed to branch: master
Review: https://review.openstack.org/112663

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to keystone (stable/icehouse)

Fix proposed to branch: stable/icehouse
Review: https://review.openstack.org/112920

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on keystone (master)

Change abandoned by Morgan Fainberg (<email address hidden>) on branch: master
Review: https://review.openstack.org/112663
Reason: This looks to be related to the tag-wonkyness that occurred between stable/icehouse and master (stable point release tagged to master incorrectly). Confirmed with -infra that this should be safe to abandon.

Thierry Carrez (ttx)
Changed in keystone:
status: Fix Committed → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to keystone (stable/icehouse)

Reviewed: https://review.openstack.org/112920
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=8485dbc4d6d15b8e30c82363695b2a4e13512de9
Submitter: Jenkins
Branch: stable/icehouse

commit 8485dbc4d6d15b8e30c82363695b2a4e13512de9
Author: Morgan Fainberg <email address hidden>
Date: Sat Jun 28 00:07:45 2014 -0700

    Remove `with_lockmode` use from Trust SQL backend.

    Due to the lack of support of `SELECT .. FOR UPDATE` call in
    MySQL + Galera, the use of `with_lockmode('update')` needs to be
    removed from the Trust SQL backend when utilizing the MySQL
    dialect.

    Instead of the pessimistic locking method (row lock, table lock),
    use optimistic locking. The consume_use method now attempts to
    update the remaining_uses column for the trust in question only
    if the remaining_uses equal the same number as on the initial
    query. This is done in a loop with a 10-iteration failsafe to
    prevent endless looping.

    Conflicts:
     keystone/trust/backends/sql.py

    Minor changes to the sql.py driver over the course of Juno
    caused the conflict. The resulting code for consume has
    not changed.

    Change-Id: I1b8af6ce5709f829f345cd351ec9242d0217e743
    Closes-Bug: #1325143
    (cherry-picked from commit 0724fc4d27c90461de3310b3b2cb7387e6c92bba)

Thierry Carrez (ttx)
Changed in keystone:
milestone: juno-3 → 2014.2
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.