[OVN] Log object deletion fails if related security group was simultaneously modified

Bug #2019887 reported by Elvira García Ruiz
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
neutron
In Progress
Medium
Elvira García Ruiz

Bug Description

If we run many operations in parallel related to deleting security groups and deleting network log objects (like test_log_deleted_with_corresponding_security_group [0] in neutron tempest tests), we generate an error since ovsdbapp tries to modify an entry with db_set of an ACL that is no longer there.
Although this is fixed on newer versions of ovsdbapp [1], it is nice to remove the possibility of such failure in stable branches. This can be achieved by overriding db_set in the needed stable branches.

More information: https://bugzilla.redhat.com/show_bug.cgi?id=2178618

[0] https://opendev.org/openstack/neutron-tempest-plugin/src/commit/bd7d0e8402af758a5de13ebc98dcb96ef558e61a/neutron_tempest_plugin/api/admin/test_logging.py#L79
[1] https://review.opendev.org/c/openstack/ovsdbapp/+/880687

tags: added: stable
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (stable/zed)

Fix proposed to branch: stable/zed
Review: https://review.opendev.org/c/openstack/neutron/+/883400

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (stable/yoga)

Fix proposed to branch: stable/yoga
Review: https://review.opendev.org/c/openstack/neutron/+/883401

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (stable/xena)

Fix proposed to branch: stable/xena
Review: https://review.opendev.org/c/openstack/neutron/+/883485

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (stable/wallaby)

Fix proposed to branch: stable/wallaby
Review: https://review.opendev.org/c/openstack/neutron/+/883486

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (stable/2023.1)

Reviewed: https://review.opendev.org/c/openstack/neutron/+/883102
Committed: https://opendev.org/openstack/neutron/commit/551ba73aa49c070f95ba3fd6461ab85c5f2c7bd2
Submitter: "Zuul (22348)"
Branch: stable/2023.1

commit 551ba73aa49c070f95ba3fd6461ab85c5f2c7bd2
Author: Elvira García <email address hidden>
Date: Mon May 15 01:24:27 2023 +0200

    [stable-only][ovn] Fix ovsdbapp db_set command for stable branches

    The lack of an if_exists on the db_set function in ovsdbapp causes the
    network log object deletion to not be a asynchronous operation on the
    ML2/OVN plugin. The if_exists was recently added to ovsdbapp, but since
    that is an API change[0] it will not be backported to older branches,
    so this patch overrides the function. This way we avoid the
    possibility of getting errors when concurrently making operations
    with security groups and network log objects.

    Closes-bug: #2019887

    [0] https://review.opendev.org/c/openstack/ovsdbapp/+/880687

    Change-Id: I628591da75c1cc367076f5a3051ca3c1e131d216

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (stable/zed)

Reviewed: https://review.opendev.org/c/openstack/neutron/+/883400
Committed: https://opendev.org/openstack/neutron/commit/1529c0a29728955064a53d616a118bc8a7ca1b4c
Submitter: "Zuul (22348)"
Branch: stable/zed

commit 1529c0a29728955064a53d616a118bc8a7ca1b4c
Author: Elvira García <email address hidden>
Date: Mon May 15 01:24:27 2023 +0200

    [stable-only][ovn] Fix ovsdbapp db_set command for stable branches

    The lack of an if_exists on the db_set function in ovsdbapp causes the
    network log object deletion to not be a asynchronous operation on the
    ML2/OVN plugin. The if_exists was recently added to ovsdbapp, but since
    that is an API change[0] it will not be backported to older branches,
    so this patch overrides the function. This way we avoid the
    possibility of getting errors when concurrently making operations
    with security groups and network log objects.

    Closes-bug: #2019887

    [0] https://review.opendev.org/c/openstack/ovsdbapp/+/880687

    Change-Id: I628591da75c1cc367076f5a3051ca3c1e131d216
    (cherry picked from commit 551ba73aa49c070f95ba3fd6461ab85c5f2c7bd2)

tags: added: in-stable-zed
tags: added: in-stable-yoga
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (stable/yoga)

Reviewed: https://review.opendev.org/c/openstack/neutron/+/883401
Committed: https://opendev.org/openstack/neutron/commit/99b712b39707189ae92110afd58eb6a5796e9220
Submitter: "Zuul (22348)"
Branch: stable/yoga

commit 99b712b39707189ae92110afd58eb6a5796e9220
Author: Elvira García <email address hidden>
Date: Mon May 15 01:24:27 2023 +0200

    [stable-only][ovn] Fix ovsdbapp db_set command for stable branches

    The lack of an if_exists on the db_set function in ovsdbapp causes the
    network log object deletion to not be a asynchronous operation on the
    ML2/OVN plugin. The if_exists was recently added to ovsdbapp, but since
    that is an API change[0] it will not be backported to older branches,
    so this patch overrides the function. This way we avoid the
    possibility of getting errors when concurrently making operations
    with security groups and network log objects.

    Closes-bug: #2019887

    [0] https://review.opendev.org/c/openstack/ovsdbapp/+/880687

    Change-Id: I628591da75c1cc367076f5a3051ca3c1e131d216
    (cherry picked from commit 551ba73aa49c070f95ba3fd6461ab85c5f2c7bd2)

tags: added: in-stable-xena
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (stable/xena)

Reviewed: https://review.opendev.org/c/openstack/neutron/+/883485
Committed: https://opendev.org/openstack/neutron/commit/f68426366d536a51d57a8a26f4185a81f543771c
Submitter: "Zuul (22348)"
Branch: stable/xena

commit f68426366d536a51d57a8a26f4185a81f543771c
Author: Elvira García <email address hidden>
Date: Mon May 15 01:24:27 2023 +0200

    [stable-only][ovn] Fix ovsdbapp db_set command for stable branches

    The lack of an if_exists on the db_set function in ovsdbapp causes the
    network log object deletion to not be a asynchronous operation on the
    ML2/OVN plugin. The if_exists was recently added to ovsdbapp, but since
    that is an API change[0] it will not be backported to older branches,
    so this patch overrides the function. This way we avoid the
    possibility of getting errors when concurrently making operations
    with security groups and network log objects.

    Closes-bug: #2019887

    [0] https://review.opendev.org/c/openstack/ovsdbapp/+/880687

    Conflicts:
          neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py

    Change-Id: I628591da75c1cc367076f5a3051ca3c1e131d216
    (cherry picked from commit 551ba73aa49c070f95ba3fd6461ab85c5f2c7bd2)

tags: added: in-stable-wallaby
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (stable/wallaby)

Reviewed: https://review.opendev.org/c/openstack/neutron/+/883486
Committed: https://opendev.org/openstack/neutron/commit/5d0f7a3a5c59693439c533b2b0e5d16b364b658b
Submitter: "Zuul (22348)"
Branch: stable/wallaby

commit 5d0f7a3a5c59693439c533b2b0e5d16b364b658b
Author: Elvira García <email address hidden>
Date: Mon May 15 01:24:27 2023 +0200

    [stable-only][ovn] Fix ovsdbapp db_set command for stable branches

    The lack of an if_exists on the db_set function in ovsdbapp causes the
    network log object deletion to not be a asynchronous operation on the
    ML2/OVN plugin. The if_exists was recently added to ovsdbapp, but since
    that is an API change[0] it will not be backported to older branches,
    so this patch overrides the function. This way we avoid the
    possibility of getting errors when concurrently making operations
    with security groups and network log objects.

    Closes-bug: #2019887

    [0] https://review.opendev.org/c/openstack/ovsdbapp/+/880687

    Conflicts:
          neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py

    Change-Id: I628591da75c1cc367076f5a3051ca3c1e131d216
    (cherry picked from commit 551ba73aa49c070f95ba3fd6461ab85c5f2c7bd2)

Revision history for this message
Elvira García Ruiz (elviragr) wrote :

This is also occurring in the db_remove command of the acl removal function.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to neutron (master)

Related fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/neutron/+/883693

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to neutron (master)

Reviewed: https://review.opendev.org/c/openstack/neutron/+/883693
Committed: https://opendev.org/openstack/neutron/commit/e0a2427a2f701adae14cd8196e3c2f064416d044
Submitter: "Zuul (22348)"
Branch: master

commit e0a2427a2f701adae14cd8196e3c2f064416d044
Author: Elvira García <email address hidden>
Date: Fri May 19 20:30:47 2023 +0200

    [ovn] Avoid unwanted ACL_NOT_FOUND error when deleting log objects

    There is the possibility that db_remove raises an error if an ACL was
    deleted on parallel for other reasons while deleting a log object. On a
    normal situation, this command would remove the no-longer-needed
    'log-related' property, but since the ACL is no longer there, it will
    raise an error. For this case, it is not problematic to have an ACL
    deleted mid-operation, so it should not raise any error.

    Related-Bug: #2019887

    Signed-off-by: Elvira García <email address hidden>
    Change-Id: I154393529134b5861e0ef0283257ef964fe057fd

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to neutron (stable/2023.1)

Related fix proposed to branch: stable/2023.1
Review: https://review.opendev.org/c/openstack/neutron/+/883717

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to neutron (stable/zed)

Related fix proposed to branch: stable/zed
Review: https://review.opendev.org/c/openstack/neutron/+/883718

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to neutron (stable/yoga)

Related fix proposed to branch: stable/yoga
Review: https://review.opendev.org/c/openstack/neutron/+/883719

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to neutron (stable/xena)

Related fix proposed to branch: stable/xena
Review: https://review.opendev.org/c/openstack/neutron/+/883720

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to neutron (stable/wallaby)

Related fix proposed to branch: stable/wallaby
Review: https://review.opendev.org/c/openstack/neutron/+/883721

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to neutron (stable/wallaby)

Reviewed: https://review.opendev.org/c/openstack/neutron/+/883721
Committed: https://opendev.org/openstack/neutron/commit/acbb2927e36c5459a1d283aca36d0c71ad620227
Submitter: "Zuul (22348)"
Branch: stable/wallaby

commit acbb2927e36c5459a1d283aca36d0c71ad620227
Author: Elvira García <email address hidden>
Date: Fri May 19 20:30:47 2023 +0200

    [ovn] Avoid unwanted ACL_NOT_FOUND error when deleting log objects

    There is the possibility that db_remove raises an error if an ACL was
    deleted on parallel for other reasons while deleting a log object. On a
    normal situation, this command would remove the no-longer-needed
    'log-related' property, but since the ACL is no longer there, it will
    raise an error. For this case, it is not problematic to have an ACL
    deleted mid-operation, so it should not raise any error.

    Related-Bug: #2019887

    Signed-off-by: Elvira García <email address hidden>
    Change-Id: I154393529134b5861e0ef0283257ef964fe057fd
    (cherry picked from commit e0a2427a2f701adae14cd8196e3c2f064416d044)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to neutron (stable/xena)

Reviewed: https://review.opendev.org/c/openstack/neutron/+/883720
Committed: https://opendev.org/openstack/neutron/commit/440e9596520ae7bda5ce3c4773ad48805da6cba5
Submitter: "Zuul (22348)"
Branch: stable/xena

commit 440e9596520ae7bda5ce3c4773ad48805da6cba5
Author: Elvira García <email address hidden>
Date: Fri May 19 20:30:47 2023 +0200

    [ovn] Avoid unwanted ACL_NOT_FOUND error when deleting log objects

    There is the possibility that db_remove raises an error if an ACL was
    deleted on parallel for other reasons while deleting a log object. On a
    normal situation, this command would remove the no-longer-needed
    'log-related' property, but since the ACL is no longer there, it will
    raise an error. For this case, it is not problematic to have an ACL
    deleted mid-operation, so it should not raise any error.

    Related-Bug: #2019887

    Signed-off-by: Elvira García <email address hidden>
    Change-Id: I154393529134b5861e0ef0283257ef964fe057fd
    (cherry picked from commit e0a2427a2f701adae14cd8196e3c2f064416d044)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to neutron (stable/yoga)

Reviewed: https://review.opendev.org/c/openstack/neutron/+/883719
Committed: https://opendev.org/openstack/neutron/commit/bbbafe1681effbc0dd6b6f80c6236596bad3e90f
Submitter: "Zuul (22348)"
Branch: stable/yoga

commit bbbafe1681effbc0dd6b6f80c6236596bad3e90f
Author: Elvira García <email address hidden>
Date: Fri May 19 20:30:47 2023 +0200

    [ovn] Avoid unwanted ACL_NOT_FOUND error when deleting log objects

    There is the possibility that db_remove raises an error if an ACL was
    deleted on parallel for other reasons while deleting a log object. On a
    normal situation, this command would remove the no-longer-needed
    'log-related' property, but since the ACL is no longer there, it will
    raise an error. For this case, it is not problematic to have an ACL
    deleted mid-operation, so it should not raise any error.

    Related-Bug: #2019887

    Signed-off-by: Elvira García <email address hidden>
    Change-Id: I154393529134b5861e0ef0283257ef964fe057fd
    (cherry picked from commit e0a2427a2f701adae14cd8196e3c2f064416d044)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to neutron (stable/zed)

Reviewed: https://review.opendev.org/c/openstack/neutron/+/883718
Committed: https://opendev.org/openstack/neutron/commit/8def3b694dd800da09e44f332c0ea11e802dbde4
Submitter: "Zuul (22348)"
Branch: stable/zed

commit 8def3b694dd800da09e44f332c0ea11e802dbde4
Author: Elvira García <email address hidden>
Date: Fri May 19 20:30:47 2023 +0200

    [ovn] Avoid unwanted ACL_NOT_FOUND error when deleting log objects

    There is the possibility that db_remove raises an error if an ACL was
    deleted on parallel for other reasons while deleting a log object. On a
    normal situation, this command would remove the no-longer-needed
    'log-related' property, but since the ACL is no longer there, it will
    raise an error. For this case, it is not problematic to have an ACL
    deleted mid-operation, so it should not raise any error.

    Related-Bug: #2019887

    Signed-off-by: Elvira García <email address hidden>
    Change-Id: I154393529134b5861e0ef0283257ef964fe057fd
    (cherry picked from commit e0a2427a2f701adae14cd8196e3c2f064416d044)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to neutron (stable/2023.1)

Reviewed: https://review.opendev.org/c/openstack/neutron/+/883717
Committed: https://opendev.org/openstack/neutron/commit/97e478dcf5483a0a5581dbb7923106a7615c7d5b
Submitter: "Zuul (22348)"
Branch: stable/2023.1

commit 97e478dcf5483a0a5581dbb7923106a7615c7d5b
Author: Elvira García <email address hidden>
Date: Fri May 19 20:30:47 2023 +0200

    [ovn] Avoid unwanted ACL_NOT_FOUND error when deleting log objects

    There is the possibility that db_remove raises an error if an ACL was
    deleted on parallel for other reasons while deleting a log object. On a
    normal situation, this command would remove the no-longer-needed
    'log-related' property, but since the ACL is no longer there, it will
    raise an error. For this case, it is not problematic to have an ACL
    deleted mid-operation, so it should not raise any error.

    Related-Bug: #2019887

    Signed-off-by: Elvira García <email address hidden>
    Change-Id: I154393529134b5861e0ef0283257ef964fe057fd
    (cherry picked from commit e0a2427a2f701adae14cd8196e3c2f064416d044)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/neutron 22.0.1

This issue was fixed in the openstack/neutron 22.0.1 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/neutron 21.1.1

This issue was fixed in the openstack/neutron 21.1.1 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/neutron 20.3.1

This issue was fixed in the openstack/neutron 20.3.1 release.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.