shard audit exaggerates reports of overlapping shard ranges

Bug #1913332 reported by Alistair Coles
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Object Storage (swift)
Confirmed
Undecided
Matthew Oliver

Bug Description

When sharder audits a root container and finds overlapping ranges, there is a tendency for each overlap to be logged more than once. For, example the following might be logged:

Audit failed for root var/folders/n4/nf8krx110f39llj82fbw_xjm0000gp/T/tmpFbb92K/sda/containers/0/f33/4a8a08f09d37b73795649038408b5f33/4a8a08f09d37b73795649038408b5f33.db (a/c): overlapping ranges in state created: s-z k-t, overlapping ranges in state created: k-t s-z

This can be illustrated by changing a unit test to assert that both variations of the same overlap are logged:

diff --git a/test/unit/container/test_sharder.py b/test/unit/container/test_sharder.py
index 22cd86797..7df1d145b 100644
--- a/test/unit/container/test_sharder.py
+++ b/test/unit/container/test_sharder.py
@@ -4397,6 +4397,9 @@ class TestSharder(BaseTestSharder):
             self.assertIn(
                 'overlapping ranges in state %s: k-t s-z' % state_text,
                 line)
+ self.assertIn(
+ 'overlapping ranges in state %s: s-z k-t' % state_text,
+ line)

         expected_stats = {'attempted': 1, 'success': 0, 'failure': 1}
         shard_bounds = (('a', 'j'), ('k', 't'), ('s', 'z'))

(based on commit 257041aff)

Revision history for this message
Matthew Oliver (matt-0) wrote :

This is because your ShardRange.__lt__ isn't sensitive to overlaps. Just whole range differences. Ie. self.upper < other.lower. Which doesn't work in an overlap

Matthew Oliver (matt-0)
Changed in swift:
assignee: nobody → Matthew Oliver (matt-0)
Changed in swift:
status: New → Confirmed
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to swift (master)

Related fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/swift/+/791485

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to swift (master)

Reviewed: https://review.opendev.org/c/openstack/swift/+/791485
Committed: https://opendev.org/openstack/swift/commit/1b183f221d258819c82ee58961afc765529314ac
Submitter: "Zuul (22348)"
Branch: master

commit 1b183f221d258819c82ee58961afc765529314ac
Author: Alistair Coles <email address hidden>
Date: Fri May 14 13:59:37 2021 +0100

    sharder: report perfectly overlapping shard ranges

    Previously, shard ranges that overlapped perfectly (i.e. their bounds
    were equal) were erroneously excluded from overlapping shard reports.
    With this patch they will be reported in logs and cause shard audit
    failures.

    Also, the format of the audit log line is simplified to avoid
    unnecessary duplication of information.

    Change-Id: Ie15c9f40a132374c89337f0009fb5cf5a8e62c51
    Closes-Bug: #1928459
    Related-Bug: #1913332

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.