OpenStack Object Storage (swift)

Reconstructor remaining time is incorrect, because total jobs number is increase continually

Bug #1468298 reported by Charles Hsu on 2015-06-24

This bug affects 1 person

Affects		Status	Importance	Assigned to	Milestone
	OpenStack Object Storage (swift)	Fix Released	Low	Charles Hsu	OpenStack Object Storage (swift) 2.4.0

Bug Description

Here is a lot of reconstructed jobs need to be process. But the total jobs count increases continually, so that makes the remaining time is no meaning. The user don't know the reconstructed process will take how much time to finish.

It seems reconstructor counting jobs number while checking each partation.
https://github.com/openstack/swift/blob/0009a43eb45fdee6716d1272b346cfc76d946e4b/swift/obj/reconstructor.py#L866-L884

Jun 24 09:48:32 localhost.localdomain object-reconstructor: 255/278 (91.73%) partitions reconstructed in 2826.57s (0.09/sec, 4m remaining)
Jun 24 09:48:32 localhost.localdomain object-reconstructor: Partition times: max 2638.2300s, min 0.0126s, med 92.2764s
Jun 24 09:49:03 localhost.localdomain object-reconstructor: 256/280 (91.43%) partitions reconstructed in 2857.90s (0.09/sec, 4m remaining)
Jun 24 09:49:03 localhost.localdomain object-reconstructor: Partition times: max 2638.2300s, min 0.0126s, med 92.2764s
Jun 24 09:49:34 localhost.localdomain object-reconstructor: 256/282 (90.78%) partitions reconstructed in 2888.71s (0.09/sec, 4m remaining)
Jun 24 09:49:34 localhost.localdomain object-reconstructor: Partition times: max 2638.2300s, min 0.0126s, med 92.2764s
Jun 24 09:50:10 localhost.localdomain object-reconstructor: 260/283 (91.87%) partitions reconstructed in 2924.78s (0.09/sec, 4m remaining)
Jun 24 09:50:10 localhost.localdomain object-reconstructor: Partition times: max 2638.2300s, min 0.0126s, med 92.2764s
Jun 24 09:50:10 localhost.localdomain object-reconstructor: Removing partition: /srv/node/d40/objects-3/2921
Jun 24 09:50:54 localhost.localdomain object-reconstructor: 262/285 (91.93%) partitions reconstructed in 2968.50s (0.09/sec, 4m remaining)
Jun 24 09:50:54 localhost.localdomain object-reconstructor: Partition times: max 2741.8954s, min 0.0126s, med 92.2860s
Jun 24 09:51:43 localhost.localdomain object-reconstructor: 265/289 (91.70%) partitions reconstructed in 3018.14s (0.09/sec, 4m remaining)
Jun 24 09:51:43 localhost.localdomain object-reconstructor: Partition times: max 2741.8954s, min 0.0126s, med 92.2860s
Jun 24 09:52:14 localhost.localdomain object-reconstructor: 268/295 (90.85%) partitions reconstructed in 3049.26s (0.09/sec, 5m remaining)
Jun 24 09:52:14 localhost.localdomain object-reconstructor: Partition times: max 2741.8954s, min 0.0126s, med 94.9995s
Jun 24 09:52:46 localhost.localdomain object-reconstructor: 269/295 (91.19%) partitions reconstructed in 3081.00s (0.09/sec, 4m remaining)
Jun 24 09:52:46 localhost.localdomain object-reconstructor: Partition times: max 2741.8954s, min 0.0126s, med 94.9995s
Jun 24 09:53:17 localhost.localdomain object-reconstructor: 272/295 (92.20%) partitions reconstructed in 3111.83s (0.09/sec, 4m remaining)
Jun 24 09:53:17 localhost.localdomain object-reconstructor: Partition times: max 2741.8954s, min 0.0126s, med 94.9995s
Jun 24 09:53:17 localhost.localdomain object-reconstructor: Removing partition: /srv/node/d40/objects-3/93
Jun 24 09:53:47 localhost.localdomain object-reconstructor: 274/299 (91.64%) partitions reconstructed in 3142.11s (0.09/sec, 4m remaining)
Jun 24 09:53:47 localhost.localdomain object-reconstructor: Partition times: max 2741.8954s, min 0.0126s, med 95.2591s
Jun 24 09:54:20 localhost.localdomain object-reconstructor: 276/302 (91.39%) partitions reconstructed in 3175.10s (0.09/sec, 4m remaining)
Jun 24 09:54:20 localhost.localdomain object-reconstructor: Partition times: max 2741.8954s, min 0.0126s, med 95.2591s
Jun 24 09:54:52 localhost.localdomain object-reconstructor: 278/304 (91.45%) partitions reconstructed in 3207.38s (0.09/sec, 4m remaining)
Jun 24 09:54:52 localhost.localdomain object-reconstructor: Partition times: max 2741.8954s, min 0.0126s, med 94.9995s
Jun 24 09:55:36 localhost.localdomain object-reconstructor: 284/308 (92.21%) partitions reconstructed in 3250.66s (0.09/sec, 4m remaining)
Jun 24 09:55:36 localhost.localdomain object-reconstructor: Partition times: max 2741.8954s, min 0.0126s, med 95.2591s
Jun 24 09:56:06 localhost.localdomain object-reconstructor: 285/310 (91.94%) partitions reconstructed in 3281.01s (0.09/sec, 4m remaining)
Jun 24 09:56:06 localhost.localdomain object-reconstructor: Partition times: max 2741.8954s, min 0.0126s, med 95.2591s
Jun 24 09:56:12 localhost.localdomain object-reconstructor: Removing partition: /srv/node/d40/objects-3/3741
Jun 24 09:57:31 localhost.localdomain object-reconstructor: 288/314 (91.72%) partitions reconstructed in 3366.22s (0.09/sec, 5m remaining)
Jun 24 09:57:31 localhost.localdomain object-reconstructor: Partition times: max 2741.8954s, min 0.0126s, med 95.2591s
Jun 24 09:58:07 localhost.localdomain object-reconstructor: 290/316 (91.77%) partitions reconstructed in 3402.30s (0.09/sec, 5m remaining)
Jun 24 09:58:07 localhost.localdomain object-reconstructor: Partition times: max 2920.7774s, min 0.0126s, med 99.2620s
Jun 24 09:58:40 localhost.localdomain object-reconstructor: 297/323 (91.95%) partitions reconstructed in 3434.70s (0.09/sec, 5m remaining)
Jun 24 09:58:40 localhost.localdomain object-reconstructor: Partition times: max 2920.7774s, min 0.0126s, med 101.9376s
Jun 24 09:59:10 localhost.localdomain object-reconstructor: 300/324 (92.59%) partitions reconstructed in 3464.94s (0.09/sec, 4m remaining)
Jun 24 09:59:10 localhost.localdomain object-reconstructor: Partition times: max 2920.7774s, min 0.0126s, med 109.4912s
Jun 24 09:59:40 localhost.localdomain object-reconstructor: 305/331 (92.15%) partitions reconstructed in 3495.04s (0.09/sec, 4m remaining)
Jun 24 09:59:40 localhost.localdomain object-reconstructor: Partition times: max 3398.9383s, min 0.0126s, med 110.4927s
Jun 24 10:00:22 localhost.localdomain object-reconstructor: 313/340 (92.06%) partitions reconstructed in 3537.11s (0.09/sec, 5m remaining)
Jun 24 10:00:22 localhost.localdomain object-reconstructor: Partition times: max 3398.9383s, min 0.0126s, med 102.8712s
Jun 24 10:01:06 localhost.localdomain object-reconstructor: 313/340 (92.06%) partitions reconstructed in 3580.96s (0.09/sec, 5m remaining)
Jun 24 10:01:06 localhost.localdomain object-reconstructor: Partition times: max 3398.9383s, min 0.0126s, med 102.8712s
Jun 24 10:01:36 localhost.localdomain object-reconstructor: 315/343 (91.84%) partitions reconstructed in 3611.08s (0.09/sec, 5m remaining)
Jun 24 10:01:36 localhost.localdomain object-reconstructor: Partition times: max 3398.9383s, min 0.0126s, med 109.4912s
Jun 24 10:02:06 localhost.localdomain object-reconstructor: 318/344 (92.44%) partitions reconstructed in 3641.09s (0.09/sec, 4m remaining)
Jun 24 10:02:06 localhost.localdomain object-reconstructor: Partition times: max 3398.9383s, min 0.0126s, med 111.0776s
Jun 24 10:02:40 localhost.localdomain object-reconstructor: 321/348 (92.24%) partitions reconstructed in 3675.14s (0.09/sec, 5m remaining)
Jun 24 10:02:40 localhost.localdomain object-reconstructor: Partition times: max 3398.9383s, min 0.0126s, med 111.0776s
Jun 24 10:03:14 localhost.localdomain object-reconstructor: 323/350 (92.29%) partitions reconstructed in 3708.94s (0.09/sec, 5m remaining)
Jun 24 10:03:14 localhost.localdomain object-reconstructor: Partition times: max 3398.9383s, min 0.0126s, med 111.0776s
Jun 24 10:03:57 localhost.localdomain object-reconstructor: 329/352 (93.47%) partitions reconstructed in 3751.59s (0.09/sec, 4m remaining)
Jun 24 10:03:57 localhost.localdomain object-reconstructor: Partition times: max 3398.9383s, min 0.0126s, med 112.1430s
Jun 24 10:04:30 localhost.localdomain object-reconstructor: 330/356 (92.70%) partitions reconstructed in 3784.79s (0.09/sec, 4m remaining)
Jun 24 10:04:30 localhost.localdomain object-reconstructor: Partition times: max 3398.9383s, min 0.0126s, med 112.1430s
Jun 24 10:04:49 localhost.localdomain object-reconstructor: Removing partition: /srv/node/d40/objects-3/4049
Jun 24 10:05:04 localhost.localdomain object-reconstructor: 334/358 (93.30%) partitions reconstructed in 3819.12s (0.09/sec, 4m remaining)
Jun 24 10:05:04 localhost.localdomain object-reconstructor: Partition times: max 3398.9383s, min 0.0126s, med 112.3085s
Jun 24 10:05:35 localhost.localdomain object-reconstructor: 334/360 (92.78%) partitions reconstructed in 3850.00s (0.09/sec, 4m remaining)
Jun 24 10:05:35 localhost.localdomain object-reconstructor: Partition times: max 3398.9383s, min 0.0126s, med 112.3085s
Jun 24 10:06:05 localhost.localdomain object-reconstructor: 335/361 (92.80%) partitions reconstructed in 3880.02s (0.09/sec, 5m remaining)
Jun 24 10:06:05 localhost.localdomain object-reconstructor: Partition times: max 3398.9383s, min 0.0126s, med 112.3085s
Jun 24 10:06:36 localhost.localdomain object-reconstructor: 339/367 (92.37%) partitions reconstructed in 3910.78s (0.09/sec, 5m remaining)
Jun 24 10:06:36 localhost.localdomain object-reconstructor: Partition times: max 3398.9383s, min 0.0126s, med 112.3085s
Jun 24 10:07:06 localhost.localdomain object-reconstructor: 342/368 (92.93%) partitions reconstructed in 3940.80s (0.09/sec, 4m remaining)
Jun 24 10:07:06 localhost.localdomain object-reconstructor: Partition times: max 3398.9383s, min 0.0126s, med 114.2424s
Jun 24 10:07:36 localhost.localdomain object-reconstructor: 345/371 (92.99%) partitions reconstructed in 3970.86s (0.09/sec, 4m remaining)
Jun 24 10:07:36 localhost.localdomain object-reconstructor: Partition times: max 3398.9383s, min 0.0126s, med 117.7179s
Jun 24 10:08:06 localhost.localdomain object-reconstructor: 348/374 (93.05%) partitions reconstructed in 4000.88s (0.09/sec, 4m remaining)
Jun 24 10:08:06 localhost.localdomain object-reconstructor: Partition times: max 3398.9383s, min 0.0126s, med 118.5659s
Jun 24 10:08:36 localhost.localdomain object-reconstructor: 349/376 (92.82%) partitions reconstructed in 4030.95s (0.09/sec, 5m remaining)
Jun 24 10:08:36 localhost.localdomain object-reconstructor: Partition times: max 3398.9383s, min 0.0126s, med 118.5659s

Tags:

CVE References

2015-5223

Charles Hsu (charles0126) on 2015-06-24

tags:

added: ec

clayg (clay-gerrard) on 2015-06-25

Changed in swift:
status:	New → Confirmed
importance:	Undecided → Low

Revision history for this message

clayg (clay-gerrard) wrote on 2015-06-26:

patch is

https://review.openstack.org/#/c/195275/

Changed in swift:
assignee:	nobody → Charles Hsu (charles0126)
status:	Confirmed → In Progress

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2015-07-28: Fix merged to swift (master)

Reviewed: https://review.openstack.org/195275
Committed: https://git.openstack.org/cgit/openstack/swift/commit/?id=39b6ef6e4fd515d81e688ef365d26d5e0499be7c
Submitter: Jenkins
Branch: master

commit 39b6ef6e4fd515d81e688ef365d26d5e0499be7c
Author: Charles Hsu <email address hidden>
Date: Thu Jun 25 02:06:54 2015 +0800

Fix reconstructor stats mssage.

    Calculate reconstruction job count and remaining time that
    would be inappropriate for user. Use real partition count would
    be suitable for user.

Change-Id: I6b025854baf4757dddf9d7fe7bc2cece58a49157
Closes-Bug: #1468298

Changed in swift:
status:	In Progress → Fix Committed

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2015-08-03: Fix proposed to swift (feature/crypto)

Fix proposed to branch: feature/crypto
Review: https://review.openstack.org/208513

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2015-08-04: Fix merged to swift (feature/crypto)

Download full text (17.4 KiB)

Reviewed: https://review.openstack.org/208513
Committed: https://git.openstack.org/cgit/openstack/swift/commit/?id=942c9bb45c6b8124bcaa407eb9b9ac7f0589c743
Submitter: Jenkins
Branch: feature/crypto

commit 207dd9b49d7d53a9faa4849af2c40bb875416fce
Author: Darrell Bishop <email address hidden>
Date: Thu Jul 30 14:32:08 2015 -0700

Fix regression in WSGI server SIGHUP behavior

    The SIGHUP receipt used to pop us out of an os.wait() where now, it's in
    a "green" wait() and Timeout() combo, some part of which eats the signal
    receipt. This causes the while loop condition to never get checked and
    SIGHUP no longer works as a server reload command.

    The fix is to loop at least every 0.5 seconds, as a trade-off between
    not busy-waiting and checking the "keep running" condition often enough
    to feel responsive.

Change-Id: I95283b8b7cfc2998ab5813e0ad3ca1fa231696c8
Closes-Bug: #1479972

commit bcd00d9461603db1477c5f1e9f8dd6405a319eb9
Author: Alistair Coles <email address hidden>
Date: Mon Jun 8 19:40:56 2015 +0100

Refactor diskfile

    This patch mostly eliminates the duplicate code that was
    deliberately left in place during EC review to avoid major
    churn of the diskfile module prior to the kilo release.

    This focuses on obvious de-duplication and shuffling code
    between classes. It deliberately does not attempt to
    hammer out every last piece of de-duplication where that
    would introduce more complex changes - that can come later.

Code is moved from the module level and from ECDiskFile*
classes into new BaseDiskFile* classes.

Knock-on changes:

- fix bug whereby get_hashes was ignoring self.reclaim_age
and always using the default arg value.

- replication diskfile manager now deletes a tombstone that is older
than reclaim_age even when there is a newer .meta file.

    - replication diskfile manager will no longer raise an
      AssertionError if only a .meta file is found during
      hash_cleanup_listdir.

    - fix stale test in test_auditor.py: test_with_tombstone test
      setup was convoluted (probably dates back to when object puts
      did not clean up the object dir). Now that they do you have to
      try harder to create a dir with a tombstone and a data file.

Change-Id: I963e0d0ae0d6569ad1de605034c529529cbb4f9a

commit 9cb7eb4a4b6cdab8a5f16b3dc800b39ab4068522
Author: Victor Stinner <email address hidden>
Date: Mon Jul 27 11:34:07 2015 +0200

Update hacking to 0.10.0

    Replace the whitelist of flake8 checks (select) with a blacklist
    (ignore). It makes possible to disable a single check, which was not
    possible before. This new approach permits to enable new tests more
    easily and see which checks are currently disabled.

    Only new checks are disabled, this change doesn't run less checks than
    before. Currently, many checks are disabled, but following changes will
    ...

Reviewed:  https://review.openstack.org/208513
Committed: https://git.openstack.org/cgit/openstack/swift/commit/?id=942c9bb45c6b8124bcaa407eb9b9ac7f0589c743
Submitter: Jenkins
Branch:    feature/crypto

commit 207dd9b49d7d53a9faa4849af2c40bb875416fce
Author: Darrell Bishop <darrell@swiftstack.com>
Date:   Thu Jul 30 14:32:08 2015 -0700

Fix regression in WSGI server SIGHUP behavior
    
    The SIGHUP receipt used to pop us out of an os.wait() where now, it's in
    a "green" wait() and Timeout() combo, some part of which eats the signal
    receipt.  This causes the while loop condition to never get checked and
    SIGHUP no longer works as a server reload command.
    
    The fix is to loop at least every 0.5 seconds, as a trade-off between
    not busy-waiting and checking the "keep running" condition often enough
    to feel responsive.
    
    Change-Id: I95283b8b7cfc2998ab5813e0ad3ca1fa231696c8
    Closes-Bug: #1479972

commit bcd00d9461603db1477c5f1e9f8dd6405a319eb9
Author: Alistair Coles <alistair.coles@hp.com>
Date:   Mon Jun 8 19:40:56 2015 +0100

Refactor diskfile
    
    This patch mostly eliminates the duplicate code that was
    deliberately left in place during EC review to avoid major
    churn of the diskfile module prior to the kilo release.
    
    This focuses on obvious de-duplication and shuffling code
    between classes. It deliberately does not attempt to
    hammer out every last piece of de-duplication where that
    would introduce more complex changes - that can come later.
    
    Code is moved from the module level and from ECDiskFile*
    classes into new BaseDiskFile* classes.
    
    Concrete classes for replication and EC policy retain their
    existing names i.e. DiskFile[Manager|Writer|Reader|] and
    ECDiskFile[Manager|Writer|Reader|] respectively.
    
    Knock-on changes:
    
    - fix bug whereby get_hashes was ignoring self.reclaim_age
      and always using the default arg value.
    
    - replication diskfile manager now deletes a tombstone that is older
      than reclaim_age even when there is a newer .meta file.
    
    - replication diskfile manager will no longer raise an
      AssertionError if only a .meta file is found during
      hash_cleanup_listdir.
    
    - fix stale test in test_auditor.py: test_with_tombstone test
      setup was convoluted (probably dates back to when object puts
      did not clean up the object dir). Now that they do you have to
      try harder to create a dir with a tombstone and a data file.
    
    Change-Id: I963e0d0ae0d6569ad1de605034c529529cbb4f9a

commit 9cb7eb4a4b6cdab8a5f16b3dc800b39ab4068522
Author: Victor Stinner <vstinner@redhat.com>
Date:   Mon Jul 27 11:34:07 2015 +0200

Update hacking to 0.10.0
    
    Replace the whitelist of flake8 checks (select) with a blacklist
    (ignore).  It makes possible to disable a single check, which was not
    possible before. This new approach permits to enable new tests more
    easily and see which checks are currently disabled.
    
    Only new checks are disabled, this change doesn't run less checks than
    before. Currently, many checks are disabled, but following changes will
    fix these new warnings.
    
    When the "select" option of flake8 is used in tox.ini, the "ignore"
    option is ignored. It is not possible to use "select" and "ignore"
    options at the same time. See the code in pep8 (flake8 is based on
    pep8):
    
    https://github.com/PyCQA/pep8/blob/d48eef071003952ed0f98445dc3dc6bc29618f9c/pep8.py#L1808
    
    The purpose of flake8 is to enhance the coding style, but also to help
    to detect common bugs. Blacklisting checks permits to detect new bugs
    when upgrading the tools. For example, new checks were recently added to
    help porting code to Python 3.
    
    This change permits to use pbr 1.0 and newer which is required to
    support Python 3. Previously, hacking 0.8.1 was used, but hacking 0.8 is
    incompatible with pbr 1.0 and newer (hacking 0.8 requires pbr<1.0).
    
    pbr 1.0 is required to use environment markers on requirements to
    install dnspython on Python 2 and dnspython3 on Python 3.
    
    Change-Id: I41d63b9467083d7606ad18aaa398ca7738b27fae

commit 389a97e3526df68121cecfaf1909343b5d381cbc
Author: Victor Stinner <vstinner@redhat.com>
Date:   Wed Jul 29 23:46:58 2015 +0200

Fix pep8 E warning for hacking 0.10
    
    Fix pep8 warnings of the E category of hacking 0.10:
    
    * E113: unexpected indentation
    * E121: continuation line under-indented for hanging indent
    * E122: continuation line missing indentation or outdented
    * E123: closing bracket does not match indentation of opening bracket's
      line
    * E126: continuation line over-indented for hanging indent
    * E251: unexpected spaces around keyword / parameter equals
    
    Change-Id: I0b24eebdf1a37dc1b572b6c9a3d3d4832d050237

commit d719064e788ec5fa70305fadaa46affcb603a9bb
Author: Victor Stinner <vstinner@redhat.com>
Date:   Thu Jul 30 00:28:44 2015 +0200

Fix warning pep8 E128 warning of hacking 0.10
    
    Fix the warning E128: "continuation line under-indented for visual
    indent" of pep8.
    
    Change-Id: Ie6c6ae341fe3d6281f2095c1d756d552fa5937f9

commit a0db56dcde83096708e6fe64ae9338098e0ba98a
Author: Victor Stinner <vstinner@redhat.com>
Date:   Thu Jul 30 00:16:06 2015 +0200

Fix pep8 E265 warning of hacking 0.10
    
    Fix the warning E265 "block comment should start with '# '" added in pep
    1.5.
    
    Change-Id: Ib57282e958be9c7cddffc7bca34fbbf1d4c460fd

commit 345785837f1caa01fec5bca091c70481d7069bea
Author: Charles Hsu <charles0126@gmail.com>
Date:   Thu Jul 30 13:11:00 2015 +0800

Remove error_suppression_interval, error_suppression_limit options.
    
    These two options is belong to proxy-server, not account-replicator.
    
    Change-Id: Ie030ecffd213e56db32df77c69b847479d96308f

commit 5a7383313b1c555df5fd47050f7ee3810b5e31fc
Author: Carlos Cavanna <ccavanna@ca.ibm.com>
Date:   Tue Jul 28 19:15:25 2015 -0400

Swift documentation for first-time contributors.
    
    In this change, there are proposed improvement to Swift documentation aimed at
    first-time contributors. They include a simplification of the Getting
    Started page and a new page with some basic instructions/commands that the
    first-time contributor should know. In addition, it shows some common errors
    that the first-time contributor may find when executing git rebase and
    information on how to track your changes.
    
    Change-Id: I704202955093736b2f3b4102a649690a0392c6b0

commit 5b5c8820024141bc6ae49d7646a92fb53667ea9b
Author: Alistair Coles <alistair.coles@hp.com>
Date:   Wed Jul 29 12:36:35 2015 +0100

Remove unused code in swift/obj/server.py
    
    keep_cache was defined twice
    
    Change-Id: I5f775f618669e3dabfc3bc54dd7bd2cd7f00b2f0

commit 0c27cb07caf1027cd9734e8bd34d9759da33e667
Author: OpenStack Proposal Bot <openstack-infra@lists.openstack.org>
Date:   Wed Jul 29 06:36:01 2015 +0000

Imported Translations from Transifex
    
    For more information about this automatic import see:
    https://wiki.openstack.org/wiki/Translations/Infrastructure
    
    Change-Id: Ic30cb1dc8f12280a52cc1f07a1cf516397ed680e

commit 117b61976fbf14d8a0d2157256f9a3230421ee72
Author: Samuel Merritt <sam@swiftstack.com>
Date:   Wed Jul 22 15:40:55 2015 -0700

functests: use assertIn and assertNotIn
    
    We have a bunch of assertions like
    
        self.assertTrue(resp.status in (200, 204))
    
    Sometimes we get smart about failure messages and have something like
    
        self.assertTrue(resp.status in (200, 204), resp.status)
    
    so we can see what the status was when it failed.
    
    Since we don't have to support Python 2.6 any more, we can use
    assertIn/assertNotIn and get nice failure messages for free.
    
    Change-Id: I2d46c9969d41207a89e01017b4c2bc533c3d744f

commit 8fe8bee7e0c5c9d34d7a8e6ab090c8d65c5ca78e
Author: janonymous <janonymous.codevulture@gmail.com>
Date:   Tue Jul 21 18:06:32 2015 +0530

test/(functional, probe): Replace "self.assert_" by "self.assertTrue"
    
    The assert_() method is deprecated and can be safely replaced by assertTrue().
    This patch makes sure that running the tests does not generate warnings
    all over the screen.
    
    Change-Id: I74705c6498249337bfdf955d62e0ad972035bc1f

commit 4a6e0ccc69302a5384b988149e294656323d7f20
Author: Samuel Merritt <sam@swiftstack.com>
Date:   Mon Jul 20 11:58:35 2015 -0700

Rename WsgiStringIO -> WsgiBytesIO.
    
    If we're going to have a subclass of BytesIO, having "StringIO" in its
    name is just asking for confusion.
    
    Change-Id: I695ab3105b1a02eb158dcf0399ae91888bc1c0ac

commit 5ee742c17d9ea92ca7332cb67db0cce64b7a0d8f
Author: janonymous <janonymous.codevulture@gmail.com>
Date:   Fri Jul 24 21:19:47 2015 +0530

Modifying Lambda function tuple usage for python3
    
    Current tuple unpacking is not supported in python3
    
    Change-Id: I15a8e977684d19154cf2233caf1174f6983ac9e2

commit 6f89f71f9ba11f7b4126b80076a0f13792332e1e
Author: Alistair Coles <alistair.coles@hp.com>
Date:   Wed Apr 15 16:17:23 2015 +0100

Filter Etag key from ssync replication-headers
    
    ssync rx sends a header X-Backend-Replication-Headers whose value is a
    list of headers that the source object has. This list extends the list
    of allowed headers on the target object server, so that the target
    object metadata is faithfully reconstructed to match the source.
    
    Unfortunately the combination of lower() and title() operations on
    header keys results in the source 'ETag' value being added to the target
    metadata under the key 'Etag' in addition to the 'ETag' key that the
    receiving server adds (note different capitilization), both having
    the same value.
    
    The spurious 'Etag' metadata is potentially confusing for humans
    inspecting the object metadata and complicates tests that wish to
    assert the equality of two object metadata dicts. See for example the
    test in test_ssync_sender.py that this patch cleans up.
    
    Furthermore, the possibility of having both Etag and ETag keys has
    required a workaround in the EC reconstructor [1].
    
    [1] reconstructor fix change id: Ie59ad93a67a7f439c9a84cd9cff31540f97f334a
    
    Change-Id: I0c89cf7924a4471bb6d268b5ef3884e2d2cb4286

commit e2f69138bf5fcdd84ac23be6b0bec1bb006b41f5
Author: Clay Gerrard <clay.gerrard@gmail.com>
Date:   Thu Jul 23 16:46:48 2015 -0700

Fix string formatting in dispersion cli command
    
     ... and add a basic test that would have prevented the regression
    
    Change-Id: I4c5f643ee291dcc1397ca951450459d8b8ad0bbd

commit 83025ff38dfea3d4e3f27dc045128a7aa4ba11e4
Author: Clay Gerrard <clay.gerrard@gmail.com>
Date:   Fri Jul 24 11:09:10 2015 -0700

Use right most occurrence object dir in diskfile.extract_policy
    
      I like using the rightmost one more; it's basically
      /operator-defined/mountpoint/objects/part/suffix/hash/ts.data, so I
      don't see any opportunity for other things named "objects" to creep in on the
      right of the real objects-N dir; but I could see some admin using
      /srv/object-storage/ or something
    
        -- Torgomatic The Wise
    
    Change-Id: I0a63a3e02df091a5ee2e110a345183012e357a2f

commit 363a256e58ee0f3e9f059352c2de968e3dc398e3
Author: Falk Reimann <falk.reimann@sap.com>
Date:   Wed Jun 24 16:54:02 2015 +0200

Support keystone v3 domains in swift-dispersion
    
    This provides the capability to specify a project_name,
    project_domain_name and user_domain_name in /etc/swift/dispersion.conf.
    If this values are set in dispersion.conf they get populated to the
    swift-client.  With this it is possible to have a specific dispersion
    project specified, which is not the keystone default domain.  Changes
    were applied to swift-dispersion-populate and swift-dispersion-report.
    Relevant man pages, the example dispersion.conf and the admin guide were
    updated accordingly.
    
    DocImpact
    Closes-Bug: #1468374
    
    Change-Id: I0e716f8d281b4d0f510bc568bcee4a13fc480ff7

commit 3d9b2d077a157279fb6cd107a0c83727106af3ef
Author: Daisuke Morita <morita.daisuke@lab.ntt.co.jp>
Date:   Fri Jul 24 10:42:10 2015 +0900

Fix swift-object-info to show policy index at deep working directory
    
    If swift-object-info command is executed at deeper working directory
    than 'objects-*' directory, it cannot parse policy index from file path
    so it does not show appropriate policy index. This patch fixes this
    problem by simply extracting the full path of a target object file.
    
    Change-Id: Idb734106a44b6121119c9b1dc8cdaaf4c6c28c31
    Closes-Bug: 1469951

commit e24d7c36fa9758618ef88ed1a8e19f4d7bb0404f
Author: Victor Stinner <vstinner@redhat.com>
Date:   Mon May 25 18:26:38 2015 +0200

Use six to fix imports on Python 3
    
    Get configparser, queue, http_client modules from six.moves.
    
    Patch generated by the six_moves operation of the sixer tool:
    https://pypi.python.org/pypi/sixer
    
    Change-Id: I666241ab50101b8cc6f992dd80134ce27327bd7d

commit 964869accce8836bd2587c4b07559fb05fe580b2
Author: Atsushi SAKAI <sakaia@jp.fujitsu.com>
Date:   Fri Jul 24 17:09:48 2015 +0900

Fix six typos on swift documentation
    
    mechanisim => mechanism
        http://docs.openstack.org/developer/swift/cors.html
    overridde => override
       http://docs.openstack.org/developer/swift/deployment_guide.html
    extentsions => extensions
      http://docs.openstack.org/developer/swift/development_ondisk_backends.html
    reuqest => request
      http://docs.openstack.org/developer/swift/logs.html
    suport => support
      http://docs.openstack.org/developer/swift/overview_architecture.html
    mininum => minimum
      http://docs.openstack.org/developer/swift/overview_erasure_code.html
    
    $ git diff | diffstat
     cors.rst | 2 +-
     deployment_guide.rst | 2 +-
     development_ondisk_backends.rst | 2 +-
     logs.rst | 2 +-
     overview_architecture.rst | 2 +-
     overview_erasure_code.rst | 2 +-
     6 files changed, 6 insertions(+), 6 deletions(-)
    
    Change-Id: I8e095f4c216b2cfae48dff1e17d387048349f73c
    Closes-Bug: #1477877

commit f563a423bf2534268c2e785094d58dbbe0ebd390
Author: Hamdi Roumani <roumani@ca.ibm.com>
Date:   Wed Jul 22 15:27:26 2015 -0400

Updated tox setup instructions on Development Guidelines
    
    The instructions suggest updating the ownership of the swift.egg-info
    directory may be required and provide a cmd to do so.  Said cmd assumes user
    'swift' exists, but tox will require the current user to have access to the
    directory to perform updates, moreover user swift may not exist at all.
    
    Change-Id: I63951970553c086ff4f1ac65a81559ba69166494

commit 39621b8629c7f378fc197f6ab1ddc8e2e797d1fa
Author: janonymous <janonymous.codevulture@gmail.com>
Date:   Sat Jul 18 14:12:58 2015 +0530

Deprecated xreadlines() usage fixed
    
    Change-Id: Iadf42c2f86f78c11259e21c88b4aef51db3441ad

commit dde34c4b442a27761f2e6eca46ed9ec47a110392
Author: janonymous <janonymous.codevulture@gmail.com>
Date:   Sat Jul 11 22:46:41 2015 +0530

Modify zip usage for python3 where necessary.
    
    py2 zip() is eager but py3 zip() and six.moves.zip() are lazy,
    changing ones that require eager evaluation.
    
    Change-Id: Ic9f6bccd7f57772158581905794f8d23b05f4223

commit c366ecf5c85c2797582bf79f7faab1d75b99a70e
Author: janonymous <janonymous.codevulture@gmail.com>
Date:   Mon Jul 13 00:07:13 2015 +0530

Fix a minor python issue related with string.split
    
    Replacing string.split(value, ',') with value.split(','):
    the string.split() function was removed in Python 3
    
    Change-Id: I9ed241a894ac698da33df75a774a56ae2824fadf

commit 39b6ef6e4fd515d81e688ef365d26d5e0499be7c
Author: Charles Hsu <charles0126@gmail.com>
Date:   Thu Jun 25 02:06:54 2015 +0800

Fix reconstructor stats mssage.
    
    Calculate reconstruction job count and remaining time that
    would be inappropriate for user. Use real partition count would
    be suitable for user.
    
    Change-Id: I6b025854baf4757dddf9d7fe7bc2cece58a49157
    Closes-Bug: #1468298

commit bfbda38db999ccd881509c68f0319c3d3dae4274
Author: Clay Gerrard <clay.gerrard@gmail.com>
Date:   Tue Jul 7 15:13:20 2015 -0700

Add save command to ring-builder-analyzer
    
     * cleanup command => ring builder function mapping
     * cleanup ParseCommandError/ValueError message building
    
    Change-Id: I4be2aa963ce4f43035f02371d8388abd7a76536c

commit c907107fe4eacb378a92799de915234bd72d5d7c
Author: janonymous <janonymous.codevulture@gmail.com>
Date:   Tue Jul 7 22:46:37 2015 +0530

cPickle is deprecated in py3, replacing it from six.moves
    
    cPickle is deprecated and should be replaced with six.moves
    to provide py2 and py3 compatibility.
    
    Change-Id: Ibad990708722360d188c641e61444d50a16a1e93

commit 86033ba4dcd7b86670306b6e66345ae464caca9d
Author: Alistair Coles <alistair.coles@hp.com>
Date:   Tue Jun 30 18:56:23 2015 +0100

EC tests should use ECDiskFileManager
    
    Some EC tests in proxy/test_server.py use the replication
    DiskFileManager to open diskfiles. That works because it
    turns out that DiskFileManager is robust to the frag index
    being stitched onto the end of the timestamp in filenames,
    but it would be better to test using the ECDiskFileManager
    for EC diskfiles.
    
    Also, drive-by deletion of some duplicated lines.
    
    Change-Id: Ie3adb607d8c34a469df7d0f6f542ed5ba9a2dac0

tags:

added: in-feature-crypto

Thierry Carrez (ttx) on 2015-09-01

Changed in swift:
milestone:	none → 2.4.0
status:	Fix Committed → Fix Released

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2015-09-08: Fix proposed to swift (feature/hummingbird)

Fix proposed to branch: feature/hummingbird
Review: https://review.openstack.org/221410

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2015-09-08: Fix merged to swift (feature/hummingbird)

Download full text (70.7 KiB)

Reviewed: https://review.openstack.org/221410
Committed: https://git.openstack.org/cgit/openstack/swift/commit/?id=eb8f1f83f1cfc63d8452bc30096fd1c145781527
Submitter: Jenkins
Branch: feature/hummingbird

commit cb683d391cb66d0f52830de16760c80fd2afedf9
Author: OpenStack Proposal Bot <email address hidden>
Date: Sat Sep 5 06:17:51 2015 +0000

Imported Translations from Transifex

For more information about this automatic import see:
https://wiki.openstack.org/wiki/Translations/Infrastructure

Change-Id: I2d92b8e34a665fb0bb4c048cfb0c59de295dfce6

commit e4542455c8a07b7981c247df8b737816062c1655
Author: Emett Speer <email address hidden>
Date: Wed Sep 2 17:18:03 2015 -0700

[Labs] Update links to Cloud Admin Guide

Update links to the Cloud Admin Guide after the
RST conversion of that book altered URLs.

Change-Id: I899f8938498b744e62887968a65e58c00ef27f1b

commit 58fcc07523978306cd3889ada73af5d9e664cf59
Author: Christian Schwede <email address hidden>
Date: Wed Sep 2 10:52:34 2015 +0000

Test if container_sweep is executed on unmounted devices

This change ensures that container_sweep is not run if a device is not mounted
and mount_check is set to True.

Change-Id: I823083c8431d9e61fd426508033ec9188503957b

commit e02609c66a804845672413b06830b87395afef31
Author: Samuel Merritt <email address hidden>
Date: Tue Sep 1 15:19:50 2015 -0700

Preserve traceback in swift-dispersion-report

    Commit c690bcb fixed a bug in the dispersion report, but changed this
    from a bare "raise" to "raise err", which loses the traceback. Not a
    big deal, but worth putting back IMO.

Change-Id: Id5b72153a4b8df8e3faaf1fa3fb2040e28ba85cc

commit d06d4ad0fd2dfe69da8008e729651264522c6c06
Author: Minwoo Bae <email address hidden>
Date: Tue Sep 1 15:08:44 2015 -0500

Included reference in swift.obj.diskfile to enumerate the string
used for data file paths.

Change-Id: Ie22caa678bc00dfc43fabec7efbbb9f34490f1b5

commit 615c7a204b9386e05c5bab658bfe96766ad1e680
Author: Brian Cline <email address hidden>
Date: Tue Sep 1 10:51:20 2015 -0500

Adds useful dispersion info from changelog

Change-Id: I1a45088fc32620b02ff9a754b02ec1eb75a59d6e

commit 3b8755098a1786c5447abf158bd686293a82977c
Author: janonymous <email address hidden>
Date: Sun Aug 2 21:29:13 2015 +0530

Replace a / b with a // b to use integer division where needed

Change-Id: I72c81faa62786e140b0de00e3a04934bf1b5adbd

commit 524c89b7eeff037b8a6b421888771e15f98c2da2
Author: John Dickinson <email address hidden>
Date: Fri Aug 21 13:39:41 2015 -0700

Updated CHANGELOG, AUTHORS, and .mailmap for 2.4.0 release.

Change-Id: Ic6301146b839c9921bb85c4f4c1e585c9ab66661

commit 05de1305a903ee4ce9c8c50fde53c552d5b90d51
Author: Clay Gerrard <email address hidden>
Date: Thu Aug 27 18:35:09 2015 -0700

Make ssync_sender send valid chunked requests

    The connect method of ssync_sender tells the remote connection that it's
    going to send a valid HTTP chunked request, but if the remote end needs
    to respond with an error of any kind sender th...

Reviewed:  https://review.openstack.org/221410
Committed: https://git.openstack.org/cgit/openstack/swift/commit/?id=eb8f1f83f1cfc63d8452bc30096fd1c145781527
Submitter: Jenkins
Branch:    feature/hummingbird

commit cb683d391cb66d0f52830de16760c80fd2afedf9
Author: OpenStack Proposal Bot <openstack-infra@lists.openstack.org>
Date:   Sat Sep 5 06:17:51 2015 +0000

Imported Translations from Transifex
    
    For more information about this automatic import see:
    https://wiki.openstack.org/wiki/Translations/Infrastructure
    
    Change-Id: I2d92b8e34a665fb0bb4c048cfb0c59de295dfce6

commit e4542455c8a07b7981c247df8b737816062c1655
Author: Emett Speer <speer.emett@gmail.com>
Date:   Wed Sep 2 17:18:03 2015 -0700

[Labs] Update links to Cloud Admin Guide
    
    Update links to the Cloud Admin Guide after the
    RST conversion of that book altered URLs.
    
    Change-Id: I899f8938498b744e62887968a65e58c00ef27f1b

commit 58fcc07523978306cd3889ada73af5d9e664cf59
Author: Christian Schwede <cschwede@redhat.com>
Date:   Wed Sep 2 10:52:34 2015 +0000

Test if container_sweep is executed on unmounted devices
    
    This change ensures that container_sweep is not run if a device is not mounted
    and mount_check is set to True.
    
    Change-Id: I823083c8431d9e61fd426508033ec9188503957b

commit e02609c66a804845672413b06830b87395afef31
Author: Samuel Merritt <sam@swiftstack.com>
Date:   Tue Sep 1 15:19:50 2015 -0700

Preserve traceback in swift-dispersion-report
    
    Commit c690bcb fixed a bug in the dispersion report, but changed this
    from a bare "raise" to "raise err", which loses the traceback. Not a
    big deal, but worth putting back IMO.
    
    Change-Id: Id5b72153a4b8df8e3faaf1fa3fb2040e28ba85cc

commit d06d4ad0fd2dfe69da8008e729651264522c6c06
Author: Minwoo Bae <minwoob@us.ibm.com>
Date:   Tue Sep 1 15:08:44 2015 -0500

Included reference in swift.obj.diskfile to enumerate the string
    used for data file paths.
    
    Change-Id: Ie22caa678bc00dfc43fabec7efbbb9f34490f1b5

commit 615c7a204b9386e05c5bab658bfe96766ad1e680
Author: Brian Cline <bcline@softlayer.com>
Date:   Tue Sep 1 10:51:20 2015 -0500

Adds useful dispersion info from changelog
    
    Change-Id: I1a45088fc32620b02ff9a754b02ec1eb75a59d6e

commit 3b8755098a1786c5447abf158bd686293a82977c
Author: janonymous <janonymous.codevulture@gmail.com>
Date:   Sun Aug 2 21:29:13 2015 +0530

Replace a / b with a // b to use integer division where needed
    
    Change-Id: I72c81faa62786e140b0de00e3a04934bf1b5adbd

commit 524c89b7eeff037b8a6b421888771e15f98c2da2
Author: John Dickinson <me@not.mn>
Date:   Fri Aug 21 13:39:41 2015 -0700

Updated CHANGELOG, AUTHORS, and .mailmap for 2.4.0 release.
    
    Change-Id: Ic6301146b839c9921bb85c4f4c1e585c9ab66661

commit 05de1305a903ee4ce9c8c50fde53c552d5b90d51
Author: Clay Gerrard <clay.gerrard@gmail.com>
Date:   Thu Aug 27 18:35:09 2015 -0700

Make ssync_sender send valid chunked requests
    
    The connect method of ssync_sender tells the remote connection that it's
    going to send a valid HTTP chunked request, but if the remote end needs
    to respond with an error of any kind sender throws HTTP right out the
    window, picks up his ball, and closes the socket down hard - much to the
    surprise of the eventlet.wsgi server who up to this point had been
    playing along quite nicely with this 'SSYNC' nonsense assuming that
    everyone here is consenting mature adults.
    
    If you're going to make a "Transfer-Encoding: chunked" request have the
    good decency to finish the job with a proper '0\r\n\r\n'. [1]
    
    N.B. It might be possible to handle an error status during the
    initialize_request phase with some sort of 100-continue support, but
    honestly it's not entirely clear to me when the server isn't going to
    close the connection if the client is still expected to send the body
    [2] - further if the error comes later during missing_check or updates
    we'll for sure want to send the chunk transfer termination line before
    we close down the socket and this way we cover both.
    
    1. Really, eventlet.wsgi shouldn't be so blasted brittle about this [3]
    2. https://lists.w3.org/Archives/Public/ietf-http-wg/2005AprJun/0007.html
    3. https://github.com/eventlet/eventlet/commit/c3ce3eef0b4d0dfdbfb1ec0186d4bb204fb8ecd5
    
    Closes-Bug #1489587
    Change-Id: Ic17c6c3075553f8cf6ef6213e62a00282f0d01cf

commit 993ee4e37af1961adba2047d5aa2eb210e423eb3
Author: nakagawamsa <nakagawamsa@nttdata.co.jp>
Date:   Fri Aug 28 11:49:43 2015 +0900

Remove duplicate X-Backend-Storage-Policy-Index key
    
    There is duplicate 'X-Backend-Storage-Policy-Index' dictionary key in unit.obj.test_server.py.
    One key has fixed policy index value, and another has random value.
    Unittest should done with random policy index, so remove key which is set fixed value.
    
    Change-Id: Ic91fcf44d48297d0feee33c928ca682def9790a3

commit 893f30c61d280804e417790dd34ba7bc3fb4f6fc
Author: paul luse <paul.e.luse@intel.com>
Date:   Wed Aug 12 13:32:50 2015 -0700

EC GET path: require fragments to be of same set
    
    And if they are not, exhaust the node iter to go get more.  The
    problem without this implementation is a simple overwrite where
    a GET follows before the handoff has put the newer obj back on
    the 'alive again' node such that the proxy gets n-1 fragments
    of the newest set and 1 of the older.
    
    This patch bucketizes the fragments by etag and if it doesn't
    have enough continues to exhaust the node iterator until it
    has a large enough matching set.
    
    Change-Id: Ib710a133ce1be278365067fd0d6610d80f1f7372
    Co-Authored-By: Clay Gerrard <clay.gerrard@gmail.com>
    Co-Authored-By: Alistair Coles <alistair.coles@hp.com>
    Closes-Bug: 1457691

commit 58a10a5fffec69304d7bcce0f1c43bd2a9f9ff52
Author: Alistair Coles <alistair.coles@hp.com>
Date:   Wed Aug 26 16:30:23 2015 +0100

Add test that a tempurl POST cannot set a DLO manifest header
    
    Follow up to [1] to add tests for tempurl POSTs not being allowed
    to set a DLO manifest header.
    
    [1] I11e68830009d3f6bff44ae4011a41b67139146f6
    
    Change-Id: I7c0ad5a936f71e56c599b8495a586913d3334422
    Related-Bug: 1453948

commit d4409c0a046c6ce0e14e18c95efe2cd16cf120e8
Author: Samuel Merritt <sam@swiftstack.com>
Date:   Tue Aug 11 09:10:13 2015 -0500

Better scoping for tempurls, especially container tempurls
    
    It used to be that a GET of a tempurl referencing a large object would
    let you download that large object regardless of where its segments
    lived. However, this led to some violated user expectations around
    container tempurls.
    
    (Note on shorthand: all tempurls reference objects. However, "account
    tempurl" and "container tempurl" are shorthand meaning tempurls
    generated using a key on the account or container, respectively.)
    
    Let's say an application is given tempurl keys to a particular
    container, and it does all its work therein using those keys. The user
    expects that, if the application is compromised, then the attacker
    only gains access to the "compromised-container". However, with the old
    behavior, the attacker could read data from *any* container like so:
    
    1) Choose a "victim-container" to download
    
    2) Create PUT and GET tempurl for any object name within the
       "compromised-container". The object doesn't need to exist;
       we'll create it.
    
    3) Using the PUT tempurl, upload a DLO manifest with
       "X-Object-Manifest: /victim-container/"
    
    4) Using the GET tempurl, download the object created in step 3. The
       result will be the concatenation of all objects in the
       "victim-container".
    
    Step 3 need not be for all objects in the "victim-container"; for
    example, a value "X-Object-Manifest: /victim-container/abc" would only
    be the concatenation of all objects whose names begin with "abc". By
    probing for object names in this way, individual objects may be found
    and extracted.
    
    A similar bug would exist for manifests referencing other accounts
    except that neither the X-Object-Manifest (DLO) nor the JSON manifest
    document (SLO) have a way of specifying a different account.
    
    This change makes it so that a container tempurl only grants access to
    objects within its container, *including* large-object segments. This
    breaks backward compatibility for container tempurls that may have
    pointed to cross container *LO's, but (a) there are security
    implications, and (b) container tempurls are a relatively new feature.
    
    This works by having the tempurl middleware install an authorization
    callback ('swift.authorize' in the WSGI environment) that limits the
    scope of any requests to the account or container from which the key
    came.
    
    This requires swift.authorize to persist for both the manifest request
    and all segment requests; this is done by having the proxy server
    restore it to the WSGI environment prior to returning from __call__.
    
    [CVE-2015-5223]
    
    Co-Authored-By: Clay Gerrard <clayg@swiftstack.com>
    Co-Authored-By: Alistair Coles <alistair.coles@hp.com>
    Co-Authored-By: Christian Schwede <cschwede@redhat.com>
    Co-Authored-By: Matthew Oliver <matt@oliver.net.au>
    
    Change-Id: Ie6d52f7a07e87f6fec21ed8b0ec1d84be8b2b11c
    Closes-Bug: 1449212

commit 10b2939b433a4a79b4f7b97640b3d208cacfeffb
Author: Clay Gerrard <clay.gerrard@gmail.com>
Date:   Thu Jul 23 22:36:21 2015 -0700

Disallow unsafe tempurl operations to point to unauthorized data
    
    Do not allow PUT tempurls to create pointers to other data. Specifically
    disallow the creation of DLO object manifests by returning an error if a
    non-safe tempurl request includes an X-Object-Manifest header regardless of
    the value of the header.
    
    This prevents discoverability attacks which can use any PUT tempurl to probe
    for private data by creating a DLO object manifest and then using the PUT
    tempurl to head the object which would 404 if the prefix does not match any
    object data or form a valid DLO HEAD response if it does.
    
    This also prevents a tricky and potentially unexpected consequence of PUT
    tempurls which would make it unsafe to allow a user to download objects
    created by tempurl (even if they just created them) because the result of
    reading the object created via tempurl may not be the data which was uploaded.
    
    [CVE-2015-5223]
    
    Co-Authored-By: Kota Tsuyuzaki <tsuyuzaki.kota@lab.ntt.co.jp>
    
    Change-Id: I11e68830009d3f6bff44ae4011a41b67139146f6
    Closes-Bug: 1453948

commit c690bcb68331818a04c94741b25f40cc40f7b3c4
Author: Kazuhiro MIYAHARA <miyahara.kazuhiro@lab.ntt.co.jp>
Date:   Mon Aug 17 16:50:56 2015 +0900

Fix dispersion-reports error message
    
    This patch fixes Swift to show message
    "No objects to query. Has swift-dispersion-populate been run?"
    for "swift-dispersion-report —object-only”
    with no container for object dispersion.
    
    Change-Id: I82da56709cfc296a27f5180681709bc56adbc13d
    Closes-Bug: #1468120

commit 8086a0e53406dc95856df2d3e015d3eaaf81380a
Author: Tushar Gohad <tushar.gohad@intel.com>
Date:   Tue Aug 18 07:24:19 2015 +0000

Restrict PyECLib version to 1.0.7
    
    v1.0.9 rev of PyECLib replaces Jerasure with a native EC
    implementation (liberasurecode_rs_vand) as the default
    EC scheme.  Going forward, Jerasure will not be bundled
    with PyPI version of PyECLib as it used to be, until
    v1.0.7.
    
    This is an interim change to Swift requirements until we
    get v1.0.9 PyECLib included into global-requirements and
    ready patches that change Swift default ec_type (for doc,
    config samples and unit tests) from "jerasure_rs_vand"
    to "liberasurecode_rs_vand."
    
    Change-Id: Ica4fee2cdea2bc7f5edd0e51ad637a4457faf3b4

commit 17efa343c605d0361b3f423696babbab3f3d972d
Author: Kota Tsuyuzaki <tsuyuzaki.kota@lab.ntt.co.jp>
Date:   Mon Jul 6 13:21:40 2015 -0700

Fix EC GET backend stream iteration state
    
    In EC case, When GET object requested, proxy-server always makes a log
    line "Client disconnected on read" even though the request succeeded.
    
    That is because ECAppIter class doesn't maintain a bunch of backend
    stream when closing the app_iter. It will cause unfortunately
    GeneratorExit on backend stream ResumingGetter.
    
    This patch fixes to set non_client_disconnected to propagate the state
    to the backend streams when the range iteration stopped successful.
    
    Co-Authored-By: Clay Gerrard <clay.gerrard@gmail.com>
    
    Change-Id: I77af9807816bea1444d66534a17e2a210bcf09f8
    Closes-Bug: #1472201

commit 25dc7224b63b039cb3778cca7f9f81f954c6d94e
Author: Alistair Coles <alistair.coles@hp.com>
Date:   Tue Aug 25 11:05:41 2015 +0100

Fix swob.Range docstring
    
    Bad ranges cause a ValueError to be raised, not an empty
    ranges list.
    
    Change-Id: I118bd2f7dc08ff5198870f4093c6eb350506c8ed

commit a7b84f4c51207608adc3a5de7f4f9a629053c0ab
Author: Samuel Merritt <sam@swiftstack.com>
Date:   Mon Aug 24 23:11:34 2015 -0700

Allow pep8 of a single file
    
    Now you can run
    
       $ tox -e pep8 path/to/file.py [path/to/file2.py [...]]
    
    to run pep8 against just those files[1]. This is quite a bit faster
    than a full pep8 run, and the faster feedback is nice when you're
    fiddling with some formatting to placate pep8.
    
    Of course, you can still run "tox -e pep8" to check the whole source
    tree, just as before this commit.
    
    [1] It'll still run against bin/swift* as well, but that's still a lot
    faster than running against all our .py files.
    
    Change-Id: I81b4363fb95a34ff0f5c346b2b24f2047154f502

commit 6a35d479e8952c854fee6fbef9fe0397a289a6e5
Author: Matthew Oliver <matt@oliver.net.au>
Date:   Tue Aug 25 11:24:49 2015 +1000

Follow up patch to fix a multiline import NITPIC
    
    This change cleans up test/unit/obj/test_replicator.py's imports
    to use only 1 version of multiline import syntaxes (' \' vs '()').
    I don't really mind which, but we should be consistant, at least
    in the same file.
    
    This is a follow up for patch 215857.
    
    Change-Id: Ie2d328c25865b19092c493981a803ee246a9d7a5

commit edde5584affaa983d8db1d294bf1e20a2d4bbb50
Author: Hisashi Osanai <osanai.hisashi@jp.fujitsu.com>
Date:   Tue Aug 25 07:52:18 2015 +0900

Fix typo of a comment in replicator
    
    The typo was introduced by patch 138342 (sorry) so I fix it.
    
    Change-Id: Id5126802d281ef7ee9be128bd2152c0d2584160e

commit 6151c6c45da481c0ea9a4fb876d14b39c1e347bf
Author: Tim Burke <tim.burke@gmail.com>
Date:   Mon Aug 24 20:36:22 2015 +0000

Fix use of delimiter in account listings
    
    Previously, account listings that used the delimiter query param could
    omit some containers if they ended with the character that follows the
    delimiter.
    
    See If196e3075612b121ef8da4a9128167d00a248c27 for the corresponding fix
    for container listings.
    
    Change-Id: I57fcb97e51f653f5f4e306a632fcb3a0fb148c4e

commit a38f63e1c6b8b85b1675aa900e239a2e9906811e
Author: Clay Gerrard <clay.gerrard@gmail.com>
Date:   Fri Aug 21 18:14:55 2015 -0700

Use correct Storage-Policy header for REPLICATE requests
    
    Under some concurrency the object-replicator could potentially send the
    wrong X-Backed-Storage-Policy-Index header to it's partner nodes during
    replication if there were multiple storage policies on the same node
    because of a race where multiple jobs being processed concurrently would
    mutate some shared state on the ObjectReplicator instance.
    
    Instead of using shared stated on the ObjectReplicator instance when
    mutating the default headers send with REPLICATION requests each job
    will copy them into a local where they can safely be updated.
    
    Change-Id: I5522db57af7e308b1f9d4181f14ea14e386a71fd

commit ae129bf5e8c918efd8ee66279f98ee85e7c97b24
Author: Kota Tsuyuzaki <tsuyuzaki.kota@lab.ntt.co.jp>
Date:   Fri May 15 06:44:12 2015 -0700

eventlet.posthook related code needn't anymore
    
    In the past, older Swift seemed to use eventlet.posthooks scheme for the
    logging on proxy with posthooklogger method. However, for now, Swift
    attaches no method to eventlet.posthooks everywhere so we don't have
    to maintain the posthooks anymore.
    
    Change-Id: Ie63941a202d448532d980252a4d25575f8edab9c

commit 4500ff340f021e4e5f6fd53960de03cb3c207925
Author: Kota Tsuyuzaki <tsuyuzaki.kota@lab.ntt.co.jp>
Date:   Wed May 13 00:43:59 2015 -0700

Fix the missing SLO state on fast-post
    
    When using fast-post and POST (i.e. metadata update) is requested to
    a SLO manifest files, current Swift drops the 'X-Static-Large-Object'
    header from the existing metadata. It results in breaking the SLO
    state because the manifest missing the 'X-Static-Large-Object' metadata
    will be maintained as a normal files.
    
    This patch fixes object-server to keep the existing
    'X-Static-Large-Object' flag and then keep the SLO state.
    
    Change-Id: Ib1eb569071372c322dd105c52baeeb094003291e
    Closes-bug: #1453807

commit 69e7424d3cb6c59341d739c3d41f42397031d96a
Author: Eran Rom <eranr@il.ibm.com>
Date:   Tue Jul 28 09:41:12 2015 +0300

Add container sync probe test to SAIO default set
    
    SAIO Configuration and documentation changes enabling to run the
    container sync probe test by default
    
    Change-Id: Iccf59533d0d4fe72549d318339ab125d04dde006
    Related-Bug: #1476623

commit 8b1df9918bf6c22b49b161a402199b532bfd8266
Author: Clay Gerrard <clay.gerrard@gmail.com>
Date:   Fri Aug 21 14:04:46 2015 -0700

Minor cleanup handoff mode warnings
    
      * message is a little clearer
      * test is a little stronger
    
    Change-Id: I745cde7f4a46dafc80ab42d39e6ccc92aa3b746e

commit a1ceab5a92dfdafd5abffed272ac2cece5302f32
Author: Carlos Cavanna <ccavanna@ca.ibm.com>
Date:   Fri Aug 21 14:14:31 2015 -0400

New troubleshooting case in documentation.
    
    Added a new troubleshooting case for the "First Contribution to Swift"
    documentation page.
    
    Change-Id: I182ba702b49b28409fe56becae93326e5f63dcd0

commit ab163702de733be39ba5e7024c7a8dd4c86bc29b
Author: Pradeep Kumar Singh <pradeep.singh@nectechnologies.in>
Date:   Tue Jul 14 10:03:18 2015 +0530

Emit warning log in object replicator
    
    When the object-replicator encounters handoffs_first and
    handoff_delete options as enabled it should emit a log
    warning indicating that it should be changed back to the
    default before the next "normal" rebalance.
    
    Closes-Bug: #1457262
    
    Change-Id: If9dc2796c18ed3cf13da920831e2d5c2ae9f12a0

commit be66aa8e76d2994eb8a0a944e0dea26ffbb3c159
Author: Samuel Merritt <sam@swiftstack.com>
Date:   Thu Aug 20 13:24:38 2015 -0700

Fix 500 for bogus Range request to 0-byte object.
    
    The proxy was trying to pop a byterange off a Range header that didn't
    contain syntactically-valid byteranges. This worked about as well as
    you'd expect. Now we detect the bogus value and remove the header
    entirely.
    
    Change-Id: I24b92f900d33ec79880c7db2870378489d5a6810

commit 4b6836b3bc465e9c2c6d2974c193a49b973c4e4e
Author: Akihito Takai <takaiak@nttdata.co.jp>
Date:   Thu Aug 20 18:40:52 2015 +0900

Fix the comment in [1].
    
    (line 259) parameter(op) of object_update method is 'PUT' or
    'DELETE' not 'POST' or 'DELETE'.
    
    [1]: swift/obj/updater.py
    
    Change-Id: I876a620ba8e09e69fba7156b12e69445c229e160

commit 923238aa1ba0963c414ba5321cd3910b2910f4ed
Author: janonymous <janonymous.codevulture@gmail.com>
Date:   Tue Jul 28 20:35:25 2015 +0530

test/(functional/probe):Replace python print operator with print function (pep H233, py33)
    
    'print' function is compatible with 2.x and 3.x python versions
    Link : https://www.python.org/dev/peps/pep-3105/
    
    Python 2.6 has a __future__ import that removes print as language syntax,
    letting you use the functional form instead
    
    Change-Id: I416c6ac21ccbfb91ec328ffb1ed21e492ef52d58

commit 47dc31940d64b0c87e99cc73eddcdd01dd8b3ad4
Author: Samuel Merritt <sam@swiftstack.com>
Date:   Wed Aug 19 10:42:28 2015 -0700

Add OpenStack release names to changelog
    
    I can never remember the mapping of real version numbers to the
    OpenStack names.
    
    Change-Id: Ib7c5ae4ff2a33018364698eb34f8df1622719fab

commit 183508710d011ec30ebba2186c58fd28d449e4bb
Author: Victor Stinner <vstinner@redhat.com>
Date:   Sun Aug 16 11:18:49 2015 +0200

pep8: Don't override '_' symbol
    
    Fix pep8 warning F402 "import '_' from line 51 shadowed by loop
    variable".
    
    Change-Id: I139060ff5d298a8b0f0f8e529a5737478fb5daf5

commit c7eb589c6ca2fe4f2950cae22a18513e4dec6c46
Author: Victor Stinner <vstinner@redhat.com>
Date:   Sun Aug 16 11:06:52 2015 +0200

pep8: Fix usage of the l10n _('...') function
    
    Fix the pep8 warning H702 "Formatting operation should be outside of
    localization method call".
    
    For the logger, pass parameters as indexed parameters instead of using
    the string str%args operator, the logger is more reliable in case of
    formatting error.
    
    Change-Id: If418bc155f6a6c0a00f63e3d87ebe4addf4aae55

commit 7bea148d2fe22daa9d1fbbc5cd16ebd390b64b1b
Author: Victor Stinner <vstinner@redhat.com>
Date:   Sun Aug 16 10:34:26 2015 +0200

pep8: replace deprecated calls to assert_()
    
    The TestCase.assert_() has been deprecated in Python 2.7. Replace it
    with assertTrue() or even better methods (assertIn, assertNotIn,
    assertIsInstance) which provide better error messages.
    
    Change-Id: I21c730351470031a2dabe5238693095eabdb8964

commit 8aaacbf88d9ea4432e58dd261b98ff95d7c69be6
Author: Victor Stinner <vstinner@redhat.com>
Date:   Mon Jul 27 18:55:01 2015 +0200

pep8: Fix hacking H232 warnings (octal)
    
    Fix warnings "H232: Python 3.x incompatible octal 000001234 should be
    written as 0o1234".
    
    Change-Id: I9a7bbb034357783885ac3e18fe1e9e32a5951616

commit 5b01c5eadc3a13607d2581a55007700fca0b00af
Author: Bill Huber <wbhuber@us.ibm.com>
Date:   Fri Aug 7 16:24:28 2015 -0500

Add unit test for container_update for unmounted device
    
    This unit test case gains coverage for container/update.py
    where a device (i.e. sda1) is not mounted and the code
    responds with a warning logger that the device is not
    mounted and continues.  Coverage increases 2% as a result.
    
    Change-Id: I33d247a930b28604093df4ade1ce7dbbd24aac54

commit d6267c1f95a7fcd2b0e4f15d5efc6bcfe2114de0
Author: Clément Contini <ccontini@cloudops.com>
Date:   Wed Aug 12 15:00:45 2015 -0400

Keep user id and project id in subrequests env
    
    Keep HTTP_X_USER_ID and HTTP_X_PROJECT_ID to be available as
    user_id and project_id in storage.objects.outgoing.bytes in
    ceilometer when downloading a multipart object.
    
    Change-Id: I0f4734f021e5d6e84d48ed9bebeb321d7a9590ad
    Closes-Bug: #1477283

commit eaa006464cb98a2f36809edfa13bd3dcaebc9952
Author: John Dickinson <me@not.mn>
Date:   Mon Aug 17 22:13:42 2015 -0700

move global statement up a few lines
    
    Change-Id: I190d2d530c6c0525d988cc88b0965b48a291fffb

commit 1a81cda8b7eba0d25ea7341da756bb588cae8d73
Author: Hamdi Roumani <roumani@ca.ibm.com>
Date:   Mon Aug 3 17:45:56 2015 -0400

Doc instructions for post rebase steps
    
    Improve the 'first_contribution_swift' by adding instructions for how to
    rebuild swift following a rebase.
    
    Change-Id: If5c91dc4e1e8d1712bbd8b326c675967fb4b8c15

commit 79ba4a85983641e539b620bd143e62673c98416e
Author: Hisashi Osanai <osanai.hisashi@jp.fujitsu.com>
Date:   Wed Dec 3 06:15:16 2014 +0900

Enable Object Replicator's failure count in recon
    
    This patch makes the count of object replication failure in recon.
    And "failure_nodes" is added to Account Replicator and
    Container Replicator.
    
    Recon shows the count of object repliction failure as follows:
    $ curl http://<ip>:<port>/recon/replication/object
    {
        "replication_last": 1416334368.60865,
        "replication_stats": {
            "attempted": 13346,
            "failure": 870,
    	"failure_nodes": {
                "192.168.0.1": {"sdb1": 3},
                "192.168.0.2": {"sdb1": 851,
                                "sdc1": 1,
                                "sdd1": 8},
                "192.168.0.3": {"sdb1": 3,
                                "sdc1": 4}
    	},
            "hashmatch": 0,
            "remove": 0,
            "rsync": 0,
            "start": 1416354240.9761429,
            "success": 1908
        },
        "replication_time": 2316.5563162644703,
        "object_replication_last": 1416334368.60865,
        "object_replication_time": 2316.5563162644703
    }
    
    Note that 'object_replication_last' and 'object_replication_time' are
    considered to be transitional and will be removed in the subsequent
    releases. Use 'replication_last' and 'replication_time' instead.
    
    Additionaly this patch adds the count in swift-recon and it will be
    showed as follows:
    $ swift-recon object -r
    ========================================================================
    =======
    --> Starting reconnaissance on 4 hosts
    ========================================================================
    =======
    [2014-11-27 16:14:09] Checking on replication
    [replication_failure] low: 0, high: 0, avg: 0.0, total: 0, Failed: 0.0%,
    no_result: 0, reported: 4
    [replication_success] low: 3, high: 3, avg: 3.0, total: 12,
    Failed: 0.0%, no_result: 0, reported: 4
    [replication_time] low: 0, high: 0, avg: 0.0, total: 0, Failed: 0.0%,
    no_result: 0, reported: 4
    [replication_attempted] low: 1, high: 1, avg: 1.0, total: 4,
    Failed: 0.0%, no_result: 0, reported: 4
    Oldest completion was 2014-11-27 16:09:45 (4 minutes ago) by
    192.168.0.4:6002.
    Most recent completion was 2014-11-27 16:14:19 (-10 seconds ago) by
    192.168.0.1:6002.
    ========================================================================
    =======
    
    In case there is a cluster which has servers, a server runs with this
    patch and the other servers run without this patch. If swift-recon
    executes on the server which runs with this patch, there are unnecessary
    information on the output such as [failure], [success] and [attempted].
    Because other servers which run without this patch are not able to
    send a response with information that this patch needs.
    Therefore once you apply this patch, you also apply this patch to other
    servers before you execute swift-recon.
    
    DocImpact
    Change-Id: Iecd33655ae2568482833131f422679996c374d78
    Co-Authored-By: Kenichiro Matsuda <matsuda_kenichi@jp.fujitsu.com>
    Co-Authored-By: Brian Cline <bcline@softlayer.com>
    Implements: blueprint enable-object-replication-failure-in-recon

commit eeb0fa40a19917e6548f95f0bd3c08736928449b
Author: Christian Schwede <cschwede@redhat.com>
Date:   Thu Aug 6 07:21:15 2015 +0000

Make swift-ring-builder filename usage more consistent
    
    Sometimes the given argument is internally altered and another filename is used
    without a note to the operator. Even worse, a given .ring.gz filename is
    sometimes written out as builder file, without updating the corresponding
    .builder file.
    
    There is already a method to parse the given argv and return the name of the
    builder and ring file. However, it's rarely used and no warning is given to the
    user if it is altered. This patch uses the already parsed builder and ring file
    name instead of argv[1], and also adds a note to the user if the used filename
    is differently to the one given as argument.
    
    Closes-Bug: 1482096
    Change-Id: I2f8ef23aeab8b07caaa799f7dcd57e684b4b2ad2

commit b75d2a4e37d4c86763a2cc56c6dd53ebe2e0de19
Author: Bill Huber <wbhuber@us.ibm.com>
Date:   Mon Aug 17 13:54:44 2015 -0500

Quorum on durable response is too low
    
    Increase the .durable quorum from 2 to "parity + 1" to guarantee
    that we will never fail to rebuild an object.  Otherwise, with
    low durable responses back (< parity + 1), the putter objects
    return with failed attribute set to true, thereby failing the
    rebuild of fragments for an object.
    
    Change-Id: I80d666f61273e589d0990baa78fd657b3470785d
    Closes-Bug: 1484565

commit fa35e38c9f83f704eb087f80157cbe33b23b9db2
Author: Kai Zhang <zakir.exe@gmail.com>
Date:   Fri Aug 14 16:46:35 2015 -0700

Fix some minor typos
    
    Fixed some typos in function name and comments.
    
    Change-Id: Ida76ab4b331a51b71e57650702acc136e66ba4b2

commit 5b246e875fe9ac3d764ea581ad52b04238f5bcc8
Author: Kota Tsuyuzaki <tsuyuzaki.kota@lab.ntt.co.jp>
Date:   Wed Jul 22 10:39:22 2015 -0700

Fix EC range GET/COPY handling
    
    When range GET (or COPY) for an EC object requested, if the requested range
    starts from more than last segments alignment (i.e.
    ceil(object_size/segment_size) * segment_size), proxy server will return
    the original content length w/o body, though Swift should return an error
    massage as a body and the length of message as the content length.
    The current behavior will cause stuck on some client. (e.g. curl)
    
    This patch fixes that proxy enables to return correct response, even if such
    an over range requested.
    
    Co-Authored-By: Clay Gerrard <clay.gerrard@gmail.com>
    
    Change-Id: I21f81c842f563ac4dddc69011ed759b744bb20bd
    Closes-Bug: #1475499

commit 7064706b2721c36c5181267f58704b9344ad4ac0
Author: Samuel Merritt <sam@swiftstack.com>
Date:   Tue Aug 11 10:17:20 2015 -0500

Fix 500 in versioned writes with bad Destination
    
    When this code lived in the proxy, it was protected by an "except
    HTTPException" clause in proxy.Application.handle_request(). When it
    moved to middleware, it lost that, and then things like
    constraints.check_name_format that raised HTTPException would cause
    500s. The HTTPException would make it all the way out to catch_errors
    and get translated to a 500.
    
    This commit just wraps a couple try/excepts around the bits in
    versioned writes that can raise HTTPException. I tried to make it use
    wsgify so I could get that for free, but that wound up being a real
    pain because env/start_response are plumbed through pretty much the
    whole versioned-writes middleware.
    
    Closes-Bug: 1483705
    
    Change-Id: Ife165bf709e64f313ed07c779b21914045e51f25

commit 035a411660ca02983cd486312266c67d78a2359c
Author: Thiago da Silva <thiago@redhat.com>
Date:   Sun Nov 9 13:13:27 2014 -0500

versioned writes middleware
    
    Rewrite object versioning as middleware to simplify the PUT method
    in the object controller.
    
    The functionality remains basically the
    same with the only major difference being the ability to now
    version slo manifest files. dlo manifests are still not
    supported as part of this patch.
    
    Co-Authored-By: Clay Gerrard <clay.gerrard@gmail.com>
    
    DocImpact
    Change-Id: Ie899290b3312e201979eafefb253d1a60b65b837
    Signed-off-by: Thiago da Silva <thiago@redhat.com>
    Signed-off-by: Prashanth Pai <ppai@redhat.com>

commit 968c91a465627bb9317ed6e2ae9ccd818c83c0c4
Author: Alistair Coles <alistair.coles@hp.com>
Date:   Tue Jul 28 10:31:54 2015 +0100

Replace assertTrue(not <condition>) with assertFalse(<condition>)
    
    The replacement of assert_ with assertTrue [1] resulted in a number of
    tests using calls of the form assertTrue(not <condition>). This patch
    replaces those with assertFalse(<condition>).
    
    [1] change I74705c6498249337bfdf955d62e0ad972035bc1f
    
    Change-Id: I78b49558f4425c2335df187b1793d1e4b3c514b1

commit 4ac1fea5d111c669ff827f4eb29c0735cbad6ba5
Author: Zhao Lei <zhaolei@cn.fujitsu.com>
Date:   Fri Aug 7 22:07:01 2015 +0800

Fix some spelling typo in comments
    
    s/overide/override for object-expirer.conf and sample.
    s/automaticaly/automatically for swift/proxy/controllers/obj.py
    
    Change-Id: Ife107c7a1005a5d4959288db50a7f8f33c522dd4
    Signed-off-by: Zhao Lei <zhaolei@cn.fujitsu.com>

commit 9456af35a2832aeafc4a4e78ebd0d4142ead71cd
Author: janonymous <janonymous.codevulture@gmail.com>
Date:   Thu Aug 6 00:55:36 2015 +0530

pep8 fix: assertEquals -> assertEqual
    
    assertEquals is deprecated in py3,changes in
    dir:
    *test/unit/obj/*
    *test/unit/test_locale/*
    
    Change-Id: I3dd0c1107165ac529f1cd967363e5cf408a1d02b

commit 1952451ed7e97e2bfa3b11dfa5000cc39f712084
Author: Zhao Lei <zhaolei@cn.fujitsu.com>
Date:   Fri Aug 7 21:57:08 2015 +0800

Fix a spelling typo in comment
    
    s/automaticaly/automatically/ for swift/proxy/controllers/obj.py
    
    Change-Id: I405441383739637e0b13746d6f4e2bad1c874b4d
    Signed-off-by: Zhao Lei <zhaolei@cn.fujitsu.com>

commit c35cc13b8abeb97400632cbe8ec56fc1040d8210
Author: Bill Huber <wbhuber@us.ibm.com>
Date:   Thu Aug 6 10:01:17 2015 -0500

pep8 fix: assertEquals -> assertEqual
    
    assertEquals is deprecated in py3 in the following dir:
    test/unit/proxy/*
    
    Change-Id: Ie2c7e73e1096233a10ee7fbf6f88386fa4d469d6

commit 239e94e62518f36601fb18119700269753d02a01
Author: Bill Huber <wbhuber@us.ibm.com>
Date:   Thu Aug 6 09:28:51 2015 -0500

pep8 fix: assertEquals -> assertEqual
    
    assertEquals is deprecated in py3 in the following dir:
    test/probe/*
    
    Change-Id: Ie08dd7a8a6c48e3452dfe4f2b41676330ce455d5

commit 49b9ba37ac522502ccad66fba7bfef375eab145d
Author: Bill Huber <wbhuber@us.ibm.com>
Date:   Wed Aug 5 14:51:32 2015 -0500

pep8 fix: assertEquals -> assertEqual
    
    assertEquals is deprecated in py3 in the following dir:
    test/functional/*
    
    Change-Id: Iee7f8ffca9838ccc521107180697d91ac9559405

commit 81816bebe6b3506af8cabaa7ebe265c70f3499cd
Author: kenichiro matsuda <matsuda_kenichi@jp.fujitsu.com>
Date:   Thu Aug 6 10:13:02 2015 +0900

Fix shebang of commands
    
    Fix shebang of following commands.
      $ grep '#!/usr/bin/python' swift/bin/*
      swift/bin/swift-account-info:#!/usr/bin/python
      swift/bin/swift-container-info:#!/usr/bin/python
      swift/bin/swift-container-sync:#!/usr/bin/python
      swift/bin/swift-recon:#!/usr/bin/python
      swift/bin/swift-ring-builder:#!/usr/bin/python
      swift/bin/swift-ring-builder-analyzer:#!/usr/bin/python
    
    Change-Id: I564d1d8abd76eba57730fc2f30263b0a0f809867
    Closes-Bug: #1481623

commit 652f0f9da408e955843f87a504b3dab5be3cff80
Author: Pete Zaitcev <zaitcev@kotori.zaitcev.us>
Date:   Wed Aug 5 15:33:29 2015 -0600

Having said H, I, J, we ought to say K
    
    In the long run, we might want to stop these updates. But this
    decision is above my pay grade.
    
    Change-Id: I335558e1da8052be1b215fbad51244b47af3d81b

commit ed3aec2146b114c85bb3b196c999bf3f44fe159e
Author: janonymous <janonymous.codevulture@gmail.com>
Date:   Thu Aug 6 00:18:52 2015 +0530

pep8 fix: assertEquals -> assertEqual
    
    assertEquals is deprecated in py3 in
    dir: test/unit/container/*
    
    Change-Id: I3333022ed63ce03198bc73147246d91d2442a440

commit f449e914728fdcaf0870c15b9fc5ebd16a9df916
Author: janonymous <janonymous.codevulture@gmail.com>
Date:   Wed Aug 5 22:32:02 2015 +0530

pep8 fix: assertEquals -> assertEqual
    
    assertEquals is deprecated in py3, fixing in:
    dir: test/unit/cli/*
    
    Change-Id: I9a2fc1f717beafd5fa8408942046e310e8de0318

commit 6594bbebb582801c069c4966352fee19eadef2f8
Author: janonymous <janonymous.codevulture@gmail.com>
Date:   Wed Aug 5 22:25:46 2015 +0530

pep8 fix: assertEquals -> assertEqual
    
    assertEquals is deprecated in py3 in
    dir : test/unit/account/*
    
    Change-Id: I70415197df1b0d58d4d6e219733c13a9c9cdfff7

commit 89f59062864e5cbfc839a6084c323ce35438aa57
Author: Ben Martin <blmartin@us.ibm.com>
Date:   Mon Jul 27 14:19:09 2015 -0500

+Document method to avoid rsync filling root drive
    
    When rsync pushes to a remote node with an unmounted drive and if
    certain steps are not taken, rsync may attempt to write files to
    the local drive at the location where the drive was mounted.
    
    There are two suggested solutions for this issue:
      1) Set the permissions for all mount points in /srv/node/
           to root:root 755
      2) Mount the drives elsewhere and symlink the drives to /srv/.../
    
    The first method ensures that only root and not the swift user
    can write in the /srv/.../ directories.
    
    The second method will prompt a broken link issue if rsync
    attempts to write to an unmounted drive.
    
    Change-Id: I60ce4ed9ef8401768d5f78b6806cbb2e2a65303e
    Closes-Bug: #1470576

commit f0d51882b921075f5309ed6ff07385dc6103d4dc
Author: Charles Hsu <charles0126@gmail.com>
Date:   Wed Aug 5 22:09:40 2015 +0800

Add extra_header_count to document and config.
    
    Change-Id: Iec86b488d71553c295afe7098822ce2046df9546

commit 7071762d3698c59fabdee76890e300f81417be74
Author: Clay Gerrard <clay.gerrard@gmail.com>
Date:   Tue Aug 4 23:15:37 2015 -0700

Fix TypeError if backend response doesn't have expected headers
    
    There was some debug logging mixed in with some error handling on PUTs
    that was relying on a very specific edge would only encounter a set of
    backend responses that included the expected set of headers to diagnoise
    the failure.
    
    But the backend responses may not always have the expected headers.
    
    The proxy debug logging should be more robust to missing headers.
    
    It's a little hard to follow, but if you look `_connect_put_node` in
    swift.proxy.controller.obj - you'll see that only a few connections can
    make their way out of the initial put connection handling with a "resp"
    attribute that is not None.  In the happy path (e.g. 100-Continue) it's
    explictly set to None, and in most errors (Timeout, 503, 413, etc) a new
    connection will be established to the next node in the node iter.
    
    Some status code will however allow a conn to be returned for validation
    in `_check_failure_put_connections`, i.e.
    
      * 2XX (e.g. 0-byte PUT would not send Expect 100-Continue)
      * 409 - Conflict with another timestamp
      * 412 - If-None-Match that encounters another object
    
    ... so I added tests for those - fixing a TypeError along the way.
    
    Change-Id: Ibdad5a90fa14ce62d081e6aaf40aacfca31b94d2

commit 01848c565e5821d2500cb234261c70871e562ebe
Author: David Goetz <dpgoetz@gmail.com>
Date:   Fri Jun 26 17:27:31 2015 -0700

Speed up reaper for a big account delete and some better error handling
    
    In run_forever mode, run the reaper on all three copies of the account
    DB instead of just the first one and shard the containers between them.
    
    Change-Id: Id182e89dad70df9d6d9988523861b8085d74c765

commit 5b24b2249878e25ee702ee4237227c984036c6ec
Author: OpenStack Proposal Bot <openstack-infra@lists.openstack.org>
Date:   Tue Aug 4 06:29:11 2015 +0000

Imported Translations from Transifex
    
    For more information about this automatic import see:
    https://wiki.openstack.org/wiki/Translations/Infrastructure
    
    Change-Id: I9f0e5fd25ec143fb87de1895794764dea4587f93

commit 8d06f56c1fac23bca463d3a93353da6b50ad5bed
Author: Takashi NATSUME <natsume.takashi@lab.ntt.co.jp>
Date:   Tue Aug 4 10:14:25 2015 +0900

Fix typo of 'receive'
    
    Change-Id: Ic01defa8d48ba4b7f4bd6cbe4d59b9bab468a36c

commit 310ac18d61b001bdf297505243d8aa62e8fde8fe
Author: Brian Reitz <brian.reitz@oracle.com>
Date:   Mon Aug 3 16:28:31 2015 -0500

Convert docstring to a comment in test_denied_DELETE_of_versioned_object unit test.
    
    Docstrings cause issues for nose and created non standard test output for this test.
    
    Change-Id: I91e6450480567de0ee0b18d3efae01a4b94f0a86

commit 21a7b4aaa6f991c0eeb8f74876b14fe22713586b
Author: Alistair Coles <alistair.coles@hp.com>
Date:   Mon Aug 3 14:58:24 2015 +0100

Test that get_hashes ignores only removed hash dir
    
    Add test for case not yet covered by unit tests: suffix dir
    has two hash dirs, one with expired tombstone. That hash dir gets
    removed and its hash is not included in the suffix hash, but the
    remaining hash dir's hash is reported.
    
    Change-Id: I031a022daed6b8a66dfd04bea1b4d5eebcb882b3

commit 44917c8a90e79a85cd6c6821d8b6d043f83b322d
Author: Clay Gerrard <clay.gerrard@gmail.com>
Date:   Mon Jul 20 13:06:48 2015 -0700

Handle removed suffix dirs the same as empty suffix dirs
    
    When hashes suffix dirs, the directory might have gotten cleaned up while
    still appearing in hashes.pkl.  It would even get cleaned up the next time.
    
    For example, given this really old tombstone:
    
        objects/846/3d0/d3a20154d0a828a032aba6860397c3d0/1432267961.41808.ts
    
    Prior to this commit, a call to get_hashes() would reap the old
    tombstone and any empty containing dirs, but the resulting hashes.pkl
    would still contain {'3d0': 'd41d8cd98f00b204e9800998ecf8427e'} even
    though there's no such suffix dir any more.
    ("d41d8cd98f00b204e9800998ecf8427e" is the MD5 of the empty string.)
    
    Then, a *subsequent* get_hashes() call would omit 3d0 from the resulting
    hash, so then hashes.pkl would no longer contain 3d0. This difference
    would result in a little useless replication traffic while nodes without
    a particular part/suffix pair, but who disagreed on how that showed up
    in hashes.pkl, tried to push their version of nothing to one another.
    
    Now, an empty suffix dir doesn't appear in hashes.pkl at all, whether
    it's for replication or EC, or whether it's for the get_hashes() call
    that reaped the suffix dirs or not.
    
    Co-Author: Samuel Merritt <sam@swiftstack.com>
    
    Change-Id: Ie1bfb1cc56d0fc030c6db42f97b55d140695cf1f

commit cf4d50bd68b658d35759fb56446557858dcd9572
Author: Alistair Coles <alistair.coles@hp.com>
Date:   Fri Jun 5 13:06:29 2015 +0100

Make test_proxy work independent of env vars
    
    test.unit.common.test_internal_client.TestSimpleClient.test_proxy
    will fail if the environment has http_proxy set and does not have
    no_proxy=127.0.0.1 set. This seems to be because urllib overrides
    any proxy arg passed to the Request with the env http_proxy var,
    unless the Request host is in no_proxy.
    
    We don't need to test urllib, so this patch changes the test to
    simply ensure that swift code does pass the correct proxy arg to
    urllib2.urlopen. That avoids testers having to make sure their
    env vars are compliant to successfully run unit tests. While it is
    reasonable to think that 127.0.0.1 is in the no_proxy list, it
    shouldn't be a requirement pass swift tests.
    
    Change-Id: Iff91acdb76fabe7056b3e90e2756e27fe5105817

commit 67513fc17c23066f6547b5e3c4ab2dc852e4f7f0
Author: janonymous <janonymous.codevulture@gmail.com>
Date:   Sat Jun 27 23:26:47 2015 +0530

Adding bandit for security static analysis testing in swift
    
    This change adds a basic bandit config for Swift. It can be invoked
    by running the tox environment for bandit;
        tox -e bandit
    
    This is an initial step for using bandit with Swift
    and it should be revisited to improve the testing as more is learned
    about the specific needs of the Swift code base.As per now some tests
    are excluded as they were used on purpose.
    
    https://wiki.openstack.org/wiki/Security/Projects/Bandit
    
    Implements: blueprint swift-bandit
    
    Change-Id: I621be9a68ae9311f3a6eadd1636b05e646260cf2

commit 768d7ab074807175471d589de96d091e3239052e
Author: Clay Gerrard <clay.gerrard@gmail.com>
Date:   Thu Jul 30 15:30:35 2015 -0700

Add a probetest for HUP/reload
    
    This would have been enough to catch the regression, and we can extend
    them as we work on any future ehancements to our process management.
    
    Change-Id: I9a1b57aa15663380c45cf783afc8212ab4ffbace