Comment 29 for bug 1813725

Reviewed: https://review.opendev.org/749532
Committed: https://git.openstack.org/cgit/openstack/swift/commit/?id=db485392fc238b32e29c8aca12d2e70dd9561004
Submitter: Zuul
Branch: stable/stein

commit db485392fc238b32e29c8aca12d2e70dd9561004
Author: Tim Burke <email address hidden>
Date: Mon Jul 20 14:18:33 2020 -0700

    py3: Work with proper native string paths in crypto meta

    Previously, we would work with these paths as WSGI strings -- this would
    work fine when all data were read and written on the same major version
    of Python, but fail pretty badly during and after upgrading Python.

    In particular, if a py3 proxy-server tried to read existing data that
    was written down by a py2 proxy-server, it would hit an error and
    respond 500. Worse, if an un-upgraded py2 proxy tried to read data that
    was freshly-written by a py3 proxy, it would serve corrupt data back to
    the client (including a corrupt/invalid ETag and Content-Type).

    Now, ensure that both py2 and py3 write down paths as native strings.
    Make an effort to still work with WSGI-string metadata, though it can be
    ambiguous as to whether a string is a WSGI string or not. The heuristic
    used is if

     * the path from metadata does not match the (native-string) request
       path and
     * the path from metadata (when interpreted as a WSGI string) can be
       "un-wsgi-fied" without any encode/decode errors and
     * the native-string path from metadata *does* match the native-string
       request path

    then trust the path from the request. By contrast, we usually prefer the
    path from metadata in case there was a pipeline misconfiguration (see
    related bug).

    Add the ability to read and write a new, unambiguous version of metadata
    that always has the path as a native string. To support rolling
    upgrades, a new config option is added: meta_version_to_write. This
    defaults to 2 to support rolling upgrades without configuration changes,
    but the default may change to 3 in a future release.

    UpgradeImpact
    =============
    When upgrading from Swift 2.20.0 or Swift 2.19.1 or earlier, set

        meta_version_to_write = 1

    in your keymaster's configuration. Regardless of prior Swift version, set

        meta_version_to_write = 3

    after upgrading all proxy servers.

    When switching from Python 2 to Python 3, first upgrade Swift while on
    Python 2, then upgrade to Python 3.

    Change-Id: I00c6693c42c1a0220b64d8016d380d5985339658
    Closes-Bug: #1888037
    Related-Bug: #1813725
    (cherry picked from commit 7d429318ddb854a23cdecfe35721b1ecbe8bcccc)