Privsep uses system libraries instead of the ones in virtual env

Related bug: https://bugs.launchpad.net/cinderlib/+bug/1958159

Fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/cinderlib/+/847170

Reviewed: https://review.opendev.org/c/openstack/cinderlib/+/847170
Committed: https://opendev.org/openstack/cinderlib/commit/4d784d23a91f789918929ca397910f3121fa166e
Submitter: "Zuul (22348)"
Branch: master

commit 4d784d23a91f789918929ca397910f3121fa166e
Author: Gorka Eguileor <email address hidden>
Date: Wed Jun 22 14:58:55 2022 +0200

    Fix privsep issues inside virtual env

    When a virtual environment is created with the "--system-site-packages"
    option and privsep is installed on the system privsep will only use the
    system packages and completely ignore the ones in the virtual
    environment.

    This results in errors such as the ones we see:

    - In the Ussuri gate: ModuleNotFoundError: No module named
      'os_brick.privileged.rootwrap'

    - In the Wallaby gate: ModuleNotFoundError: No module named
      'os_brick.privileged.nvmeof'

    This happens because os-brick and cinder are starting privsep using the
    "privsep-helper" mechanism, and privsep was not installed in the virtual
    env because it was already present system wide, so the "privsep-helper"
    that is executed is the one from "/usr/local/bin/privsep-helper".

    This python script "privsep-helper" ignores the virtual environment and
    forces usage of the system's python, for example in a Wallaby
    installation this could be "#!/usr/bin/python3.6".

    Since it ignores the virtual environment it won't use its packages and
    anything that's not present on system wide will not be found, and if
    found it may be executing different code.

    This patch fixes this issue by replacing the helper used to start
    privsep with our own command.

    This command is the same as the one usually installed in /usr/local/bin
    but using /usr/bin/env to select the python to use.

    This new script has been included as data in the cinderlib namespace
    instead of making it install as a system script (like the original
    privsep command) because we don't want to polute the system wide
    binaries directory just for a corner case.

    We also need to preserve user site-packages for the running Python when
    calling root from the virtual environment, since the packages installed
    on the virtual environment with "--system-site-packages" would have
    taken those into consideration during the installation and not the ones
    present on the root user.

    To help debug issues at the gate all functional tests are now running
    with debug logs.

    Change-Id: I0278b42785d14f92a521e6deff872dcba6505270
    Related-Bug: #1958159
    Closes-Bug: #1979534

Fix proposed to branch: stable/xena
Review: https://review.opendev.org/c/openstack/cinderlib/+/847622

Fix proposed to branch: stable/wallaby
Review: https://review.opendev.org/c/openstack/cinderlib/+/847623

This issue was fixed in the openstack/cinderlib 4.2.0 release.

Reviewed: https://review.opendev.org/c/openstack/cinderlib/+/847622
Committed: https://opendev.org/openstack/cinderlib/commit/4fc56c815b0e9e4d3706dca14bbe54fbb4866272
Submitter: "Zuul (22348)"
Branch: stable/xena

commit 4fc56c815b0e9e4d3706dca14bbe54fbb4866272
Author: Gorka Eguileor <email address hidden>
Date: Wed Jun 22 14:58:55 2022 +0200

    Fix privsep issues inside virtual env

    When a virtual environment is created with the "--system-site-packages"
    option and privsep is installed on the system privsep will only use the
    system packages and completely ignore the ones in the virtual
    environment.

    This results in errors such as the ones we see:

    - In the Ussuri gate: ModuleNotFoundError: No module named
      'os_brick.privileged.rootwrap'

    - In the Wallaby gate: ModuleNotFoundError: No module named
      'os_brick.privileged.nvmeof'

    This happens because os-brick and cinder are starting privsep using the
    "privsep-helper" mechanism, and privsep was not installed in the virtual
    env because it was already present system wide, so the "privsep-helper"
    that is executed is the one from "/usr/local/bin/privsep-helper".

    This python script "privsep-helper" ignores the virtual environment and
    forces usage of the system's python, for example in a Wallaby
    installation this could be "#!/usr/bin/python3.6".

    Since it ignores the virtual environment it won't use its packages and
    anything that's not present on system wide will not be found, and if
    found it may be executing different code.

    This patch fixes this issue by replacing the helper used to start
    privsep with our own command.

    This command is the same as the one usually installed in /usr/local/bin
    but using /usr/bin/env to select the python to use.

    This new script has been included as data in the cinderlib namespace
    instead of making it install as a system script (like the original
    privsep command) because we don't want to polute the system wide
    binaries directory just for a corner case.

    We also need to preserve user site-packages for the running Python when
    calling root from the virtual environment, since the packages installed
    on the virtual environment with "--system-site-packages" would have
    taken those into consideration during the installation and not the ones
    present on the root user.

    To help debug issues at the gate all functional tests are now running
    with debug logs.

    Change-Id: I0278b42785d14f92a521e6deff872dcba6505270
    Related-Bug: #1958159
    Closes-Bug: #1979534
    (cherry picked from commit 4d784d23a91f789918929ca397910f3121fa166e)

Reviewed: https://review.opendev.org/c/openstack/cinderlib/+/847623
Committed: https://opendev.org/openstack/cinderlib/commit/1b58a5fb9af8d0beca964eee91f0e0040f21cb23
Submitter: "Zuul (22348)"
Branch: stable/wallaby

commit 1b58a5fb9af8d0beca964eee91f0e0040f21cb23
Author: Gorka Eguileor <email address hidden>
Date: Wed Jun 22 14:58:55 2022 +0200

    Fix privsep issues inside virtual env

    When a virtual environment is created with the "--system-site-packages"
    option and privsep is installed on the system privsep will only use the
    system packages and completely ignore the ones in the virtual
    environment.

    This results in errors such as the ones we see:

    - In the Ussuri gate: ModuleNotFoundError: No module named
      'os_brick.privileged.rootwrap'

    - In the Wallaby gate: ModuleNotFoundError: No module named
      'os_brick.privileged.nvmeof'

    This happens because os-brick and cinder are starting privsep using the
    "privsep-helper" mechanism, and privsep was not installed in the virtual
    env because it was already present system wide, so the "privsep-helper"
    that is executed is the one from "/usr/local/bin/privsep-helper".

    This python script "privsep-helper" ignores the virtual environment and
    forces usage of the system's python, for example in a Wallaby
    installation this could be "#!/usr/bin/python3.6".

    Since it ignores the virtual environment it won't use its packages and
    anything that's not present on system wide will not be found, and if
    found it may be executing different code.

    This patch fixes this issue by replacing the helper used to start
    privsep with our own command.

    This command is the same as the one usually installed in /usr/local/bin
    but using /usr/bin/env to select the python to use.

    This new script has been included as data in the cinderlib namespace
    instead of making it install as a system script (like the original
    privsep command) because we don't want to polute the system wide
    binaries directory just for a corner case.

    We also need to preserve user site-packages for the running Python when
    calling root from the virtual environment, since the packages installed
    on the virtual environment with "--system-site-packages" would have
    taken those into consideration during the installation and not the ones
    present on the root user.

    To help debug issues at the gate all functional tests are now running
    with debug logs.

    Change-Id: I0278b42785d14f92a521e6deff872dcba6505270
    Related-Bug: #1958159
    Closes-Bug: #1979534
    (cherry picked from commit 4d784d23a91f789918929ca397910f3121fa166e)
    (cherry picked from commit 4fc56c815b0e9e4d3706dca14bbe54fbb4866272)