python-pip >= 21.3.1+dfsg-3 fails pip3-root.sh autopkgtest with python3.10

As I understand it, this is because pip expects the installation layouts in sysconfig to be correct since Python 3.10. Previously it used the layouts in distutils.sysconfig.

See:
* LP: #1940705
* LP: #1408092
* https://github.com/pypa/pip/pull/10358

The correct resolution here is to enable sysconfig-debian-schemes.diff in Python 3.10. I'm not sure why this hasn't been enabled, yet.

I talked with doko about that, and he told me to give it a try. The patch does not apply directly, I had to rebase it: it seems to fix this issue (i.e.: I can install a package to the right location with my new python 3.10 package here [1]) but then when installed, it fails to build python-pip package with the following error:

make[1]: Leaving directory '/tmp/autopkgtest.hzRvXB/build.kr1/real-tree'
   dh_installsystemduser -O--buildsystem=pybuild
   dh_lintian -O--buildsystem=pybuild
   dh_perl -O--buildsystem=pybuild
   dh_usrlocal -O--buildsystem=pybuild
dh_usrlocal: error: debian/python3-pip/usr/local/bin/pip is not a directory
make: *** [debian/rules:8: binary] Error 255
dpkg-buildpackage: error: debian/rules binary subprocess returned exit status 2
blame: python-pip-22.0.2+dfsg/
badpkg: rules build failed with exit code 2

I attach my rebase that may be wrong if you can take a look.

(I'll send the same info by mail as requested by doko.)

[1] https://launchpad.net/~alexghiti/+archive/ubuntu/riscv/+sourcepub/13261732/+listing-archive-extra

OK, that seems to be caused by the default scheme changing from posix_prefix to posix_local, dh-python was making silly assumptions, there.

Your updated sysconfig patch looks good.

This bug was fixed in the package dh-python - 5.20220215

---------------
dh-python (5.20220215) unstable; urgency=medium

  * pybuild plugin: Explicitly unpack wheels using the "deb_system" sysconfig
    scheme, allowing python3.10 to change the default sysconfig scheme to
    "posix_local". (LP: #1960608)

 -- Stefano Rivera <email address hidden> Tue, 15 Feb 2022 20:26:39 -0400

I'm not sure I understand: do I need to enable the sysconfig patch for python3.10 in addition to your fix?

Thanks

Yes, it's required. My fix was resolving the python-pip FTBFS you reported with that patch.

python3.10 now in -proposed

Hello, there are also bugs in python.m4, the same fix as https://github.com/pybind/pybind11/pull/3764/files fixes the issue in m4 too

I patched locally my python.m4 with something like this:

- sitedir = sysconfig.get_path('purelib', vars={'base':'$am_py_prefix'})
- sitedir = sysconfig.get_path('purelib', scheme='posix_prefix', vars={'base':'$am_py_prefix'})

- sitedir = sysconfig.get_path('platlib', vars={'platbase':'$am_py_exec_prefix'})
to
+ sitedir = sysconfig.get_path('platlib', scheme='posix_prefix', vars={'platbase':'$am_py_exec_prefix'})

And firewalld was building correctly again.

This bug was fixed in the package automake-1.16 - 1:1.16.5-1.1ubuntu1

---------------
automake-1.16 (1:1.16.5-1.1ubuntu1) jammy; urgency=medium

  * Upload to jammy with a fix for LP: #1960608

 -- Gianfranco Costamagna <email address hidden> Fri, 04 Mar 2022 10:29:43 +0100

<BTS> Opened #1006784 in src:automake-1.16 1:1.16.5-1.1 by Gianfranco Costamagna (locutusofborg) «automake: fixes for python3.10 distutils changes». https://bugs.debian.org/1006784

Please have a look at it if possible.

python3.10 was fixed in 3.10.2-4, which is in the release pocket.

pip was fixed by fixing python3.10 and dh-python.

This new sysconfig-debian-schemes.diff has broken virtual environments (venv and virtualenv). Please take a look at the fix in https://bugs.launchpad.net/ubuntu/+source/python3.10/+bug/1962791.