#!/usr/bin/env python breaks Python-based Ubuntu packages in the presence of virtualenvs, local installations

David, thank you for the bug report. Such a report needs to be assigned to the individual packages to make the appropriate maintainers aware of the problem. I have done so now.

To the HP developers at HP: Please use "#!/usr/bin/python" in your Python executables, the distro's Python interpreter is always at that path, or use "@PYTHONPATH@" and let the "configure" script determine the correct Python path and replace the placeholder with it.

I would like to second this request as in Fedora we also change the shebang
'#!/usr/bin/env python' to '#!/usr/bin/python'
in python executables when building hplip packages.
For more details see
http://fedoraproject.org/wiki/Features/SystemPythonExecutablesUseSystemPython

Thanks.

--
Jiri

This bug was fixed in the package hplip - 3.11.10-1ubuntu4

---------------
hplip (3.11.10-1ubuntu4) precise; urgency=low

  * debian/rules: Use "#!/usr/bin/python" instead of "#!/usr/bin/env python"
    to specify the path to the Python interpreter, to assure that always
    the system's interpreter is used and never a possibly incompatible
    alternative Python interpreter (LP: #912625).
 -- Till Kamppeter <email address hidden> Fri, 6 Jan 2012 09:37:00 +0100

OdyX, can you have a look at the Debian packages of c2esp, foo2zjs, and pyppd? Thanks.

I might take a look at c2esp, foo2zjs and pyppd in due time.

So far I'm not sure this bug is as uncontroversial as it might seem and would really prefer a discussion about it on <email address hidden> before rushing out any actions. The Debian Python Policy currently says "Maintainers _should_ not use env ..": this doesn't mean it's (currently) forbidden; this even means it discouraged, but allowed.

I don't think creating a Ubuntu-specific diff for this type of bug is the way to go: either there is a valid reason to do it and it should be done in Debian first (with appropriate Debian Python Policy change, lintian warning, etc) or there is none and, well, the cost of the diff will have to be carried by Ubuntu hands. (Note that given bugs on the respective packages, such a change could be done in very few days in Debian.) Not doing it in "Debian first" now means that it might eventually be done there later and that a merge will be needed; that sounds like unneccessary work on both sides (and I like to avoid unneccessary work).

In particular, this topic was previously discussed there: http://lists.debian.org/debian-python/2009/09/msg00132.html and there seemed to be reasonable agreement around a lintian warning and/or python policy amendment (although without actions); this should really come before blindly fixing all affected packages in Ubuntu only. (IMHO, eh).

Cheers, OdyX

This isn't a security issue. A regular user can't modify which python gets used by other users.

/usr/bin/env is a perfectly good solution for *development* branches of packages, but very definitely not for deployed production versions of applications, for exactly the reasons described in this bug report. Meaning: if you are developing a Python application, by all means use /usr/bin/env in your own code, since this will make it easier to test against a variety of Python versions. But packaging should always install the application using the explicit path to the appropriate Python executable. I'm pretty sure distribute and setuptools do this munging automatically.

I'm not convinced this is a bug at all. Debian Python policy (which Ubuntu uses) gives a preference to /usr/bin/python*, but says /usr/bin/env python is allowable. You shouldn't put a non-system python in your system's python path.

Marc: Fair enough. I guess the same kind of hijacking I mentioned could be accomplished in a lot of ways, including setting PYTHONPATH, so it's probably alright.

Scott: The Debian policy reads:

> Maintainers should not override the Debian Python interpreter using /usr/bin/env

The word "should" is somewhat ambiguous in its level of severity, but I would read that as a "strongly discouraged" even if not a hard and fast "you must not do this". I would say there ought to be a really good reason if a system-installed executable is figuring out which interpreter to use at runtime.

> You shouldn't put a non-system python in your system's python path.

I assume you mean on your shell's PATH as the PYTHONPATH is something different -- at any rate, this is an unworkable demand for just about anyone who does anything resembling serious Python development. In addition to virtualenv being a ubiquitous tool for deployment management and environment isolation, several specialized Python distributions exist (both commercial and FLOSS) such as Enthought Python Distribution, ActivePython, FEMhub, Sage, etc. and isolate themselves from the system Python (as they should).

Placing the bin directory of one of these distributions, or of a virtualenv, on your shell's PATH (i.e. adding it to your PATH in ~/.bash_profile) should not cause random system-installed executable scripts to start breaking, and I would very much consider it a bug in the package that installed the executable if this does happen.

Furthermore, most of the Python scripts in /usr/bin on my machine follow the "hard code which interpreter you want" convention, and as Barry pointed out above, even the native Python packaging system, broken as it is in many ways, performs this kind of munging. I would consider this a strong case for not using #!/usr/bin/env python.

The mercurial task is fixed in precise:

$ head -1 /usr/bin/hg-ssh
#!/usr/bin/python

Can someone please check this is still an issue? I though we are meant to use /env shebangs now