Comment 9 for bug 1326813

While this bug describes an old issue it still happens these days, even after PYPI migrated to a CDN for serving its wheels.

Behaviour changed a little bit since original times because now pip supports retries which works really nicesly, unles... the CDN randomly gives a 404 response for stuff that should never get a 404, like core python packages, ones that installed on most virtual environments like pip, pbr,,..

We all know that this is lie, as these packages never vanished from the pypi server, still one response like this would ruin even a 4 hour build job on CI.

Using proxies, transparent or not for accelerating access pypi does not help because they will respect the 404 response.

By HTTP specification 404 is not retriable so what we need is to add an optional feature in pip that would enable PIP_RETRY_ON_404, just like a timeout error.

This would address the issue of flakiness. I know this for sure because I did something similar for python-jira library where Atlassian Cloud was randomly giving invalid HTTP codes. In that case I was able to subclass python requests session class to add retries, and it worked very well.

I thik we need to raise a bug on pip for this to track it there because that is the only place wheere this can really be fixed. We cannot really fix CDNs or people mirrors, but we probably can make pip more resilient.