downloading through proxy not working

Bug #484735 reported by Huub Bouma
18
This bug affects 4 people
Affects Status Importance Assigned to Milestone
Buildout
New
Undecided
Unassigned

Bug Description

I'm working on a suse unix system, and I have an environment variable:

http_proxy=http://user:passwd@localhost:11233/

When I do a python ./bootstrap, I get the following error:

Traceback (most recent call last):
  File "/tmp/tmpXwJvtd/zc.buildout-1.4.2-py2.4.egg/zc/buildout/buildout.py", line 1659, in main
    user_defaults, windows_restart, command)
  File "/tmp/tmpXwJvtd/zc.buildout-1.4.2-py2.4.egg/zc/buildout/buildout.py", line 182, in __init__
    data['buildout'].copy(), override))
  File "/tmp/tmpXwJvtd/zc.buildout-1.4.2-py2.4.egg/zc/buildout/buildout.py", line 1333, in _open
    eresult = _open(base, extends.pop(0), seen, dl_options, override)
  File "/tmp/tmpXwJvtd/zc.buildout-1.4.2-py2.4.egg/zc/buildout/buildout.py", line 1333, in _open
    eresult = _open(base, extends.pop(0), seen, dl_options, override)
  File "/tmp/tmpXwJvtd/zc.buildout-1.4.2-py2.4.egg/zc/buildout/buildout.py", line 1283, in _open
    path, is_temp = download(filename)
  File "/tmp/tmpXwJvtd/zc.buildout-1.4.2-py2.4.egg/zc/buildout/download.py", line 96, in __call__
    local_path, is_temp = self.download(url, md5sum, path)
  File "/tmp/tmpXwJvtd/zc.buildout-1.4.2-py2.4.egg/zc/buildout/download.py", line 173, in download
    tmp_path, headers = urllib.urlretrieve(url, tmp_path)
  File "/usr/lib64/python2.4/urllib.py", line 85, in urlretrieve
    return _urlopener.retrieve(url, filename, reporthook, data)
  File "/usr/lib64/python2.4/urllib.py", line 218, in retrieve
    fp = self.open(url, data)
  File "/usr/lib64/python2.4/urllib.py", line 186, in open
    return getattr(self, name)(url)
  File "/usr/lib64/python2.4/urllib.py", line 309, in open_http
    h.endheaders()
  File "/usr/lib64/python2.4/httplib.py", line 795, in endheaders
    self._send_output()
  File "/usr/lib64/python2.4/httplib.py", line 676, in _send_output
    self.send(msg)
  File "/usr/lib64/python2.4/httplib.py", line 643, in send
    self.connect()
  File "/usr/lib64/python2.4/httplib.py", line 611, in connect
    socket.SOCK_STREAM):
IOError: [Errno socket error] (-2, 'Name or service not known')

After a little debugging it seems that urllib.urlretrieve seems to be the culprit. It can't handle this kind of proxy setting. I have also tried this in a little standalone python script, and it has the same bad result.

urllib2 handles the proxy setting just fine. So if I change this line:

  tmp_path, headers = urllib.urlretrieve(url, tmp_path)

into this:

  import urllib2
  tmp_sock = urllib2.urlopen(url)
  tmp_file = open(tmp_path, 'w')
  tmp_file.write(tmp_sock.read())
  tmp_file.close()

then all works fine.
Now this code might not be great, but is it possible that urllib2 will be used as a workaround for this proxy issue?

Cheers, Huub

Revision history for this message
Clayton (claytonc-sousa) wrote :

This error is not detected in machines with python2.4.

The error is detected in zc.buildout 1.4.4 and 1.5.2 with python2.6.

When I use cntlm version 0.35 I dont't detect the problem, but when i use cntlm version > 0.9 the error occurs.

With ntlmaps it's working.

I did what is described above and it worked.

Clayton

Revision history for this message
Clayton (claytonc-sousa) wrote :

The above tests were done on Ubuntu 11.04 and 12.04.

Clayton

Revision history for this message
Marius Gedminas (mgedmin) wrote :

This sounds like a bug in the Python standard library. Have you reported it at http://bugs.python.org, or found an existing bug report for this issue? Can you please provide a link to it here?

Revision history for this message
Alessandro Pisa (alessandro-pisa) wrote :
Revision history for this message
Huub Bouma (bouma-w20e) wrote :

The problem I think is that I used a proxy with authentication, which is not supported in urllib:

"Proxies which require authentication for use are not currently supported; this is considered an implementation limitation."

http://docs.python.org/2/library/urllib.html

Revision history for this message
Marius Gedminas (mgedmin) wrote :

Still, the error message from urllib is misleading. Is it trying to resolve a host named 'user:password@localhost' and failing? Why doesn't urllib.urlopen() fail in the same fashion?

Please file a urllib bug at http://bugs.python.org/

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.