NotImplementedError: "should resend request to http://feeds.edge.launchpad.net/bazaar/, but this isn't implemented" during lp name lookup

Tried without proxy, it works... I think it's a duplicate of #186920

This apparently happens when the XML-RPC request from the lp plugin gets a 301 response, presumably the Location header in that response should be tried.

Does Launchpad really want us to get bzr stuff from
feeds.edge.launchpad.net? It seems like a server side error.

Lorenzo, if you can reproduce this, can you please try again with the
-Dhttp option and attach the log file from that.

It seem so, it's perfectly reproducible when http_proxy/https_proxy is set. I'll attach the log

Thanks, that helps a lot; it's interesting that someone else hit this recently.

I see you have a proxy at 192.168.0.222, and we're trying to CONNECT through it, which is ok. But this request looks wrong to me:

0.829 > POST /bazaar/
0.829 > Content-Length: 164
> Connection: Keep-Alive
> Accept: */*
> User-Agent: bzr/2.1.0 (urllib)
> Host: 192.168.0.222:3128
> Pragma: no-cache
> Cache-Control: max-age=0
> Content-Type: text/xml

Surely the 'Host' header should be xmlrpc.edge.launchpad.net, not that of the proxy?

I think Launchpad is then being a bit weird by sending a redirect to a nonexistent feeds url rather than just giving an error that there is no host 192.168.0.222. Probably feeds.l.n is the default vhost. That should be fixed too.

I agree with Martin that the Host header looks broken.

If the only problem on the Launchpad side is the weirdness he describes, I'm triaging this as low on our side. I'll keep an eye on this to see if there is a diagnosis that points to something more serious.

I did do a quick once-over to see if I could find the pertinent code path in Launchpad that would cause this behavior. I didn't see anything obvious. An AssertionError or a KeyError seem to by the likely thing to happen on the Launchpad side with an XMLRPC request pointing to an unknown host.

So the fix on the Bazaar side is probably that we should make sure xmlrpclib (or whatever) puts in the correct destination Host when it's using a proxy.

Why this has been moved to low importance: to me it looks a showstopper.

bumping to high in bazaar - duplicates are starting to accrue

def request(self, host, handler, request_body, verbose=0) in the lp registration module recieves 'xmlrpc.edge.launchpad.net' as the host.

To reproduce:
apt-get install squid3
http_proxy=http://127.0.0.1:3128/ https_proxy=http://127.0.0.1:3128/ /usr/bin/bzr info lp:epics-base/3-14

We do the following:
CONNECT to tunnel through the proxy
SSL after that, so wireshark isn't all that helpful.

In the failing method I see:
(Pdb) print proxy._ServerProxy__host
xmlrpc.edge.launchpad.net
(Pdb) proxy._ServerProxy__transport._opener
<bzrlib.transport.http._urllib2_wrappers.Opener object at 0x1e781d0>

Deep in urllib2, in do_request, line 1080, request.has_proxy is False - this is interesting and seems to be due to the urllib2 set_proxy methods behaviour. I'll need to stare harder at this to understand it I think.

and online 1084 is where the wrong Host header is set.

Now, Its my first time looking at this code, but I think the simplest thing up front is to set Host ourselves. The backtrace is:
  /home/robertc/source/baz/pending/working/bzrlib/plugins/launchpad/lp_registration.py(74)request()
-> response = self._opener.open(request)
  /usr/lib/python2.6/urllib2.py(389)open()
-> req = meth(req)
> /usr/lib/python2.6/urllib2.py(1084)do_request_()
-> if not request.has_header('Host'):
(Pdb)

More later, a quick attempt at that hasn't worked.

Ok, found an incantation that appears to work for me. Feedback solicited!

Could/should we backport this to 2.1 or 2.0?

https://code.edge.launchpad.net/~lifeless/bzr/bug-558343-wrong-host-with-proxy/+merge/24938
is the patch - its pretty self contained.

I suspect it would work in 2.1, and not in 2.0 due to other https
support requirements vila added.

When will this be available in lucid updates? It makes bzr unusable on my system.

Paul: that's a good question. The fix hasn't been backported to the 2.1 branch, but as comment #17 says it's probably a good candidate for backporting. After that we'd need to kick off the SRU process (possibly making a 2.1.4 release would ease that)... So I'd guess it'd be weeks away at least.

In the meantime you may find the bzr PPA to be helpful, it has 2.2.0 for lucid, which includes this fix: <https://edge.launchpad.net/~bzr/+archive/ppa>

@Paul, as a workaround you can also use bzr+ssh urls, eg bzr+ssh://bazaar.launchpad.net/~bzr-pqm/bzr/trunk

Bug description:
root@ubuntu2:/opt/openstack# bzr branch lp:nova
bzr: ERROR: exceptions.NotImplementedError: should resend request to http://feeds.edge.launchpad.net/bazaar/, but this isn't implemented

*** Bazaar has encountered an internal error. This probably indicates a
    bug in Bazaar. You can help us fix it by filing a bug report at
        https://bugs.launchpad.net/bzr/+filebug
    attaching the crash file
        /root/.cache/crash/bzr-20101111061506-7466.crash
    and including a description of the problem.

    The crash file is plain text and you can inspect or edit it to remove
    private information.

@Lakshmi: You need to upgrade your bzr version to get the fix that's why I marked your report as a duplicate of this one.
This isn't related to your branch so there is no need to add more comments here unless you're using an upgraded bzr and still encounter the problem.

'bzr version' will tell you which version you are currently using (2.1.1 from your crash report)

See coment #19 above too.

Accepted bzr into maverick-proposed, the package will build now and be available in a few hours. Please test and give feedback here. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you in advance!

verification done:

I set up a squid proxy per Robert's comment #13, and ran

http_proxy=http://127.0.0.1:3128/ https_proxy=http://127.0.0.1:3128/ /usr/bin/bzr info lp:epics-base

using maverick's original 2.2.0-1. It worked. This bug was actually fixed prior to the bzr release shipped in maverick, and so is not especially relevant to the maverick SRU. It still works in 2.2.4-0ubuntu1.

It would, however, be good to do a separate lucid sru to get this fix from bzr 2.1.4.

Accepted bzr into lucid-proposed, the package will build now and be available in a few hours. Please test and give feedback here. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you in advance!

This bug was fixed in the package bzr - 2.1.4-0ubuntu1

---------------
bzr (2.1.4-0ubuntu1) lucid-proposed; urgency=low

  * Update watch file to use 2.1 series.
  * New upstream release.
   + Fix file descriptors leaks in dirstate compiled extension. LP: #583486
   + Refuse to stack on repositories in incompatible formats. LP: #562380
   + Don't delete nested trees/repos. LP: #572098
   + Fix 'bzr switch' crash when a 'ConfigurableFileMerger' is used. LP: #559436
   + Fix compatibility with older smart servers. LP: #528041
   + Fix symlinks addition. LP: #192859
   + Properly unversion children of unversioned directories. LP: #494221
   + Lock configuration files in '~/.bazaar' for updates. LP: #525571
   + Fix 'bzr commit <symlink>'. LP: #128562
   + Fix `lp:` urls when behind an http proxy. LP: #558343
   + Stop using edge.launchpad.net. LP: #583667
 -- Max Bowsher <email address hidden> Tue, 17 May 2011 09:54:17 +0100