Bazaar

bzr 2.0.0 hang on merge & pull from location when trying to read directory as a file

Bug #484558 reported by Wade Williams
12
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Bazaar
Won't Fix
Low
Unassigned

Bug Description

Worked with jam on this quite a while in #bzr and he asked me to report a bug about bzr trying to read a directory as a file.

When trying to merge or bzr pull $location, using sftp, bzr just hangs indefinitely.

"Server" is running OpenBSD 4.6. I tried using both the built-in openssh (OpenBSD has it integrated into the operating system) and also installing the latest OpenSSH (5.3). Neither had an effect.

-bash-4.0$ uname -a
OpenBSD anagram.dogwatchsw.com 4.6 GENERIC#58 i386

-bash-4.0$ ssh -v
OpenSSH_5.3, OpenSSL 0.9.8k 25 Mar 2009

-bash-4.0$ bzr --version
Bazaar (bzr) 2.0.2
  Python interpreter: /usr/local/bin/python 2.5.4
  Python standard library: /usr/local/lib/python2.5
  Platform: OpenBSD-4.6-i386-Intel_Pentium_II_-GenuineIntel-_686-class,_512KB_L2

Client:

Mac OS X. Tried both 10.6.2 (on Intel) and 10.5.8 (on PowerPC).

Here's the Intel 10.6.2 machine:
xanadu:XSquawkBox_mirror wwilliam$ uname -a
Darwin xanadu.local 10.2.0 Darwin Kernel Version 10.2.0: Tue Nov 3 10:37:10 PST 2009; root:xnu-1486.2.11~1/RELEASE_I386 i386

bzr --version
Bazaar (bzr) 2.0.0
  Python interpreter: /usr/bin/python2.6 2.6.1
  Python standard library: /System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6
  Platform: Darwin-10.2.0-i386-64bit
  bzrlib: /Library/Python/2.6/site-packages/bzrlib
  Bazaar configuration: /Users/wwilliam/.bazaar
  Bazaar log file: /Users/wwilliam/.bzr.log

xanadu:XSquawkBox_mirror wwilliam$ ssh -v
OpenSSH_5.2p1, OpenSSL 0.9.8k 25 Mar 2009

Log from a failed merge attempt:

Tue 2009-11-17 17:41:07 -0600
0.046 bzr arguments: [u'merge', u'-Dsftp', u'sftp://wade@10.1.1.2/home/bzr/XSquawkBox/branches/HEAD/']
0.062 looking for plugins in /Users/wwilliam/.bazaar/plugins
0.063 looking for plugins in /Library/Python/2.6/site-packages/bzrlib/plugins
0.674 opening working tree '/Users/wwilliam/Development/xsb/XSquawkBox_mirror'
1.975 ssh implementation is OpenSSH
85.344 Traceback (most recent call last):
  File "/Library/Python/2.6/site-packages/bzrlib/commands.py", line 842, in exception_to_return_code
    return the_callable(*args, **kwargs)
  File "/Library/Python/2.6/site-packages/bzrlib/commands.py", line 1037, in run_bzr
    ret = run(*run_argv)
  File "/Library/Python/2.6/site-packages/bzrlib/commands.py", line 654, in run_argv_aliases
    return self.run(**all_cmd_args)
  File "/Library/Python/2.6/site-packages/bzrlib/builtins.py", line 3674, in run
    possible_transports=possible_transports)
  File "/Library/Python/2.6/site-packages/bzrlib/bundle/__init__.py", line 49, in read_mergeable_from_url
    _do_directive)
  File "/Library/Python/2.6/site-packages/bzrlib/bundle/__init__.py", line 96, in read_mergeable_from_transport
    return _serializer.read_bundle(f), transport
  File "/Library/Python/2.6/site-packages/bzrlib/bundle/serializer/__init__.py", line 57, in read_bundle
    for line in f:
  File "/Library/Python/2.6/site-packages/paramiko/file.py", line 106, in next
    line = self.readline()
  File "/Library/Python/2.6/site-packages/paramiko/file.py", line 214, in readline
    new_data = self._read(n)
  File "/Library/Python/2.6/site-packages/paramiko/sftp_file.py", line 152, in _read
    data = self._read_prefetch(size)
  File "/Library/Python/2.6/site-packages/paramiko/sftp_file.py", line 132, in _read_prefetch
    self.sftp._read_response()
  File "/Library/Python/2.6/site-packages/paramiko/sftp_client.py", line 654, in _read_response
    t, data = self._read_packet()
  File "/Library/Python/2.6/site-packages/paramiko/sftp.py", line 175, in _read_packet
    x = self._read_all(4)
  File "/Library/Python/2.6/site-packages/paramiko/sftp.py", line 159, in _read_all
    x = self.sock.recv(n)
  File "/Library/Python/2.6/site-packages/bzrlib/transport/ssh.py", line 193, in recv
    return self.__socket.recv(n)
  File "/Library/Python/2.6/site-packages/bzrlib/transport/ssh.py", line 632, in recv
    return os.read(self.proc.stdout.fileno(), count)
  File "/Library/Python/2.6/site-packages/bzrlib/breakin.py", line 41, in _debug
    signal.signal(_breakin_signal_number, _debug)
  File "/Library/Python/2.6/site-packages/bzrlib/breakin.py", line 41, in _debug
    signal.signal(_breakin_signal_number, _debug)
  File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/bdb.py", line 46, in trace_dispatch
    return self.dispatch_line(frame)
  File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/bdb.py", line 65, in dispatch_line
    if self.quitting: raise BdbQuit
BdbQuit

Log from a failed pull $location attempt:

Tue 2009-11-17 19:13:30 -0600
0.053 bzr arguments: [u'pull', u'-Dsftp', u'sftp://wade@10.1.1.2/home/bzr/XSquawkBox/branches/HEAD/']
0.069 looking for plugins in /Users/wwilliam/.bazaar/plugins
0.070 looking for plugins in /Library/Python/2.6/site-packages/bzrlib/plugins
0.359 encoding stdout as sys.stdout encoding 'UTF-8'
0.682 opening working tree '/Users/wwilliam/Development/xsb/XSquawkBox_mirror'
2.751 ssh implementation is OpenSSH
51.858 Traceback (most recent call last):
  File "/Library/Python/2.6/site-packages/bzrlib/commands.py", line 842, in exception_to_return_code
    return the_callable(*args, **kwargs)
  File "/Library/Python/2.6/site-packages/bzrlib/commands.py", line 1037, in run_bzr
    ret = run(*run_argv)
  File "/Library/Python/2.6/site-packages/bzrlib/commands.py", line 654, in run_argv_aliases
    return self.run(**all_cmd_args)
  File "/Library/Python/2.6/site-packages/bzrlib/builtins.py", line 970, in run
    possible_transports=possible_transports)
  File "/Library/Python/2.6/site-packages/bzrlib/bundle/__init__.py", line 49, in read_mergeable_from_url
    _do_directive)
  File "/Library/Python/2.6/site-packages/bzrlib/bundle/__init__.py", line 96, in read_mergeable_from_transport
    return _serializer.read_bundle(f), transport
  File "/Library/Python/2.6/site-packages/bzrlib/bundle/serializer/__init__.py", line 57, in read_bundle
    for line in f:
  File "/Library/Python/2.6/site-packages/paramiko/file.py", line 106, in next
    line = self.readline()
  File "/Library/Python/2.6/site-packages/paramiko/file.py", line 214, in readline
    new_data = self._read(n)
  File "/Library/Python/2.6/site-packages/paramiko/sftp_file.py", line 152, in _read
    data = self._read_prefetch(size)
  File "/Library/Python/2.6/site-packages/paramiko/sftp_file.py", line 132, in _read_prefetch
    self.sftp._read_response()
  File "/Library/Python/2.6/site-packages/paramiko/sftp_client.py", line 654, in _read_response
    t, data = self._read_packet()
  File "/Library/Python/2.6/site-packages/paramiko/sftp.py", line 175, in _read_packet
    x = self._read_all(4)
  File "/Library/Python/2.6/site-packages/paramiko/sftp.py", line 159, in _read_all
    x = self.sock.recv(n)
  File "/Library/Python/2.6/site-packages/bzrlib/transport/ssh.py", line 193, in recv
    return self.__socket.recv(n)
  File "/Library/Python/2.6/site-packages/bzrlib/transport/ssh.py", line 632, in recv
    return os.read(self.proc.stdout.fileno(), count)
  File "/Library/Python/2.6/site-packages/bzrlib/breakin.py", line 41, in _debug
    signal.signal(_breakin_signal_number, _debug)
  File "/Library/Python/2.6/site-packages/bzrlib/breakin.py", line 41, in _debug
    signal.signal(_breakin_signal_number, _debug)
  File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/bdb.py", line 46, in trace_dispatch
    return self.dispatch_line(frame)
  File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/bdb.py", line 65, in dispatch_line
    if self.quitting: raise BdbQuit
BdbQuit

Related branches

lp://staging/~mbp/bzr/484558-merge-directory
On hold for merging into lp://staging/bzr
John A Meinel: Approve
Diff: 132 lines (+27/-34) (has conflicts)
4 files modified
NEWS (+10/-0)
bzrlib/bundle/__init__.py (+17/-5)
bzrlib/tests/test_http.py (+0/-8)
bzrlib/tests/test_read_bundle.py (+0/-21)
Revision history for this message
Martin Pool (mbp) wrote :

Thanks for the report. What happens if you try to cat a directory using an sftp client?

Changed in bzr:
status: New → Incomplete
importance: Undecided → Medium
importance: Medium → Low
status: Incomplete → Confirmed
Revision history for this message
Wade Williams (wadesworld) wrote :

Do you mean what happens if I try to list a directory? (There is no cat command in the sftp client). Listing works fine.

FYI - If a developer needs it, I can give access to my OpenBSD box.

Revision history for this message
John A Meinel (jameinel) wrote : Re: [Bug 484558] Re: bzr 2.0.0 hang on merge & pull from location when trying to read directory as a file

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Wade Williams wrote:
> Do you mean what happens if I try to list a directory? (There is no cat
> command in the sftp client). Listing works fine.
>
> FYI - If a developer needs it, I can give access to my OpenBSD box.
>

Well, 'get' a directory. Specifically:

$ sftp host
sftp> get bzr
Fetching /home/jameinel/bzr to bzr
Cannot download non-regular file: /home/jameinel/bzr

Is what I get.

John
=:->
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Cygwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAksEDq8ACgkQJdeBCYSNAANg6QCgwxJIUlXYGjMN7Kr8YtlQXWnn
RU8Anjb+NSLaXReq9WW7EDusxfy+IkJS
=3JPc
-----END PGP SIGNATURE-----

Revision history for this message
Wade Williams (wadesworld) wrote :

Yep, I get same thing:

sftp> get bzr-2.0.2
Fetching /home/wade/bzr-2.0.2 to bzr-2.0.2
Cannot download non-regular file: /home/wade/bzr-2.0.2

Revision history for this message
Wade Williams (wadesworld) wrote :

Just wondering if this bug is going to assigned? I'm not sure if two months for a bug to get assigned is a normal timeframe or not. But I'm obviously still interested in a fix.

Revision history for this message
Robert Collins (lifeless) wrote : Re: [Bug 484558] Re: bzr 2.0.0 hang on merge & pull from location when trying to read directory as a file

On Thu, 2010-01-14 at 21:11 +0000, Wade Williams wrote:
> Just wondering if this bug is going to assigned? I'm not sure if two
> months for a bug to get assigned is a normal timeframe or not. But I'm
> obviously still interested in a fix.

This bug has been assessed as a low priority for the project as a whole.
As such, core developers are only likely to get to it once critical,
high & medium bugs are all cleared out, or it gets reassessed to a
higher priority (e.g. because it completely stops you using bzr) (does
it ? - can you use bzr+ssh instead, which is much faster and more
reliable than sftp?)

You can see the main bug list at
https://bugs.edge.launchpad.net/bzr/+bugs

I'm sure someone would be happy to help you if you want to try fixing it
yourself.

Cheers,
Rob

Revision history for this message
Martin Pool (mbp) wrote :

I would surmise this descends from the bsd behaviour of letting you read() directories, whereas linux gives EISDIR. I'm not sure why that causes a hang rather than just giving back garbage; it may in fact be a bug in the sftp server.

I would further surmise that the sftp client is checking the type of the file before trying to read from it. I don't think we'd want Transport.read to do an extra roundtrip there in the general case but perhaps we should look before we leap when trying to read a bundle, by trying to stat the location.

Wade it could be useful if you could run a debugger or system call trace on the sftpd while this is hung and report what it's trying to do.

Revision history for this message
Martin Pool (mbp) wrote :

Hi

The attached branch should fix this. Could you try it and see?

Changed in bzr:
status: Confirmed → In Progress
Martin Pool (mbp)
Changed in bzr:
assignee: nobody → Martin Pool (mbp)
Revision history for this message
Wade Williams (wadesworld) wrote :

My apologies for not responding to this. My spam filter was catching launchpad email so I never saw the response.

I tried to replicate the previous behavior on the latest stable version of bzr and was unable to do so. I can't say for 100% certain that there are no differences between the test I was doing today and the previous situation, but I tried everything I could think of to recreate it and was unable to do so. So, I don't know that this fix is necessary (or perhaps it was integrated already).

Thanks so much for help. As far as I know, I'm in good shape now.

Revision history for this message
Martin Pool (mbp) wrote :

Proposed for merge https://code.edge.launchpad.net/~mbp/bzr/484558-merge-directory/+merge/22430

Revision history for this message
Martin Pool (mbp) wrote :

After writing a patch for it, I think it's not really worth merging, because there is more risk of breaking something on another platform and the problem no longer seems to be live on openbsd. If it recurs we can resurrect the patch.

Changed in bzr:
assignee: Martin Pool (mbp) → nobody
status: In Progress → Invalid
status: Invalid → Won't Fix
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.