Slow transfer over sftp connection

Bug #710585 reported by Anton
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
sbackup
Confirmed
Low
Unassigned

Bug Description

I noticed that backups were taking much longer than I expected. Using scp to the same server, I get about 20MB/s (which is about my max HD throughput). However, sbackup writes at about only 2MB/s, a factor of 10 (!) slower (as reported by iotop and network traffic and manual measurement of file growth). It isn't the compression, because that only runs at only 20% CPU. So there seems room for a 5-fold improvement here.

This is a serious issue, because a full backup now can take up to 5 hours or so (in stead of about one hour), which significantly increases the odds that the backup does not finish before I shutdown my laptop, or take it offline.

I have tracked the problem to what I think may be the origin; a bug in glib/gvfs:
https://bugs.launchpad.net/ubuntu/+source/glib2.0/+bug/250517
In short, all writes on a gvfs volume are acknowledged, causing a network-latency dependent bottleneck, which may be quite severe even on networks with a high (intrinsic) bandwidth. In my case, it's 1G from my laptop all the way to the server, but with several routers in between (university setup).

The proper solution would be to have gvfs fixed, however according to the bug that isn't going to be any time soon.

An alternative may be to see if a reasonable workaround fix can be implemented for sbackup. Any ideas?

(I tried to use the 'fuse' plugin in stead of 'gvfs' for sbackup to see if the issue exists with fuse as well, but fuse will not connect without password, and does not accept my password since it has a '?' in it. Will get back on this if I found time to try with a different password.)

Revision history for this message
Jean-Peer Lorenz (peer.loz) wrote :

Still valid? How is the status?

Changed in sbackup:
status: New → Incomplete
Revision history for this message
Anton (feenstra) wrote :

I'm running Natty at the moment (gvfs 1.9.0-0ubuntu1, glib 2.28.6-0ubuntu1), and it does seem to run much faster now, although I haven't done any proper timings now. However, the glib bug I refer to above is still open, so I don't straight away see how things may have approved (maybe the latency in our university network went down?).

Revision history for this message
Launchpad Janitor (janitor) wrote :

[Expired for sbackup because there has been no activity for 60 days.]

Changed in sbackup:
status: Incomplete → Expired
Revision history for this message
Anton (feenstra) wrote :

Hi Jean-Peer,
Unfortunately, on Trusty, this problem is still there. The other day a full backup took almost the whole working day (this is a university laptop, and I've tweaked scripts so its starts a backup as soon as it comes online when I arrive at work).
As a workaround, I'd be happy to patch in a backend that uses an ssh pipe (something like tar | ssh user@remote dd of=/where/my/backups/are.tgz). But I don't understand the structure of the file backend that sbackup uses. Is this worth looking in to (I'd certainly like to have this feature - it would also allow me to do full backups from home over fiber optics - high bandwith but much longer latencies).
Please give me some pointers to where I could put this into sbackup. I'll try to do it nicely enough - preferably as a separate backend - so that you may be inclined to include it in trunk.

Changed in sbackup:
status: Expired → Incomplete
importance: Undecided → Low
Changed in sbackup:
status: Incomplete → Confirmed
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.