Duplicity

Progress not working with many backends

Bug #1482841 reported by Artur Bodera
162
This bug affects 34 people
Affects Status Importance Assigned to Milestone
Duplicity
Confirmed
Medium
Unassigned
Debian
New
Undecided
Unassigned

Bug Description

The --progress flag and progress bar seem to be broken with a lot of backends.
So far I've tested CF+Hubic and onedrive, both behave the same way. With --verbosity debug I can see that subsequent volumes are being sent to backend for transfer, backend keeps transferring them, but the progress bar after 8 hours of work looks the same:

0.0KB 08:00:00 [0.0B/s] [> ] 0% ETA Stalled!
0.0KB 08:00:03 [0.0B/s] [> ] 0% ETA Stalled!
0.0KB 08:00:06 [0.0B/s] [> ] 0% ETA Stalled!
0.0KB 08:00:09 [0.0B/s] [> ] 0% ETA Stalled!

It never moves.

I understand that some backends might not be good at reporting speed or file transfer progress, but I don't see why the progress bar would not be updated at least after each volume has been transferred ( no. MB transferred / no MB total to transfer = progress).

Revision history for this message
Artur Bodera (abodera) wrote :

Same happens with onedrive.

Revision history for this message
Cysioland (cysioland) wrote :

Same happens with plain, old file:///. So I guess it counts as confirmed, amirite?

Changed in duplicity:
status: New → Confirmed
Revision history for this message
dmtroyer (dmtroyer) wrote :

happening with b2 on 0.7.06.

Revision history for this message
Courtney Rosenthal (courtneyr) wrote :

I can confirm this problem with rsync backend on 0.7.06.

Revision history for this message
Samic (i-samic) wrote :

Same thing with rsync backend on 0.6.23

Revision history for this message
Xytime (xytime) wrote :

Happens with 0.7.07.1 (ArchLinux) with the sftp:// backend

Revision history for this message
Guy Rutenberg (guyrutenberg) wrote :

Also affects the pydrive backend (Google Drive).

Revision history for this message
Paweł Krawczyk (pawel-krawczyk) wrote :

Also with S3.

Revision history for this message
Paweł Krawczyk (pawel-krawczyk) wrote :

In my case this was caused by iptables blocking packets with "invalid" state (-m state --state INVALID) which seems to block some packets from AWS. Removing that rule restored the full connection speed.

Revision history for this message
Artur Puzio (cytadela8-g) wrote :

For me also happens with 0.7.07.1(Gentoo) with the sftp:// backend.

Revision history for this message
cortocopy (cortocopy) wrote :

It happens with Swift too. S3 works fine however

using duplicity 0.7.10

Revision history for this message
orest (piahoo-u) wrote :

same issue on fedora26, duplicity 0.7.15

Revision history for this message
Kevin (wittyman37) wrote :

Same issue with duplicity 0.7.06, Ubuntu GNOME 16.04.4, and webdav server

no longer affects: ubuntu
Revision history for this message
Ignacy (ignacy-sawa) wrote :

PARTIAL SOLUTION:

Progress can be monitored using `pv` function. It's especially useful when backing up large files.
First grab the process id, and then put it as parameter to pv.

Example:
`ps -ef | grep duplicity`
We read the PID (first number after the username).
pv -d PID

Or a one-liner*:
*assuming we are copying a file named "disk". The second grep narrows the search results.

pv -d $(ps -ef | grep duplicity | grep disk | tr -s ' ' | cut -d ' ' -f 2)

Revision history for this message
Kevin (wittyman37) wrote :

Here's a slightly shorter version

pv -d "$(pgrep -u $USER duplicity)"

Revision history for this message
iBart (bart-) wrote :

Still present with rsync version 0.8.09

Revision history for this message
Kenneth Loafman (kenneth-loafman) wrote : Update bug #1482841, status confirmed, importance medium

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

 status confirmed
 importance medium
 done
-----BEGIN PGP SIGNATURE-----

iQEzBAEBCAAdFiEExwf+p6/3mDvUSsaGH1G0ZBEHM/EFAmBmFhgACgkQH1G0ZBEH
M/EelQgArjMyCrUPB6SDdzRm0HaIRaCWp868f0bnCAulW1CYoQBCPe3lYPdYjmMH
WU62hzntxnQxKOYvZ2tNV7uzmaZtIvyliCFcu++PTvIKpZhDTf+bi3lDbW6h6vq2
CRx2zTkE865ev8AwbFmwD2yXntZ43BDL4x6Jp9tEw52e9nSW8qIsS/oGsWarWNcx
W5E8hi99ObGcv7/KFOxtosa9/meH3NyjmIzXokEseOvwUPux4qSJpufzDiaVCds9
djHmZbJpffQKbqKihZhdpfO7iHQECTMIzrrFiViJclbGVs3F1FXU8AKXDSnN2z1q
6i6K1N9gpd4ddcTc/YiIgy4NOf0EYw==
=/Q/F
-----END PGP SIGNATURE-----

Changed in duplicity:
importance: Undecided → Medium
Revision history for this message
ChieftainY2k (chieftainy2k) wrote (last edit ):

Hello, I am also affected by this bug.

But since this issue is alive and well for so long :) I wonder if there is any pay-per-bug plan for this project ?

I would gladly pay someone to fix this bug. I'm serious.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Related questions

Remote bug watches

Bug watches keep track of this bug in other bug trackers.