Backup fails with ReadError("raise ReadError("unexpected end of data")
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
Duplicity |
Confirmed
|
Undecided
|
Unassigned |
Bug Description
Hey team. I've been using duplicity and duply for about a year now with few problems. I recently upgraded my Debian system (I can probably find out the exact upgrade details if you need to know) and duplicity started crashing. You can see the full trace below.
As far as I can tell, it's attempting to convert an incomplete tarfile into a GPG version. I've paused the code at that point and verified that if I do a "tar tzvf" on the input tar file, it returns an EOF error.
Note that I modified /usr/lib/
I'm going to continue to debug independently, but if you can point me at some things to try that would help me. Happy to run whatever diagnostics will help narrow this down.
OS INFO
-------
Linux fidelity 4.9.0-1-686-pae #1 SMP Debian 4.9.6-3 (2017-01-28) i686 GNU/Linux
PACKAGE INFO
------------
ii duplicity 0.7.12-1 i386 encrypted bandwidth-efficient backup
ii duply 1.11.3-1 all easy to use frontend to the duplicity backup system
OUTPUT FROM A RUN
-----------------
# TMPDIR='/tmp' PASSPHRASE=
gpg: WARNING: unsafe ownership on homedir '/home/
Using archive dir: /var/cache/
Using backup name: duply_backups
GPG binary is gpg, version 2.1.18
Import of duplicity.
Import of duplicity.
Import of duplicity.
Import of duplicity.
Import of duplicity.
Import of duplicity.
Import of duplicity.
Import of duplicity.
Import of duplicity.
Import of duplicity.
Import of duplicity.
Import of duplicity.
Import of duplicity.
Import of duplicity.
Import of duplicity.
Import of duplicity.
Import of duplicity.
Import of duplicity.
Import of duplicity.
Import of duplicity.
Import of duplicity.
Import of duplicity.
Import of duplicity.
Import of duplicity.
Import of duplicity.
Import of duplicity.
Import of duplicity.
Import of duplicity.
Reading globbing filelist /etc/duply/
Main action: inc
=======
duplicity 0.7.12 (March 21, 2017)
Args: /usr/bin/duplicity --archive-dir /var/cache/duply --name duply_backups --verbosity 9 --volsize 1024 --full-
Linux fidelity 4.9.0-1-686-pae #1 SMP Debian 4.9.6-3 (2017-01-28) i686
/usr/bin/python 2.7.13 (default, Jan 19 2017, 14:48:08)
[GCC 6.3.0 20170118]
=======
Using temporary directory /var/cache/
Registering (mkstemp) temporary file /var/cache/
Temp has 172445667328 available, backup will use approx 2469606195.
....
Comparing usr/lib/
Selection: examining path /usr/lib/
Selection: result: None from function: Command-line exclude glob: /dev
Selection: result: None from function: Command-line exclude glob: /tmp
Selection: result: None from function: Command-line exclude glob: /proc
Selection: result: None from function: Command-line exclude glob: /sys
Selection: result: None from function: Command-line exclude glob: /var/lib/
Selection: result: None from function: Command-line exclude glob: /var/lib/
Selection: result: None from function: Command-line exclude glob: /var/lib/mlocate
Selection: result: None from function: Command-line exclude glob: /var/cache/apt
Selection: result: None from function: Command-line exclude glob: /home/duply
Selection: result: None from function: Command-line exclude glob: /home/virtualbo
Selection: result: None from function: Command-line exclude glob: /home/virtualbo
Selection: + including file
Selecting /usr/lib/
<TarInfo 'signature/
Comparing usr/lib/
Selection: examining path /usr/lib/
Selection: result: None from function: Command-line exclude glob: /dev
Selection: result: None from function: Command-line exclude glob: /tmp
Selection: result: None from function: Command-line exclude glob: /proc
Selection: result: None from function: Command-line exclude glob: /sys
Selection: result: None from function: Command-line exclude glob: /var/lib/
Selection: result: None from function: Command-line exclude glob: /var/lib/
Selection: result: None from function: Command-line exclude glob: /var/lib/mlocate
Selection: result: None from function: Command-line exclude glob: /var/cache/apt
Selection: result: None from function: Command-line exclude glob: /home/duply
Selection: result: None from function: Command-line exclude glob: /home/virtualbo
Selection: result: None from function: Command-line exclude glob: /home/virtualbo
Selection: + including file
Selecting /usr/lib/
<TarInfo 'signature/
Comparing usr/lib/
Selection: examining path /usr/lib/
Selection: result: None from function: Command-line exclude glob: /dev
Selection: result: None from function: Command-line exclude glob: /tmp
Selection: result: None from function: Command-line exclude glob: /proc
Selection: result: None from function: Command-line exclude glob: /sys
Selection: result: None from function: Command-line exclude glob: /var/lib/
Selection: result: None from function: Command-line exclude glob: /var/lib/
Selection: result: None from function: Command-line exclude glob: /var/lib/mlocate
Selection: result: None from function: Command-line exclude glob: /var/cache/apt
Selection: result: None from function: Command-line exclude glob: /home/duply
Selection: result: None from function: Command-line exclude glob: /home/virtualbo
Selection: result: None from function: Command-line exclude glob: /home/virtualbo
Selection: + including file
Selecting /usr/lib/
<TarInfo 'snapshot/
Comparing usr/lib/
Selection: examining path /usr/lib/
Selection: result: None from function: Command-line exclude glob: /dev
Selection: result: None from function: Command-line exclude glob: /tmp
Selection: result: None from function: Command-line exclude glob: /proc
Selection: result: None from function: Command-line exclude glob: /sys
Selection: result: None from function: Command-line exclude glob: /var/lib/
Selection: result: None from function: Command-line exclude glob: /var/lib/
Selection: result: None from function: Command-line exclude glob: /var/lib/mlocate
Selection: result: None from function: Command-line exclude glob: /var/cache/apt
Selection: result: None from function: Command-line exclude glob: /home/duply
Selection: result: None from function: Command-line exclude glob: /home/virtualbo
Selection: result: None from function: Command-line exclude glob: /home/virtualbo
Selection: + including file
Selecting /usr/lib/
<TarInfo 'signature/
Comparing usr/lib/
Selection: examining path /usr/lib/
Selection: result: None from function: Command-line exclude glob: /dev
Selection: result: None from function: Command-line exclude glob: /tmp
Selection: result: None from function: Command-line exclude glob: /proc
Selection: result: None from function: Command-line exclude glob: /sys
Selection: result: None from function: Command-line exclude glob: /var/lib/
Selection: result: None from function: Command-line exclude glob: /var/lib/
Selection: result: None from function: Command-line exclude glob: /var/lib/mlocate
Selection: result: None from function: Command-line exclude glob: /var/cache/apt
Selection: result: None from function: Command-line exclude glob: /home/duply
Selection: result: None from function: Command-line exclude glob: /home/virtualbo
Selection: result: None from function: Command-line exclude glob: /home/virtualbo
Selection: + including file
Selecting /usr/lib/
<TarInfo 'signature/
Comparing usr/lib/
Selection: examining path /usr/lib/
Selection: result: None from function: Command-line exclude glob: /dev
Selection: result: None from function: Command-line exclude glob: /tmp
Selection: result: None from function: Command-line exclude glob: /proc
Selection: result: None from function: Command-line exclude glob: /sys
Selection: result: None from function: Command-line exclude glob: /var/lib/
Selection: result: None from function: Command-line exclude glob: /var/lib/
Selection: result: None from function: Command-line exclude glob: /var/lib/mlocate
Selection: result: None from function: Command-line exclude glob: /var/cache/apt
Selection: result: None from function: Command-line exclude glob: /home/duply
Selection: result: None from function: Command-line exclude glob: /home/virtualbo
Selection: result: None from function: Command-line exclude glob: /home/virtualbo
Selection: + including file
Selecting /usr/lib/
<TarInfo 'signature/
Comparing usr/lib/
Selection: examining path /usr/lib/
Selection: result: None from function: Command-line exclude glob: /dev
Selection: result: None from function: Command-line exclude glob: /tmp
Selection: result: None from function: Command-line exclude glob: /proc
Selection: result: None from function: Command-line exclude glob: /sys
Selection: result: None from function: Command-line exclude glob: /var/lib/
Selection: result: None from function: Command-line exclude glob: /var/lib/
Selection: result: None from function: Command-line exclude glob: /var/lib/mlocate
Selection: result: None from function: Command-line exclude glob: /var/cache/apt
Selection: result: None from function: Command-line exclude glob: /home/duply
Selection: result: None from function: Command-line exclude glob: /home/virtualbo
Selection: result: None from function: Command-line exclude glob: /home/virtualbo
Selection: + including file
Selecting /usr/lib/
Releasing lockfile /var/cache/
Removing still remembered temporary file /var/cache/
Removing still remembered temporary file /var/cache/
Removing still remembered temporary file /var/cache/
Traceback (most recent call last):
File "/usr/bin/
with_
File "/usr/bin/
fn()
File "/usr/bin/
do_
File "/usr/bin/
incremental
File "/usr/bin/
globals.
File "/usr/bin/
globals.
File "/usr/lib/
data = block_iter.
File "/usr/lib/
result = self.process(
File "/usr/lib/
for new_path, sig_path in collated:
File "/usr/lib/
relem2 = riter2.next()
File "/usr/lib/
refresh_
File "/usr/lib/
new_triple = get_triple(
File "/usr/lib/
path = path_iter_
File "/usr/lib/
for tarinfo in tf:
File "/usr/lib/
tarinfo = self.tarfile.next()
File "/usr/lib/
raise ReadError(
ReadError: unexpected end of data in /usr/lib/
Changed in duplicity: | |
status: | New → In Progress |
importance: | Undecided → Medium |
milestone: | none → 0.8.13 |
Changed in duplicity: | |
importance: | Medium → Undecided |
milestone: | 0.8.13 → none |
I did a little more digging. It appears that the sigtar for the full backup is corrupt:
# tar tzf /home/duply/ duply_backups/ duplicity- full-signatures .20171001T10000 5Z.sigtar. gz | wc -l
tar: Unexpected EOF in archive
tar: Error is not recoverable: exiting now
213827
So that opens up a few questions:
1. Why is it corrupt? I don't have a crash log handy, but I could try to find it
2. How do I recover? Is it safe to just delete the local files and try again?
3. How can I help improve the system to do a better job of error recovery in this case?
thanks!