Duplicity fails to start

I notice that this problem (using deja-dup to run duplicity) only occurs when I run as sudo or pkexec (I need to do so to back up some kernel files).

You are on duplicity 0.8.04, the latest is 0.8.07, please upgrade.

There are three options:

* Release tarball Install - https://launchpad.net/duplicity/+download
* Daily duplicity builds - https://launchpad.net/~duplicity-team/+archive/ubuntu/daily-dev-trunk
* Stable snap builds - “sudo snap install duplicity —classic"

NOTE: UNinstall duplicity first if it was installed via the distribution repository. For Ubuntu, that would be "sudo apt-get purge duplicity".

Whoops, latest is 0.8.08. Please let me know if this fixes the error.

I had looked for a ppa but failed to find yours.

Unfortunately deja-dup doesn't recognise this new version, saying "Could not understand duplicity version."

I tend to use GUIs for almost everything. I guess I will have to try to translate my deja-dup instructions into duplicity command line. This may take me quite a while.

If there is a simple fix to deja-dup (or duplicity) to get round this I'd be grateful.

After a quick web search I tried running

duplicity --version

That told me I needed to install 'future' with pip. So I installed python-pip, future, and fasteners.

I now fail at:

duplicity --version
Traceback (most recent call last):
  File "/usr/bin/duplicity", line 55, in <module>
    from duplicity.dup_main import main
ImportError: No module named duplicity.dup_main

And then:

pip install duplicity.dup_main
Collecting duplicity.dup_main
Could not install packages due to an EnvironmentError: 404 Client Error: Not Found for url: https://pypi.org/simple/duplicity-dup-main/

Are there some comprehensive installation instructions somewhere?

How did you install it? The version should be OK for Deja Dup.

On Tue, Dec 10, 2019 at 11:45 AM Tim Passingham <email address hidden>
wrote:

> After a quick web search I tried running
>
> duplicity --version
>
> That told me I needed to install 'future' with pip. So I installed
> python-pip, future, and fasteners.
>
> I now fail at:
>
> duplicity --version
> Traceback (most recent call last):
> File "/usr/bin/duplicity", line 55, in <module>
> from duplicity.dup_main import main
> ImportError: No module named duplicity.dup_main
>
> And then:
>
> pip install duplicity.dup_main
> Collecting duplicity.dup_main
> Could not install packages due to an EnvironmentError: 404 Client Error:
> Not Found for url: https://pypi.org/simple/duplicity-dup-main/
>
> Are there some comprehensive installation instructions somewhere?
>
> --
> You received this bug notification because you are subscribed to
> duplicity in Ubuntu.
> https://bugs.launchpad.net/bugs/1855736
>
> Title:
> Duplicity fails to start
>
> Status in Duplicity:
> Incomplete
> Status in duplicity package in Ubuntu:
> New
>
> Bug description:
> I am on 19.10. Duplicity crashes as soon as I try to start a backup,
> as below. It was running fine on 19.04.
>
> Traceback (innermost last):
> File "/usr/bin/duplicity", line 107, in <module>
> with_tempdir(main)
> File "/usr/bin/duplicity", line 93, in with_tempdir
> fn()
> File "/usr/lib/python3/dist-packages/duplicity/dup_main.py", line
> 1522, in main
> action = commandline.ProcessCommandLine(sys.argv[1:])
> File "/usr/lib/python3/dist-packages/duplicity/commandline.py", line
> 1200, in ProcessCommandLine
> globals.backend = backend.get_backend(args[0])
> File "/usr/lib/python3/dist-packages/duplicity/backend.py", line 225,
> in get_backend
> obj = get_backend_object(url_string)
> File "/usr/lib/python3/dist-packages/duplicity/backend.py", line 211,
> in get_backend_object
> return factory(pu)
> File
> "/usr/lib/python3/dist-packages/duplicity/backends/giobackend.py", line 81,
> in __init__
> ensure_dbus()
> File
> "/usr/lib/python3/dist-packages/duplicity/backends/giobackend.py", line 37,
> in ensure_dbus
> lines = output.split(u'\n')
> TypeError: a bytes-like object is required, not 'str'
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/duplicity/+bug/1855736/+subscriptions
>

I think this patch should fix the duplicity error you're seeing.

But deja-dup doesn't work well under root so much - it'll ask for root permissions as appropriate when restoring anyway?

I'm not sure whether the fix mentioned is for the last problenm I encountered, but having updated duplicity at 20:20GMT (now version 0.8.09-0ubuntu0ppa1524~ubuntu19.10.1), when I type:

duplicity --version

I still get:

Traceback (most recent call last):
  File "/usr/bin/duplicity", line 55, in <module>
    from duplicity.dup_main import main
ImportError: No module named duplicity.dup_main

Did you remove the repo version of duplicity ($ sudo apt purge duplicity)?

If not, do the purge now, then do ($ sudo apt reinstall duplicity). It should install from the ppa.

I had done, but just to be sure I purged it again, autoremove the un-needed items (python fasteners etc), reinstalled, rebooted, and get the same error on "duplicity --version"

Try this:

$ PYTHONPATH='.' duplicity ...

Looks like the current directory is not in the path. It should be.

Sorry to be difficult, but:

PYTHONPATH='.' duplicity --version
Traceback (most recent call last):
  File "/usr/bin/duplicity", line 55, in <module>
    from duplicity.dup_main import main
ImportError: No module named duplicity.dup_main

You're not being difficult, the problem is vexing!

Try this:

    $ PYTHONPATH='.'duplicity/dup_main.py

It should work like this too:

    $ duplicity/dup_main.py

In either case there should be no output. Please log whatever happens. Perhaps there's a missing module or something. Python imports don't tell you a lot when they go wrong.

Sorry, that should be:

$ python3 duplicity/dup_main.py

in both cases.

I'm not quite sure what your last comment means, so here are 4 attempts.

PYTHONPATH='.' duplicity/dup_main.py --version
bash: duplicity/dup_main.py: No such file or directory

duplicity/dup_main.py --version
bash: duplicity/dup_main.py: No such file or directory

PYTHONPATH='.' python3 duplicity/dup_main.py --version
python3: can't open file 'duplicity/dup_main.py': [Errno 2] No such file or directory

python3 duplicity/dup_main.py --version
python3: can't open file 'duplicity/dup_main.py': [Errno 2] No such file or directory

The results are the same with or without --version

Sorry, my bad. You needed to be in the /usr/lib/python3.X/site-packages/ directory. This is not trying to execute duplicity, merely to import the installed duplicity/dup_main.py.

Well that may explain something. I have no python3.X directory. I have python3, and it contains dist-packages.

I did try:

/usr/lib/python3/dist-packages$ python3 duplicity/dup_main.py
Traceback (most recent call last):
  File "duplicity/dup_main.py", line 54, in <module>
    from . import asyncscheduler
ImportError: attempted relative import with no known parent package

I got that update, and things have changed a bit.

usr/lib/python3/dist-packages$ python3 duplicity/dup_main.py

returns nothing, which I think is what you wanted.

But:

/usr/lib/python3/dist-packages$ duplicity
Traceback (most recent call last):
  File "/usr/bin/duplicity", line 55, in <module>
    from duplicity.dup_main import main
ImportError: No module named duplicity.dup_main

(the same if I repeat the command from my home directory)

Or:

/usr/lib/python3/dist-packages$ duplicity/dup_main.py
bash: duplicity/dup_main.py: Permission denied

Or:

sudo duplicity/dup_main.py
[sudo] password for ...:
sudo: duplicity/dup_main.py: command not found

As you can tell, I've no idea what to try, or what's really going on.

To be clear, I'm now on version 0.8.09-0ubuntu0ppa1526~ubuntu19.10.1 (eoan)

I rebooted after installing (just in case). I didn't purge beforehand, so I then did so, did an apt autoremove, and reinstalled it. No change.

The attachment "decode.patch" seems to be a patch. If it isn't, please remove the "patch" flag from the attachment, remove the "patch" tag, and if you are a member of the ~ubuntu-reviewers, unsubscribe the team.

[This is an automated message performed by a Launchpad user owned by ~brian-murray, for any issues please contact him.]

Do this and report the results:

$ python3 /usr/bin/duplicity --version

I think I know the problem, but want to validate.

If that fails for any reason, do these and report the results:

$ find /usr -type d -name duplicity -ls
$ find /usr -type f -name duplicity -ls

python3 /usr/bin/duplicity --version
duplicity 0.8.09

Success!

But as before:

duplicity --version
Traceback (most recent call last):
  File "/usr/bin/duplicity", line 55, in <module>
    from duplicity.dup_main import main
ImportError: No module named duplicity.dup_main

Ditto for "/usr/bin/duplicity --version"

So something about not being explicity about python3 is now the issue?

OK, what's happening is two factor:

1) Something in setup is changing the shebang line to Python 2 and,
2) It's libraries are being installed on Python 3.

which means duplicity can't find its libraries.

Please run the following:

$ find /usr -type d -name duplicity -ls
$ find /usr -type f -name duplicity -ls

OK (I didn't do so before because the python3 command seemed to work).

find /usr -type d -name duplicity -ls
  1602725 4 drwxr-xr-x 2 root root 4096 Dec 15 20:10 /usr/share/doc/duplicity
  1602642 4 drwxr-xr-x 4 root root 4096 Dec 15 20:10 /usr/lib/python3/dist-packages/duplicity

find /usr -type f -name duplicity -ls
   658314 8 -rwxr-xr-x 1 root root 5265 Dec 15 17:37 /usr/bin/duplicity

I would have replied earlier but I don't seem to be getting emails regarding comments as quickly as I used to (in fact none about your last 2 posts).

OK, execute the following to fix the problem:

$ sudo sed -i s/python2/python3/g /usr/bin/duplicity

This will make duplicity run under Python3.

$ duplicity --version

should respond with: duplicity 0.8.09

Yes, that's fixed. Thanks very much.

Now all I have to do is get deja dup to work:

Traceback (innermost last):
  File "/usr/bin/duplicity", line 101, in <module>
    with_tempdir(main)
  File "/usr/bin/duplicity", line 87, in with_tempdir
    fn()
  File "/usr/lib/python3/dist-packages/duplicity/dup_main.py", line 1526, in main
    action = commandline.ProcessCommandLine(sys.argv[1:])
  File "/usr/lib/python3/dist-packages/duplicity/commandline.py", line 1172, in ProcessCommandLine
    globals.backend = backend.get_backend(args[0])
  File "/usr/lib/python3/dist-packages/duplicity/backend.py", line 225, in get_backend
    obj = get_backend_object(url_string)
  File "/usr/lib/python3/dist-packages/duplicity/backend.py", line 211, in get_backend_object
    return factory(pu)
  File "/usr/lib/python3/dist-packages/duplicity/backends/giobackend.py", line 82, in __init__
    ensure_dbus()
  File "/usr/lib/python3/dist-packages/duplicity/backends/giobackend.py", line 38, in ensure_dbus
    lines = output.decode(u'utf8').split(u'\n')
 AttributeError: 'str' object has no attribute 'decode'

Which particular fix has been applied?

The shebang points to python2 on all scripts. It was my fault. setup pointed to python3, duplicity pointed to python2. That caused all the problems.

Thanks very much. I'm glad I was able to help a little by testing things.

Have you any thoughts about why deja-dup fails as I reported in #26? Do I need to raise a separate report somewhere else?

A few more deja-dup issues:

deja-dup (brings up GUI, I select backup...)
Traceback (most recent call last):
  File "/usr/bin/duplicity", line 55, in <module>
    from duplicity.dup_main import main
ImportError: No module named duplicity.dup_main

sudo deja-dup (brings up GUI, I select backup...)
Traceback (most recent call last):
  File "/usr/bin/duplicity", line 31, in <module>
    from future import standard_library
ImportError: No module named future

So:

pip install future
Requirement already satisfied: future in ./.local/lib/python2.7/site-packages (0.18.2)

But isn't in python3 now?

Tim, your issue in comment 26 is confusing to me. It looks like it's caused by my patch in comment 6?

Specifically, it looks like subprocess.Popen returns a unicode string as stdout for you. I can't reproduce locally, in either python2.7 or python3.7:

>>> import subprocess
>>> subprocess.Popen([u'ls'], stdout=subprocess.PIPE).communicate()[0].decode('utf8')

The documentation for Popen says:
"If encoding or errors are specified, or text is true, the file objects stdin, stdout and stderr are opened in text mode with the specified encoding and errors, as described above in Frequently Used Arguments. The universal_newlines argument is equivalent to text and is provided for backwards compatibility. By default, file objects are opened in binary mode."

None of the arguments that would cause Popen to return a unicode string seem to be passed to it by duplicity...?

It is indeed strange, because I can't reproduce it now either. I get the errors in #30 instead, with deja-dup saying it can't understand the duplicity version.

I think what may have happened is that because I couldn't get deja-dup to work I used the duplicity command line, and encrypted this particular backup. I don't normally encrypt this backup (which is mainly system files, config files etc). Is 'decode' something to do with encryption?

Once duplicity seemed to be sorted I redid the backup without encryption, and now I get the issues in #30.

With the latest update duplicity still fails. Version 0.8.09-0ubuntu0ppa1531-ubuntu19.10.1 (eoan).

I had to redo the sed from #25.

Which way does it fail? I thought we had that fixed for you.

It was fixed. My system later updated to version 1531 as above, and this then failed in the same way as before. I had to apply your sed patch to make it work.

If you redid the sed from #25, that caused the problem. You need to reinstall and let it run under Python 2.

Sadly not. I purged, installed, and it failed as before. I had to redo the sed.

1532 this morning fixed it. Thanks! :-)

With version 0.8.09-0ubuntu0ppa1535-ubuntu19.10.1 (eoan) deja-dup now seems to start to work as a normal user, but I need to backup some root stuff, and when run under sudo or pkexec I get the following immediately:

Traceback (innermost last):
  File "/usr/bin/duplicity", line 101, in <module>
    with_tempdir(main)
  File "/usr/bin/duplicity", line 87, in with_tempdir
    fn()
  File "/usr/lib/python3/dist-packages/duplicity/dup_main.py", line 1525, in main
    action = commandline.ProcessCommandLine(sys.argv[1:])
  File "/usr/lib/python3/dist-packages/duplicity/commandline.py", line 1172, in ProcessCommandLine
    globals.backend = backend.get_backend(args[0])
  File "/usr/lib/python3/dist-packages/duplicity/backend.py", line 225, in get_backend
    obj = get_backend_object(url_string)
  File "/usr/lib/python3/dist-packages/duplicity/backend.py", line 211, in get_backend_object
    return factory(pu)
  File "/usr/lib/python3/dist-packages/duplicity/backends/giobackend.py", line 82, in __init__
    ensure_dbus()
  File "/usr/lib/python3/dist-packages/duplicity/backends/giobackend.py", line 38, in ensure_dbus
    lines = output.decode(u'utf8').split(u'\n')
 AttributeError: 'str' object has no attribute 'decode'

Thanks very much indeed. I really appreciate all the work you and others do in supporting us, the users.

It all works fine with deja-dup now. I'm now on duplicity version 0.8.09-0ubuntu0ppa1542-ubuntu19.10.1 (eoan), and deja-dup version 40.1-1ubuntu2

Should I close this report myself, or leave that you you?

You're welcome. The report is closed.

I had/have similar issue as "resolved" in comment #25:

Initially, after building and installing version 0.8.09, I would try
duplicity --version
and get
Traceback (most recent call last):
  File "/usr/local/bin/duplicity", line 55, in <module>
    from duplicity.dup_main import main
ImportError: No module named 'duplicity'

I came upon this bug, saw comment #25 where it was suggested to change
the shebang to 'python3'. I checked, and mine was ALREADY saying 'python3', but
I changed it to 'python2' and then
duplicity --version responds correctly with
duplicity 0.8.09

If it helps, one of the first messages from running
$ sudo python setup.py install --prefix=/usr/local
is
/usr/lib/python2.7/dist-packages/setuptools/dist.py:333: UserWarning: Normalizing '0.8.09' to '0.8.9'
  normalized_version,

So it appears to be using python2 (2.7, to be exact) "during the build"?

Let me know if you'd like another bug open, or if there already is one to improve build scripts?
OR, better yet, maybe your "fix" was a fix for 0.8.10 and it has already been taken care of?

Thanks,

If it helps, the output of the 'find' commands, mentioned prior to comment #25 are as you might guess:

$ sudo find /usr -type d -name duplicity -ls
  1471833 4 drwxr-xr-x 3 root root 4096 Jun 9 2017 /usr/lib/python2.7/dist-packages/duplicity
    54981 4 drwxr-xr-x 3 root root 4096 Jan 10 20:01 /usr/local/lib/python2.7/dist-packages/duplicity

$ sudo find /usr -type f -name duplicity -ls
    55524 8 -rwxr-xr-x 1 root root 5276 Jan 10 20:38 /usr/local/bin/duplicity
  1177994 64 -rwxr-xr-x 1 root root 62300 Jan 10 20:19 /usr/bin/duplicity

Terrible app. You have backup complite, but when You really need use it:
deja-dup file "/usr/bin/duplicity", line 107, in <module>

Kubuntu 19.10

All of this Opensource must works like that? This is not first time when: python - Duplicity error while restoring - Ask Ubuntu

Maybe this app need a button to test environment compatibility. Just for be sure.

Getting similar issue after upgrading to Ubuntu 20.10:

Traceback (innermost last):
  File "/usr/bin/duplicity", line 106, in <module>
    with_tempdir(main)
  File "/usr/bin/duplicity", line 92, in with_tempdir
    fn()
  File "/usr/lib/python3/dist-packages/duplicity/dup_main.py", line 1538, in main
    do_backup(action)
  File "/usr/lib/python3/dist-packages/duplicity/dup_main.py", line 1674, in do_backup
    incremental_backup(sig_chain)
  File "/usr/lib/python3/dist-packages/duplicity/dup_main.py", line 655, in incremental_backup
    bytes_written = dummy_backup(tarblock_iter)
  File "/usr/lib/python3/dist-packages/duplicity/dup_main.py", line 224, in dummy_backup
    while next(tarblock_iter):
  File "/usr/lib/python3/dist-packages/duplicity/diffdir.py", line 544, in __next__
    result = self.process(next(self.input_iter)) # pylint: disable=assignment-from-no-return
  File "/usr/lib/python3/dist-packages/duplicity/diffdir.py", line 209, in get_delta_iter
    for new_path, sig_path in collated:
  File "/usr/lib/python3/dist-packages/duplicity/diffdir.py", line 296, in collate2iters
    relem1 = next(riter1)
  File "/usr/lib/python3/dist-packages/duplicity/selection.py", line 97, in __next__
    return next(self.iter)
  File "/usr/lib/python3/dist-packages/duplicity/selection.py", line 181, in Iterate
    subpath, val = next(diryield_stack[-1])
  File "/usr/lib/python3/dist-packages/duplicity/selection.py", line 148, in diryield
    s = self.Select(new_path)
  File "/usr/lib/python3/dist-packages/duplicity/selection.py", line 210, in Select
    result = sf(path)
  File "/usr/lib/python3/dist-packages/duplicity/selection.py", line 458, in exclude_sel_func
    elif path.append(filename).exists():
  File "/usr/lib/python3/dist-packages/duplicity/path.py", line 566, in append
    return self.__class__(self.base, self.index + (ext,))
  File "/usr/lib/python3/dist-packages/duplicity/path.py", line 539, in __init__
    self.setdata()
  File "/usr/lib/python3/dist-packages/duplicity/path.py", line 549, in setdata
    self.stat = os.lstat(self.name)
 PermissionError: [Errno 13] Permission denied: b'/home/nikodll/.hardinfo/.deja-dup-ignore'

---------

$ python3 /usr/bin/duplicity --version
> duplicity 0.8.12

First, this is not the same bug, but a support issue.

duplicity can't stat /home/nikodll/.hardinfo/.deja-dup-ignore, so you need to run duplicity as root, i.e. 'sudo duplicity ...'.

Thanks! I needed something simple today! :-)

Nikita, your issue actually looks a lot like something fixed in duplicity 0.8.16:
https://gitlab.com/duplicity/duplicity/-/commit/f6809e8a4e09b70138a0902434d8c88d3c4c8ccb

Try installing the snap version of deja-dup or duplicity, both of which have the fix for this.

Hello,
I previously have this problem (with sudo), and solve it, I don't remember how.
Today the bug also here but it is strange, I use the ppa with the 2.1.2 and duplicity folder on /usr/local/lib/python3.10/dist-packages/ is empty. So I think the bug is link to that.
I have the same message write in bug description.

I try to build duplicity : https://gitlab.com/duplicity/duplicity
python3 setup.py install
It work but when I start backup I have a new error message : "Backups requires at least version 0.7.14 of duplicity, but only found version 0.0.00"
The "duplicity --version" return "duplicity $version" so I found __init__.py in /usr/local/lib/python3.10/dist-packages/duplicity and replace "__version__ = '$version'" by "__version__ = '2.1.2'".
It work's.

Do you think it is really correct ?

How it could work without root ?

Can anybody look the problem and found another solution ?

Thanks in advance

Hello,
Not sure that is is correct, the full backup and the incremental seem to work but I have this at the end :
CommandLineError: Wrong number of positional args for 'restore', got 3
Expected 2 positionals from ['--file-to-restore=home/......deja-dup/metadata', 'gio+file:///media/....../backups', '/home/......deja-dup/metadata'].

Any idea for if it is link ?

For the first backup, the partition of the deja-dup cache, have been very few space, don't know if at a moment, it has been full.

Thanks in advance.
David

Hi David,

This sounds like a mismatch in libraries which happens when multiple versions of duplicity are installed from different sources. I'd like you to do this:

$ sudo apt purge duplicity # to remove the distro version.
$ sudo python3 -m pip uninstall duplicity # to remove the pip3 version.
$ sudo snap remove duplicity # to remove the snap version.

You may get errors in any of those. That's OK.

Then reinstall from Gitlab by doing:

$ sudo python3 -m pip install https://gitlab.com/duplicity/duplicity/uploads/3e1efd3ebe74bbbf2109f3d9c239f4be/duplicity-2.1.3.tar.gz

This is the versioned source from https://gitlab.com/duplicity/duplicity/-/releases.

BTW, bug reporting has moved to https://gitlab.com/duplicity/duplicity/-/issues. Please report an issue there if you're still having problems.

Hello,
Thanks a lot, I have totally remove them and I have an error with root or without root :
Traceback (innermost last):
  File "/usr/local/bin/duplicity", line 62, in <module>
    with_tempdir(main)
  File "/usr/local/bin/duplicity", line 53, in with_tempdir
    fn()
  File "/usr/local/lib/python3.10/dist-packages/duplicity/dup_main.py", line 1501, in main
    action = cli_main.process_command_line(sys.argv[1:])
  File "/usr/local/lib/python3.10/dist-packages/duplicity/cli_main.py", line 247, in process_command_line
    args = parse_cmdline_options(cmdline_list)
  File "/usr/local/lib/python3.10/dist-packages/duplicity/cli_main.py", line 204, in parse_cmdline_options
    if remainder[0] in all_commands:
 IndexError: list index out of range

I try to use an old version :
sudo apt install duplicity=0.8.21-1build1 (default of ubuntu 22.04)
It work well !
sudo apt install duplicity, install the 2.1.3-ppa202310081632~ubuntu22.04.1 and say the same error message.

So I remove the ppa and keep the Ubuntu version it seem work !

Thanks for your previous answer :)

The previous error is in command line.
In command line the 0.8.21 work, but not with deja-dup it is the error :
Traceback (innermost last):
  File "/usr/bin/duplicity", line 92, in <module>
    with_tempdir(main)
  File "/usr/bin/duplicity", line 75, in with_tempdir
    fn()
  File "/usr/lib/python3/dist-packages/duplicity/dup_main.py", line 1555, in main
    action = commandline.ProcessCommandLine(sys.argv[1:])
  File "/usr/lib/python3/dist-packages/duplicity/commandline.py", line 1223, in ProcessCommandLine
    config.backend = backend.get_backend(args[0])
  File "/usr/lib/python3/dist-packages/duplicity/backend.py", line 223, in get_backend
    obj = get_backend_object(url_string)
  File "/usr/lib/python3/dist-packages/duplicity/backend.py", line 209, in get_backend_object
    return factory(pu)
  File "/usr/lib/python3/dist-packages/duplicity/backends/giobackend.py", line 82, in __init__
    ensure_dbus()
  File "/usr/lib/python3/dist-packages/duplicity/backends/giobackend.py", line 37, in ensure_dbus
    output = p.communicate()[0].decode("utf8", errors="replace")
 AttributeError: 'str' object has no attribute 'decode'

I have found solution for the 0.8.21 version :

Don't use sudo but su and after launch deja-dup !

So I keep this version :)