Comment 4 for bug 215612

Revision history for this message
Vincent Ladeuil (vila) wrote : Re: [Bug 215612] Re: upload plugin should silently ignore deleted files on remote server

>>>>> "James" == James <email address hidden> writes:

    James> I understand your point. Maybe the upload plugin
    James> should, by default, mirror what happens for instance
    James> when one bzr branch is merged into another -- in the
    James> case where the source branch has had a file bzr
    James> removed and the target branch has had the same file
    James> manually removed.

This is where the plugin shows its limitations. bzr fully knows
the content of both branches, whereas the plugin *suppose* it
knows the content of the remote working tree via the revid.

That's why I say the remote working tree is write-only,

    James> I'm not sure if --force would be the correct verb to describe the
    James> process - maybe "--ignore" to sliently ignore upload errors ?

The more I think about it, the more I prefer to use
'--catch-up'. The idea being: ok, we told you not to update your
remote working tree, you did it anyway, but you are pretty sure
that ignoring some errors will be enough to get back in sync, so
you told me: go ahead, trust me, all will be fine ;-)

    James> Finally, the workaround described is not really
    James> valid. The point of the plugin is to upload changed
    James> files - if I wanted to upload the entire tree every
    James> time, then I'd use plain old FTP. If I have a 300MB
    James> website to upload to a remote server but only 30kb of
    James> changes, that's not an efficient process.

Exactly. That also why trying to manage the remote working tree
as a bzr manage the local one is a dead end.

You realize that uploading the whole working tree is a waste of
resources. In the other way, establishing the modifications made
on the working tree requires comparing that working tree with the
last revision. And to do that, you need to download the whole
remote working tree.

    James> I'd suggest implementing the --ignore flag would solve
    James> the issue in hand and provide a generic way of
    James> ignoring upload errors. I'm just unsure at the moment
    James> of the scope of an "--ignore" - it may ignore things
    James> that really ought to cause an upload failure.

I need to think more about it to clearly establish which use
cases we can safely handle with a '--catch-up' option, obviously
the deleted files are one, some others will be more problematic:

- added files (no way to discover them short of walking the whole
  remote working tree, filtering with .bzrignore (which is not
  required so far) for all log files or whatever working files
  existing only on the remote site,

- renamed files or dirs (the magic involved is of a higher level
  without knowledge of the content, and even then...)

- modified files (obviously need to download them)

The more important point that makes me still hesitate to implement
that option is that once we lose the synchronization between the
revid and the remote working tree, there way to get it back.

That may requires addressing bug #206770 by providing a command
to compare the remote and local working trees. May be that
command may work in two stages:

- compare tree shapes (i.e. only the names of files and directories),
- compare tree content (download remote files to compare with
  local ones)

Providing parameters to limit the comparison may also help to
limit the bandwidth needed.