Ftp servers with bogus APPE are not supported

On 7 February 2012 09:20, Fbe546 <email address hidden> wrote:
> Thanks a lot for the answer.
>
> We posted a bug report at https://bugs.launchpad.net/bzr/+bug/927929
> with the .bzr.log file.
>
> The pack files of the old repository did indeed begin with "Bazaar
> pack format 1 (introduced in 0.18)\n". There are no pack files inside
> the newly created repository (with the commands above), but I suppose
> it's normal.

In the bug you create a new repository and then try to commit to it.
In this case bzr is creating and then reading back a pack file. If
the data we wrote in to the pack is immediately missing this is seems
very likely a bug in the ftp server implementation.

--
Martin

You mean that the problem would be on the side of the FTP server ? It
would be quite strange since it is a mutualized hosting at the largest
hosting company in Europe and everything had been working for a year.

Anyway, do you have any idea of how we could get a more precise idea
of where the problem lies ?

Looking at the FTP log below (found in .bzr.log), the problem would be
that wb3yq8rbpvyrz1zlhsbm.pack isn't read correctly at 4.107 ?

Again thanks for the help and Bazaar :)

3.363 FTP stat: /repository2/.bzr
3.485 FTP put: /repository2/.bzr/repository/upload/wb3yq8rbpvyrz1zlhsbm.pack
3.733 FTP has check: wb3yq8rbpvyrz1zlhsbm.pack =>
/repository2/.bzr/repository/upload/wb3yq8rbpvyrz1zlhsbm.pack
3.765 FTP has: /repository2/.bzr/repository/upload/wb3yq8rbpvyrz1zlhsbm.pack
3.795 FTP appe to /repository2/.bzr/repository/upload/wb3yq8rbpvyrz1zlhsbm.pack
3.795 FTP appe (try 0) to
/repository2/.bzr/repository/upload/wb3yq8rbpvyrz1zlhsbm.pack
3.925 FTP has check: wb3yq8rbpvyrz1zlhsbm.pack =>
/repository2/.bzr/repository/upload/wb3yq8rbpvyrz1zlhsbm.pack
3.955 FTP has: /repository2/.bzr/repository/upload/wb3yq8rbpvyrz1zlhsbm.pack
3.985 FTP appe to /repository2/.bzr/repository/upload/wb3yq8rbpvyrz1zlhsbm.pack
3.985 FTP appe (try 0) to
/repository2/.bzr/repository/upload/wb3yq8rbpvyrz1zlhsbm.pack
4.107 FTP get: /repository2/.bzr/repository/upload/wb3yq8rbpvyrz1zlhsbm.pack
4.229 FTP rm: /repository2/.bzr/repository/upload/wb3yq8rbpvyrz1zlhsbm.pack
4.265 FTP get: /repository2/site/.bzr/branch/lock/held/info
4.389 FTP rename: /repository2/site/.bzr/branch/lock/held =>
/repository2/site/.bzr/branch/lock/releasing.obv9us6ree5bjyztw5ue.tmp
4.453 FTP rm: /repository2/site/.bzr/branch/lock/releasing.obv9us6ree5bjyztw5ue.tmp/info
4.489 FTP rmd:
/repository2/site/.bzr/branch/lock/releasing.obv9us6ree5bjyztw5ue.tmp
4.521 Transferred: 0kB (0.0kB/s r:0kB w:0kB)
4.536 Traceback (most recent call last): (...)

On Wed, Feb 8, 2012 at 12:49 AM, Fbe546 <email address hidden> wrote:
> You mean that the problem would be on the side of the FTP server ? It
> would be quite strange since it is a mutualized hosting at the largest
> hosting company in Europe and everything had been working for a year.

Unusual, yes. Unprecedented no. We had another user using FTP to a
hosting site that was build on Microsoft Windows with a cluster file
system that took 1-2 *seconds* to allow newly written content to be
read. (The details are in an old bug report, I don't remember the
number).

> Anyway, do you have any idea of how we could get a more precise idea
> of where the problem lies ?
>
> Looking at the FTP log below (found in .bzr.log), the problem would be
> that wb3yq8rbpvyrz1zlhsbm.pack isn't read correctly at 4.107 ?

Yes, exactly that spot. The file is written with APPE, then when it is
read-back it is not returning the expected content.

Cheers,
Rob

Maybe someone familiar with the source code of Bazaar could answer
this question before I begin to dig into it:

How hard would it be to replace the use of APPE with a GET and a PUT ?

It would surely be less efficient, but at least it would work...

On 8 February 2012 06:56, Fbe546 <email address hidden> wrote:
> Maybe someone familiar with the source code of Bazaar could answer
> this question before I begin to dig into it:
>
> How hard would it be to replace the use of APPE with a GET and a PUT ?
>
> It would surely be less efficient, but at least it would work...

There is already support for doing this in transport/ftp/__init__.py,
the has_append attribute. If the server doesn't support APPE at all
we fall back to this. Apparently we have not yet encountered a server
that has an append but where it doesn't work properly. You could
fairly easily check the location config stack for a new variable
ftp.disable_append and then turn that on. It would be at least worth
trying.

If it's true, as it seems, that the problem is the server is not
read-after-write coherent with itself on a single file, I don't know
why PUT would be any more coherent than APPE....

--
Martin

Yay, it works again if I set "self._has_append" to false in
transport/ftp/__init__.py!

So it means, as you said, that the servers implements APPE, but it
doesn't work correctly... We reported that to our host, let's see what
they will say...

Thanks for your help.

On 8 February 2012 23:00, Fbe546 <email address hidden> wrote:
> Yay, it works again if I set "self._has_append" to false in
> transport/ftp/__init__.py!
>
> So it means, as you said, that the servers implements APPE, but it
> doesn't work correctly... We reported that to our host, let's see what
> they will say...
>
> Thanks for your help.

OK, that's interesting. Would you like to try a patch to add a
configuration variable for avoiding append?

It may get pretty slow to read everything back as your repository gets larger.

When we write a file out in a single stream we could possibly avoid
APPE with some code changes...

--
Martin

The host told use they "reported the issue", but I don't hope much.

I can try what you want. If I have some free time soon, I might try
too to understand where exactly the server bug with APPE lies. Maybe
all the data is returned, but with something bad in addition...

Dear,

There is something really amazing I've found a couple of days ago, check it out http://hncg.com.cn/quarter.php?0302

Rushing, scott

Hi friend!

I'm sure you'll like the stuff I've just found for you, here, take a look http://raynerecording.com/implication.php?3435

Sincerely yours, scott

Yo!

This info may be very useful for you, you won't regret, just take a look at http://www.solheim.nl/intimate.php?f8f9

Yours, scott

Hello friend,

I've been writing an article and I've come across that strange stuff, do you have any ideas what could it be? Just take a look http://hazazontamar.badgleybankruptcysolutions.com

Best wishes, scott

Sent from Mail for Windows 10

Hello,

I've recently read some useful information, I guess it might be useful for you too, please read at http://pallab.andrewtam.us

Best regards, scott

From: 927929 [mailto:<email address hidden>]
Sent: Sunday, May 14, 2017 2:42 PM
To: <email address hidden>
Subject: I think so.

I'm certain that at the very, very least, the PSU was powerful enough for the new equipment at one point. My new mobo/CPU worked just fine for about 1.5 days before they abruptly stopped working. Also, I'm typing this from the old mobo/CPU combo right now, so I'm pretty confident that the PSU isn't the problem. I was just hoping that there would be some way to tell whether or not it's the CPU or the mobo that is broken. Looks like I may just have to flip and coin on returning one of them.

Sent from Mail for Windows 10