no status update after dput for ppa package

Bug #136593 reported by Neal McBurnett
58
This bug affects 8 people
Affects Status Importance Assigned to Milestone
Launchpad itself
Triaged
Low
Unassigned

Bug Description

I thought I'd take a stab at testing ppa, so I took an old bit of authtool source and banged away at following the instructions in https://help.launchpad.net/PPAQuickStart with it. Being even less than a noob at packaging, I was surprised to relatively quickly succeed with "Successfully uploaded packages and nice words about how my signatures checked out well. But after several hours I don't see any emails or any build status or anything in my repository at https://launchpad.net/~nealmcb/+archive

I've tried again twice (via dput -f), signing with key 0x2C9EBA60 which is registered for nealmcb in launchpad. Still no mail and no build status on the web.

So my first question is "What is the status of my PPA packages?"

In the broader picture, it seems that a goal of PPA is to get more folks involved in packaging, so having better documentation, which includes detailed examples, common error conditions, and fixes would be helpful.

E.g. I heard that packages signed by the "wrong key" are silently thrown away. For PPA that sounds like it will lead to lots of frustration especially for the many newbies.

Can dput somehow confirm that launchpad will provide future status info?
If not, how do people track down these sorts of problems?

Revision history for this message
Neal McBurnett (nealmcb) wrote :

Here are some other relevant details.

authtool_0.2.2~ppa1_source.upload:

Successfully uploaded ../authtool_0.2.2~ppa1.dsc to ppa.launchpad.net.
Successfully uploaded ../authtool_0.2.2~ppa1.tar.gz to ppa.launchpad.net.
Successfully uploaded ../authtool_0.2.2~ppa1_source.changes to ppa.launchpad.net.
(3 times)

.dput.cf:

[my-ppa]
fqdn = ppa.launchpad.net
incoming = ~nealmcb/ubuntu/
login = anonymous

It would seem that getting dput to provide some sort of tracking id would enable much better status tracking. Perhaps the dput output should indicate the full upload location in its output as a URL like
 ftp://ppa.launchpad.net/~nealmcb/ubuntu/authtool_0.2.2~ppa1.dsc (or whatever)

and that could be cut-and-pasted as input in some launchpad web query to get status?
Or Is ftp just the wrong sort of protocol for this queue injection function?

Revision history for this message
Christian Reis (kiko) wrote :

FTP will never work in that way -- the poppy server is a virtual incoming FTP directory that is moved out of the way as soon as your session ends.

Revision history for this message
Neal McBurnett (nealmcb) wrote :

Regardless of how ftp and poppy work, what I'm saying is that at least dput and launchpad could agree on and implement some sort of tracking id to help people track package submissions that vanish into the ether.

Revision history for this message
Mark Shuttleworth (sabdfl) wrote :

Hmm... i see no reason why we can't evolve to a protocol for uploads which is smart, efficient and authenticated. For example, if dput was having a conversation with Launchpad, it could figure out that it does not need to resend an orig.tar.gz which LP has already got from somewhere else... If we had this, then of course status could be much better handled too. For the moment, with FTP, all we can try to do is reduce the time till you get your emails regarding builds.

Changed in soyuz:
importance: Undecided → Wishlist
Revision history for this message
Cody A.W. Somerville (cody-somerville) wrote :

If the package is signed then there is an e-mail address to contact to say that a package signed with a key attached to this e-mail address was rejected. This won't work if someone doesn't enter their e-mail address correctly when creating the key but thats a corner case.

However, I also agree with Mark that a smarter protocol for uploads sounds like a great idea. I've done a lot of hacking lately on dput and would be interested in assisting the soyuz team if and when they decide to develop such a feature.

Revision history for this message
Julian Edwards (julian-edwards) wrote :

There's an outstanding blueprint for this work https://blueprints.edge.launchpad.net/soyuz/+spec/ssh-package-upload

It's on our list of priorities for towards the end of this year.

Changed in soyuz:
status: New → Triaged
Revision history for this message
LCID Fire (lcid-fire) wrote :

We just talked on irc about this - why not use Webservices (you know the stuff with http and soap messages). First of all - port 80 is open nearly anywhere and with soap you have pretty much full control over the error handling. Granted - you have to code both the server and the client (which means AFAIK extending dput/dupload) but you can customize to the process instead of working out some dirty process.

Celso Providelo (cprov)
tags: added: soyuz-upload
Celso Providelo (cprov)
tags: added: poppy
Revision history for this message
Rod Smith (rodsmith) wrote :

I just wanted to give this bug a kick, because it's still an issue. I accidentally deactivated the wrong OpenPGP key, which resulted in an apparently-successful upload that then vanished into a black hole, from my point of view -- I received no e-mail notifications of either success or failure and no indications of problems from dput. The page for the PPA showed no hint of any upload. I banged my head against this problem for several hours, and it stumped a few people (Canonical employees) who I consulted (although in their defense, they didn't know I'd done anything with my keys recently). If I'm not mistaken, the e-mail address of the person who submitted the package is in the .changes file, so it should be possible to fire off an e-mail saying that the OpenPGP key didn't check out. This could save newbies, or fumble-fingered folks like myself, much head-banging.

Revision history for this message
William Grant (wgrant) wrote : Re: [Bug 136593] Re: no status update after dput for ppa package

On 19/12/15 04:25, Rod Smith wrote:
> I just wanted to give this bug a kick, because it's still an issue. I
> accidentally deactivated the wrong OpenPGP key, which resulted in an
> apparently-successful upload that then vanished into a black hole, from
> my point of view -- I received no e-mail notifications of either success
> or failure and no indications of problems from dput. The page for the
> PPA showed no hint of any upload. I banged my head against this problem
> for several hours, and it stumped a few people (Canonical employees) who
> I consulted (although in their defense, they didn't know I'd done
> anything with my keys recently). If I'm not mistaken, the e-mail address
> of the person who submitted the package is in the .changes file, so it
> should be possible to fire off an e-mail saying that the OpenPGP key
> didn't check out. This could save newbies, or fumble-fingered folks like
> myself, much head-banging.

But we can't verify the signature, so anybody could have uploaded that
email address via anonymous FTP. That's not a spam vector that we can
sensibly open ourselves up to.

Revision history for this message
Christian Reis (kiko) wrote :

On Fri, Dec 18, 2015 at 09:33:54PM -0000, William Grant wrote:
> On 19/12/15 04:25, Rod Smith wrote:
> > anything with my keys recently). If I'm not mistaken, the e-mail address
> > of the person who submitted the package is in the .changes file, so it
> > should be possible to fire off an e-mail saying that the OpenPGP key
> > didn't check out. This could save newbies, or fumble-fingered folks like
> > myself, much head-banging.
>
> But we can't verify the signature, so anybody could have uploaded that
> email address via anonymous FTP. That's not a spam vector that we can
> sensibly open ourselves up to.

Could we get Launchpad to create a temporary upload-error URL (say 1
week) explaining what was wrong, and somehow have dput issue an error
message containing that URL? That way you don't need to spam, the
notification is synchronous, and the end-user can learn what is wrong.

I assume the hard part is getting dput to relay the error -- is there
some way of having the server provide an error message verbatim back?
--
Christian Robottom Reis | [+55 16] 3376 0125 | http://async.com.br/~kiko
                        | [+55 16] 991 126 430 | http://launchpad.net/~kiko

Revision history for this message
Colin Watson (cjwatson) wrote :

The main problem with that approach is that dput is asynchronous: it
drops the package into an upload queue, which is processed at some later
point after dput has disconnected. (It's important that it be that way,
because processing could take quite some time for large packages and we
wouldn't want to break just because somebody's connection failed after
they'd completed the upload but before we managed to process it.)

We could do better for SFTP uploads, as alluded to earlier in this bug
log: while we should still require the upload to be signed, SSH
authentication is enough that we could at least trust it for the purpose
of assuring that we can send notification mail. This would only really
be effective if dput's defaults were changed, which is difficult because
it needs to know the user's Launchpad username, but there are ways to
mitigate that (e.g. prompt at first use) and it would still be a good
start.

The other thing I can think of in addition to this would be a
dismissable notification on your Launchpad user page about upload errors
we think might be related to your account, a bit like the system we
already have for package copy failures. If we also arranged for dput to
tell you that upload errors would be notified on your user page, that
would be a good improvement.

Revision history for this message
Robert Bruce Park (robru) wrote :

Or perhaps a notification on the ppa page? There's already notifications there for copy failures

Revision history for this message
Takkat (takkat-nebuk) wrote :

Still an issue.

I had to update my GPG key because it had expired but it then was not automatically synced with the Ubuntu keyserver. Therefore an upload to my PPA using dput failed silently.

Any notification anywhere with possibly saying that there was an issue with my key would have saved me a lot of time.

Revision history for this message
Paul Goins (vultaire) wrote :

This affected me during some training exercises. I'm doing successful uploads, but things appear to be getting stuck behind the scenes and I have no idea what's going on.

* I did an initial build, and it was handled successfully. (PPA: https://launchpad.net/~vultaire/+archive/ubuntu/experiments, package: python-jblite-0.4.3-1, distro: cosmic)

* I then uploaded versions with a ~distro suffix, e.g. python-jblite-0.4.3-1~trusty. This worked for trusty, xenial, and bionic. ~cosmic did not build at all, despite a successful dput.

* After dropping the previous non-suffixed cosmic package and re-uploading the ~cosmic-suffixed package, I was able to get a new successful build for cosmic.

* However, after doing a README update and trying to upload version 0.4.3-2~<distro>, After doing a README update and trying to upload a second revision of my package for bionic and cosmic, nothing built. Trying re-uploads wouldn't work, trying a different PPA (https://launchpad.net/~vultaire/+archive/ubuntu/jblite) wouldn't work, and doing a version bump to 0.4.3-3~<distro> wouldn't work.

Revision history for this message
Paul Goins (vultaire) wrote :

Just to be clear: my point is that with this issue, I'm not sure how you'd debug what goes wrong when you do an upload and nothing ends up in the build queue. I'm at a loss personally.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Related blueprints

Remote bug watches

Bug watches keep track of this bug in other bug trackers.