no status update after dput for ppa package

Here are some other relevant details.

authtool_0.2.2~ppa1_source.upload:

Successfully uploaded ../authtool_0.2.2~ppa1.dsc to ppa.launchpad.net.
Successfully uploaded ../authtool_0.2.2~ppa1.tar.gz to ppa.launchpad.net.
Successfully uploaded ../authtool_0.2.2~ppa1_source.changes to ppa.launchpad.net.
(3 times)

.dput.cf:

[my-ppa]
fqdn = ppa.launchpad.net
incoming = ~nealmcb/ubuntu/
login = anonymous

It would seem that getting dput to provide some sort of tracking id would enable much better status tracking. Perhaps the dput output should indicate the full upload location in its output as a URL like
 ftp://ppa.launchpad.net/~nealmcb/ubuntu/authtool_0.2.2~ppa1.dsc (or whatever)

and that could be cut-and-pasted as input in some launchpad web query to get status?
Or Is ftp just the wrong sort of protocol for this queue injection function?

FTP will never work in that way -- the poppy server is a virtual incoming FTP directory that is moved out of the way as soon as your session ends.

Regardless of how ftp and poppy work, what I'm saying is that at least dput and launchpad could agree on and implement some sort of tracking id to help people track package submissions that vanish into the ether.

Hmm... i see no reason why we can't evolve to a protocol for uploads which is smart, efficient and authenticated. For example, if dput was having a conversation with Launchpad, it could figure out that it does not need to resend an orig.tar.gz which LP has already got from somewhere else... If we had this, then of course status could be much better handled too. For the moment, with FTP, all we can try to do is reduce the time till you get your emails regarding builds.

If the package is signed then there is an e-mail address to contact to say that a package signed with a key attached to this e-mail address was rejected. This won't work if someone doesn't enter their e-mail address correctly when creating the key but thats a corner case.

However, I also agree with Mark that a smarter protocol for uploads sounds like a great idea. I've done a lot of hacking lately on dput and would be interested in assisting the soyuz team if and when they decide to develop such a feature.

There's an outstanding blueprint for this work https://blueprints.edge.launchpad.net/soyuz/+spec/ssh-package-upload

It's on our list of priorities for towards the end of this year.

We just talked on irc about this - why not use Webservices (you know the stuff with http and soap messages). First of all - port 80 is open nearly anywhere and with soap you have pretty much full control over the error handling. Granted - you have to code both the server and the client (which means AFAIK extending dput/dupload) but you can customize to the process instead of working out some dirty process.

I just wanted to give this bug a kick, because it's still an issue. I accidentally deactivated the wrong OpenPGP key, which resulted in an apparently-successful upload that then vanished into a black hole, from my point of view -- I received no e-mail notifications of either success or failure and no indications of problems from dput. The page for the PPA showed no hint of any upload. I banged my head against this problem for several hours, and it stumped a few people (Canonical employees) who I consulted (although in their defense, they didn't know I'd done anything with my keys recently). If I'm not mistaken, the e-mail address of the person who submitted the package is in the .changes file, so it should be possible to fire off an e-mail saying that the OpenPGP key didn't check out. This could save newbies, or fumble-fingered folks like myself, much head-banging.

On 19/12/15 04:25, Rod Smith wrote:
> I just wanted to give this bug a kick, because it's still an issue. I
> accidentally deactivated the wrong OpenPGP key, which resulted in an
> apparently-successful upload that then vanished into a black hole, from
> my point of view -- I received no e-mail notifications of either success
> or failure and no indications of problems from dput. The page for the
> PPA showed no hint of any upload. I banged my head against this problem
> for several hours, and it stumped a few people (Canonical employees) who
> I consulted (although in their defense, they didn't know I'd done
> anything with my keys recently). If I'm not mistaken, the e-mail address
> of the person who submitted the package is in the .changes file, so it
> should be possible to fire off an e-mail saying that the OpenPGP key
> didn't check out. This could save newbies, or fumble-fingered folks like
> myself, much head-banging.

But we can't verify the signature, so anybody could have uploaded that
email address via anonymous FTP. That's not a spam vector that we can
sensibly open ourselves up to.

On Fri, Dec 18, 2015 at 09:33:54PM -0000, William Grant wrote:
> On 19/12/15 04:25, Rod Smith wrote:
> > anything with my keys recently). If I'm not mistaken, the e-mail address
> > of the person who submitted the package is in the .changes file, so it
> > should be possible to fire off an e-mail saying that the OpenPGP key
> > didn't check out. This could save newbies, or fumble-fingered folks like
> > myself, much head-banging.
>
> But we can't verify the signature, so anybody could have uploaded that
> email address via anonymous FTP. That's not a spam vector that we can
> sensibly open ourselves up to.

Could we get Launchpad to create a temporary upload-error URL (say 1
week) explaining what was wrong, and somehow have dput issue an error
message containing that URL? That way you don't need to spam, the
notification is synchronous, and the end-user can learn what is wrong.

I assume the hard part is getting dput to relay the error -- is there
some way of having the server provide an error message verbatim back?
--
Christian Robottom Reis | [+55 16] 3376 0125 | http://async.com.br/~kiko
                        | [+55 16] 991 126 430 | http://launchpad.net/~kiko

The main problem with that approach is that dput is asynchronous: it
drops the package into an upload queue, which is processed at some later
point after dput has disconnected. (It's important that it be that way,
because processing could take quite some time for large packages and we
wouldn't want to break just because somebody's connection failed after
they'd completed the upload but before we managed to process it.)

We could do better for SFTP uploads, as alluded to earlier in this bug
log: while we should still require the upload to be signed, SSH
authentication is enough that we could at least trust it for the purpose
of assuring that we can send notification mail. This would only really
be effective if dput's defaults were changed, which is difficult because
it needs to know the user's Launchpad username, but there are ways to
mitigate that (e.g. prompt at first use) and it would still be a good
start.

The other thing I can think of in addition to this would be a
dismissable notification on your Launchpad user page about upload errors
we think might be related to your account, a bit like the system we
already have for package copy failures. If we also arranged for dput to
tell you that upload errors would be notified on your user page, that
would be a good improvement.

Or perhaps a notification on the ppa page? There's already notifications there for copy failures

Still an issue.

I had to update my GPG key because it had expired but it then was not automatically synced with the Ubuntu keyserver. Therefore an upload to my PPA using dput failed silently.

Any notification anywhere with possibly saying that there was an issue with my key would have saved me a lot of time.

This affected me during some training exercises. I'm doing successful uploads, but things appear to be getting stuck behind the scenes and I have no idea what's going on.

* I did an initial build, and it was handled successfully. (PPA: https://launchpad.net/~vultaire/+archive/ubuntu/experiments, package: python-jblite-0.4.3-1, distro: cosmic)

* I then uploaded versions with a ~distro suffix, e.g. python-jblite-0.4.3-1~trusty. This worked for trusty, xenial, and bionic. ~cosmic did not build at all, despite a successful dput.

* After dropping the previous non-suffixed cosmic package and re-uploading the ~cosmic-suffixed package, I was able to get a new successful build for cosmic.

* However, after doing a README update and trying to upload version 0.4.3-2~<distro>, After doing a README update and trying to upload a second revision of my package for bionic and cosmic, nothing built. Trying re-uploads wouldn't work, trying a different PPA (https://launchpad.net/~vultaire/+archive/ubuntu/jblite) wouldn't work, and doing a version bump to 0.4.3-3~<distro> wouldn't work.

Just to be clear: my point is that with this issue, I'm not sure how you'd debug what goes wrong when you do an upload and nothing ends up in the build queue. I'm at a loss personally.