Comment 3 for bug 1727693

On Fri, Nov 24, 2017 at 03:26:40PM -0000, Nicholas Skaggs wrote:
> The best protection for this would be an autopkgtest that would block
> landing of distro-info-data. It's clear the existing tests didn't stop
> this, although as mentioned, there is are future-provider tests that
> play with this idea. Ensuring juju works and responds as expected would

I suggest that you extend the future-provider tests to also manipulate
the other areas that Juju pays attention to, such as the LTS label and
the --devel label. If you aren't already, you should adopt the my
future-provider type tests into your upstream CI.

Since Juju is no longer maintained as a distribution package, I don't
think you have the option of blocking anything landing in the
distribution itself - certainly not a correct update to
distro-info-data - using autopkgtests. You have (for good reason)
de-coupled the Juju distribution from the Ubuntu distribution by using
snaps. A consequence of that de-coupling is that as the distribution
moves you need to actively keep up and not break, rather than passively
relying on autopkgtests and fixing issues as you update packages in the
distribution.

You might be able to arrange an additional hook to britney to check
and report against Juju CI before proposed migration. But I don't think
the autopkgtest infrastructure hooks are currently suitable for that.

This doesn't apply to existing Juju packages in the distribution that
you're still maintaining, but I think we need a long term solution here
rather than a short term one.

I think you were lucky that on this occasion it happened to be possible
to work around your bug by having Ubuntu revert its update temporarily.
I don't think this is necessarily going to be the case for your next
bug. It doesn't sound to me that your CI has good (or any!) coverage for
this class of bug, or that Juju's assumptions about distribution
releases is now correct.