kill-controller should take down known hosted models

kill-controller does destroy hosted models, but only if going via the API. If it fails to destroy via the API, it will use the Environ.Destroy method, but only on the admin model.

If the API is down, then there's no guarantee that you'll know about all of the hosted models; the best you could do is run through all of the models cached in models.yaml. Also, this method won't uninstall Juju from any manually provisioned machines.

destroy-controller is meant to be less fire and brimstone. It'll just error out if the API method fails.

"the best you could do is run through all of the models cached in models.yaml" - This is exactly what I was opening the bug for.

Just thinking about this again: we can't necessarily destroy hosted models in this way, because we don't store the hosted model config at the client. As an example, the Azure provider needs the following:
 - the model UUID
 - the model name
 - the controller resource group name
the last one is specific to the Azure provider, and stored in the hosted model config. That's only stored in Mongo.

I'm researching an alternative now: when env.Destroy() is called on the controller model, for it to also destroy hosted model resources. This should be doable by tagging resources with the controller UUID.

Partial fix: https://github.com/juju/juju/pull/5021

This is done for lxd. The other providers are yet to come.

ec2: https://github.com/juju/juju/pull/5197

OpenStack: https://github.com/juju/juju/pull/5290

Azure: https://bugs.launchpad.net/juju-core/+bug/1566011

Try that again...

Azure: https://github.com/juju/juju/pull/5314

Just a quick status update, as I've not been active on this. I'll unassign myself so someone else can pick it up if they have time.

We're still missing support for the following providers: cloudsigma, gce, joyent, maas, vsphere.

Last I looked, gce was going to be a bit difficult because disks cannot be tagged. So disks might leak. We could perhaps use some other API (e.g. GCE's object store) to store information about relationships.

For maas, I think we should predicate any further work on switching over to using tagging; we're still (ab)using MAAS storage. Any provider that supports tags should be fairly straight forward to do. Just apply the tags supplied to StartInstanceParams, and have Environ.DestroyController destroy all resources with the "tags.JujuController" tag matching.

I am going to create individual bugs for outstanding providers and close this one.

For QA, these providers are good to have functional tests:
* Azure;
* Openstack;
* LXD;
* ec2.

MAAS bug: https://bugs.launchpad.net/juju-core/+bug/1611267
GCE bug: https://bugs.launchpad.net/juju-core/+bug/1611269
Cloudsigma bug: https://bugs.launchpad.net/juju-core/+bug/1611271
Joyent bug: https://bugs.launchpad.net/juju-core/+bug/1611273
Vsphere: https://bugs.launchpad.net/juju-core/+bug/1611275