retry-provisioning doesn't retry failed deployments on MAAS

Why can't juju automatically retry-provisioning? It knows many cases where provisioning failed. juju is retrying hooks now; users rarely need to retry.

Removing 2.1 milestone as we will not be addressing this issue in 2.1.

I hit this today on Juju 2.1.1 and MAAS 2.1.3.
retry-provisioning does nothing and the machine is just down/pending.

I believe the underlying issue is that maas has handed us an 'instance-id' which I think means that we think we have a concrete instance that is running. Which is different from failing-to-get-an-instance. Its possible retry-provisioning handles the latter but not the former.

I should also note that MAAS doesn't hand back 'instance for the request you made' but always hands back an exact identifier for a specific machine. So we have to be a bit careful that 'retry-provisioning' properly decommissions the existing instance id, and can cope with being given back exactly the same instance ID a second time, but this time meaning it is trying again.

The same for tags updated after 'juju deploy'.

Retry-provisioning should re-query machine metadata if said so in my view. This is a manual action and you probably know what you are doing.

Instead, one has to remove-machine --force and add-unit again.

fwiw, I think the internal issue is that MAAS has already given us an
instance-id, so we think the machine is provisioned. Normally for providers
'juju retry-provisioning' probably does do some of what you want, but only
when an instance hasn't yet been assigned.

On Mon, Oct 23, 2017 at 6:59 AM, Dmitrii Shcherbakov <
<email address hidden>> wrote:

> The same for tags updated after 'juju deploy'.
>
> Retry-provisioning should re-query machine metadata if said so in my
> view. This is a manual action and you probably know what you are doing.
>
> Instead, one has to remove-machine --force and add-unit again.
>
> ** Tags added: cpe-onsite
>
> --
> You received this bug notification because you are subscribed to juju.
> Matching subscriptions: juju bugs
> https://bugs.launchpad.net/bugs/1645422
>
> Title:
> retry-provisioning doesn't retry failed deployments on MAAS
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/juju/+bug/1645422/+subscriptions
>

I think we just need to define what it means to "provision" better.

Conceptually, I would use the following definition:

provisioning = <matching a machine by constraints & other criteria> + <successfully deploying once and installing a machine agent>

At least for MAAS it is intuitive in my view.

If I have to reconfigure a machine, doing retry-provisioning also makes sense but with the following logic:

1. get a machine ID;
2. a deployment has failed either automatically or via a manual action before machine/unit agents have started;
3. a user has released the machine in MAAS;
4. reconfigured the machine/swapped out hardware etc.
5. a manual retry-provisioning detected that a given ID is no longer allocated and tried to allocate a new ID.

The target idea here would be that one could write an orchestrator/automation to talk to Juju, see if a deployment has failed, check MAAS to determine if we can recover from a failure, retry-provisioning without affecting a Juju model unit-wise or application-wise.

If a node is not suitable it would be marked as broken by an orchestrator in MAAS and a different node would be picked without making remove-machine --force && add-unit steps.

This is still an issue with Juju 2.7.8 and MAAS 2.8.2.

My occurrence is a transient MAAS failed deployment because of *reasons*, and I want Juju to retry so that I can get a working machine.

I see from bug discussion history that there is some disagreement about what retry-provisioning means or does, and I guess I'll add to the scale that to me I expected it to mean that Juju could re-use the machine slot it has in its model and either fill it with a new instance or reach out to maas and do a release+deploy dance with the instance ID it already has.

Right now nothing happens and there is zero feedback to the user.

Typo in comment #9 juju version is 2.8.7

Bumping importance to Medium to accurately reflect that this is a legitimate issue, but is not in scope for the current roadmap.

(I agree that it would be very nice to fix.)

This is a legitimate issue for us as well (currently contributing to descent into madness) - without this, failed nodes get re-numbered, and targeted placements of units aiming to be on those nodes get wonky despite the --map-machines flag on iterative overlays (up to 5 here for openstack ha with vault and a bunch of other stuff).
The problem also exists with LXD units - Juju has no way to retry those, and that's entirely within its scope of control.

I can confirm this with juju 2.9.11 interacting with MAAS 3.1.0~alpha1.

This Medium-priority bug has not been updated in 60 days, so we're marking it Low importance. If you believe this is incorrect, please update the importance.