attempting to delete locked maas node results in juju marking valid cloud credentials as invalid

Bug #1952792 reported by Steven Parker
This bug affects 6 people
Affects Status Importance Assigned to Milestone
Canonical Juju

Bug Description

Replicate with MAAS node that has been locked.

Remove node with:
    juju remove-machine and/or --force

juju status will yield
  foundations-maas maas_cloud 2.9.18 unsupported 22:01:30Z suspended since cloud credential is not valid

  juju update-credentials maas_cloud maas_cloud_credentials
  juju set-credential -m lma maas_cloud maas_cloud_credentials

NOTE: Loss of cloud credentials can lead to actions such as add-machine, remove-machine failing to complete or error out. These actions simply hang. In fact remove-machine --force will also fail.

Revision history for this message
Ian Booth (wallyworld) wrote :

If Juju makes an cloud (MAAS in this case) API call, if the cloud responses with codes 401, 403 etc then Juju will deem the credential to be invalid as the api call was rejected as unauthorised.

It seems that here MAAS might be responding to attempt to delete a locked node with 403, thus triggering the juju to deem the credential as invalid. We need to audit what return codes MAAS uses and see what's appropriate to use to trigger the invalid credential behaviour in juju.

tags: added: maas-provider
Changed in juju:
status: New → Triaged
importance: Undecided → High
milestone: none → 2.9.23
Changed in juju:
milestone: 2.9.23 → 2.9.24
Changed in juju:
milestone: 2.9.24 → 2.9.25
Changed in juju:
milestone: 2.9.25 → 2.9.26
Changed in juju:
milestone: 2.9.26 → 2.9.27
Revision history for this message
Steven Parker (sbparke) wrote :

Further work around if needed is

If nodes are out of the model you'll get an error

Credential invalid for:
    couldn't find instance "bga76y" for machine 14
    couldn't find instance "h3hbrd" for machine 13
Failed models may require a different credential.
Use ‘juju set-credential’ to change credential for these models before repeating this update.

So you'll need to force in this case.

juju update-credentials maas_cloud maas_cloud_credentials --force

Changed in juju:
milestone: 2.9.27 → 2.9.28
Changed in juju:
milestone: 2.9.28 → 2.9.29
Changed in juju:
milestone: 2.9.29 → 2.9.30
Ian Booth (wallyworld)
Changed in juju:
milestone: 2.9.30 → 2.9-next
Steven Parker (sbparke)
description: updated
description: updated
Tom Haddon (mthaddon)
tags: added: canonical-is
Ian Booth (wallyworld)
Changed in juju:
milestone: 2.9-next → none
Revision history for this message
John Lettman (jplettman) wrote :

It would be a nice improvement for Juju to warn that the machine is locked.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.