ERROR some models are no longer visible

Bug #1795495 reported by Adam Israel
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Canonical Juju
Fix Released
Medium
Anastasia

Bug Description

Version: Juju 2.5-beta1-bionic-amd64 installed from the snap edge channel (2.5-beta1+develop-c25711a)

# Expected behavior:

I find it useful to destroy and recreate a model when I'm doing testing, and I typically squash the two commands on one line. This has worked fine until I switched to the edge channel

$ juju destroy-model default -y && juju add-model default
Destroying model
Waiting on model to be removed...
Waiting on model to be removed...
Model destroyed.
Uploading credential 'localhost/admin/localhost' to controller
ERROR some models are no longer visible

The default model is destroyed, but no new model has been created.

It seems like a race condition; if I run the two commands separately, with a wait between them, it works as expected.

# This will fail with the same error above
juju destroy-model default -y
juju add-model default

# This works as expected
juju destroy-model default -y
sleep 5
juju add-model default

Tags: osm
Adam Israel (aisrael)
description: updated
Revision history for this message
Tim Penhey (thumper) wrote :

This looks like a race in the uploading of the credentials where it is trying to update the ones that are there. The existing default model is in the process of being removed, so is probably marked as dead, and the credential update isn't allowing for the dead model, or it is iterating over them and the model is removed from underneath it by the undertaker.

Changed in juju:
status: New → Triaged
importance: Undecided → Medium
Changed in juju:
assignee: nobody → Anastasia (anastasia-macmood)
status: Triaged → In Progress
Revision history for this message
Anastasia (anastasia-macmood) wrote :

This is a great report \o/ It highlighted couple of things... Thank you, @Adam Israel!

For one, the failure itself... We have recently added a check to verify that a cloud credential works for all the models that use it whenever we update credential content. This check should work on all models. Of course, in CI environment models get destroyed and added quickly. So, the failure observed highlighted that a dead model was being picked up for this check. Thinking about it, a dead model is of no interest when checking credential - dead model's instances are already been terminated or are in the process of being terminated. The PR that excludes dead models from credential's models is https://github.com/juju/juju/pull/9284

For two, when adding a model we should not be implicitly updating cloud credential content stored on the controller from the client. This update (and/or upload) should only happen when a user explicitly states a credential via --credential option on 'juju add-model' command. When this option is not supplied, we should be using a CONTROLLER stored default credential for this user, bug # 1796618.

Changed in juju:
status: In Progress → Fix Committed
milestone: none → 2.5-beta1
Changed in juju:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.