Migrating model between controllers in conjure-up/novalxd openstack not complete after 3 hours stuck on transferring ownership of cloud resources to target controller
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
Canonical Juju |
Fix Released
|
High
|
Christian Muirhead |
Bug Description
Migrating a simple model comprising of 1 instance with 3 charms deployed still running, apparently stuck after 3 hours with juju show-model showing status: transferring ownership of cloud resources to target controller
Juju status on the source controller shows:
gareth@
Model Controller Cloud/Region Version Notes
mojo-how-to mojo-test-migrate mystack/RegionOne 2.1.2.1 migrating: successful, timed out waiting for agents to report
App Version Status Scale Charm Store Rev OS Notes
apache2 unknown 0/1 apache2 local 0 ubuntu exposed
content-fetcher unknown 0/1 content-fetcher local 0 ubuntu
nrpe unknown 0/1 nrpe local 0 ubuntu
Unit Workload Agent Machine Public address Ports Message
apache2/0 unknown lost 0 10.101.0.75 80/tcp agent lost, see 'juju show-status-log apache2/0'
content-fetcher/0 unknown lost 10.101.0.75 agent lost, see 'juju show-status-log content-fetcher/0'
nrpe/0 unknown lost 10.101.0.75 agent lost, see 'juju show-status-log nrpe/0'
Machine State DNS Inst id Series AZ
0 down 10.101.0.75 291524ae-
Relation Provides Consumes Type
general-info apache2 content-fetcher subordinate
nrpe-external-
gareth@
mojo-how-to:
name: mojo-how-to
model-uuid: 5ec26299-
controller-uuid: 5431e276-
controller-name: mojo-test-migrate
owner: admin
cloud: mystack
region: RegionOne
type: openstack
life: alive
status:
current: available
since: 3 hours ago
migration: transferring ownership of cloud resources to target controller
migration-
users:
admin:
display-name: admin
access: admin
last-
machines:
"0":
cores: 1
If I juju switch to the "new" controller the model status looks good and juju status etc work:
gareth@
mojo-test-
gareth@
mojo-test (controller) -> mojo-test:
gareth@
Model Controller Cloud/Region Version
mojo-how-to mojo-test mystack/RegionOne 2.1.2.1
App Version Status Scale Charm Store Rev OS Notes
apache2 waiting 1 apache2 local 0 ubuntu exposed
content-fetcher waiting 1 content-fetcher local 0 ubuntu
nrpe waiting 1 nrpe local 0 ubuntu
Unit Workload Agent Machine Public address Ports Message
apache2/0* unknown idle 0 10.101.0.75 80/tcp
content-
nrpe/0* unknown idle 10.101.0.75
Machine State DNS Inst id Series AZ
0 started 10.101.0.75 291524ae-
Relation Provides Consumes Type
general-info apache2 content-fetcher subordinate
nrpe-external-
gareth@
mojo-how-to:
name: mojo-how-to
model-uuid: 5ec26299-
controller-uuid: 719cc4d6-
controller-name: mojo-test
owner: admin
cloud: mystack
region: RegionOne
type: openstack
life: alive
status:
current: available
since: 3 hours ago
users:
admin:
display-name: admin
access: admin
last-
machines:
"0":
cores: 1
Checking machine-0 on the old controller this appears to be an issue with volumes not being available, which is understandable given a conjure-up/nova-lxd environment doesn't have block storage deployed.
2017-03-29 09:13:44 ERROR juju.worker.
2017-03-29 09:13:47 ERROR juju.worker.
2017-03-29 09:13:47 ERROR juju.worker.
The above seemingly loops forever.
This shouldn't affect migration though?!?
Changed in juju: | |
status: | New → Triaged |
importance: | Undecided → High |
milestone: | none → 2.2-beta3 |
tags: | added: eda model-migration |
Changed in juju: | |
status: | Triaged → Incomplete |
milestone: | 2.2-beta3 → none |
Changed in juju: | |
status: | New → In Progress |
assignee: | nobody → Christian Muirhead (2-xtian) |
Changed in juju: | |
milestone: | none → 2.2-beta3 |
Changed in juju: | |
status: | In Progress → Fix Committed |
Changed in juju: | |
status: | Fix Committed → Fix Released |
We're looking into this. Could you provide some of the controller logs please?
First turn up the log level for the controller: config= "<root> =DEBUG"
juju model-config -m controller logging-
Then wait 30s for the migrationmaster to restart a few times and grab the relevant logs:
juju debug-log -m controller --replay | grep migrat > migration.log
Please compress and attach the log here.