Machine stuck in 'pending' state cannot be terminated

Bug #1214651 reported by Adam Gandelman
This bug report is a duplicate of:  Bug #1089291: destroy-machine --force. Edit Remove
16
This bug affects 3 people
Affects Status Importance Assigned to Milestone
juju-core
Triaged
High
Unassigned

Bug Description

Using juju-core 1.13.1-1~1628~raring1 against an OpenStack provider. Sometimes when provisioning many machines, some instances fail to get an IP. The result is an instance that is 'Active' in OpenStack's eyes, but still 'Pending' in Juju's. If I later attempt to destroy the service unit and terminate the machine, the unit is destroyed but the machine never terminates. Instead, I end up with something like:

environment: precise
machines:
  "0":
    agent-state: started
    agent-version: 1.13.1
    dns-name: 10.5.0.4
    instance-id: 982d3feb-fa72-4c26-911b-48023316d5d1
    series: precise
    hardware: arch=amd64 cpu-cores=1 mem=512M
  "9":
    agent-state: pending
    dns-name: 10.5.0.14
    instance-id: c97fd63a-fb33-4071-bcf6-4d001cc21013
    life: dying
    series: precise
    hardware: arch=amd64 cpu-cores=1 mem=2048M

Attempting to re-terminate the machine does not help. If I terminate the instance manually via nova, the machine stays in the environment as pending.

FWIW, I am using juju-deployer (+ python-jujuclient) to reset the enviornment.

Attached is machine-0.log.

Revision history for this message
Adam Gandelman (gandelman-a) wrote :
William Reade (fwereade)
Changed in juju-core:
status: New → Triaged
importance: Undecided → High
Revision history for this message
Dave Cheney (dave-cheney) wrote :
Revision history for this message
Canonical Solutions QA Bot (oil-ci-bot) wrote :

Seems to also happen when MAAS provider returns errors that block the machine from provisioning. Seems to be a general issue with unreliable provisioning.

tags: added: theme-oil
Revision history for this message
Curtis Hovey (sinzui) wrote :

We might want to treat this as dupe of bug 1089291. When Juju/machine is unresponsive, we need the power to make it leap to its death.

Curtis Hovey (sinzui)
tags: added: destroy-machine
Revision history for this message
Curtis Hovey (sinzui) wrote :

This bug also overlaps with bug 1205451.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.