Deleting RSD Pod fails because it tries deleting composed node twice.
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
MAAS |
Fix Released
|
Critical
|
Blake Rouse | ||
MAAS RSD |
Fix Released
|
Critical
|
Blake Rouse |
Bug Description
If an RSD Pod contains more than four composed nodes, Pod deletion will fail. This has to do with a timeout that is being exceeded on the region. Pod deletion doesn't seem to fail when there are four or less composed machines.
When deleting the RSD Pod like so:
$ maas admin pod delete <pod_id>
It fails. Here are the logs from the rackd.log showing this with some additional print statements for debugging:
2017-02-14 21:43:13 stdout: [info] In decompose
2017-02-14 21:43:13 stdout: [info] context: {'node_id': '94', 'power_user': 'admin', 'power_pass': 'admin', 'power_address': '10.3.0.1:8443'}
2017-02-14 21:43:13 stdout: [info] endpoint: b'redfish/
2017-02-14 21:43:13 stdout: [info] method: b'DELETE'
2017-02-14 21:43:13 stdout: [info] uri: b'https:/
2017-02-14 21:43:14 stdout: [info] method: b'GET'
2017-02-14 21:43:14 stdout: [info] uri: b'https:/
2017-02-14 21:43:14 stdout: [info] method: b'GET'
2017-02-14 21:43:14 stdout: [info] uri: b'https:/
2017-02-14 21:43:14 stdout: [info] method: b'GET'
2017-02-14 21:43:14 stdout: [info] uri: b'https:/
2017-02-14 21:43:14 stdout: [info] method: b'GET'
2017-02-14 21:43:14 stdout: [info] uri: b'https:/
2017-02-14 21:43:14 stdout: [info] method: b'GET'
2017-02-14 21:43:14 stdout: [info] uri: b'https:/
2017-02-14 21:43:14 stdout: [info] method: b'GET'
2017-02-14 21:43:14 stdout: [info] uri: b'https:/
2017-02-14 21:43:14 stdout: [info] method: b'GET'
2017-02-14 21:43:14 stdout: [info] uri: b'https:/
2017-02-14 21:43:14 stdout: [info] method: b'GET'
2017-02-14 21:43:14 stdout: [info] uri: b'https:/
2017-02-14 21:43:14 stdout: [info] method: b'GET'
2017-02-14 21:43:14 stdout: [info] uri: b'https:/
2017-02-14 21:43:14 stdout: [info] method: b'GET'
2017-02-14 21:43:14 stdout: [info] uri: b'https:/
2017-02-14 21:43:14 stdout: [info] method: b'GET'
2017-02-14 21:43:14 stdout: [info] uri: b'https:/
2017-02-14 21:43:14 stdout: [info] method: b'GET'
2017-02-14 21:43:14 stdout: [info] uri: b'https:/
2017-02-14 21:43:14 stdout: [info] method: b'GET'
2017-02-14 21:43:14 stdout: [info] uri: b'https:/
2017-02-14 21:43:14 stdout: [info] method: b'GET'
2017-02-14 21:43:14 stdout: [info] uri: b'https:/
2017-02-14 21:43:14 stdout: [info] method: b'GET'
2017-02-14 21:43:14 stdout: [info] uri: b'https:/
2017-02-14 21:43:14 stdout: [info] method: b'GET'
2017-02-14 21:43:14 stdout: [info] uri: b'https:/
2017-02-14 21:43:14 stdout: [info] method: b'GET'
2017-02-14 21:43:14 stdout: [info] uri: b'https:/
2017-02-14 21:43:15 stdout: [info] method: b'GET'
2017-02-14 21:43:15 stdout: [info] uri: b'https:/
2017-02-14 21:43:15 stdout: [info] method: b'GET'
2017-02-14 21:43:15 stdout: [info] uri: b'https:/
2017-02-14 21:43:15 stdout: [info] method: b'GET'
2017-02-14 21:43:15 stdout: [info] uri: b'https:/
2017-02-14 21:43:15 stdout: [info] method: b'GET'
2017-02-14 21:43:15 stdout: [info] uri: b'https:/
2017-02-14 21:43:15 stdout: [info] method: b'GET'
2017-02-14 21:43:15 stdout: [info] uri: b'https:/
2017-02-14 21:43:15 stdout: [info] method: b'GET'
2017-02-14 21:43:15 stdout: [info] uri: b'https:/
2017-02-14 21:43:15 stdout: [info] method: b'GET'
2017-02-14 21:43:15 stdout: [info] uri: b'https:/
2017-02-14 21:43:15 stdout: [info] method: b'GET'
2017-02-14 21:43:15 stdout: [info] uri: b'https:/
2017-02-14 21:43:15 stdout: [info] method: b'GET'
2017-02-14 21:43:15 stdout: [info] uri: b'https:/
2017-02-14 21:43:15 stdout: [info] method: b'GET'
2017-02-14 21:43:15 stdout: [info] uri: b'https:/
2017-02-14 21:43:15 stdout: [info] method: b'GET'
2017-02-14 21:43:15 stdout: [info] uri: b'https:/
2017-02-14 21:43:15 stdout: [info] method: b'GET'
2017-02-14 21:43:15 stdout: [info] uri: b'https:/
2017-02-14 21:43:15 stdout: [info] method: b'GET'
2017-02-14 21:43:15 stdout: [info] uri: b'https:/
2017-02-14 21:43:15 stdout: [info] method: b'GET'
2017-02-14 21:43:15 stdout: [info] uri: b'https:/
2017-02-14 21:43:15 stdout: [info] method: b'GET'
2017-02-14 21:43:15 stdout: [info] uri: b'https:/
2017-02-14 21:43:15 stdout: [info] method: b'GET'
2017-02-14 21:43:15 stdout: [info] uri: b'https:/
2017-02-14 21:43:15 stdout: [info] method: b'GET'
2017-02-14 21:43:15 stdout: [info] uri: b'https:/
2017-02-14 21:43:15 stdout: [info] method: b'GET'
2017-02-14 21:43:15 stdout: [info] uri: b'https:/
2017-02-14 21:43:15 stdout: [info] method: b'GET'
2017-02-14 21:43:15 stdout: [info] uri: b'https:/
2017-02-14 21:43:15 stdout: [info] method: b'GET'
2017-02-14 21:43:15 stdout: [info] uri: b'https:/
2017-02-14 21:43:16 stdout: [info] method: b'GET'
2017-02-14 21:43:16 stdout: [info] uri: b'https:/
2017-02-14 21:43:16 stdout: [info] method: b'GET'
2017-02-14 21:43:16 stdout: [info] uri: b'https:/
2017-02-14 21:43:16 stdout: [info] method: b'GET'
2017-02-14 21:43:16 stdout: [info] uri: b'https:/
2017-02-14 21:43:16 stdout: [info] method: b'GET'
2017-02-14 21:43:16 stdout: [info] uri: b'https:/
2017-02-14 21:43:16 stdout: [info] method: b'GET'
2017-02-14 21:43:16 stdout: [info] uri: b'https:/
2017-02-14 21:43:16 stdout: [info] method: b'GET'
2017-02-14 21:43:16 stdout: [info] uri: b'https:/
2017-02-14 21:43:16 stdout: [info] method: b'GET'
2017-02-14 21:43:16 stdout: [info] uri: b'https:/
2017-02-14 21:43:16 stdout: [info] method: b'GET'
2017-02-14 21:43:16 stdout: [info] uri: b'https:/
2017-02-14 21:43:16 stdout: [info] method: b'GET'
2017-02-14 21:43:16 stdout: [info] uri: b'https:/
2017-02-14 21:43:16 stdout: [info] method: b'GET'
2017-02-14 21:43:16 stdout: [info] uri: b'https:/
2017-02-14 21:43:16 stdout: [info] method: b'GET'
2017-02-14 21:43:16 stdout: [info] uri: b'https:/
2017-02-14 21:43:16 stdout: [info] method: b'GET'
2017-02-14 21:43:16 stdout: [info] uri: b'https:/
2017-02-14 21:43:16 stdout: [info] method: b'GET'
2017-02-14 21:43:16 stdout: [info] uri: b'https:/
2017-02-14 21:43:16 stdout: [info] method: b'GET'
2017-02-14 21:43:16 stdout: [info] uri: b'https:/
2017-02-14 21:43:16 stdout: [info] method: b'GET'
2017-02-14 21:43:16 stdout: [info] uri: b'https:/
2017-02-14 21:43:16 stdout: [info] method: b'GET'
2017-02-14 21:43:16 stdout: [info] uri: b'https:/
2017-02-14 21:43:16 stdout: [info] method: b'GET'
2017-02-14 21:43:16 stdout: [info] uri: b'https:/
2017-02-14 21:43:16 stdout: [info] method: b'GET'
2017-02-14 21:43:16 stdout: [info] uri: b'https:/
2017-02-14 21:43:16 stdout: [info] method: b'GET'
2017-02-14 21:43:16 stdout: [info] uri: b'https:/
2017-02-14 21:43:16 stdout: [info] method: b'GET'
2017-02-14 21:43:16 stdout: [info] uri: b'https:/
2017-02-14 21:43:16 stdout: [info] method: b'GET'
2017-02-14 21:43:16 stdout: [info] uri: b'https:/
2017-02-14 21:43:17 stdout: [info] method: b'GET'
2017-02-14 21:43:17 stdout: [info] uri: b'https:/
2017-02-14 21:43:17 stdout: [info] method: b'GET'
2017-02-14 21:43:17 stdout: [info] uri: b'https:/
2017-02-14 21:43:17 stdout: [info] method: b'GET'
2017-02-14 21:43:17 stdout: [info] uri: b'https:/
2017-02-14 21:43:17 stdout: [info] method: b'GET'
2017-02-14 21:43:17 stdout: [info] uri: b'https:/
2017-02-14 21:43:17 stdout: [info] method: b'GET'
2017-02-14 21:43:17 stdout: [info] uri: b'https:/
2017-02-14 21:43:17 stdout: [info] method: b'GET'
2017-02-14 21:43:17 stdout: [info] uri: b'https:/
2017-02-14 21:43:17 stdout: [info] method: b'GET'
2017-02-14 21:43:17 stdout: [info] uri: b'https:/
2017-02-14 21:43:17 stdout: [info] method: b'GET'
2017-02-14 21:43:17 stdout: [info] uri: b'https:/
2017-02-14 21:43:17 stdout: [info] method: b'GET'
2017-02-14 21:43:17 stdout: [info] uri: b'https:/
2017-02-14 21:43:17 stdout: [info] method: b'GET'
2017-02-14 21:43:17 stdout: [info] uri: b'https:/
2017-02-14 21:43:17 stdout: [info] method: b'GET'
2017-02-14 21:43:17 stdout: [info] uri: b'https:/
2017-02-14 21:43:17 stdout: [info] method: b'GET'
2017-02-14 21:43:17 stdout: [info] uri: b'https:/
2017-02-14 21:43:17 stdout: [info] method: b'GET'
2017-02-14 21:43:17 stdout: [info] uri: b'https:/
2017-02-14 21:43:17 stdout: [info] method: b'GET'
2017-02-14 21:43:17 stdout: [info] uri: b'https:/
2017-02-14 21:43:17 stdout: [info] method: b'GET'
2017-02-14 21:43:17 stdout: [info] uri: b'https:/
2017-02-14 21:43:17 stdout: [info] method: b'GET'
2017-02-14 21:43:17 stdout: [info] uri: b'https:/
2017-02-14 21:43:17 stdout: [info] method: b'GET'
2017-02-14 21:43:17 stdout: [info] uri: b'https:/
2017-02-14 21:43:17 stdout: [info] method: b'GET'
2017-02-14 21:43:17 stdout: [info] uri: b'https:/
2017-02-14 21:43:17 stdout: [info] method: b'GET'
2017-02-14 21:43:17 stdout: [info] uri: b'https:/
2017-02-14 21:43:17 stdout: [info] method: b'GET'
2017-02-14 21:43:17 stdout: [info] uri: b'https:/
2017-02-14 21:43:18 stdout: [info] method: b'GET'
2017-02-14 21:43:18 stdout: [info] uri: b'https:/
2017-02-14 21:43:18 stdout: [info] method: b'GET'
2017-02-14 21:43:18 stdout: [info] uri: b'https:/
2017-02-14 21:43:18 stdout: [info] method: b'GET'
2017-02-14 21:43:18 stdout: [info] uri: b'https:/
2017-02-14 21:43:18 stdout: [info] method: b'GET'
2017-02-14 21:43:18 stdout: [info] uri: b'https:/
2017-02-14 21:43:18 stdout: [info] method: b'GET'
2017-02-14 21:43:18 stdout: [info] uri: b'https:/
2017-02-14 21:43:18 stdout: [info] method: b'GET'
2017-02-14 21:43:18 stdout: [info] uri: b'https:/
2017-02-14 21:43:18 stdout: [info] method: b'GET'
2017-02-14 21:43:18 stdout: [info] uri: b'https:/
2017-02-14 21:43:18 stdout: [info] method: b'GET'
2017-02-14 21:43:18 stdout: [info] uri: b'https:/
2017-02-14 21:43:18 stdout: [info] method: b'GET'
2017-02-14 21:43:18 stdout: [info] uri: b'https:/
2017-02-14 21:43:18 stdout: [info] method: b'GET'
2017-02-14 21:43:18 stdout: [info] uri: b'https:/
2017-02-14 21:43:18 stdout: [info] method: b'GET'
2017-02-14 21:43:18 stdout: [info] uri: b'https:/
2017-02-14 21:43:18 stdout: [info] method: b'GET'
2017-02-14 21:43:18 stdout: [info] uri: b'https:/
2017-02-14 21:43:18 stdout: [info] method: b'GET'
2017-02-14 21:43:18 stdout: [info] uri: b'https:/
2017-02-14 21:43:18 stdout: [info] method: b'GET'
2017-02-14 21:43:18 stdout: [info] uri: b'https:/
2017-02-14 21:43:18 stdout: [info] method: b'GET'
2017-02-14 21:43:18 stdout: [info] uri: b'https:/
2017-02-14 21:43:18 stdout: [info] method: b'GET'
2017-02-14 21:43:18 stdout: [info] uri: b'https:/
2017-02-14 21:43:18 stdout: [info] method: b'GET'
2017-02-14 21:43:18 stdout: [info] uri: b'https:/
2017-02-14 21:43:18 stdout: [info] method: b'GET'
2017-02-14 21:43:18 stdout: [info] uri: b'https:/
2017-02-14 21:43:18 stdout: [info] method: b'GET'
2017-02-14 21:43:18 stdout: [info] uri: b'https:/
2017-02-14 21:43:19 stdout: [info] In decompose
2017-02-14 21:43:19 stdout: [info] context: {'node_id': '94', 'power_user': 'admin', 'power_pass': 'admin', 'power_address': '10.3.0.1:8443'}
2017-02-14 21:43:19 stdout: [info] endpoint: b'redfish/
2017-02-14 21:43:19 stdout: [info] method: b'DELETE'
2017-02-14 21:43:19 stdout: [info] uri: b'https:/
2017-02-14 21:43:19 provisioningser
Traceback (most recent call last):
File "/usr/lib/
self.
File "/usr/lib/
self.
File "/usr/lib/
current.result = callback(
File "/usr/lib/
_inlineCall
--- <exception caught here> ---
File "/usr/lib/
result = result.
File "/usr/lib/
return g.throw(self.type, self.value, self.tb)
File "/usr/lib/
b"DELETE", join(url, endpoint), headers)
twisted.
==> /var/log/
2017-02-14 21:43:19 twisted.python.log: [info] ::ffff:10.150.105.2 - - [14/Feb/
==> /var/log/
sudo: unable to resolve host maas-dev: Connection timed out
==> /var/log/
2017-02-14 21:43:19 maasserver.
==> /var/log/
sudo: unable to resolve host maas-dev: Connection timed out
==> /var/log/
2017-02-14 21:43:21 maasserver.dhcp: [info] Successfully configured DHCPv4 on rack controller 'ssapke'.
2017-02-14 21:43:21 maasserver.dhcp: [info] Successfully configured DHCPv6 on rack controller 'ssapke'.
==> /var/log/
Feb 14 21:43:33 maas-dev maas.node: [info] 138 (No IP): Deleting BMC
==> /var/log/
2017-02-14 21:43:34 stdout: [info] In decompose
2017-02-14 21:43:34 stdout: [info] context: {'node_id': '89', 'power_user': 'admin', 'power_pass': 'admin', 'power_address': '10.3.0.1:8443'}
2017-02-14 21:43:34 stdout: [info] endpoint: b'redfish/
2017-02-14 21:43:34 stdout: [info] method: b'DELETE'
2017-02-14 21:43:34 stdout: [info] uri: b'https:/
2017-02-14 21:43:34 provisioningser
Traceback (most recent call last):
File "/usr/lib/
self.
File "/usr/lib/
self.
File "/usr/lib/
current.result = callback(
File "/usr/lib/
_inlineCall
--- <exception caught here> ---
File "/usr/lib/
result = result.
File "/usr/lib/
return g.throw(self.type, self.value, self.tb)
File "/usr/lib/
b"DELETE", join(url, endpoint), headers)
twisted.
Related branches
- Newell Jensen (community): Approve
-
Diff: 684 lines (+324/-24)14 files modifiedsrc/maasserver/api/pods.py (+9/-1)
src/maasserver/api/tests/test_pods.py (+17/-2)
src/maasserver/enum.py (+11/-0)
src/maasserver/forms/pods.py (+6/-2)
src/maasserver/migrations/builtin/maasserver/0114_node_dynamic_to_creation_type.py (+26/-0)
src/maasserver/models/bmc.py (+115/-3)
src/maasserver/models/node.py (+6/-5)
src/maasserver/models/signals/nodes.py (+3/-1)
src/maasserver/models/signals/tests/test_nodes.py (+15/-3)
src/maasserver/models/tests/test_bmc.py (+109/-2)
src/maasserver/models/tests/test_node.py (+3/-1)
src/maasserver/testing/factory.py (+2/-2)
src/maasserver/websockets/handlers/device.py (+1/-1)
src/maasserver/websockets/handlers/machine.py (+1/-1)
Changed in maas: | |
status: | New → Confirmed |
importance: | Undecided → Critical |
assignee: | nobody → Blake Rouse (blake-rouse) |
description: | updated |
Changed in maas: | |
status: | Confirmed → In Progress |
Changed in maasrsd: | |
status: | Confirmed → In Progress |
Changed in maas: | |
milestone: | none → 2.2.0 |
Changed in maasrsd: | |
milestone: | none → beta |
Changed in maas: | |
status: | In Progress → Fix Committed |
Changed in maasrsd: | |
status: | In Progress → Fix Committed |
Changed in maas: | |
status: | Fix Committed → Fix Released |
Changed in maasrsd: | |
status: | Fix Committed → Fix Released |