MAAS RSD

  1. Bug #1664732
  2. Activity log

Activity log for bug #1664732

Date Who What changed Old value New value Message
2017-02-14 22:15:01 Newell Jensen bug added bug
2017-02-14 22:15:11 Newell Jensen bug task added maas
2017-02-14 22:15:17 Newell Jensen maas: status New Confirmed
2017-02-14 22:15:19 Newell Jensen maas: importance Undecided Critical
2017-02-14 22:15:26 Newell Jensen maas: assignee Blake Rouse (blake-rouse)
2017-02-15 19:27:12 Newell Jensen description When deleting the RSD Pod like so: $ maas admin pod delete <pod_id> It fails. Here are the logs from the rackd.log showing this with some additional print statements for debugging: 2017-02-14 21:43:13 stdout: [info] In decompose 2017-02-14 21:43:13 stdout: [info] context: {'node_id': '94', 'power_user': 'admin', 'power_pass': 'admin', 'power_address': '10.3.0.1:8443'} 2017-02-14 21:43:13 stdout: [info] endpoint: b'redfish/v1/Nodes/94' 2017-02-14 21:43:13 stdout: [info] method: b'DELETE' 2017-02-14 21:43:13 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Nodes/94' 2017-02-14 21:43:14 stdout: [info] method: b'GET' 2017-02-14 21:43:14 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems' 2017-02-14 21:43:14 stdout: [info] method: b'GET' 2017-02-14 21:43:14 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/113/Memory' 2017-02-14 21:43:14 stdout: [info] method: b'GET' 2017-02-14 21:43:14 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/113/Memory/373' 2017-02-14 21:43:14 stdout: [info] method: b'GET' 2017-02-14 21:43:14 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/113/Memory/374' 2017-02-14 21:43:14 stdout: [info] method: b'GET' 2017-02-14 21:43:14 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/113/Memory/375' 2017-02-14 21:43:14 stdout: [info] method: b'GET' 2017-02-14 21:43:14 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/113/Memory/376' 2017-02-14 21:43:14 stdout: [info] method: b'GET' 2017-02-14 21:43:14 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/113/Processors' 2017-02-14 21:43:14 stdout: [info] method: b'GET' 2017-02-14 21:43:14 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/113/Processors/225' 2017-02-14 21:43:14 stdout: [info] method: b'GET' 2017-02-14 21:43:14 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/113/Processors/226' 2017-02-14 21:43:14 stdout: [info] method: b'GET' 2017-02-14 21:43:14 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/113/Adapters' 2017-02-14 21:43:14 stdout: [info] method: b'GET' 2017-02-14 21:43:14 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/113/Adapters/51/Devices' 2017-02-14 21:43:14 stdout: [info] method: b'GET' 2017-02-14 21:43:14 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/113/Adapters/52/Devices' 2017-02-14 21:43:14 stdout: [info] method: b'GET' 2017-02-14 21:43:14 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/113/Adapters/52/Devices/136' 2017-02-14 21:43:14 stdout: [info] method: b'GET' 2017-02-14 21:43:14 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/120/Memory' 2017-02-14 21:43:14 stdout: [info] method: b'GET' 2017-02-14 21:43:14 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/120/Memory/413' 2017-02-14 21:43:14 stdout: [info] method: b'GET' 2017-02-14 21:43:14 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/120/Memory/414' 2017-02-14 21:43:14 stdout: [info] method: b'GET' 2017-02-14 21:43:14 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/120/Memory/415' 2017-02-14 21:43:15 stdout: [info] method: b'GET' 2017-02-14 21:43:15 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/120/Memory/416' 2017-02-14 21:43:15 stdout: [info] method: b'GET' 2017-02-14 21:43:15 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/120/Processors' 2017-02-14 21:43:15 stdout: [info] method: b'GET' 2017-02-14 21:43:15 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/120/Processors/239' 2017-02-14 21:43:15 stdout: [info] method: b'GET' 2017-02-14 21:43:15 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/120/Processors/240' 2017-02-14 21:43:15 stdout: [info] method: b'GET' 2017-02-14 21:43:15 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/120/Adapters' 2017-02-14 21:43:15 stdout: [info] method: b'GET' 2017-02-14 21:43:15 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/120/Adapters/45/Devices' 2017-02-14 21:43:15 stdout: [info] method: b'GET' 2017-02-14 21:43:15 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/120/Adapters/45/Devices/133' 2017-02-14 21:43:15 stdout: [info] method: b'GET' 2017-02-14 21:43:15 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/120/Adapters/46/Devices' 2017-02-14 21:43:15 stdout: [info] method: b'GET' 2017-02-14 21:43:15 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/121/Memory' 2017-02-14 21:43:15 stdout: [info] method: b'GET' 2017-02-14 21:43:15 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/121/Memory/418' 2017-02-14 21:43:15 stdout: [info] method: b'GET' 2017-02-14 21:43:15 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/121/Memory/419' 2017-02-14 21:43:15 stdout: [info] method: b'GET' 2017-02-14 21:43:15 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/121/Memory/420' 2017-02-14 21:43:15 stdout: [info] method: b'GET' 2017-02-14 21:43:15 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/121/Memory/417' 2017-02-14 21:43:15 stdout: [info] method: b'GET' 2017-02-14 21:43:15 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/121/Processors' 2017-02-14 21:43:15 stdout: [info] method: b'GET' 2017-02-14 21:43:15 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/121/Processors/241' 2017-02-14 21:43:15 stdout: [info] method: b'GET' 2017-02-14 21:43:15 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/121/Processors/242' 2017-02-14 21:43:15 stdout: [info] method: b'GET' 2017-02-14 21:43:15 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/121/Adapters' 2017-02-14 21:43:15 stdout: [info] method: b'GET' 2017-02-14 21:43:15 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/121/Adapters/41/Devices' 2017-02-14 21:43:15 stdout: [info] method: b'GET' 2017-02-14 21:43:15 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/121/Adapters/42/Devices' 2017-02-14 21:43:15 stdout: [info] method: b'GET' 2017-02-14 21:43:15 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/121/Adapters/42/Devices/131' 2017-02-14 21:43:16 stdout: [info] method: b'GET' 2017-02-14 21:43:16 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/122/Memory' 2017-02-14 21:43:16 stdout: [info] method: b'GET' 2017-02-14 21:43:16 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/122/Memory/422' 2017-02-14 21:43:16 stdout: [info] method: b'GET' 2017-02-14 21:43:16 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/122/Memory/423' 2017-02-14 21:43:16 stdout: [info] method: b'GET' 2017-02-14 21:43:16 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/122/Memory/424' 2017-02-14 21:43:16 stdout: [info] method: b'GET' 2017-02-14 21:43:16 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/122/Memory/421' 2017-02-14 21:43:16 stdout: [info] method: b'GET' 2017-02-14 21:43:16 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/122/Processors' 2017-02-14 21:43:16 stdout: [info] method: b'GET' 2017-02-14 21:43:16 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/122/Processors/243' 2017-02-14 21:43:16 stdout: [info] method: b'GET' 2017-02-14 21:43:16 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/122/Processors/244' 2017-02-14 21:43:16 stdout: [info] method: b'GET' 2017-02-14 21:43:16 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/122/Adapters' 2017-02-14 21:43:16 stdout: [info] method: b'GET' 2017-02-14 21:43:16 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/122/Adapters/43/Devices' 2017-02-14 21:43:16 stdout: [info] method: b'GET' 2017-02-14 21:43:16 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/122/Adapters/43/Devices/132' 2017-02-14 21:43:16 stdout: [info] method: b'GET' 2017-02-14 21:43:16 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/122/Adapters/44/Devices' 2017-02-14 21:43:16 stdout: [info] method: b'GET' 2017-02-14 21:43:16 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/123/Memory' 2017-02-14 21:43:16 stdout: [info] method: b'GET' 2017-02-14 21:43:16 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/123/Memory/425' 2017-02-14 21:43:16 stdout: [info] method: b'GET' 2017-02-14 21:43:16 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/123/Memory/426' 2017-02-14 21:43:16 stdout: [info] method: b'GET' 2017-02-14 21:43:16 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/123/Memory/427' 2017-02-14 21:43:16 stdout: [info] method: b'GET' 2017-02-14 21:43:16 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/123/Memory/428' 2017-02-14 21:43:16 stdout: [info] method: b'GET' 2017-02-14 21:43:16 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/123/Processors' 2017-02-14 21:43:16 stdout: [info] method: b'GET' 2017-02-14 21:43:16 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/123/Processors/246' 2017-02-14 21:43:16 stdout: [info] method: b'GET' 2017-02-14 21:43:16 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/123/Processors/245' 2017-02-14 21:43:16 stdout: [info] method: b'GET' 2017-02-14 21:43:16 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/123/Adapters' 2017-02-14 21:43:17 stdout: [info] method: b'GET' 2017-02-14 21:43:17 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/123/Adapters/47/Devices' 2017-02-14 21:43:17 stdout: [info] method: b'GET' 2017-02-14 21:43:17 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/123/Adapters/47/Devices/134' 2017-02-14 21:43:17 stdout: [info] method: b'GET' 2017-02-14 21:43:17 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/123/Adapters/48/Devices' 2017-02-14 21:43:17 stdout: [info] method: b'GET' 2017-02-14 21:43:17 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/127/Memory' 2017-02-14 21:43:17 stdout: [info] method: b'GET' 2017-02-14 21:43:17 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/127/Memory/437' 2017-02-14 21:43:17 stdout: [info] method: b'GET' 2017-02-14 21:43:17 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/127/Memory/438' 2017-02-14 21:43:17 stdout: [info] method: b'GET' 2017-02-14 21:43:17 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/127/Memory/439' 2017-02-14 21:43:17 stdout: [info] method: b'GET' 2017-02-14 21:43:17 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/127/Memory/440' 2017-02-14 21:43:17 stdout: [info] method: b'GET' 2017-02-14 21:43:17 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/127/Processors' 2017-02-14 21:43:17 stdout: [info] method: b'GET' 2017-02-14 21:43:17 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/127/Processors/253' 2017-02-14 21:43:17 stdout: [info] method: b'GET' 2017-02-14 21:43:17 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/127/Processors/254' 2017-02-14 21:43:17 stdout: [info] method: b'GET' 2017-02-14 21:43:17 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/127/Adapters' 2017-02-14 21:43:17 stdout: [info] method: b'GET' 2017-02-14 21:43:17 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/127/Adapters/55/Devices' 2017-02-14 21:43:17 stdout: [info] method: b'GET' 2017-02-14 21:43:17 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/127/Adapters/55/Devices/138' 2017-02-14 21:43:17 stdout: [info] method: b'GET' 2017-02-14 21:43:17 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/127/Adapters/56/Devices' 2017-02-14 21:43:17 stdout: [info] method: b'GET' 2017-02-14 21:43:17 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/128/Memory' 2017-02-14 21:43:17 stdout: [info] method: b'GET' 2017-02-14 21:43:17 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/128/Memory/446' 2017-02-14 21:43:17 stdout: [info] method: b'GET' 2017-02-14 21:43:17 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/128/Memory/447' 2017-02-14 21:43:17 stdout: [info] method: b'GET' 2017-02-14 21:43:17 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/128/Memory/448' 2017-02-14 21:43:17 stdout: [info] method: b'GET' 2017-02-14 21:43:17 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/128/Memory/445' 2017-02-14 21:43:18 stdout: [info] method: b'GET' 2017-02-14 21:43:18 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/128/Processors' 2017-02-14 21:43:18 stdout: [info] method: b'GET' 2017-02-14 21:43:18 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/128/Processors/255' 2017-02-14 21:43:18 stdout: [info] method: b'GET' 2017-02-14 21:43:18 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/128/Processors/256' 2017-02-14 21:43:18 stdout: [info] method: b'GET' 2017-02-14 21:43:18 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/128/Adapters' 2017-02-14 21:43:18 stdout: [info] method: b'GET' 2017-02-14 21:43:18 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/128/Adapters/57/Devices' 2017-02-14 21:43:18 stdout: [info] method: b'GET' 2017-02-14 21:43:18 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/128/Adapters/58/Devices' 2017-02-14 21:43:18 stdout: [info] method: b'GET' 2017-02-14 21:43:18 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/128/Adapters/58/Devices/139' 2017-02-14 21:43:18 stdout: [info] method: b'GET' 2017-02-14 21:43:18 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/129/Memory' 2017-02-14 21:43:18 stdout: [info] method: b'GET' 2017-02-14 21:43:18 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/129/Memory/449' 2017-02-14 21:43:18 stdout: [info] method: b'GET' 2017-02-14 21:43:18 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/129/Memory/450' 2017-02-14 21:43:18 stdout: [info] method: b'GET' 2017-02-14 21:43:18 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/129/Memory/451' 2017-02-14 21:43:18 stdout: [info] method: b'GET' 2017-02-14 21:43:18 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/129/Memory/452' 2017-02-14 21:43:18 stdout: [info] method: b'GET' 2017-02-14 21:43:18 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/129/Processors' 2017-02-14 21:43:18 stdout: [info] method: b'GET' 2017-02-14 21:43:18 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/129/Processors/257' 2017-02-14 21:43:18 stdout: [info] method: b'GET' 2017-02-14 21:43:18 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/129/Processors/258' 2017-02-14 21:43:18 stdout: [info] method: b'GET' 2017-02-14 21:43:18 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/129/Adapters' 2017-02-14 21:43:18 stdout: [info] method: b'GET' 2017-02-14 21:43:18 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/129/Adapters/60/Devices' 2017-02-14 21:43:18 stdout: [info] method: b'GET' 2017-02-14 21:43:18 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/129/Adapters/59/Devices' 2017-02-14 21:43:18 stdout: [info] method: b'GET' 2017-02-14 21:43:18 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/129/Adapters/59/Devices/140' 2017-02-14 21:43:19 stdout: [info] In decompose 2017-02-14 21:43:19 stdout: [info] context: {'node_id': '94', 'power_user': 'admin', 'power_pass': 'admin', 'power_address': '10.3.0.1:8443'} 2017-02-14 21:43:19 stdout: [info] endpoint: b'redfish/v1/Nodes/94' 2017-02-14 21:43:19 stdout: [info] method: b'DELETE' 2017-02-14 21:43:19 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Nodes/94' 2017-02-14 21:43:19 provisioningserver.rpc.pods: [critical] Failed to decompose machine. Traceback (most recent call last): File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 434, in errback self._startRunCallbacks(fail) File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 501, in _startRunCallbacks self._runCallbacks() File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 588, in _runCallbacks current.result = callback(current.result, *args, **kw) File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 1184, in gotResult _inlineCallbacks(r, g, deferred) --- <exception caught here> --- File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 1126, in _inlineCallbacks result = result.throwExceptionIntoGenerator(g) File "/usr/lib/python3/dist-packages/twisted/python/failure.py", line 389, in throwExceptionIntoGenerator return g.throw(self.type, self.value, self.tb) File "/usr/lib/python3/dist-packages/provisioningserver/drivers/pod/rsd.py", line 683, in decompose b"DELETE", join(url, endpoint), headers) twisted.web.client.PartialDownloadError: 404 Not Found ==> /var/log/maas/regiond.log <== 2017-02-14 21:43:19 twisted.python.log: [info] ::ffff:10.150.105.2 - - [14/Feb/2017:21:43:18 +0000] "DELETE /MAAS/api/2.0/pods/138/ HTTP/1.1" 503 74 "-" "Python-httplib2/0.9.1 (gzip)" ==> /var/log/maas/rackd.log <== sudo: unable to resolve host maas-dev: Connection timed out ==> /var/log/maas/regiond.log <== 2017-02-14 21:43:19 maasserver.region_controller: [info] Successfully configured DNS. ==> /var/log/maas/rackd.log <== sudo: unable to resolve host maas-dev: Connection timed out ==> /var/log/maas/regiond.log <== 2017-02-14 21:43:21 maasserver.dhcp: [info] Successfully configured DHCPv4 on rack controller 'ssapke'. 2017-02-14 21:43:21 maasserver.dhcp: [info] Successfully configured DHCPv6 on rack controller 'ssapke'. ==> /var/log/maas/maas.log <== Feb 14 21:43:33 maas-dev maas.node: [info] 138 (No IP): Deleting BMC ==> /var/log/maas/rackd.log <== 2017-02-14 21:43:34 stdout: [info] In decompose 2017-02-14 21:43:34 stdout: [info] context: {'node_id': '89', 'power_user': 'admin', 'power_pass': 'admin', 'power_address': '10.3.0.1:8443'} 2017-02-14 21:43:34 stdout: [info] endpoint: b'redfish/v1/Nodes/89' 2017-02-14 21:43:34 stdout: [info] method: b'DELETE' 2017-02-14 21:43:34 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Nodes/89' 2017-02-14 21:43:34 provisioningserver.rpc.pods: [critical] Failed to decompose machine. Traceback (most recent call last): File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 434, in errback self._startRunCallbacks(fail) File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 501, in _startRunCallbacks self._runCallbacks() File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 588, in _runCallbacks current.result = callback(current.result, *args, **kw) File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 1184, in gotResult _inlineCallbacks(r, g, deferred) --- <exception caught here> --- File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 1126, in _inlineCallbacks result = result.throwExceptionIntoGenerator(g) File "/usr/lib/python3/dist-packages/twisted/python/failure.py", line 389, in throwExceptionIntoGenerator return g.throw(self.type, self.value, self.tb) File "/usr/lib/python3/dist-packages/provisioningserver/drivers/pod/rsd.py", line 683, in decompose b"DELETE", join(url, endpoint), headers) twisted.web.client.PartialDownloadError: 404 Not Found If an RSD Pod contains more than four composed nodes, Pod deletion will fail. This has to do with a timeout that is being exceeded on the region. Pod deletion doesn't seem to fail when there are four or less composed machines. When deleting the RSD Pod like so: $ maas admin pod delete <pod_id> It fails. Here are the logs from the rackd.log showing this with some additional print statements for debugging: 2017-02-14 21:43:13 stdout: [info] In decompose 2017-02-14 21:43:13 stdout: [info] context: {'node_id': '94', 'power_user': 'admin', 'power_pass': 'admin', 'power_address': '10.3.0.1:8443'} 2017-02-14 21:43:13 stdout: [info] endpoint: b'redfish/v1/Nodes/94' 2017-02-14 21:43:13 stdout: [info] method: b'DELETE' 2017-02-14 21:43:13 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Nodes/94' 2017-02-14 21:43:14 stdout: [info] method: b'GET' 2017-02-14 21:43:14 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems' 2017-02-14 21:43:14 stdout: [info] method: b'GET' 2017-02-14 21:43:14 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/113/Memory' 2017-02-14 21:43:14 stdout: [info] method: b'GET' 2017-02-14 21:43:14 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/113/Memory/373' 2017-02-14 21:43:14 stdout: [info] method: b'GET' 2017-02-14 21:43:14 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/113/Memory/374' 2017-02-14 21:43:14 stdout: [info] method: b'GET' 2017-02-14 21:43:14 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/113/Memory/375' 2017-02-14 21:43:14 stdout: [info] method: b'GET' 2017-02-14 21:43:14 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/113/Memory/376' 2017-02-14 21:43:14 stdout: [info] method: b'GET' 2017-02-14 21:43:14 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/113/Processors' 2017-02-14 21:43:14 stdout: [info] method: b'GET' 2017-02-14 21:43:14 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/113/Processors/225' 2017-02-14 21:43:14 stdout: [info] method: b'GET' 2017-02-14 21:43:14 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/113/Processors/226' 2017-02-14 21:43:14 stdout: [info] method: b'GET' 2017-02-14 21:43:14 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/113/Adapters' 2017-02-14 21:43:14 stdout: [info] method: b'GET' 2017-02-14 21:43:14 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/113/Adapters/51/Devices' 2017-02-14 21:43:14 stdout: [info] method: b'GET' 2017-02-14 21:43:14 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/113/Adapters/52/Devices' 2017-02-14 21:43:14 stdout: [info] method: b'GET' 2017-02-14 21:43:14 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/113/Adapters/52/Devices/136' 2017-02-14 21:43:14 stdout: [info] method: b'GET' 2017-02-14 21:43:14 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/120/Memory' 2017-02-14 21:43:14 stdout: [info] method: b'GET' 2017-02-14 21:43:14 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/120/Memory/413' 2017-02-14 21:43:14 stdout: [info] method: b'GET' 2017-02-14 21:43:14 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/120/Memory/414' 2017-02-14 21:43:14 stdout: [info] method: b'GET' 2017-02-14 21:43:14 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/120/Memory/415' 2017-02-14 21:43:15 stdout: [info] method: b'GET' 2017-02-14 21:43:15 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/120/Memory/416' 2017-02-14 21:43:15 stdout: [info] method: b'GET' 2017-02-14 21:43:15 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/120/Processors' 2017-02-14 21:43:15 stdout: [info] method: b'GET' 2017-02-14 21:43:15 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/120/Processors/239' 2017-02-14 21:43:15 stdout: [info] method: b'GET' 2017-02-14 21:43:15 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/120/Processors/240' 2017-02-14 21:43:15 stdout: [info] method: b'GET' 2017-02-14 21:43:15 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/120/Adapters' 2017-02-14 21:43:15 stdout: [info] method: b'GET' 2017-02-14 21:43:15 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/120/Adapters/45/Devices' 2017-02-14 21:43:15 stdout: [info] method: b'GET' 2017-02-14 21:43:15 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/120/Adapters/45/Devices/133' 2017-02-14 21:43:15 stdout: [info] method: b'GET' 2017-02-14 21:43:15 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/120/Adapters/46/Devices' 2017-02-14 21:43:15 stdout: [info] method: b'GET' 2017-02-14 21:43:15 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/121/Memory' 2017-02-14 21:43:15 stdout: [info] method: b'GET' 2017-02-14 21:43:15 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/121/Memory/418' 2017-02-14 21:43:15 stdout: [info] method: b'GET' 2017-02-14 21:43:15 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/121/Memory/419' 2017-02-14 21:43:15 stdout: [info] method: b'GET' 2017-02-14 21:43:15 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/121/Memory/420' 2017-02-14 21:43:15 stdout: [info] method: b'GET' 2017-02-14 21:43:15 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/121/Memory/417' 2017-02-14 21:43:15 stdout: [info] method: b'GET' 2017-02-14 21:43:15 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/121/Processors' 2017-02-14 21:43:15 stdout: [info] method: b'GET' 2017-02-14 21:43:15 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/121/Processors/241' 2017-02-14 21:43:15 stdout: [info] method: b'GET' 2017-02-14 21:43:15 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/121/Processors/242' 2017-02-14 21:43:15 stdout: [info] method: b'GET' 2017-02-14 21:43:15 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/121/Adapters' 2017-02-14 21:43:15 stdout: [info] method: b'GET' 2017-02-14 21:43:15 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/121/Adapters/41/Devices' 2017-02-14 21:43:15 stdout: [info] method: b'GET' 2017-02-14 21:43:15 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/121/Adapters/42/Devices' 2017-02-14 21:43:15 stdout: [info] method: b'GET' 2017-02-14 21:43:15 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/121/Adapters/42/Devices/131' 2017-02-14 21:43:16 stdout: [info] method: b'GET' 2017-02-14 21:43:16 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/122/Memory' 2017-02-14 21:43:16 stdout: [info] method: b'GET' 2017-02-14 21:43:16 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/122/Memory/422' 2017-02-14 21:43:16 stdout: [info] method: b'GET' 2017-02-14 21:43:16 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/122/Memory/423' 2017-02-14 21:43:16 stdout: [info] method: b'GET' 2017-02-14 21:43:16 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/122/Memory/424' 2017-02-14 21:43:16 stdout: [info] method: b'GET' 2017-02-14 21:43:16 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/122/Memory/421' 2017-02-14 21:43:16 stdout: [info] method: b'GET' 2017-02-14 21:43:16 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/122/Processors' 2017-02-14 21:43:16 stdout: [info] method: b'GET' 2017-02-14 21:43:16 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/122/Processors/243' 2017-02-14 21:43:16 stdout: [info] method: b'GET' 2017-02-14 21:43:16 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/122/Processors/244' 2017-02-14 21:43:16 stdout: [info] method: b'GET' 2017-02-14 21:43:16 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/122/Adapters' 2017-02-14 21:43:16 stdout: [info] method: b'GET' 2017-02-14 21:43:16 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/122/Adapters/43/Devices' 2017-02-14 21:43:16 stdout: [info] method: b'GET' 2017-02-14 21:43:16 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/122/Adapters/43/Devices/132' 2017-02-14 21:43:16 stdout: [info] method: b'GET' 2017-02-14 21:43:16 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/122/Adapters/44/Devices' 2017-02-14 21:43:16 stdout: [info] method: b'GET' 2017-02-14 21:43:16 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/123/Memory' 2017-02-14 21:43:16 stdout: [info] method: b'GET' 2017-02-14 21:43:16 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/123/Memory/425' 2017-02-14 21:43:16 stdout: [info] method: b'GET' 2017-02-14 21:43:16 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/123/Memory/426' 2017-02-14 21:43:16 stdout: [info] method: b'GET' 2017-02-14 21:43:16 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/123/Memory/427' 2017-02-14 21:43:16 stdout: [info] method: b'GET' 2017-02-14 21:43:16 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/123/Memory/428' 2017-02-14 21:43:16 stdout: [info] method: b'GET' 2017-02-14 21:43:16 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/123/Processors' 2017-02-14 21:43:16 stdout: [info] method: b'GET' 2017-02-14 21:43:16 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/123/Processors/246' 2017-02-14 21:43:16 stdout: [info] method: b'GET' 2017-02-14 21:43:16 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/123/Processors/245' 2017-02-14 21:43:16 stdout: [info] method: b'GET' 2017-02-14 21:43:16 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/123/Adapters' 2017-02-14 21:43:17 stdout: [info] method: b'GET' 2017-02-14 21:43:17 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/123/Adapters/47/Devices' 2017-02-14 21:43:17 stdout: [info] method: b'GET' 2017-02-14 21:43:17 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/123/Adapters/47/Devices/134' 2017-02-14 21:43:17 stdout: [info] method: b'GET' 2017-02-14 21:43:17 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/123/Adapters/48/Devices' 2017-02-14 21:43:17 stdout: [info] method: b'GET' 2017-02-14 21:43:17 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/127/Memory' 2017-02-14 21:43:17 stdout: [info] method: b'GET' 2017-02-14 21:43:17 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/127/Memory/437' 2017-02-14 21:43:17 stdout: [info] method: b'GET' 2017-02-14 21:43:17 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/127/Memory/438' 2017-02-14 21:43:17 stdout: [info] method: b'GET' 2017-02-14 21:43:17 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/127/Memory/439' 2017-02-14 21:43:17 stdout: [info] method: b'GET' 2017-02-14 21:43:17 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/127/Memory/440' 2017-02-14 21:43:17 stdout: [info] method: b'GET' 2017-02-14 21:43:17 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/127/Processors' 2017-02-14 21:43:17 stdout: [info] method: b'GET' 2017-02-14 21:43:17 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/127/Processors/253' 2017-02-14 21:43:17 stdout: [info] method: b'GET' 2017-02-14 21:43:17 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/127/Processors/254' 2017-02-14 21:43:17 stdout: [info] method: b'GET' 2017-02-14 21:43:17 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/127/Adapters' 2017-02-14 21:43:17 stdout: [info] method: b'GET' 2017-02-14 21:43:17 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/127/Adapters/55/Devices' 2017-02-14 21:43:17 stdout: [info] method: b'GET' 2017-02-14 21:43:17 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/127/Adapters/55/Devices/138' 2017-02-14 21:43:17 stdout: [info] method: b'GET' 2017-02-14 21:43:17 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/127/Adapters/56/Devices' 2017-02-14 21:43:17 stdout: [info] method: b'GET' 2017-02-14 21:43:17 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/128/Memory' 2017-02-14 21:43:17 stdout: [info] method: b'GET' 2017-02-14 21:43:17 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/128/Memory/446' 2017-02-14 21:43:17 stdout: [info] method: b'GET' 2017-02-14 21:43:17 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/128/Memory/447' 2017-02-14 21:43:17 stdout: [info] method: b'GET' 2017-02-14 21:43:17 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/128/Memory/448' 2017-02-14 21:43:17 stdout: [info] method: b'GET' 2017-02-14 21:43:17 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/128/Memory/445' 2017-02-14 21:43:18 stdout: [info] method: b'GET' 2017-02-14 21:43:18 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/128/Processors' 2017-02-14 21:43:18 stdout: [info] method: b'GET' 2017-02-14 21:43:18 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/128/Processors/255' 2017-02-14 21:43:18 stdout: [info] method: b'GET' 2017-02-14 21:43:18 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/128/Processors/256' 2017-02-14 21:43:18 stdout: [info] method: b'GET' 2017-02-14 21:43:18 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/128/Adapters' 2017-02-14 21:43:18 stdout: [info] method: b'GET' 2017-02-14 21:43:18 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/128/Adapters/57/Devices' 2017-02-14 21:43:18 stdout: [info] method: b'GET' 2017-02-14 21:43:18 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/128/Adapters/58/Devices' 2017-02-14 21:43:18 stdout: [info] method: b'GET' 2017-02-14 21:43:18 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/128/Adapters/58/Devices/139' 2017-02-14 21:43:18 stdout: [info] method: b'GET' 2017-02-14 21:43:18 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/129/Memory' 2017-02-14 21:43:18 stdout: [info] method: b'GET' 2017-02-14 21:43:18 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/129/Memory/449' 2017-02-14 21:43:18 stdout: [info] method: b'GET' 2017-02-14 21:43:18 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/129/Memory/450' 2017-02-14 21:43:18 stdout: [info] method: b'GET' 2017-02-14 21:43:18 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/129/Memory/451' 2017-02-14 21:43:18 stdout: [info] method: b'GET' 2017-02-14 21:43:18 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/129/Memory/452' 2017-02-14 21:43:18 stdout: [info] method: b'GET' 2017-02-14 21:43:18 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/129/Processors' 2017-02-14 21:43:18 stdout: [info] method: b'GET' 2017-02-14 21:43:18 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/129/Processors/257' 2017-02-14 21:43:18 stdout: [info] method: b'GET' 2017-02-14 21:43:18 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/129/Processors/258' 2017-02-14 21:43:18 stdout: [info] method: b'GET' 2017-02-14 21:43:18 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/129/Adapters' 2017-02-14 21:43:18 stdout: [info] method: b'GET' 2017-02-14 21:43:18 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/129/Adapters/60/Devices' 2017-02-14 21:43:18 stdout: [info] method: b'GET' 2017-02-14 21:43:18 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/129/Adapters/59/Devices' 2017-02-14 21:43:18 stdout: [info] method: b'GET' 2017-02-14 21:43:18 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Systems/129/Adapters/59/Devices/140' 2017-02-14 21:43:19 stdout: [info] In decompose 2017-02-14 21:43:19 stdout: [info] context: {'node_id': '94', 'power_user': 'admin', 'power_pass': 'admin', 'power_address': '10.3.0.1:8443'} 2017-02-14 21:43:19 stdout: [info] endpoint: b'redfish/v1/Nodes/94' 2017-02-14 21:43:19 stdout: [info] method: b'DELETE' 2017-02-14 21:43:19 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Nodes/94' 2017-02-14 21:43:19 provisioningserver.rpc.pods: [critical] Failed to decompose machine. Traceback (most recent call last):   File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 434, in errback     self._startRunCallbacks(fail)   File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 501, in _startRunCallbacks     self._runCallbacks()   File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 588, in _runCallbacks     current.result = callback(current.result, *args, **kw)   File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 1184, in gotResult     _inlineCallbacks(r, g, deferred) --- <exception caught here> ---   File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 1126, in _inlineCallbacks     result = result.throwExceptionIntoGenerator(g)   File "/usr/lib/python3/dist-packages/twisted/python/failure.py", line 389, in throwExceptionIntoGenerator     return g.throw(self.type, self.value, self.tb)   File "/usr/lib/python3/dist-packages/provisioningserver/drivers/pod/rsd.py", line 683, in decompose     b"DELETE", join(url, endpoint), headers) twisted.web.client.PartialDownloadError: 404 Not Found ==> /var/log/maas/regiond.log <== 2017-02-14 21:43:19 twisted.python.log: [info] ::ffff:10.150.105.2 - - [14/Feb/2017:21:43:18 +0000] "DELETE /MAAS/api/2.0/pods/138/ HTTP/1.1" 503 74 "-" "Python-httplib2/0.9.1 (gzip)" ==> /var/log/maas/rackd.log <== sudo: unable to resolve host maas-dev: Connection timed out ==> /var/log/maas/regiond.log <== 2017-02-14 21:43:19 maasserver.region_controller: [info] Successfully configured DNS. ==> /var/log/maas/rackd.log <== sudo: unable to resolve host maas-dev: Connection timed out ==> /var/log/maas/regiond.log <== 2017-02-14 21:43:21 maasserver.dhcp: [info] Successfully configured DHCPv4 on rack controller 'ssapke'. 2017-02-14 21:43:21 maasserver.dhcp: [info] Successfully configured DHCPv6 on rack controller 'ssapke'. ==> /var/log/maas/maas.log <== Feb 14 21:43:33 maas-dev maas.node: [info] 138 (No IP): Deleting BMC ==> /var/log/maas/rackd.log <== 2017-02-14 21:43:34 stdout: [info] In decompose 2017-02-14 21:43:34 stdout: [info] context: {'node_id': '89', 'power_user': 'admin', 'power_pass': 'admin', 'power_address': '10.3.0.1:8443'} 2017-02-14 21:43:34 stdout: [info] endpoint: b'redfish/v1/Nodes/89' 2017-02-14 21:43:34 stdout: [info] method: b'DELETE' 2017-02-14 21:43:34 stdout: [info] uri: b'https://10.3.0.1:8443/redfish/v1/Nodes/89' 2017-02-14 21:43:34 provisioningserver.rpc.pods: [critical] Failed to decompose machine. Traceback (most recent call last):   File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 434, in errback     self._startRunCallbacks(fail)   File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 501, in _startRunCallbacks     self._runCallbacks()   File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 588, in _runCallbacks     current.result = callback(current.result, *args, **kw)   File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 1184, in gotResult     _inlineCallbacks(r, g, deferred) --- <exception caught here> ---   File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 1126, in _inlineCallbacks     result = result.throwExceptionIntoGenerator(g)   File "/usr/lib/python3/dist-packages/twisted/python/failure.py", line 389, in throwExceptionIntoGenerator     return g.throw(self.type, self.value, self.tb)   File "/usr/lib/python3/dist-packages/provisioningserver/drivers/pod/rsd.py", line 683, in decompose     b"DELETE", join(url, endpoint), headers) twisted.web.client.PartialDownloadError: 404 Not Found
2017-02-17 21:03:31 Blake Rouse maas: status Confirmed In Progress
2017-02-17 21:03:33 Blake Rouse maasrsd: status Confirmed In Progress
2017-02-17 21:03:35 Blake Rouse maas: milestone 2.2.0
2017-02-17 21:03:41 Blake Rouse maasrsd: milestone beta
2017-02-17 21:03:39 Launchpad Janitor branch linked lp:~blake-rouse/maas/fix-pod-delete
2017-02-18 13:38:59 MAAS Lander maas: status In Progress Fix Committed
2017-02-27 15:15:45 Blake Rouse maasrsd: status In Progress Fix Committed
2017-03-15 09:10:11 Andres Rodriguez maas: status Fix Committed Fix Released
2017-05-17 00:48:28 Newell Jensen maasrsd: status Fix Committed Fix Released