When Blazar fails to remove a host from an aggregate, it immediately stops executing the end_lease event, for example:
ERROR blazar.manager.service [-] Error occurred while handling event.
ERROR blazar.manager.service Traceback (most recent call last):
ERROR blazar.manager.service File "/usr/lib/python2.7/site-packages/blazar/manager/service.py", line 242, in _exec_event
ERROR blazar.manager.service event_fn(lease_id=event['lease_id'], event_id=event['id'])
ERROR blazar.manager.service File "/usr/lib/python2.7/site-packages/blazar/status.py", line 225, in wrapper
ERROR blazar.manager.service {'status': cls.ERROR})
ERROR blazar.manager.service File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
ERROR blazar.manager.service self.force_reraise()
ERROR blazar.manager.service File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
ERROR blazar.manager.service six.reraise(self.type_, self.value, self.tb)
ERROR blazar.manager.service File "/usr/lib/python2.7/site-packages/blazar/status.py", line 209, in wrapper
ERROR blazar.manager.service result = func(*args, **kwargs)
ERROR blazar.manager.service File "/usr/lib/python2.7/site-packages/blazar/manager/service.py", line 623, in end_lease
ERROR blazar.manager.service status.reservation.DELETED)
ERROR blazar.manager.service File "/usr/lib/python2.7/site-packages/blazar/manager/service.py", line 644, in _basic_action
ERROR blazar.manager.service reservation['resource_id']
ERROR blazar.manager.service File "/usr/lib/python2.7/site-packages/blazar/plugins/oshosts/host_plugin.py", line 282, in on_end
ERROR blazar.manager.service pool.delete(host_reservation['aggregate_id'])
ERROR blazar.manager.service File "/usr/lib/python2.7/site-packages/blazar/utils/openstack/nova.py", line 304, in delete
ERROR blazar.manager.service self.nova.aggregates.remove_host(agg.id, host)
ERROR blazar.manager.service File "/usr/lib/python2.7/site-packages/novaclient/v2/aggregates.py", line 90, in remove_host
ERROR blazar.manager.service body, "aggregate")
ERROR blazar.manager.service File "/usr/lib/python2.7/site-packages/novaclient/base.py", line 361, in _create
ERROR blazar.manager.service resp, body = self.api.client.post(url, body=body)
ERROR blazar.manager.service File "/usr/lib/python2.7/site-packages/keystoneauth1/adapter.py", line 223, in post
ERROR blazar.manager.service return self.request(url, 'POST', **kwargs)
ERROR blazar.manager.service File "/usr/lib/python2.7/site-packages/novaclient/client.py", line 80, in request
ERROR blazar.manager.service raise exceptions.from_response(resp, body, url, method)
ERROR blazar.manager.service NotFound: Cannot remove host e5fcdcf3-8355-4705-9a0b-371e75ea1732 in aggregate 9657 (HTTP 404) (Request-ID: req-401c46d4-9159-4880-a4ca-0d8d898c924e)
This causes issues with future leases using the same hosts, as they are still in the uncleaned aggregate instead of the freepool.
Blazar should continue removing hosts from the aggregate even when it gets a failure, while logging a warning that not all hosts have been removed.
