Blazar

  1. Bug #1794481
  2. Activity log

Activity log for bug #1794481

Date Who What changed Old value New value Message
2018-09-26 10:00:03 Pierre Riteau bug added bug
2018-09-26 10:20:16 Pierre Riteau blazar: importance Undecided High
2018-09-26 10:20:19 Pierre Riteau blazar: status New Confirmed
2018-09-26 10:20:21 Pierre Riteau blazar: assignee Pierre Riteau (priteau)
2018-09-26 10:20:25 Pierre Riteau blazar: milestone stein-rc1
2018-09-26 10:20:30 Pierre Riteau blazar: milestone stein-rc1 stein-1
2018-10-23 10:18:04 Pierre Riteau blazar: milestone stein-1 stein-2
2018-12-17 21:42:54 Pierre Riteau description When Blazar fails to remove a host from an aggregate, it immediately stops executing the end_lease vent, for example: ERROR blazar.manager.service [-] Error occurred while handling event. ERROR blazar.manager.service Traceback (most recent call last): ERROR blazar.manager.service File "/usr/lib/python2.7/site-packages/blazar/manager/service.py", line 242, in _exec_event ERROR blazar.manager.service event_fn(lease_id=event['lease_id'], event_id=event['id']) ERROR blazar.manager.service File "/usr/lib/python2.7/site-packages/blazar/status.py", line 225, in wrapper ERROR blazar.manager.service {'status': cls.ERROR}) ERROR blazar.manager.service File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ ERROR blazar.manager.service self.force_reraise() ERROR blazar.manager.service File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise ERROR blazar.manager.service six.reraise(self.type_, self.value, self.tb) ERROR blazar.manager.service File "/usr/lib/python2.7/site-packages/blazar/status.py", line 209, in wrapper ERROR blazar.manager.service result = func(*args, **kwargs) ERROR blazar.manager.service File "/usr/lib/python2.7/site-packages/blazar/manager/service.py", line 623, in end_lease ERROR blazar.manager.service status.reservation.DELETED) ERROR blazar.manager.service File "/usr/lib/python2.7/site-packages/blazar/manager/service.py", line 644, in _basic_action ERROR blazar.manager.service reservation['resource_id'] ERROR blazar.manager.service File "/usr/lib/python2.7/site-packages/blazar/plugins/oshosts/host_plugin.py", line 282, in on_end ERROR blazar.manager.service pool.delete(host_reservation['aggregate_id']) ERROR blazar.manager.service File "/usr/lib/python2.7/site-packages/blazar/utils/openstack/nova.py", line 304, in delete ERROR blazar.manager.service self.nova.aggregates.remove_host(agg.id, host) ERROR blazar.manager.service File "/usr/lib/python2.7/site-packages/novaclient/v2/aggregates.py", line 90, in remove_host ERROR blazar.manager.service body, "aggregate") ERROR blazar.manager.service File "/usr/lib/python2.7/site-packages/novaclient/base.py", line 361, in _create ERROR blazar.manager.service resp, body = self.api.client.post(url, body=body) ERROR blazar.manager.service File "/usr/lib/python2.7/site-packages/keystoneauth1/adapter.py", line 223, in post ERROR blazar.manager.service return self.request(url, 'POST', **kwargs) ERROR blazar.manager.service File "/usr/lib/python2.7/site-packages/novaclient/client.py", line 80, in request ERROR blazar.manager.service raise exceptions.from_response(resp, body, url, method) ERROR blazar.manager.service NotFound: Cannot remove host e5fcdcf3-8355-4705-9a0b-371e75ea1732 in aggregate 9657 (HTTP 404) (Request-ID: req-401c46d4-9159-4880-a4ca-0d8d898c924e) This causes issues with future leases using the same hosts, as they are still in the uncleaned aggregate instead of the freepool. Blazar should continue removing hosts from the aggregate even when it gets a failure, while logging a warning that not all hosts have been removed. When Blazar fails to remove a host from an aggregate, it immediately stops executing the end_lease event, for example: ERROR blazar.manager.service [-] Error occurred while handling event. ERROR blazar.manager.service Traceback (most recent call last): ERROR blazar.manager.service File "/usr/lib/python2.7/site-packages/blazar/manager/service.py", line 242, in _exec_event ERROR blazar.manager.service event_fn(lease_id=event['lease_id'], event_id=event['id']) ERROR blazar.manager.service File "/usr/lib/python2.7/site-packages/blazar/status.py", line 225, in wrapper ERROR blazar.manager.service {'status': cls.ERROR}) ERROR blazar.manager.service File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ ERROR blazar.manager.service self.force_reraise() ERROR blazar.manager.service File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise ERROR blazar.manager.service six.reraise(self.type_, self.value, self.tb) ERROR blazar.manager.service File "/usr/lib/python2.7/site-packages/blazar/status.py", line 209, in wrapper ERROR blazar.manager.service result = func(*args, **kwargs) ERROR blazar.manager.service File "/usr/lib/python2.7/site-packages/blazar/manager/service.py", line 623, in end_lease ERROR blazar.manager.service status.reservation.DELETED) ERROR blazar.manager.service File "/usr/lib/python2.7/site-packages/blazar/manager/service.py", line 644, in _basic_action ERROR blazar.manager.service reservation['resource_id'] ERROR blazar.manager.service File "/usr/lib/python2.7/site-packages/blazar/plugins/oshosts/host_plugin.py", line 282, in on_end ERROR blazar.manager.service pool.delete(host_reservation['aggregate_id']) ERROR blazar.manager.service File "/usr/lib/python2.7/site-packages/blazar/utils/openstack/nova.py", line 304, in delete ERROR blazar.manager.service self.nova.aggregates.remove_host(agg.id, host) ERROR blazar.manager.service File "/usr/lib/python2.7/site-packages/novaclient/v2/aggregates.py", line 90, in remove_host ERROR blazar.manager.service body, "aggregate") ERROR blazar.manager.service File "/usr/lib/python2.7/site-packages/novaclient/base.py", line 361, in _create ERROR blazar.manager.service resp, body = self.api.client.post(url, body=body) ERROR blazar.manager.service File "/usr/lib/python2.7/site-packages/keystoneauth1/adapter.py", line 223, in post ERROR blazar.manager.service return self.request(url, 'POST', **kwargs) ERROR blazar.manager.service File "/usr/lib/python2.7/site-packages/novaclient/client.py", line 80, in request ERROR blazar.manager.service raise exceptions.from_response(resp, body, url, method) ERROR blazar.manager.service NotFound: Cannot remove host e5fcdcf3-8355-4705-9a0b-371e75ea1732 in aggregate 9657 (HTTP 404) (Request-ID: req-401c46d4-9159-4880-a4ca-0d8d898c924e) This causes issues with future leases using the same hosts, as they are still in the uncleaned aggregate instead of the freepool. Blazar should continue removing hosts from the aggregate even when it gets a failure, while logging a warning that not all hosts have been removed.
2019-01-10 14:50:02 Pierre Riteau blazar: milestone stein-2 stein-3
2019-01-23 11:49:33 Pierre Riteau blazar: status Confirmed In Progress
2019-04-15 17:59:15 Pierre Riteau blazar: milestone stein-3 train-1