My senin version is master.
My cluster bind health_policy_poll policy, when I delete vm that belong this cluster, node cannot auto recover
through policy.
Below is error log. I hope node can auto recover, because that is health policy meaning.
2017-04-07 15:04:32.331 21107 INFO senlin.engine.service [req-9801a74c-e95d-4ed3-9748-ff8feabbfdc5 - - - - -] Checking cluster '25018444-961b-4301-a86a-52452ec6a718'. 2017-04-07 15:04:33.738 21107 INFO senlin.engine.service [req-9801a74c-e95d-4ed3-9748-ff8feabbfdc5 - - - - -] Cluster check action queued: 21dd55d7-eb13-43be-a30d-14039f27fa15. 2017-04-07 15:04:35.103 21107 INFO senlin.engine.event [req-9801a74c-e95d-4ed3-9748-ff8feabbfdc5 - - - - -] test_cluster [25018444] CLUSTER_CHECK - start: None 2017-04-07 15:04:36.205 21107 INFO senlin.engine.event [req-9801a74c-e95d-4ed3-9748-ff8feabbfdc5 - - - - -] node-25018444-232 [81187992] NODE_CHECK - start: None 2017-04-07 15:04:36.578 21107 ERROR senlin.drivers.openstack.sdk [req-9801a74c-e95d-4ed3-9748-ff8feabbfdc5 - - - - -] ResourceNotFound: No Server found for 42377008-aca4-4bb0-829e-5afcf9c93a61, Instance 42377008-aca4-4bb0-829e-5afcf9c93a61 could not be found. 2017-04-07 15:04:36.578 21107 ERROR senlin.drivers.openstack.sdk Traceback (most recent call last): 2017-04-07 15:04:36.578 21107 ERROR senlin.drivers.openstack.sdk File "/opt/senlin/senlin/drivers/openstack/sdk.py", line 96, in invoke_with_catch 2017-04-07 15:04:36.578 21107 ERROR senlin.drivers.openstack.sdk return func(driver, *args, **kwargs) 2017-04-07 15:04:36.578 21107 ERROR senlin.drivers.openstack.sdk File "/opt/senlin/senlin/drivers/openstack/nova_v2.py", line 50, in server_get 2017-04-07 15:04:36.578 21107 ERROR senlin.drivers.openstack.sdk return self.conn.compute.servers(details=details, **query) 2017-04-07 15:04:36.578 21107 ERROR senlin.drivers.openstack.sdk File "/usr/lib/python2.7/site-packages/openstack/compute/v2/_proxy.py", line 371, in get_server 2017-04-07 15:04:36.578 21107 ERROR senlin.drivers.openstack.sdk return self._get(_server.Server, server) 2017-04-07 15:04:36.578 21107 ERROR senlin.drivers.openstack.sdk File "/usr/lib/python2.7/site-packages/openstack/proxy2.py", line 37, in check 2017-04-07 15:04:36.578 21107 ERROR senlin.drivers.openstack.sdk return method(self, expected, actual, *args, **kwargs) 2017-04-07 15:04:36.578 21107 ERROR senlin.drivers.openstack.sdk File "/usr/lib/python2.7/site-packages/openstack/proxy2.py", line 225, in _get 2017-04-07 15:04:36.578 21107 ERROR senlin.drivers.openstack.sdk http_status=e.http_status, cause=e.cause) 2017-04-07 15:04:36.578 21107 ERROR senlin.drivers.openstack.sdk ResourceNotFound: ResourceNotFound: No Server found for 42377008-aca4-4bb0-829e-5afcf9c93a61, Instance 42377008-aca4-4bb0-829e-5afcf9c93a61 could not be found. 2017-04-07 15:04:36.578 21107 ERROR senlin.drivers.openstack.sdk 2017-04-07 15:04:36.705 21107 ERROR senlin.engine.event [req-9801a74c-e95d-4ed3-9748-ff8feabbfdc5 - - - - -] node-25018444-232 [81187992] NODE_CHECK - error: Node check failed. 2017-04-07 15:04:37.082 21107 WARNING senlin.engine.health_manager [req-9801a74c-e95d-4ed3-9748-ff8feabbfdc5 - - - - -] Cluster check action failed
My senin version is master.
My cluster bind health_policy_poll policy, when I delete vm that belong this cluster, node cannot auto recover
through policy.
Below is error log. I hope node can auto recover, because that is health policy meaning.
2017-04-07 15:04:32.331 21107 INFO senlin. engine. service [req-9801a74c- e95d-4ed3- 9748-ff8feabbfd c5 - - - - -] Checking cluster '25018444- 961b-4301- a86a-52452ec6a7 18'. engine. service [req-9801a74c- e95d-4ed3- 9748-ff8feabbfd c5 - - - - -] Cluster check action queued: 21dd55d7- eb13-43be- a30d-14039f27fa 15. e95d-4ed3- 9748-ff8feabbfd c5 - - - - -] test_cluster [25018444] CLUSTER_CHECK - start: None e95d-4ed3- 9748-ff8feabbfd c5 - - - - -] node-25018444-232 [81187992] NODE_CHECK - start: None drivers. openstack. sdk [req-9801a74c- e95d-4ed3- 9748-ff8feabbfd c5 - - - - -] ResourceNotFound: No Server found for 42377008- aca4-4bb0- 829e-5afcf9c93a 61, Instance 42377008- aca4-4bb0- 829e-5afcf9c93a 61 could not be found. drivers. openstack. sdk Traceback (most recent call last): drivers. openstack. sdk File "/opt/senlin/ senlin/ drivers/ openstack/ sdk.py" , line 96, in invoke_with_catch drivers. openstack. sdk return func(driver, *args, **kwargs) drivers. openstack. sdk File "/opt/senlin/ senlin/ drivers/ openstack/ nova_v2. py", line 50, in server_get drivers. openstack. sdk return self.conn. compute. servers( details= details, **query) drivers. openstack. sdk File "/usr/lib/ python2. 7/site- packages/ openstack/ compute/ v2/_proxy. py", line 371, in get_server drivers. openstack. sdk return self._get( _server. Server, server) drivers. openstack. sdk File "/usr/lib/ python2. 7/site- packages/ openstack/ proxy2. py", line 37, in check drivers. openstack. sdk return method(self, expected, actual, *args, **kwargs) drivers. openstack. sdk File "/usr/lib/ python2. 7/site- packages/ openstack/ proxy2. py", line 225, in _get drivers. openstack. sdk http_status= e.http_ status, cause=e.cause) drivers. openstack. sdk ResourceNotFound: ResourceNotFound: No Server found for 42377008- aca4-4bb0- 829e-5afcf9c93a 61, Instance 42377008- aca4-4bb0- 829e-5afcf9c93a 61 could not be found. drivers. openstack. sdk e95d-4ed3- 9748-ff8feabbfd c5 - - - - -] node-25018444-232 [81187992] NODE_CHECK - error: Node check failed. engine. health_ manager [req-9801a74c- e95d-4ed3- 9748-ff8feabbfd c5 - - - - -] Cluster check action failed
2017-04-07 15:04:33.738 21107 INFO senlin.
2017-04-07 15:04:35.103 21107 INFO senlin.engine.event [req-9801a74c-
2017-04-07 15:04:36.205 21107 INFO senlin.engine.event [req-9801a74c-
2017-04-07 15:04:36.578 21107 ERROR senlin.
2017-04-07 15:04:36.578 21107 ERROR senlin.
2017-04-07 15:04:36.578 21107 ERROR senlin.
2017-04-07 15:04:36.578 21107 ERROR senlin.
2017-04-07 15:04:36.578 21107 ERROR senlin.
2017-04-07 15:04:36.578 21107 ERROR senlin.
2017-04-07 15:04:36.578 21107 ERROR senlin.
2017-04-07 15:04:36.578 21107 ERROR senlin.
2017-04-07 15:04:36.578 21107 ERROR senlin.
2017-04-07 15:04:36.578 21107 ERROR senlin.
2017-04-07 15:04:36.578 21107 ERROR senlin.
2017-04-07 15:04:36.578 21107 ERROR senlin.
2017-04-07 15:04:36.578 21107 ERROR senlin.
2017-04-07 15:04:36.578 21107 ERROR senlin.
2017-04-07 15:04:36.705 21107 ERROR senlin.engine.event [req-9801a74c-
2017-04-07 15:04:37.082 21107 WARNING senlin.