Comment 0 for bug 1752152

Revision history for this message
Divya K Konoor (dikonoor) wrote :

It is found that when cinder endpoint is configured to use https, attach volume flow fails with the stack trace seen below (seen in nova api log) because it fails to make a secure call from nova to cinder. Secure calls perform certificate validation and in this particular flow, certificate validation is completely skipped

File "/usr/lib/python2.7/site-packages/nova/compute/api.py", line 3971, in attach_volume
2018-02-27 08:16:51.338 1324 ERROR cinder.is_microversion_supported(context, '3.44')
2018-02-27 08:16:51.338 1324 ERROR File "/usr/lib/python2.7/site-packages/nova/volume/cinder.py", line 138, in is_microversion_supported
2018-02-27 08:16:51.338 1324 ERROR _check_microversion(url, microversion)
2018-02-27 08:16:51.338 1324 ERROR File "/usr/lib/python2.7/site-packages/nova/volume/cinder.py", line 86, in _check_microversion
2018-02-27 08:16:51.338 1324 ERROR max_api_version = cinder_client.get_highest_client_server_version(url)
2018-02-27 08:16:51.338 1324 ERROR File "/usr/lib/python2.7/site-packages/cinderclient/client.py", line 126, in get_highest_client_server_version
2018-02-27 08:16:51.338 1324 ERROR min_server, max_server = get_server_version(url)
2018-02-27 08:16:51.338 1324 ERROR File "/usr/lib/python2.7/site-packages/cinderclient/client.py", line 109, in get_server_version
2018-02-27 08:16:51.338 1324 ERROR response = requests.get(version_url)
2018-02-27 08:16:51.338 1324 ERROR File "/usr/lib/python2.7/site-packages/requests/api.py", line 72, in get
2018-02-27 08:16:51.338 1324 ERROR return request('get', url, params=params, **kwargs)
2018-02-27 08:16:51.338 1324 ERROR File "/usr/lib/python2.7/site-packages/requests/api.py", line 58, in request
2018-02-27 08:16:51.338 1324 ERROR return session.request(method=method, url=url, **kwargs)
2018-02-27 08:16:51.338 1324 ERROR File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 502, in request
2018-02-27 08:16:51.338 1324 ERROR resp = self.send(prep, **send_kwargs)
2018-02-27 08:16:51.338 1324 ERROR File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 612, in send
2018-02-27 08:16:51.338 1324 ERROR r = adapter.send(request, **kwargs)
2018-02-27 08:16:51.338 1324 ERROR File "/usr/lib/python2.7/site-packages/requests/adapters.py", line 504, in send
2018-02-27 08:16:51.338 1324 ERROR raise ConnectionError(e, request=request)
2018-02-27 08:16:51.338 1324 ERROR ConnectionError: HTTPSConnectionPool(host='ip9-114-192-132.pok.stglabs.ibm.com', port=9000): Max retries exceeded with url: / (Caused by SSLError(SSLError("bad handshake: Error([('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')],)",),))

This is a regression bug introduced as part of changeset https://review.openstack.org/#/c/469579/, which was merged way back in June 2017. As part of this changeset, a new function namely _check_microversion was introduced, which then makes a cinderclient call , which finally makes a cinder https REST api call without passing the certificate. This leads to the problem listed above.

https://github.com/openstack/nova/blob/stable/queens/nova/volume/cinder.py#L75
https://github.com/openstack/nova/blob/stable/queens/nova/volume/cinder.py#L86

https://github.com/openstack/python-cinderclient/blob/stable/queens/cinderclient/client.py#L126
https://github.com/openstack/python-cinderclient/blob/stable/queens/cinderclient/client.py#L109