OpenStack Compute (nova)

Bug #1441235
Comment #0

Comment 0 for bug 1441235

Revision history for this message

Chinmaya Bharadwaj (acbharadwaj) wrote on 2015-04-07:

With the latest code, cluster as the compute driver, Taking a snapshot is failing with following error :

ERROR oslo_messaging.rpc.dispatcher [req-9c49bfdd-641c-4022-90cc-aae697e387f3 admin admin] Exception during message handling: Error occurred while opening URL: https://10.1.214.7/nfc/52135ab3-7409-6a9c-30ba-65f00d264262/disk-1.vmdk for reading.
Cause: HTTPSConnectionPool(host='10.1.214.7', port=443): Max retries exceeded with url: /nfc/52135ab3-7409-6a9c-30ba-65f00d264262/disk-1.vmdk (Caused by ProxyError('Cannot connect to proxy.', error('Tunnel connection failed: 503 Service Unavailable',)))
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher Traceback (most recent call last):
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 142, in _dispatch_and_reply
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher executor_callback))
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 186, in _dispatch
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher executor_callback)
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 130, in _do_dispatch
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher result = func(ctxt, **new_args)
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher File "/opt/stack/nova/nova/exception.py", line 88, in wrapped
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher payload)
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 85, in __exit__
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb)
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher File "/opt/stack/nova/nova/exception.py", line 71, in wrapped
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher return f(self, context, *args, **kw)
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher File "/opt/stack/nova/nova/compute/manager.py", line 324, in decorated_function
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher LOG.warning(msg, e, instance_uuid=instance_uuid)
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 85, in __exit__
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb)
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher File "/opt/stack/nova/nova/compute/manager.py", line 295, in decorated_function
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher return function(self, context, *args, **kwargs)
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher File "/opt/stack/nova/nova/compute/manager.py", line 352, in decorated_function
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher kwargs['instance'], e, sys.exc_info())
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 85, in __exit__
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb)
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher File "/opt/stack/nova/nova/compute/manager.py", line 340, in decorated_function
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher return function(self, context, *args, **kwargs)
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher File "/opt/stack/nova/nova/compute/manager.py", line 400, in decorated_function
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher instance=instance)
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 85, in __exit__
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb)
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher File "/opt/stack/nova/nova/compute/manager.py", line 390, in decorated_function
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher *args, **kwargs)
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher File "/opt/stack/nova/nova/compute/manager.py", line 3201, in snapshot_instance
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher task_states.IMAGE_SNAPSHOT)
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher File "/opt/stack/nova/nova/compute/manager.py", line 3231, in _snapshot_instance
2015-04-07 2

1:16:47.166 TRACE oslo_messaging.rpc.dispatcher update_task_state)
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 506, in snapshot
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher self._vmops.snapshot(context, instance, image_id, update_task_state)
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 833, in snapshot
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher vmdk_size=vmdk.capacity_in_bytes)
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher File "/opt/stack/nova/nova/virt/vmwareapi/images.py", line 457, in upload_image_stream_optimized
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher vmdk_size)
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo_vmware/rw_handles.py", line 500, in __init__
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher cacerts=cacerts)
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo_vmware/rw_handles.py", line 79, in _create_read_connection
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher raise exceptions.VimException(excep_msg, excep)
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher VimException: Error occurred while opening URL: https://10.1.214.7/nfc/52135ab3-7409-6a9c-30ba-65f00d264262/disk-1.vmdk for reading.
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher Cause: HTTPSConnectionPool(host='10.1.214.7', port=443): Max retries exceeded with url: /nfc/52135ab3-7409-6a9c-30ba-65f00d264262/disk-1.vmdk (Caused by ProxyError('Cannot connect to proxy.', error('Tunnel connection failed: 503 Service Unavailable',)))
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher

git log -1
commit bf70df295b0529da9f4381f52fc94d328fa2fdb3
Merge: ad329b0 7560620
Author: Jenkins <email address hidden>
Date: Mon Apr 6 23:57:10 2015 +0000

Merge "neutronv2: only create client once when adding/removing fixed IPs"

Attaching errors shown in vcenter

With the latest code, cluster as  the compute driver, Taking a  snapshot is failing with following error :

ERROR oslo_messaging.rpc.dispatcher [req-9c49bfdd-641c-4022-90cc-aae697e387f3 admin admin] Exception during message handling: Error occurred while opening URL: https://10.1.214.7/nfc/52135ab3-7409-6a9c-30ba-65f00d264262/disk-1.vmdk for reading.
Cause: HTTPSConnectionPool(host='10.1.214.7', port=443): Max retries exceeded with url: /nfc/52135ab3-7409-6a9c-30ba-65f00d264262/disk-1.vmdk (Caused by ProxyError('Cannot connect to proxy.', error('Tunnel connection failed: 503 Service Unavailable',)))
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher Traceback (most recent call last):
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher   File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 142, in _dispatch_and_reply
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher     executor_callback))
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher   File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 186, in _dispatch
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher     executor_callback)
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher   File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 130, in _do_dispatch
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher     result = func(ctxt, **new_args)
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher   File "/opt/stack/nova/nova/exception.py", line 88, in wrapped
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher     payload)
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 85, in __exit__
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher     six.reraise(self.type_, self.value, self.tb)
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher   File "/opt/stack/nova/nova/exception.py", line 71, in wrapped
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher     return f(self, context, *args, **kw)
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher   File "/opt/stack/nova/nova/compute/manager.py", line 324, in decorated_function
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher     LOG.warning(msg, e, instance_uuid=instance_uuid)
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 85, in __exit__
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher     six.reraise(self.type_, self.value, self.tb)
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher   File "/opt/stack/nova/nova/compute/manager.py", line 295, in decorated_function
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher     return function(self, context, *args, **kwargs)
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher   File "/opt/stack/nova/nova/compute/manager.py", line 352, in decorated_function
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher     kwargs['instance'], e, sys.exc_info())
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 85, in __exit__
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher     six.reraise(self.type_, self.value, self.tb)
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher   File "/opt/stack/nova/nova/compute/manager.py", line 340, in decorated_function
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher     return function(self, context, *args, **kwargs)
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher   File "/opt/stack/nova/nova/compute/manager.py", line 400, in decorated_function
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher     instance=instance)
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 85, in __exit__
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher     six.reraise(self.type_, self.value, self.tb)
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher   File "/opt/stack/nova/nova/compute/manager.py", line 390, in decorated_function
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher     *args, **kwargs)
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher   File "/opt/stack/nova/nova/compute/manager.py", line 3201, in snapshot_instance
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher     task_states.IMAGE_SNAPSHOT)
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher   File "/opt/stack/nova/nova/compute/manager.py", line 3231, in _snapshot_instance
2015-04-07 2

1:16:47.166 TRACE oslo_messaging.rpc.dispatcher     update_task_state)
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher   File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 506, in snapshot
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher     self._vmops.snapshot(context, instance, image_id, update_task_state)
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher   File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 833, in snapshot
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher     vmdk_size=vmdk.capacity_in_bytes)
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher   File "/opt/stack/nova/nova/virt/vmwareapi/images.py", line 457, in upload_image_stream_optimized
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher     vmdk_size)
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher   File "/usr/local/lib/python2.7/dist-packages/oslo_vmware/rw_handles.py", line 500, in __init__
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher     cacerts=cacerts)
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher   File "/usr/local/lib/python2.7/dist-packages/oslo_vmware/rw_handles.py", line 79, in _create_read_connection
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher     raise exceptions.VimException(excep_msg, excep)
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher VimException: Error occurred while opening URL: https://10.1.214.7/nfc/52135ab3-7409-6a9c-30ba-65f00d264262/disk-1.vmdk for reading.
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher Cause: HTTPSConnectionPool(host='10.1.214.7', port=443): Max retries exceeded with url: /nfc/52135ab3-7409-6a9c-30ba-65f00d264262/disk-1.vmdk (Caused by ProxyError('Cannot connect to proxy.', error('Tunnel connection failed: 503 Service Unavailable',)))
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher

git log -1
commit bf70df295b0529da9f4381f52fc94d328fa2fdb3
Merge: ad329b0 7560620
Author: Jenkins <jenkins@review.openstack.org>
Date:   Mon Apr 6 23:57:10 2015 +0000

Merge "neutronv2: only create client once when adding/removing fixed IPs"

Attaching errors shown in vcenter