Instance 2b580249-9467-44d7-9e6b-73693534a5d4 was first scheduled to node-6: 2016-05-26 02:37:34.645 14354 DEBUG nova.compute.manager [req-81751add-78d4-4f53-8042-a04596d0a040 becfc7903e7e4917ab927eae3fb3bb3d 420cb14a42c84b3fbbb8891fc9329816 - - -] [instance: 2b580249-9467-44d7-9e6b-73693534a5d4] Starting instance... _do_build_and_run_instance /usr/lib/python2.7/dist-packages/nova/compute/manager.py:1897 port fa843bf8-01b7-424e-a705-ff8453ec743e created: 2016-05-26 02:37:37.614 14354 DEBUG keystoneauth.session [req-81751add-78d4-4f53-8042-a04596d0a040 becfc7903e7e4917ab927eae3fb3bb3d 420cb14a42c84b3fbbb8891fc9329816 - - -] RESP: [201] Date: Thu, 26 May 2016 02:37:37 GMT Connection: close Content-Type: application/json; charset=UTF-8 Content-Length: 886 X-Openstack-Request-Id: req-c801de97-029d-4927-a983-3d3db3e6f443 RESP BODY: {"port": {"status": "DOWN", "binding:host_id": "node-6.test.domain.local", "description": "", "allowed_address_pairs": [], "extra_dhcp_opts": [], "updated_at": "2016-05-26T02:37:37", "device_owner": "compute:None", "port_security_enabled": true, "binding:profile": {}, "fixed_ips": [{"subnet_id": "b888bc63-4e17-43e1-885f-aa7ba3511d33", "ip_address": "10.100.0.6"}], "id": "fa843bf8-01b7-424e-a705-ff8453ec743e", "security_groups": ["f1cb0412-1db4-4332-911b-1f9a0362f3bc"], "device_id": "2b580249-9467-44d7-9e6b-73693534a5d4", "name": "", "admin_state_up": true, "network_id": "5db2908b-a315-45e2-92ac-37472d4bdf76", "dns_name": null, "binding:vif_details": {"port_filter": true, "ovs_hybrid_plug": true}, "binding:vnic_type": "normal", "binding:vif_type": "ovs", "tenant_id": "420cb14a42c84b3fbbb8891fc9329816", "mac_address": "fa:16:3e:2c:5a:c4", "created_at": "2016-05-26T02:37:36"}} _http_log_response /usr/lib/python2.7/dist-packages/keystoneauth1/session.py:277 2016-05-26 02:37:37.615 14354 DEBUG nova.network.neutronv2.api [req-81751add-78d4-4f53-8042-a04596d0a040 becfc7903e7e4917ab927eae3fb3bb3d 420cb14a42c84b3fbbb8891fc9329816 - - -] [instance: 2b580249-9467-44d7-9e6b-73693534a5d4] Successfully created port: fa843bf8-01b7-424e-a705-ff8453ec743e _create_port /usr/lib/python2.7/dist-packages/nova/network/neutronv2/api.py:261 But eventually instance failed to spawn: 2016-05-26 02:37:47.591 14354 ERROR nova.compute.manager [req-81751add-78d4-4f53-8042-a04596d0a040 becfc7903e7e4917ab927eae3fb3bb3d 420cb14a42c84b3fbbb8891fc9329816 - - -] [instance: 2b580249-9467-44d7-9e6b-73693534a5d4] Instance failed to spawn 2016-05-26 02:37:47.591 14354 ERROR nova.compute.manager [instance: 2b580249-9467-44d7-9e6b-73693534a5d4] Traceback (most recent call last): 2016-05-26 02:37:47.591 14354 ERROR nova.compute.manager [instance: 2b580249-9467-44d7-9e6b-73693534a5d4] File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 2218, in _build_resources 2016-05-26 02:37:47.591 14354 ERROR nova.compute.manager [instance: 2b580249-9467-44d7-9e6b-73693534a5d4] yield resources 2016-05-26 02:37:47.591 14354 ERROR nova.compute.manager [instance: 2b580249-9467-44d7-9e6b-73693534a5d4] File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 2064, in _build_and_run_instance 2016-05-26 02:37:47.591 14354 ERROR nova.compute.manager [instance: 2b580249-9467-44d7-9e6b-73693534a5d4] block_device_info=block_device_info) 2016-05-26 02:37:47.591 14354 ERROR nova.compute.manager [instance: 2b580249-9467-44d7-9e6b-73693534a5d4] File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 2780, in spawn 2016-05-26 02:37:47.591 14354 ERROR nova.compute.manager [instance: 2b580249-9467-44d7-9e6b-73693534a5d4] block_device_info=block_device_info) 2016-05-26 02:37:47.591 14354 ERROR nova.compute.manager [instance: 2b580249-9467-44d7-9e6b-73693534a5d4] File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 4915, in _create_domain_and_network 2016-05-26 02:37:47.591 14354 ERROR nova.compute.manager [instance: 2b580249-9467-44d7-9e6b-73693534a5d4] xml, pause=pause, power_on=power_on) 2016-05-26 02:37:47.591 14354 ERROR nova.compute.manager [instance: 2b580249-9467-44d7-9e6b-73693534a5d4] File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 4846, in _create_domain 2016-05-26 02:37:47.591 14354 ERROR nova.compute.manager [instance: 2b580249-9467-44d7-9e6b-73693534a5d4] guest.launch(pause=pause) 2016-05-26 02:37:47.591 14354 ERROR nova.compute.manager [instance: 2b580249-9467-44d7-9e6b-73693534a5d4] File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/guest.py", line 142, in launch 2016-05-26 02:37:47.591 14354 ERROR nova.compute.manager [instance: 2b580249-9467-44d7-9e6b-73693534a5d4] self._encoded_xml, errors='ignore') 2016-05-26 02:37:47.591 14354 ERROR nova.compute.manager [instance: 2b580249-9467-44d7-9e6b-73693534a5d4] File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__ 2016-05-26 02:37:47.591 14354 ERROR nova.compute.manager [instance: 2b580249-9467-44d7-9e6b-73693534a5d4] self.force_reraise() 2016-05-26 02:37:47.591 14354 ERROR nova.compute.manager [instance: 2b580249-9467-44d7-9e6b-73693534a5d4] File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise 2016-05-26 02:37:47.591 14354 ERROR nova.compute.manager [instance: 2b580249-9467-44d7-9e6b-73693534a5d4] six.reraise(self.type_, self.value, self.tb) 2016-05-26 02:37:47.591 14354 ERROR nova.compute.manager [instance: 2b580249-9467-44d7-9e6b-73693534a5d4] File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/guest.py", line 137, in launch 2016-05-26 02:37:47.591 14354 ERROR nova.compute.manager [instance: 2b580249-9467-44d7-9e6b-73693534a5d4] return self._domain.createWithFlags(flags) 2016-05-26 02:37:47.591 14354 ERROR nova.compute.manager [instance: 2b580249-9467-44d7-9e6b-73693534a5d4] File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 186, in doit 2016-05-26 02:37:47.591 14354 ERROR nova.compute.manager [instance: 2b580249-9467-44d7-9e6b-73693534a5d4] result = proxy_call(self._autowrap, f, *args, **kwargs) 2016-05-26 02:37:47.591 14354 ERROR nova.compute.manager [instance: 2b580249-9467-44d7-9e6b-73693534a5d4] File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 144, in proxy_call 2016-05-26 02:37:47.591 14354 ERROR nova.compute.manager [instance: 2b580249-9467-44d7-9e6b-73693534a5d4] rv = execute(f, *args, **kwargs) 2016-05-26 02:37:47.591 14354 ERROR nova.compute.manager [instance: 2b580249-9467-44d7-9e6b-73693534a5d4] File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 125, in execute 2016-05-26 02:37:47.591 14354 ERROR nova.compute.manager [instance: 2b580249-9467-44d7-9e6b-73693534a5d4] six.reraise(c, e, tb) 2016-05-26 02:37:47.591 14354 ERROR nova.compute.manager [instance: 2b580249-9467-44d7-9e6b-73693534a5d4] File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 83, in tworker 2016-05-26 02:37:47.591 14354 ERROR nova.compute.manager [instance: 2b580249-9467-44d7-9e6b-73693534a5d4] rv = meth(*args, **kwargs) 2016-05-26 02:37:47.591 14354 ERROR nova.compute.manager [instance: 2b580249-9467-44d7-9e6b-73693534a5d4] File "/usr/lib/python2.7/dist-packages/libvirt.py", line 1033, in createWithFlags 2016-05-26 02:37:47.591 14354 ERROR nova.compute.manager [instance: 2b580249-9467-44d7-9e6b-73693534a5d4] if ret == -1: raise libvirtError ('virDomainCreateWithFlags() failed', dom=self) 2016-05-26 02:37:47.591 14354 ERROR nova.compute.manager [instance: 2b580249-9467-44d7-9e6b-73693534a5d4] libvirtError: Unable to read from monitor: Connection reset by peer 2016-05-26 02:37:47.591 14354 ERROR nova.compute.manager [instance: 2b580249-9467-44d7-9e6b-73693534a5d4] 2016-05-26 02:37:47.595 14354 INFO nova.compute.manager [req-81751add-78d4-4f53-8042-a04596d0a040 becfc7903e7e4917ab927eae3fb3bb3d 420cb14a42c84b3fbbb8891fc9329816 - - -] [instance: 2b580249-9467-44d7-9e6b-73693534a5d4] Terminating instance However nova-compute did not delete the port from neutron during instance termination. Then instance was rescheduled to node-3, nova-compute created new port in neutron and successfully spawned the instance. Nova team, please check why nova-compute on node-6 did not clear neutron port.