| 2016-09-09 10:17:15 |
Joris S'heeren |
description |
Our setup consists of multiple controllers and multiple hypervisors. Our shared storage for the instances is on a nfs 4.1 export. Using Ubuntu 16.04 LTS and Openstack Mitaka
When we launch an instance, nova updates the mtime for the _base image to let the image cache manager know the image is actively used. I think this was added here: https://review.openstack.org/gitweb?p=openstack/nova.git;a=commitdiff;h=fb6ca3e7c8a38328d384cd41c061ded6623dac90
Because of this, in our setup, we are seeing unexpected input/output errors:
Stderr: u"/bin/touch: setting times of '/var/lib/nova/instances/_base/79e34519bacb47ad6f64c4baca4d33fd5c57d34d': Input/output error
A full trace can be found here: http://paste.openstack.org/show/570161/
This error particularly shows itself when launching multiple instances at once.
Also, because of this error, the instances are rescheduled. The assigned neutron ports, however, are not deleted. This results in multiple ip's assigned to the instances, with only one of them UP. This also results in attached floating ip's not working ..
This is similar to https://bugs.launchpad.net/nova/+bug/1609526, nova should tell neutron, either to delete the unused port, or update it instead of creating a new one.
Some more info on our environment:
----------------------------------
Using libvirt + kvm, neutron with openvswitch L3 HA
# dpkg -l | grep nova
ii nova-common 2:13.0.0-0ubuntu2 all OpenStack Compute - common files
ii nova-compute 2:13.0.0-0ubuntu2 all OpenStack Compute - compute node base
ii nova-compute-kvm 2:13.0.0-0ubuntu2 all OpenStack Compute - compute node (KVM)
ii nova-compute-libvirt 2:13.0.0-0ubuntu2 all OpenStack Compute - compute node libvirt support
ii python-nova 2:13.0.0-0ubuntu2 all OpenStack Compute Python libraries
ii python-novaclient 2:3.3.1-2 all client library for OpenStack Compute API - Python 2.7 |
Our setup consists of multiple controllers and multiple hypervisors. Our shared storage for the instances is on a nfs 4.1 export. Using Ubuntu 16.04 LTS and Openstack Mitaka
When we launch an instance, nova updates the mtime for the _base image to let the image cache manager know the image is actively used. I think this was added here: https://review.openstack.org/gitweb?p=openstack/nova.git;a=commitdiff;h=fb6ca3e7c8a38328d384cd41c061ded6623dac90
Because of this, in our setup, we are seeing unexpected input/output errors:
Stderr: u"/bin/touch: setting times of '/var/lib/nova/instances/_base/79e34519bacb47ad6f64c4baca4d33fd5c57d34d': Input/output error
A full trace can be found here: http://paste.openstack.org/show/570161/
This error particularly shows itself when launching multiple instances at once.
Also, because of this error, the instances are rescheduled. The assigned neutron ports, however, are not deleted. This results in multiple ip's assigned to the instances, with only one of them UP. This also results in attached floating ip's not working ..
This is similar to https://bugs.launchpad.net/nova/+bug/1609526, nova should tell neutron, either to delete the unused port, or update it instead of creating a new one.
Some more info on our environment:
----------------------------------
Using libvirt + kvm, neutron with openvswitch L3 HA
# dpkg -l | grep nova
ii nova-common 2:13.0.0-0ubuntu2 all OpenStack Compute - common files
ii nova-compute 2:13.0.0-0ubuntu2 all OpenStack Compute - compute node base
ii nova-compute-kvm 2:13.0.0-0ubuntu2 all OpenStack Compute - compute node (KVM)
ii nova-compute-libvirt 2:13.0.0-0ubuntu2 all OpenStack Compute - compute node libvirt support
ii python-nova 2:13.0.0-0ubuntu2 all OpenStack Compute Python libraries
ii python-novaclient 2:3.3.1-2 all client library for OpenStack Compute API - Python 2.7
# dpkg -l |grep libvirt
ii libvirt-bin 1.3.1-1ubuntu10.1 amd64 programs for the libvirt library
ii libvirt0:amd64 1.3.1-1ubuntu10.1 amd64 library for interfacing with different virtualization systems
ii nova-compute-libvirt 2:13.0.0-0ubuntu2 all OpenStack Compute - compute node libvirt support
ii python-libvirt 1.3.1-1ubuntu1 amd64 libvirt Python bindings |
|
