[DPDK] Nova can't boot instance: "Command: sudo nova-rootwrap /etc/nova/rootwrap.conf ip link set vhu9f1b57a7-c9 mtu 1500. Exit code: 1. Stderr: Cannot find device "vhu9f1b57a7-c9"

Bug #1575091 reported by Artem Panchenko
14
This bug affects 2 people
Affects Status Importance Assigned to Milestone
Fuel for OpenStack
Fix Committed
High
Ivan Udovichenko
Mitaka
Fix Released
High
MOS Nova

Bug Description

Fuel version info (10 build #156): http://paste.openstack.org/show/495399/

Instance creation fails on environment with enabled DPDK on computes:

179>Apr 26 09:46:24 node-3 nova-conductor: 2016-04-26 09:46:24.323 2482 ERROR nova.scheduler.utils [req-9599a7e1-230b-47d4-b5a1-89aab1bdd1ed 3fb1765cdeec4c539150406bc7af606e 33b1781e91934e81bac54daa83acc6fd - - -] [instance: 126c33
4f-9b75-4d32-9a79-c31b2aade9b8] Error from last host: node-2.test.domain.local (node node-2.test.domain.local): [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 1926
, in _do_build_and_run_instance\n filter_properties)\n', u' File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 2116, in _build_and_run_instance\n instance_uuid=instance.uuid, reason=six.text_type(e))\n', u'
RescheduledException: Build of instance 126c334f-9b75-4d32-9a79-c31b2aade9b8 was re-scheduled: Unexpected error while running command.\nCommand: sudo nova-rootwrap /etc/nova/rootwrap.conf ip link set vhu8b24aa6d-e5 mtu 1500\nExit co
de: 1\nStdout: u\'\'\nStderr: u\'Cannot find device "vhu8b24aa6d-e5"\\n\'\n']
...
<180>Apr 26 09:46:24 node-3 nova-conductor: 2016-04-26 09:46:24.416 2482 WARNING nova.scheduler.utils [req-9599a7e1-230b-47d4-b5a1-89aab1bdd1ed 3fb1765cdeec4c539150406bc7af606e 33b1781e91934e81bac54daa83acc6fd - - -] Failed to compu
te_task_build_instances: No valid host was found. There are not enough hosts available.
Traceback (most recent call last):

  File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/server.py", line 150, in inner
    return func(*args, **kwargs)

  File "/usr/lib/python2.7/dist-packages/nova/scheduler/manager.py", line 104, in select_destinations
    dests = self.driver.select_destinations(ctxt, spec_obj)

  File "/usr/lib/python2.7/dist-packages/nova/scheduler/filter_scheduler.py", line 74, in select_destinations
    raise exception.NoValidHost(reason=reason)

NoValidHost: No valid host was found. There are not enough hosts available.

Steps to reproduce:

1. Deploy environment with DPDK enabled on computes and allocated HugePages for Nova and DPDK
2. Create new Nova instance

Expected result: instance is created and has 'Active' status

Actual result: instance creation fails, Neutron/OVS on compute node is unable to create port/interface for VM

Additional info:

http://paste.openstack.org/show/495400/

Tags: feature-dpdk
Revision history for this message
Artem Panchenko (apanchenko-8) wrote :
Changed in fuel:
status: New → Confirmed
Revision history for this message
Sergey Matov (smatov) wrote :

For DPDK VHOST_USER interfaces we cant setup MTU using ip link since it's userspace device.
This bug is closing by commit
https://review.openstack.org/gitweb?p=openstack/nova.git;a=commitdiff;h=c7eb823fe73e3db5dca48df5879db18cbab5bd8d
to nova.
Please verify that you are using Mitaka release.

Redirect to MOS-Nova.

Changed in fuel:
assignee: MOS Neutron (mos-neutron) → MOS Nova (mos-nova)
Revision history for this message
Timofey Durakov (tdurakov) wrote :

fix already merged, need to verify this issue on newer iso version

Changed in fuel:
status: Confirmed → Fix Committed
Changed in fuel:
assignee: MOS Nova (mos-nova) → MOS Packaging Team (mos-packaging)
status: Fix Committed → Confirmed
Revision history for this message
Timofey Durakov (tdurakov) wrote :

rebuild nova package for 10.0 required, as patches are already being merged

Changed in fuel:
assignee: MOS Packaging Team (mos-packaging) → Ivan Udovichenko (iudovichenko)
status: Confirmed → In Progress
Revision history for this message
Kristina Berezovskaia (kkuznetsova) wrote :

Verify on iso 454. We deployed env with dpdk and run mos-integration test on it. Vm was successfully created

Revision history for this message
Ivan Udovichenko (iudovichenko) wrote :

Package has been updated with this change request:
https://review.fuel-infra.org/#/c/22584/

Changed in fuel:
status: In Progress → Fix Committed
Revision history for this message
Fuel Devops McRobotson (fuel-devops-robot) wrote : Change abandoned on openstack-build/nova-build (master)

Change abandoned by Ivan Udovichenko <email address hidden> on branch: master
Review: https://review.fuel-infra.org/20219
Reason: Not required anymore.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.