Removing smart_nic port in openstack will try to delete representor port

Bug #1859976 reported by waleed mousa
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
neutron
Invalid
Undecided
Unassigned
os-vif
Incomplete
Undecided
Unassigned

Bug Description

When remove smart_nic port ovs agent it will remove the port from the integration bridge and also will try to remove net device as the port is defined as VIFOpenvswitch device in ovs_neuton_agent

DEBUG oslo_concurrency.processutils [^[[00;36m-] [[01;35mCMD "ovs-vsctl --timeout=120 --oneline --format=json --db=tcp:127.0.0.1:6640 -- --if-exists del-port br-int pf0hpf" returned: 0 in 0.109s[[00m [[00;33m{{(pid=50) execute /usr/lib/python2.7/site-packages/oslo_concurrency/processutils.py:409}}[[00m
DEBUG oslo.privsep.daemon [^[[00;36m-] [[01;35mprivsep: reply281473324162416: (4, '')[[00m [[00;33m{{(pid=50) call_back /usr/lib/python2.7/site-packages/oslo_privsep/daemon.py:475}}[[00m
DEBUG os_vif.internal.ip.linux.impl_pyroute2 [^[[00;36m-] [[01;35mpyroute2 command del, arguments {'index': 9}[[00m [[00;33m{{(pid=50) _ip_link /usr/lib/python2.7/site-packages/os_vif/internal/ip/linux/impl_pyroute2.py:31}}[[00m
DEBUG oslo.privsep.daemon [^[[00;36m-] [[01;35mprivsep: Exception during request281473324162416: (95, 'Operation not supported')[[00m [[00;33m{{(pid=50) _process_cmd /usr/lib/python2.7/site-packages/oslo_privsep/daemon.py:454}}[[00m
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/oslo_privsep/daemon.py", line 449, in _process_cmd
ret = func(f_args, **f_kwargs)
File "/usr/lib/python2.7/site-packages/oslo_privsep/priv_context.py", line 247, in _wrap
return func(*args, **kwargs)
File "/usr/lib/python2.7/site-packages/vif_plug_ovs/linux_net.py", line 69, in delete_net_dev
ip_lib.delete(dev, check_exit_code=[0, 2, 254])
File "/usr/lib/python2.7/site-packages/os_vif/internal/ip/linux/impl_pyroute2.py", line 114, in delete
return self._ip_link(ip, 'del', check_exit_code, *{'index': idx})
File "/usr/lib/python2.7/site-packages/os_vif/internal/ip/linux/impl_pyroute2.py", line 38, in _ip_link
ctx.reraise = False
File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in exit
self.force_reraise()
File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
six.reraise(self.type, self.value, self.tb)
File "/usr/lib/python2.7/site-packages/os_vif/internal/ip/linux/impl_pyroute2.py", line 32, in ip_link
return ip.link(command, **kwargs)
File "/usr/lib/python2.7/site-packages/pyroute2/iproute/linux.py", line 1163, in link
msg_flags=msg_flags)
File "/usr/lib/python2.7/site-packages/pyroute2/netlink/nlsocket.py", line 373, in nlm_request
return tuple(self._genlm_request(*argv, **kwarg))
File "/usr/lib/python2.7/site-packages/pyroute2/netlink/nlsocket.py", line 864, in nlm_request
callback=callback):
File "/usr/lib/python2.7/site-packages/pyroute2/netlink/nlsocket.py", line 376, in get
return tuple(self._genlm_get(*argv, **kwarg))
File "/usr/lib/python2.7/site-packages/pyroute2/netlink/nlsocket.py", line 701, in get
raise msg['header']['error']
NetlinkError: (95, 'Operation not supported')
DEBUG oslo.privsep.daemon [^[[00;36m-] [[01;35mprivsep: reply281473324162416: (5, 'pyroute2.netlink.exceptions.NetlinkError', (95, 'Operation not supported'))[[00m [[00;33m{{(pid=50) _call_back /usr/lib/python2.7/site-packages/oslo_privsep/daemon.py:475}}[[00m
ERROR os_vif [^[[01;36mNone req-73d389c8-7dd3-47be-b136-ed247b6e7e47 [[00;36mNone None] ^[[01;35mFailed to unplug vif VIFOpenVSwitch(active=,address=50:6b:4b:34:a5:3a,bridge_name=,has_traffic_filtering=,id=,network=Network(UNKNOWN),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=,vif_name='pf0hpf')[[00m: NetlinkError: (95, 'Operation not supported')
ERROR os_vif [[01;35m[[00mTraceback (most recent call last):
ERROR os_vif [[01;35m[[00m File "/usr/lib/python2.7/site-packages/os_vif/_init__.py", line 110, in unplug
ERROR os_vif [[01;35m[[00m plugin.unplug(vif, instance_info)
ERROR os_vif [[01;35m[[00m File "/usr/lib/python2.7/site-packages/vif_plug_ovs/ovs.py", line 363, in unplug
ERROR os_vif [[01;35m[[00m self._unplug_vif_generic(vif, instance_info)
ERROR os_vif [[01;35m[[00m File "/usr/lib/python2.7/site-packages/vif_plug_ovs/ovs.py", line 330, in _unplug_vif_generic
ERROR os_vif [[01;35m[[00m self.ovsdb.delete_ovs_vif_port(vif.network.bridge, vif.vif_name)
ERROR os_vif [[01;35m[[00m File "/usr/lib/python2.7/site-packages/vif_plug_ovs/ovsdb/ovsdb_lib.py", line 115, in delete_ovs_vif_port
ERROR os_vif [[01;35m[[00m linux_net.delete_net_dev(dev)
ERROR os_vif [[01;35m[[00m File "/usr/lib/python2.7/site-packages/oslo_privsep/priv_context.py", line 245, in _wrap
ERROR os_vif [[01;35m[[00m return self.channel.remote_call(name, args, kwargs)
ERROR os_vif [[01;35m[[00m File "/usr/lib/python2.7/site-packages/oslo_privsep/daemon.py", line 204, in remote_call
ERROR os_vif [[01;35m[[00m raise exc_type(*result2)
ERROR os_vif [[01;35m[[00mNetlinkError: (95, 'Operation not supported')
ERROR os_vif [[01;35m[[00m
ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent [^[[01;36mNone req-73d389c8-7dd3-47be-b136-ed247b6e7e47 [[00;36mNone None] ^[[01;35mFailed to treat smart-nic port:99abc6d3-a2f4-460f-9d9b-fd2f016b9e95 , error:Failed to unplug VIF VIFOpenVSwitch(active=,address=50:6b:4b:34:a5:3a,bridge_name=,has_traffic_filtering=,id=,network=Network(UNKNOWN),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=,vif_name='pf0hpf'). Got error: (95, 'Operation not supported')[[00m: UnplugException: Failed to unplug VIF VIFOpenVSwitch(active=,address=50:6b:4b:34:a5:3a,bridge_name=,has_traffic_filtering=,id=,network=Network(UNKNOWN),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=,vif_name='pf0hpf'). Got error: (95, 'Operation not supported')

Changed in neutron:
status: New → Invalid
Revision history for this message
sean mooney (sean-k-mooney) wrote :

the traceback show that we are calling ip_lib.delete(dev, check_exit_code=[0, 2, 254]) on line 69

https://github.com/openstack/os-vif/blob/7bad0f93bbda1238fa3d853ce428f92d8606b360/vif_plug_ovs/linux_net.py#L69

however to reach that it means that "if ip_lib.exists(dev):" on line 67 would have to be true.
for a dpdk port representor there should be no kernel netdev so that shoudl not be true.

that is calling https://github.com/openstack/os-vif/blob/master/os_vif/internal/ip/linux/impl_pyroute2.py#L116-L120

def exists(self, device):
        """Return True if the device exists."""
        with iproute.IPRoute() as ip:
            idx = ip.link_lookup(ifname=device)
        return True if idx else False

which is just delegating to iproute2

can you provide the output of ip like before and after booting a vm.
we should not see any netdev with the name of the dpdk reprsentor.

Changed in os-vif:
status: New → Incomplete
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.