Ubuntu
openvswitch package

With hardware offloading enabled, OVS logs are spammed with netdev_offload_tc ERR messages

Bug #1897483 reported by Nikolay Vinogradov
10
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Neutron Open vSwitch Charm
Invalid
Undecided
Unassigned
openvswitch (Ubuntu)
Confirmed
Medium
Unassigned

Bug Description

If neutron-openvswitch charm is deployed with enable-hardware-offload=true ovs-vswitchd.log are flooded with messages like those:

2020-09-27T21:55:40.460Z|28192|netdev_offload_tc(revalidator625)|ERR|Dropped 11 log messages in last 1 seconds (most recently, 1 seconds ago) due to excessive rate
2020-09-27T21:55:40.460Z|28193|netdev_offload_tc(revalidator625)|ERR|dump_create: failed to get ifindex for qr-66e28056-8f: Operation not supported
2020-09-27T21:55:41.461Z|28194|netdev_offload_tc(revalidator625)|ERR|Dropped 11 log messages in last 1 seconds (most recently, 1 seconds ago) due to excessive rate
2020-09-27T21:55:41.461Z|28195|netdev_offload_tc(revalidator625)|ERR|dump_create: failed to get ifindex for qr-66e28056-8f: Operation not supported
2020-09-27T21:55:42.461Z|28196|netdev_offload_tc(revalidator625)|ERR|Dropped 11 log messages in last 1 seconds (most recently, 1 seconds ago) due to excessive rate
2020-09-27T21:55:42.461Z|28197|netdev_offload_tc(revalidator625)|ERR|dump_create: failed to get ifindex for qr-66e28056-8f: Operation not supported
2020-09-27T21:55:43.462Z|28198|netdev_offload_tc(revalidator625)|ERR|Dropped 11 log messages in last 1 seconds (most recently, 1 seconds ago) due to excessive rate
2020-09-27T21:55:43.462Z|28199|netdev_offload_tc(revalidator625)|ERR|dump_create: failed to get ifindex for qr-66e28056-8f: Operation not supported
2020-09-27T21:55:44.464Z|28200|netdev_offload_tc(revalidator625)|ERR|Dropped 11 log messages in last 1 seconds (most recently, 1 seconds ago) due to excessive rate
2020-09-27T21:55:44.464Z|28201|netdev_offload_tc(revalidator625)|ERR|dump_create: failed to get ifindex for qr-66e28056-8f: Operation not supported

The messages are possibly harmful, but there are thousands of them per hour. They can be disabled by executing:

sudo ovs-appctl vlog/set netdev_offload_tc:OFF

Tags: sts
Revision history for this message
James Page (james-page) wrote :

Moved to openvswitch package as this is not a charm issue.

Changed in charm-neutron-openvswitch:
status: New → Invalid
Revision history for this message
Trent Lloyd (lathiat) wrote :

There is an indication in the below RHBZ this can actually prevent openvswitch from working properly as it loses too much CPU time to this processing in large environments (100s or 1000s of ports)

https://bugzilla.redhat.com/show_bug.cgi?id=1737982

Seems to be a rejected upstream patch here, unclear if one was later accepted, we shoudl check for it:
https://lists.linuxfoundation.org/pipermail/ovs-dev/2019-March/357348.html

And potentially prioritise a fix for this.

tags: added: sts
Revision history for this message
James Page (james-page) wrote :

Triaging as enough there to confirm this is a bug and Medium as there is a workaround.

Changed in openvswitch (Ubuntu):
status: New → Triaged
importance: Undecided → Medium
Revision history for this message
James Page (james-page) wrote :

This was fixed in the 2.13 branch under commit 1028cb71d0 which makes the log level for these messages a lower priority.

Revision history for this message
James Page (james-page) wrote :

git tag --contains 1028cb71d0

v2.12.0
v2.12.1
v2.12.2
v2.12.3
v2.13.0
v2.13.1
v2.13.2
v2.13.3
v2.14.0
v2.14.1
v2.14.2
v2.15.0

Changed in openvswitch (Ubuntu):
status: Triaged → Fix Released
Revision history for this message
Erlon R. Cruz (sombrafam) wrote :

Hi James,

The fix mentioned above fixes the issue for the 'dpif_netlink' module. The problem mentioned
here is for the netdev_offload_tc. The fix should be basically the same though.

Erlon

Changed in openvswitch (Ubuntu):
status: Fix Released → Confirmed
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.