Tempest test TestNetworkBasicOps.test_port_security_macspoofing_port sometimes is failing due to ping issue

Bug #2008062 reported by Slawek Kaplonski
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
neutron
Fix Committed
Critical
yatin

Bug Description

It happened at least twice recently in the neutron-ovn-tempest-slow job:

    https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_992/periodic/opendev.org/openstack/neutron/master/neutron-ovn-tempest-slow/992d108/testr_results.html

    https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_64e/periodic/opendev.org/openstack/neutron/master/neutron-ovn-tempest-slow/64eb479/testr_results.html

Stacktrace:

Traceback (most recent call last):
  File "/opt/stack/tempest/tempest/common/utils/__init__.py", line 89, in wrapper
    return func(*func_args, **func_kwargs)
  File "/opt/stack/tempest/tempest/common/utils/__init__.py", line 70, in wrapper
    return f(*func_args, **func_kwargs)
  File "/opt/stack/tempest/tempest/scenario/test_network_basic_ops.py", line 920, in test_port_security_macspoofing_port
    self.check_remote_connectivity(ssh_client, dest=peer_address,
  File "/opt/stack/tempest/tempest/scenario/manager.py", line 1465, in check_remote_connectivity
    self.fail(msg)
  File "/usr/lib/python3.10/unittest/case.py", line 675, in fail
    raise self.failureException(msg)
AssertionError: Timed out waiting for 10.1.0.21 to become reachable from 172.24.5.84

Revision history for this message
Lajos Katona (lajos-katona) wrote :
Revision history for this message
yatin (yatinkarel) wrote :

It's seen quite frequently in tempest-slow-py3 and tempest-full-parallel(periodic so less execution) jobs. As per opensearch in last 54 hits:-

tempest-slow-py3 74.1%
tempest-full-parallel 12.0%
neutron-ovn-tempest-slow 7.4%
tempest-slow-2023-1 5.6%
tempest-all 0.9%

Changed in neutron:
importance: High → Critical
Revision history for this message
yatin (yatinkarel) wrote :

This looks to be a side effect of dhcpcd(configure all interfaces by default) available in cirros-0.6.0+ and the issue started to appear after jobs switched to use the newer cirros versions[1][2]

"test_port_security_macspoofing_port" is specifically impacted as it does manage the ip address manually, there was a slight change after the newer cirros version to it[3]

We could stop dhcpcd or reconfigure(denyinterfaces eth1 to /etc/dhcpcd.conf) so it do not interfare with the test config. And for this i found an issue in cirros and reported[4]

To fix we can update test as above or move affected jobs to use cirros 0.5.2 temporary.

[1] https://review.opendev.org/c/openstack/neutron/+/871272
[2] https://review.opendev.org/c/openstack/devstack/+/881437
[3] https://review.opendev.org/c/openstack/tempest/+/871271
[4] https://github.com/cirros-dev/cirros/issues/107

yatin (yatinkarel)
Changed in neutron:
assignee: nobody → yatin (yatinkarel)
status: Confirmed → In Progress
Revision history for this message
yatin (yatinkarel) wrote :
yatin (yatinkarel)
Changed in neutron:
status: In Progress → Fix Committed
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/tempest 36.0.0

This issue was fixed in the openstack/tempest 36.0.0 release.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.