[tempest] CI job "neutron-ovn-tempest-ipv6-only-ovs-release" unstable, most of the times because of "test_hotplug_nic"

Bug #2007166 reported by Rodolfo Alonso
12
This bug affects 2 people
Affects Status Importance Assigned to Milestone
neutron
Confirmed
Critical
yatin
ovn (Ubuntu)
New
Undecided
Unassigned
Changed in neutron:
importance: Undecided → Critical
yatin (yatinkarel)
Changed in neutron:
status: New → Confirmed
tags: added: gate-failure ovn tempest
yatin (yatinkarel)
Changed in neutron:
assignee: nobody → yatin (yatinkarel)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to neutron (master)

Related fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/neutron/+/873684

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to neutron (master)

Reviewed: https://review.opendev.org/c/openstack/neutron/+/873684
Committed: https://opendev.org/openstack/neutron/commit/5dac5f8c6c4186921a7553116c38f5f03310b1d2
Submitter: "Zuul (22348)"
Branch: master

commit 5dac5f8c6c4186921a7553116c38f5f03310b1d2
Author: yatinkarel <email address hidden>
Date: Tue Feb 14 21:58:16 2023 +0530

    [ovn-ipv6] Temporary exclude test_hotplug_nic

    The test is failing quite often with ovn
    version(v22.03.0) included in ubuntu jammy.

    Until ubuntu jammy includes fixed version
    of ovn let's exclude this test.

    It's running fine on ubuntu focal and
    ubuntu jammy jobs running with ovn from source.

    Related-Bug: #2007166
    Change-Id: I0377418bb5b4a68f9c08326bff0211f668a22b58

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to neutron (master)

Related fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/neutron/+/874253

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on neutron (master)

Change abandoned by "Rodolfo Alonso <email address hidden>" on branch: master
Review: https://review.opendev.org/c/openstack/neutron/+/874253
Reason: Superseded by https://review.opendev.org/c/openstack/neutron/+/874112

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to neutron (master)

Reviewed: https://review.opendev.org/c/openstack/neutron/+/874112
Committed: https://opendev.org/openstack/neutron/commit/f991fc2fbef6b69b97d373e7acff8038b2da3028
Submitter: "Zuul (22348)"
Branch: master

commit f991fc2fbef6b69b97d373e7acff8038b2da3028
Author: yatinkarel <email address hidden>
Date: Thu Feb 16 21:19:24 2023 +0530

    [ovn][ipv6] Skip test_update_delete_extra_route

    The issue is random and when it reproduces, any test
    involving localport(metadata access) ran just after
    test_update_delete_extra_route fails as metadata
    flows are missing.

    The issue do not reproduce in ovn-22.06+(since [1]).

    So until issue's fix is available in Ubuntu jammy,
    we can skip this test.

    [1] https://github.com/ovn-org/ovn/commit/eaf9832be2

    Related-Bug: #2007565
    Related-Bug: #2007166
    Change-Id: Ie2f5ff169bb3007694b9858465125775cfee40f4

Revision history for this message
yatin (yatinkarel) wrote :

Reported Bug against OVN to track actual fix https://bugzilla.redhat.com/show_bug.cgi?id=2172036

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to neutron (master)

Related fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/neutron/+/885074

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to neutron (master)

Reviewed: https://review.opendev.org/c/openstack/neutron/+/885074
Committed: https://opendev.org/openstack/neutron/commit/2a8c7ff4f0a27d426db6d993b373a8222372abbd
Submitter: "Zuul (22348)"
Branch: master

commit 2a8c7ff4f0a27d426db6d993b373a8222372abbd
Author: yatinkarel <email address hidden>
Date: Thu Jun 1 21:25:37 2023 +0530

    [ovn][ipv6] Skip test_update_router_admin_state

    This is same to already skipped test in [1].

    [1] https://review.opendev.org/c/openstack/neutron/+/874112

    Related-Bug: #2007166
    Change-Id: Iafc596e1ff13c7efde5c7da417db85bb562aa3f7

Revision history for this message
yatin (yatinkarel) wrote :
Download full text (4.6 KiB)

Seen a few more failures:-
 - https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_a63/886231/1/check/neutron-ovn-tempest-ipv6-only-ovs-release/a63687d/testr_results.html
 - https://a3e174dd787d1569c081-614f3af0609b64225912d2a4919c81df.ssl.cf5.rackcdn.com/882581/7/check/neutron-ovn-tempest-ipv6-only-ovs-release/feb8b96/testr_results.html

$ grep admin_state_up screen-q-svc.txt |grep "'admin_state_up': True"|grep "Request body: {'router'"
Jun 16 16:28:00.617751 np0034320529 neutron-server[57489]: DEBUG neutron.api.v2.base [None req-2a66e63a-6f5e-4abd-9326-87edec5262f5 demo demo] Request body: {'router': {'admin_state_up': True, 'name': 'router1'}} {{(pid=57489) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:731}}
Jun 16 16:56:24.073625 np0034320529 neutron-server[57489]: DEBUG neutron.api.v2.base [None req-68e7d391-f799-42d6-a156-91566ee8a7d2 tempest-NetworksTestDHCPv6-1147368735 tempest-NetworksTestDHCPv6-1147368735-project-member] Request body: {'router': {'name': 'tempest-NetworksTestDHCPv6-router-863088563', 'external_gateway_info': {}, 'admin_state_up': True}} {{(pid=57489) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:731}}
Jun 16 16:56:29.565406 np0034320529 neutron-server[57489]: DEBUG neutron.api.v2.base [None req-69ecb9e9-adad-4bab-9d88-4d6a8ec93e16 tempest-NetworksTestDHCPv6-1147368735 tempest-NetworksTestDHCPv6-1147368735-project-member] Request body: {'router': {'name': 'tempest-NetworksTestDHCPv6-router-2081586294', 'external_gateway_info': {}, 'admin_state_up': True}} {{(pid=57489) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:731}}
Jun 16 16:58:48.759733 np0034320529 neutron-server[57489]: DEBUG neutron.api.v2.base [None req-e53ee47d-05b5-44d3-ae03-cd5c743e7c2a tempest-TestNetworkBasicOps-1899229111 tempest-TestNetworkBasicOps-1899229111-project-member] Request body: {'router': {'name': 'tempest-TestNetworkBasicOps-router-1236277632', 'admin_state_up': True, 'project_id': 'ffcabcbafb6248378ecf851ad0c4a820', 'external_gateway_info': {'network_id': '738da43f-f3f1-44eb-96b9-c00cba3b25bb'}}} {{(pid=57489) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:731}}
Jun 16 17:03:08.332661 np0034320529 neutron-server[57489]: DEBUG neutron.api.v2.base [None req-602e429e-3d8a-4a4e-a29c-6dbba2a9e2ba tempest-TestNetworkBasicOps-1899229111 tempest-TestNetworkBasicOps-1899229111-project-member] Request body: {'router': {'name': 'tempest-TestNetworkBasicOps-router-742545444', 'admin_state_up': True, 'project_id': 'ffcabcbafb6248378ecf851ad0c4a820', 'external_gateway_info': {'network_id': '738da43f-f3f1-44eb-96b9-c00cba3b25bb'}}} {{(pid=57489) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:731}}
Jun 16 17:05:31.764891 np0034320529 neutron-server[57490]: DEBUG neutron.api.v2.base [None req-43142176-2858-4515-8ed7-a4d2c97a934a tempest-TestSecurityGroupsBasicOps-146927725 tempest-TestSecurityGroupsBasicOps-146927725-project-member] Request body: {'router': {'name': 'tempest-TestSecurityGroupsBasicOps-router-162865443', 'admin_state_up': True, 'project_id': '9db58a07a2a14a5f82b6894431dae303', 'external_gateway_info': {'network_id...

Read more...

Revision history for this message
yatin (yatinkarel) wrote :

Added Ubuntu(ovn) to impacted project. It would be good if OVN22.06 get's included in Ubuntu Jammy.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.