did some digging here . on the multinode jobs there is a pretty clear error from the dhcp-agent logs [1][2] "Device tapc0c2f8e7-d3 cannot be used as it has no MAC address"
2021-07-26 13:20:24.116 126380 DEBUG neutron.agent.linux.utils [req-ac28e47a-1e4e-4ef3-b310-9f17f93b9c95 - - - - -] Running command (rootwrap daemon): ['ip', 'netns', 'exec', 'qdhcp-2dd0cdd7-df67-4ee5-910b-bfa09ee0fc55', 'sysctl', '-w', 'net.ipv4.ip_nonlocal_bind=1'] execute_rootwrap_daemon /usr/lib/python3.6/site-packages/neutron/agent/linux/utils.py:104
2021-07-26 13:20:24.149 126380 DEBUG neutron.agent.linux.utils [req-ac28e47a-1e4e-4ef3-b310-9f17f93b9c95 - - - - -] Running command (rootwrap daemon): ['ip', 'netns', 'exec', 'qdhcp-2dd0cdd7-df67-4ee5-910b-bfa09ee0fc55', 'sysctl', '-w', 'net.ipv6.conf.default.accept_ra=0'] execute_rootwrap_daemon /usr/lib/python3.6/site-packages/neutron/agent/linux/utils.py:104
2021-07-26 13:20:24.198 139823 DEBUG neutron.privileged.agent.linux.ip_lib [-] Interface tapc0c2f8e7-d3 not found in namespace qdhcp-2dd0cdd7-df67-4ee5-910b-bfa09ee0fc55 get_link_id /usr/lib/python3.6/site-packages/neutron/privileged/agent/linux/ip_lib.py:290
2021-07-26 13:20:24.198 139823 DEBUG oslo.privsep.daemon [-] privsep: reply[139750036169080]: (4, False) _call_back /usr/lib/python3.6/site-packages/oslo_privsep/daemon.py:511
2021-07-26 13:20:24.200 126380 ERROR neutron.agent.linux.ip_lib [req-ac28e47a-1e4e-4ef3-b310-9f17f93b9c95 - - - - -] Device tapc0c2f8e7-d3 cannot be used as it has no MAC address
On the standalone jobs there is no dhcp-agent container so we don't have that same error, however there are different errors in neutron server.log [3]
2021-07-19 02:08:49.050 15 ERROR networking_ovn.ovsdb.ovsdb_monitor [-] HashRing is empty, error: Hash Ring returned empty when hashing "b'908a8d0b-3423-4817-bc0b-384f3f8941fc'". This should never happen in a normal situation, please check the status of your cluster: networking_ovn.common.exceptions.HashRingIsEmpty: Hash Ring returned empty when hashing "b'908a8d0b-3423-4817-bc0b-384f3f8941fc'". This should never happen in a normal situation, please check the status of your cluster
...
2021-07-19 02:16:14.759 15 INFO neutron.services.segments.plugin [-] Segment 4c047c5c-c230-4c8f-98ed-b5b15a5c2e90 resource provider not found; error: Placement Client Error (4xx): {"errors": [{"status": 404, "title": "Not Found", "detail": "The resource could not be found.\n\n No resource provider with uuid 4c047c5c-c230-4c8f-98ed-b5b15a5c2e90 found for delete ", "request_id": "req-ac607e4b-5e09-4ea0-90cf-eebcc2dcbfd3"}]}
did some digging here . on the multinode jobs there is a pretty clear error from the dhcp-agent logs [1][2] "Device tapc0c2f8e7-d3 cannot be used as it has no MAC address"
2021-07-26 13:20:24.116 126380 DEBUG neutron. agent.linux. utils [req-ac28e47a- 1e4e-4ef3- b310-9f17f93b9c 95 - - - - -] Running command (rootwrap daemon): ['ip', 'netns', 'exec', 'qdhcp- 2dd0cdd7- df67-4ee5- 910b-bfa09ee0fc 55', 'sysctl', '-w', 'net.ipv4. ip_nonlocal_ bind=1' ] execute_ rootwrap_ daemon /usr/lib/ python3. 6/site- packages/ neutron/ agent/linux/ utils.py: 104 agent.linux. utils [req-ac28e47a- 1e4e-4ef3- b310-9f17f93b9c 95 - - - - -] Running command (rootwrap daemon): ['ip', 'netns', 'exec', 'qdhcp- 2dd0cdd7- df67-4ee5- 910b-bfa09ee0fc 55', 'sysctl', '-w', 'net.ipv6. conf.default. accept_ ra=0'] execute_ rootwrap_ daemon /usr/lib/ python3. 6/site- packages/ neutron/ agent/linux/ utils.py: 104 privileged. agent.linux. ip_lib [-] Interface tapc0c2f8e7-d3 not found in namespace qdhcp-2dd0cdd7- df67-4ee5- 910b-bfa09ee0fc 55 get_link_id /usr/lib/ python3. 6/site- packages/ neutron/ privileged/ agent/linux/ ip_lib. py:290 169080] : (4, False) _call_back /usr/lib/ python3. 6/site- packages/ oslo_privsep/ daemon. py:511 agent.linux. ip_lib [req-ac28e47a- 1e4e-4ef3- b310-9f17f93b9c 95 - - - - -] Device tapc0c2f8e7-d3 cannot be used as it has no MAC address
2021-07-26 13:20:24.149 126380 DEBUG neutron.
2021-07-26 13:20:24.198 139823 DEBUG neutron.
2021-07-26 13:20:24.198 139823 DEBUG oslo.privsep.daemon [-] privsep: reply[139750036
2021-07-26 13:20:24.200 126380 ERROR neutron.
On the standalone jobs there is no dhcp-agent container so we don't have that same error, however there are different errors in neutron server.log [3]
2021-07-19 02:08:49.050 15 ERROR networking_ ovn.ovsdb. ovsdb_monitor [-] HashRing is empty, error: Hash Ring returned empty when hashing "b'908a8d0b- 3423-4817- bc0b-384f3f8941 fc'". This should never happen in a normal situation, please check the status of your cluster: networking_ ovn.common. exceptions. HashRingIsEmpty : Hash Ring returned empty when hashing "b'908a8d0b- 3423-4817- bc0b-384f3f8941 fc'". This should never happen in a normal situation, please check the status of your cluster services. segments. plugin [-] Segment 4c047c5c- c230-4c8f- 98ed-b5b15a5c2e 90 resource provider not found; error: Placement Client Error (4xx): {"errors": [{"status": 404, "title": "Not Found", "detail": "The resource could not be found.\n\n No resource provider with uuid 4c047c5c- c230-4c8f- 98ed-b5b15a5c2e 90 found for delete ", "request_id": "req-ac607e4b- 5e09-4ea0- 90cf-eebcc2dcbf d3"}]}
...
2021-07-19 02:16:14.759 15 INFO neutron.
[1] https:/ /logserver. rdoproject. org/openstack- component- network/ opendev. org/openstack/ tripleo- ci/master/ periodic- tripleo- ci-centos- 8-containers- multinode- network- train/794b86d/ logs/undercloud /var/log/ extra/errors. txt.gz /logserver. rdoproject. org/openstack- component- network/ opendev. org/openstack/ tripleo- ci/master/ periodic- tripleo- ci-centos- 8-containers- multinode- network- train/794b86d/ logs/undercloud /var/log/ containers/ neutron/ dhcp-agent. log.txt. gz /b18b498c87a707 60171a- 55be5aa52ff9f3c 19ea7817c824a7d d8.ssl. cf5.rackcdn. com/799981/ 1/check/ tripleo- ci-centos- 8-standalone/ d4c3fa4/ logs/undercloud /var/log/ containers/ neutron/ server. log
[2] https:/
[3] https:/