CS9 Tempest tests failing with "Host 'standalone.localdomain' is not mapped to any cell"

Bug #1964269 reported by Douglas Viroel
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
tripleo
Fix Released
Critical
Unassigned

Bug Description

Multiple compute and network tests are failing trying to create a testing server. The errors is always:
 "Host 'standalone.localdomain' is not mapped to any cell"

Traceback:
Traceback (most recent call last):
  File "/usr/lib/python3.9/site-packages/tempest/common/utils/__init__.py", line 70, in wrapper
    return f(*func_args, **func_kwargs)
  File "/usr/lib/python3.9/site-packages/tempest/scenario/test_network_basic_ops.py", line 435, in test_network_basic_ops
    self._setup_network_and_servers()
  File "/usr/lib/python3.9/site-packages/tempest/scenario/test_network_basic_ops.py", line 120, in _setup_network_and_servers
    server = self._create_server(self.network, port_id)
  File "/usr/lib/python3.9/site-packages/tempest/scenario/test_network_basic_ops.py", line 169, in _create_server
    server = self.create_server(
  File "/usr/lib/python3.9/site-packages/tempest/scenario/manager.py", line 309, in create_server
    body, _ = compute.create_test_server(
  File "/usr/lib/python3.9/site-packages/tempest/common/compute.py", line 294, in create_test_server
    LOG.exception('Server %s failed to delete in time',
  File "/usr/lib/python3.9/site-packages/oslo_utils/excutils.py", line 227, in __exit__
    self.force_reraise()
  File "/usr/lib/python3.9/site-packages/oslo_utils/excutils.py", line 200, in force_reraise
    raise self.value
  File "/usr/lib/python3.9/site-packages/tempest/common/compute.py", line 264, in create_test_server
    waiters.wait_for_server_status(
  File "/usr/lib/python3.9/site-packages/tempest/common/waiters.py", line 80, in wait_for_server_status
    raise exceptions.BuildErrorException(details, server_id=server_id)
tempest.exceptions.BuildErrorException: Server 58c51d2c-b202-40a9-84c5-04cf8d33b5eb failed to build and is in ERROR status
Details: Fault: {'code': 400, 'created': '2022-03-08T20:36:41Z', 'message': "Host 'standalone.localdomain' is not mapped to any cell"}. Server boot request ID: req-b6b752bb-fb9f-4214-a171-7de1e1bf61e6.

Issue is known in Main and Wallaby branches so far:

https://logserver.rdoproject.org/53/21853/62/check/periodic-tripleo-ci-centos-9-scenario004-standalone-master/f4a6d91/logs/undercloud/var/log/tempest/stestr_results.html.gz
https://logserver.rdoproject.org/openstack-periodic-integration-main/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-containers-multinode-master/51cab12/logs/undercloud/var/log/tempest/stestr_results.html.gz
https://logserver.rdoproject.org/53/21853/62/check/periodic-tripleo-ci-centos-9-standalone-wallaby/ed691f7/logs/undercloud/var/log/tempest/stestr_results.html.gz

Those errors appear on nova-conductor:
https://logserver.rdoproject.org/53/21853/62/check/periodic-tripleo-ci-centos-9-standalone-wallaby/ed691f7/logs/undercloud/var/log/containers/nova/nova-conductor.log.txt.gz

Revision history for this message
Soniya Murlidhar Vyas (svyas) wrote :
Revision history for this message
chandan kumar (chkumar246) wrote :

On CS9 wallaby, the above issue is not showing up.

Based on the results of openstack-periodic-integration-stable1-cs9
https://review.rdoproject.org/zuul/buildset/3ae5ce4e79784fab81260584860766a2

https://logserver.rdoproject.org/15/151734d416f03f3b5c9c93b1cb999f490c49589c/openstack-periodic-integration-stable1-cs9/periodic-tripleo-ci-centos-9-standalone-wallaby/43072dc/logs/undercloud/var/log/tempest/stestr_results.html.gz

Failure seems to be different
```
{0} tempest.api.compute.servers.test_attach_interfaces.AttachInterfacesUnderV243Test.test_add_remove_fixed_ip [349.895008s] ... FAILED

Captured traceback:
~~~~~~~~~~~~~~~~~~~
    Traceback (most recent call last):
      File "/usr/lib/python3.9/site-packages/tempest/lib/common/ssh.py", line 107, in _get_ssh_connection
        ssh.connect(self.host, port=self.port, username=self.username,
      File "/usr/lib/python3.9/site-packages/paramiko/client.py", line 435, in connect
        self._auth(
      File "/usr/lib/python3.9/site-packages/paramiko/client.py", line 764, in _auth
        raise saved_exception
      File "/usr/lib/python3.9/site-packages/paramiko/client.py", line 751, in _auth
        self._transport.auth_password(username, password)
      File "/usr/lib/python3.9/site-packages/paramiko/transport.py", line 1509, in auth_password
        return self.auth_handler.wait_for_response(my_event)
      File "/usr/lib/python3.9/site-packages/paramiko/auth_handler.py", line 250, in wait_for_response
        raise e
    paramiko.ssh_exception.AuthenticationException: Authentication failed.
```

tempest.api.compute.servers.test_attach_interfaces.AttachInterfacesUnderV243Test.test_add_remove_fixed_ip[id-c7e0e60b-ee45-43d0-abeb-8596fd42a2f9,network,smoke]
tempest.scenario.test_server_basic_ops.TestServerBasicOps.test_server_basic_ops[compute,id-7fff3fb3-91d8-4fd0-bd7d-0204f1f180ba,network,smoke]
tempest.api.compute.servers.test_server_actions.ServerActionsTestJSON.test_reboot_server_hard[id-2cb1baf6-ac8d-4429-bf0d-ba8a0ba53e32,smoke]
tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.test_network_basic_ops[compute,id-f323b3ba-82f8-4db7-8ea6-6a895869ec49,network,smoke]

Revision history for this message
Ronelle Landy (rlandy) wrote :

Promoted master and wallaby - closing this out

Changed in tripleo:
status: Triaged → Fix Released
Changed in tripleo:
status: Fix Released → Triaged
status: Triaged → Fix Released
Revision history for this message
chandan kumar (chkumar246) wrote :

https://logserver.rdoproject.org/openstack-periodic-integration-main/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-scenario007-multinode-oooq-container-master/8112a31/logs/undercloud/var/log/tempest/tempest_run.log.txt.gz

```
{0} tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.test_network_basic_ops [13.065515s] ... FAILED

Captured traceback:
~~~~~~~~~~~~~~~~~~~
    Traceback (most recent call last):

      File "/usr/lib/python3.9/site-packages/tempest/common/utils/__init__.py", line 70, in wrapper
    return f(*func_args, **func_kwargs)

      File "/usr/lib/python3.9/site-packages/tempest/scenario/test_network_basic_ops.py", line 435, in test_network_basic_ops
    self._setup_network_and_servers()

      File "/usr/lib/python3.9/site-packages/tempest/scenario/test_network_basic_ops.py", line 120, in _setup_network_and_servers
    server = self._create_server(self.network, port_id)

      File "/usr/lib/python3.9/site-packages/tempest/scenario/test_network_basic_ops.py", line 169, in _create_server
    server = self.create_server(

      File "/usr/lib/python3.9/site-packages/tempest/scenario/manager.py", line 309, in create_server
    body, _ = compute.create_test_server(

      File "/usr/lib/python3.9/site-packages/tempest/common/compute.py", line 327, in create_test_server
    LOG.exception('Server %s failed to delete in time',

      File "/usr/lib/python3.9/site-packages/oslo_utils/excutils.py", line 227, in __exit__
    self.force_reraise()

      File "/usr/lib/python3.9/site-packages/oslo_utils/excutils.py", line 200, in force_reraise
    raise self.value

      File "/usr/lib/python3.9/site-packages/tempest/common/compute.py", line 274, in create_test_server
    waiters.wait_for_server_status(

      File "/usr/lib/python3.9/site-packages/tempest/common/waiters.py", line 81, in wait_for_server_status
    raise exceptions.BuildErrorException(details, server_id=server_id)

    tempest.exceptions.BuildErrorException: Server d0cd008a-fe72-4ba1-a517-fafe3edea0cd failed to build and is in ERROR status
Details: Fault: {'code': 400, 'created': '2022-03-14T02:01:35Z', 'message': "Host 'node-0002228241.localdomain' is not mapped to any cell"}. Server boot request ID: req-7901f54e-0db5-4f6e-8dae-08d4af84ff79.

```

Similar bug reported at Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1562082

Changed in tripleo:
status: Fix Released → Triaged
Revision history for this message
chandan kumar (chkumar246) wrote :
Changed in tripleo:
status: Triaged → Fix Released
Revision history for this message
Marios Andreou (marios-b) wrote :

another example there from centos8 https://logserver.rdoproject.org/openstack-periodic-integration-stable1/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-scenario003-standalone-wallaby/2651c6d/logs/undercloud/var/log/tempest/stestr_results.html.gz

tempest.exceptions.BuildErrorException: Server 619e14cf-1510-4f93-ad77-db8c98af7a59 failed to build and is in ERROR status
Details: Fault: {'code': 400, 'created': '2022-03-30T18:50:23Z', 'message': "Host 'standalone.localdomain' is not mapped to any cell"}. Server boot request ID: req-2012fa3a-953c-449a-9dc9-a03934729fcd.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.