tripleo

o->p->q undercloud upgrade fails with "Cannot use v2 authentication with domain scope"

  1. Series queens
  2. Bug #1798553
Bug #1798553 reported by Sofer Athlan-Guyot
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
tripleo
Won't Fix
Medium
Sofer Athlan-Guyot
tripleo stein-1
Queens
Fix Committed
Medium
Sofer Athlan-Guyot

Bug Description

Hi,

deploy ocata, upgrade undercloud to pike then to queens and then:

Hi,

After running the openstack undercloud upgrade from pike to queens. We've encountered errors. Kindly see below logs for the errors.

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/lib/python2.7/site-packages/instack_undercloud/undercloud.py", line 2411, in pre_upgrade
    user_domain_name='Default')
  File "/usr/lib/python2.7/site-packages/os_client_config/__init__.py", line 86, in make_client
    return cloud.get_legacy_client(service_key, constructor)
  File "/usr/lib/python2.7/site-packages/os_client_config/cloud_config.py", line 370, in get_legacy_client
    service_key, min_version=min_version, max_version=max_version)
  File "/usr/lib/python2.7/site-packages/os_client_config/cloud_config.py", line 309, in get_session_endpoint
    endpoint = self._get_highest_endpoint(service_types, kwargs)
  File "/usr/lib/python2.7/site-packages/os_client_config/cloud_config.py", line 267, in _get_highest_endpoint
    return session.get_endpoint(**kwargs)
  File "/usr/lib/python2.7/site-packages/keystoneauth1/session.py", line 942, in get_endpoint
    return auth.get_endpoint(self, **kwargs)
  File "/usr/lib/python2.7/site-packages/keystoneauth1/identity/base.py", line 379, in get_endpoint
    allow_version_hack=allow_version_hack, **kwargs)
  File "/usr/lib/python2.7/site-packages/keystoneauth1/identity/base.py", line 270, in get_endpoint_data
    service_catalog = self.get_access(session).service_catalog
  File "/usr/lib/python2.7/site-packages/keystoneauth1/identity/base.py", line 134, in get_access
    self.auth_ref = self.get_auth_ref(session)
  File "/usr/lib/python2.7/site-packages/keystoneauth1/identity/generic/base.py", line 199, in get_auth_ref
    self._plugin = self._do_create_plugin(session)
  File "/usr/lib/python2.7/site-packages/keystoneauth1/identity/generic/base.py", line 188, in _do_create_plugin
    'Cannot use v2 authentication with domain scope')
keystoneauth1.exceptions.discovery.DiscoveryFailure: Cannot use v2 authentication with domain scope
Command 'instack-pre-upgrade-undercloud' returned non-zero exit status 1
clean_up UpgradeUndercloud: Command 'instack-pre-upgrade-undercloud' returned non-zero exit status 1
END return value: 1

Originally found by pvc on #tripleo.

Revision history for this message
Sofer Athlan-Guyot (sofer-athlan-guyot) wrote :

So, for the last error, it's as if we're using /v2 path[1] while we're using domain[2]

[1] https://github.com/openstack/keystoneauth/blob/master/keystoneauth1/identity/generic/base.py#L150-L153

[2] https://github.com/openstack/instack-undercloud/blob/stable/queens/instack_undercloud/undercloud.py#L2410..L2411

After updating the OS_AUTH_URL in stackrc to a v3 address, the user could proceed with the undercloud upgrade.

Changed in tripleo:
assignee: nobody → Sofer Athlan-Guyot (sofer-athlan-guyot)
Revision history for this message
p.cipriano (ctpeter) wrote :

I updated my stackrc to use auth_url v3 authentication and it is now working.

Revision history for this message
Sofer Athlan-Guyot (sofer-athlan-guyot) wrote :

This is the likely change[1] that broke this upgrade path.

[1] https://github.com/openstack/instack-undercloud/commit/96f09de818f9ce099a1db0cb8ccd95f570edc016

Revision history for this message
Bogdan Dobrelya (bogdando) wrote :

Note this is won't fix for the master (Stein), nor Rocky (as it has containerized undercloud instead of instack) and should only be fixed as a backports.

Changed in tripleo:
status: New → Triaged
status: Triaged → Won't Fix
Revision history for this message
Sofer Athlan-Guyot (sofer-athlan-guyot) wrote :

Hi Bogdan,

I don't understand the new status as it still has to be fixed in pike, so "won't fix" sounds strange to me.

Furthermore even if we don't use it the faulty code is still there in master, so even if we don't use it anymore, it's still there and faulty. I don't really get this neither, but that's a lesser concern to me.

Revision history for this message
Bogdan Dobrelya (bogdando) wrote :

It should be fixed only for Pike, thanks!

Revision history for this message
Sofer Athlan-Guyot (sofer-athlan-guyot) wrote :

Actually we need queens as well as in pike we don't use stackrc during pike undercloud upgrade so it can be unoticed and then happen only in queens where we do use authentication to check the overcloud state.

Bogdan Dobrelya (bogdando)
no longer affects: tripleo/pike
tags: added: pike-backport-potential
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to instack-undercloud (stable/queens)

Fix proposed to branch: stable/queens
Review: https://review.openstack.org/611647

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to puppet-tripleo (master)

Related fix proposed to branch: master
Review: https://review.openstack.org/611919

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on instack-undercloud (stable/queens)

Change abandoned by Athlan-Guyot sofer (<email address hidden>) on branch: stable/queens
Review: https://review.openstack.org/611647
Reason: legit error

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to instack-undercloud (stable/queens)

Reviewed: https://review.openstack.org/611647
Committed: https://git.openstack.org/cgit/openstack/instack-undercloud/commit/?id=a1038f30366f95aef7cf0b1df8f695a638fc499e
Submitter: Zuul
Branch: stable/queens

commit a1038f30366f95aef7cf0b1df8f695a638fc499e
Author: Sofer Athlan-Guyot <email address hidden>
Date: Thu Oct 18 12:21:25 2018 +0200

    Update stackrc to v3 auth values when --use-heat is false.

    During pike we have updated the deployment to use v3 authentication in
    stackrc. But we didn't upgrade the existing stackrc.

    We didn't use it in the code so it was not a problem.

    But in queens, when we use '--use-heat' is false, we check the status
    of the overcloud before starting and we load auth url from stackrc and
    force domain in the api request. So we have a v2 url and a v3
    request. That fails.

    Now, we make sure that the stackrc is in v3 version before starting
    the undercloud upgrade in queens when we use --use-heat set to false.

    Change-Id: I3f008b9466408fe797ca50ebf96651a72fc9a544
    Closes-Bug: #1798553

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/instack-undercloud 8.4.7

This issue was fixed in the openstack/instack-undercloud 8.4.7 release.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.