OpenStack Identity (keystone)

Websso fails when HTTP_REFERRER that horizon is unable to connect to gets used

Bug #1874705 reported by Georgina Shippey
14
This bug affects 3 people
Affects Status Importance Assigned to Milestone
OpenStack Dashboard (Horizon)
Fix Released
Undecided
Georgina Shippey
OpenStack Identity (keystone)
New
Undecided
Unassigned

Bug Description

I am currently having an issue where a request to Horizon's websso endpoint fails to respond in time as the token validation request fails to connect between Horizon and Keystone.

(On Openstack Train)
I am trying to login to Horizon using an external identity provider.
I have set the WEBSSO_KEYSTONE_URL to keystones external facing endpoint as the IDP is on an external network.

The POST request to https://horizon_ip/auth/websso/ that includes a keystone token for validation in its params is failing.
This request routes to the horizon view 'websso' (https://opendev.org/openstack/horizon/src/branch/master/openstack_auth/views.py#L165)
The token authentication request to keystone in this view uses the requests HTTP_REFERRER when available as the keystone endpoint to use.
The previous request was to keystone on its external endpoint (as used by the external identity provider) to its route 'auth/OS-FEDERATION/websso/openid', and therefore the HTTP_REFERRER for this POST request is the external keystone endpoint.

Our Openstack services have minimal external connectivity for security reasons.
So in our setup the horizon service is unable to make connections to the external keystone endpoint.
Therefore in the horizon apache logs I see:
  Unable to establish connection to https://keystone_external_ip:5000/v3/auth/tokens
Which eventually leads to a time out.

As this is request between Horizon and Keystone ideally for us it should be using the internal endpoint. I've had a go at setting the auth_url to be settings.OPENSTACK_KEYSTONE_URL and this lets me login successfully.

I am unsure as to why the HTTP_REFERRER gets used in preference over the settings.OPENSTACK_KEYSTONE_URL for this request?

I propose either:
1. Removing the use of HTTP_REFERRER in favor of settings.OPENSTACK_KEYSTONE_URL.
2. Providing a setting to toggle between using the HTTP_REFERRER or settings.OPENSTACK_KEYSTONE_URL to build the auth request with.

Original commit in django_openstack_auth for websso view: https://github.com/openstack/django_openstack_auth/commit/302f422568a32b513ffbb3089ba799a4416df108

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to horizon (master)

Fix proposed to branch: master
Review: https://review.opendev.org/722685

Changed in horizon:
assignee: nobody → Georgina Shippey (gshippey)
status: New → In Progress
Revision history for this message
Jeff Albert (jralbert) wrote :

This affects our installation as well, and will affect anyone running an OpenStack environment in which the control plane doesn't have outbound network connectivity to hit public endpoints, and instead relies on internal endpoints on a shared network. This is a somewhat common security control to implement, and the change the original bug reporter has proposed seems both effective and narrowly-scoped so that it should be a fairly simple thing to review. Can the proposed fix be merged?

Revision history for this message
Jonathan Rosser (jrosser) wrote (last edit ):

With a control plane that has no outbound connectivity it's very important that the correct endpoints can be used/specified.

The horizon service itself must talk to keystone on the internal endpoint, whilst the communication between the end user browser, the IdP, horizon, and the public openstack APIs only happen via the external endpoint.

The use of HTTP_REFERRER as described in the bug report mixes the internal and external physical network contexts in a way which leads to websso being broken for some deployments, but it will work for others with less strict (or no... devstack?) isolation.

We are currently having to maintain a fork of horizon with https://review.opendev.org/722685 applied in order to get websso to work with a strictly segregated network. Comment #2 shows this is not a unique situation.

Vishal Manchanda (vishalmanchanda)
affects: horizon → keystone
affects: keystone → horizon
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to horizon (master)

Reviewed: https://review.opendev.org/c/openstack/horizon/+/722685
Committed: https://opendev.org/openstack/horizon/commit/33292ca0a467637971c73f420166b4077e941e20
Submitter: "Zuul (22348)"
Branch: master

commit 33292ca0a467637971c73f420166b4077e941e20
Author: Georgina Shippey <email address hidden>
Date: Fri Apr 24 13:52:42 2020 +0100

    Use OPENSTACK_KEYSTONE_URL instead of HTTP_REFERRER

    By using OPENSTACK_KEYSTONE_URL instead of the HTTP_REFERRER
    the authentication request between Horizon and Keystone continues
    to work in situations where the HTTP_REFERRER is an external keystone
    endpoint that Horizon does not have access to.

    Change-Id: I9c5c8d59c5f5a8570dbb563ae224d45406a73ba5
    Closes-bug: #1874705

Changed in horizon:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/horizon 21.0.0

This issue was fixed in the openstack/horizon 21.0.0 release.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.