Mirantis OpenStack

novnc - novncproxy_base_url changed to https and port to 443 = 'Invalid Token' - websocketproxy.py adds '/websockify' to token

Bug #1503193 reported by Dmitry Sutyagin
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Mirantis OpenStack
Won't Fix
Medium
MOS Nova
Mirantis OpenStack 8.0
6.0.x
Won't Fix
High
MOS Maintenance
Mirantis OpenStack 6.0-updates
6.1.x
Won't Fix
Medium
MOS Maintenance
Mirantis OpenStack 6.1-updates
7.0.x
Won't Fix
Medium
MOS Maintenance
Mirantis OpenStack 7.0-updates
8.0.x
Won't Fix
Medium
MOS Nova
Mirantis OpenStack 8.0
9.x
Invalid
Medium
Timur Nurlygayanov
Mirantis OpenStack 9.0

Bug Description

Fuel 6.0

Customer tried to enable SSL termination on haproxy for all services, haproxy is configured to listen on 443 and redirect all 80 to 443, ACLs are used to redirect traffic to proper backends. Every backend except novncproxy works properly.

novncproxy_base_url is modified on computes to use https:
from "http://.../vnc_auto.html"
to "https://.../vnc_auto.html?port=443" (https - needed for initial connection, ?port=443 needed to tell javascript to use port 443 for wss, otherwise it will connect to port 80)

Result:

/var/log/nova/consoleauth.log:
2015-10-05T21:23:42.572675+00:00 warning: Checking Token: 8f9a0e2f-0b0c-4e77-b816-bb618de5e2e8/websockify, False

/var/log/nova/nova-novncproxy.log:
2015-10-05T21:23:42.577323+00:00 info: handler exception: Invalid Token
2015-10-05T21:23:42.579856+00:00 debug: exception
2015-10-05 23:23:42.578 108841 TRACE nova.console.websocketproxy Traceback (most recent call last):
2015-10-05 23:23:42.578 108841 TRACE nova.console.websocketproxy File "/usr/lib/python2.7/dist-packages/websockify/websocket.py", line 874, in top_new_client
2015-10-05 23:23:42.578 108841 TRACE nova.console.websocketproxy client = self.do_handshake(startsock, address)
2015-10-05 23:23:42.578 108841 TRACE nova.console.websocketproxy File "/usr/lib/python2.7/dist-packages/websockify/websocket.py", line 809, in do_handshake
2015-10-05 23:23:42.578 108841 TRACE nova.console.websocketproxy self.RequestHandlerClass(retsock, address, self)
2015-10-05 23:23:42.578 108841 TRACE nova.console.websocketproxy File "/usr/lib/python2.7/dist-packages/nova/console/websocketproxy.py", line 150, in __init__
2015-10-05 23:23:42.578 108841 TRACE nova.console.websocketproxy websockify.ProxyRequestHandler.__init__(self, *args, **kwargs)
2015-10-05 23:23:42.578 108841 TRACE nova.console.websocketproxy File "/usr/lib/python2.7/dist-packages/websockify/websocket.py", line 112, in __init__
2015-10-05 23:23:42.578 108841 TRACE nova.console.websocketproxy SimpleHTTPRequestHandler.__init__(self, req, addr, server)
2015-10-05 23:23:42.578 108841 TRACE nova.console.websocketproxy File "/usr/lib/python2.7/SocketServer.py", line 638, in __init__
2015-10-05 23:23:42.578 108841 TRACE nova.console.websocketproxy self.handle()
2015-10-05 23:23:42.578 108841 TRACE nova.console.websocketproxy File "/usr/lib/python2.7/dist-packages/websockify/websocket.py", line 540, in handle
2015-10-05 23:23:42.578 108841 TRACE nova.console.websocketproxy SimpleHTTPRequestHandler.handle(self)
2015-10-05 23:23:42.578 108841 TRACE nova.console.websocketproxy File "/usr/lib/python2.7/BaseHTTPServer.py", line 340, in handle
2015-10-05 23:23:42.578 108841 TRACE nova.console.websocketproxy self.handle_one_request()
2015-10-05 23:23:42.578 108841 TRACE nova.console.websocketproxy File "/usr/lib/python2.7/BaseHTTPServer.py", line 328, in handle_one_request
2015-10-05 23:23:42.578 108841 TRACE nova.console.websocketproxy method()
2015-10-05 23:23:42.578 108841 TRACE nova.console.websocketproxy File "/usr/lib/python2.7/dist-packages/websockify/websocket.py", line 506, in do_GET
2015-10-05 23:23:42.578 108841 TRACE nova.console.websocketproxy if not self.handle_websocket():
2015-10-05 23:23:42.578 108841 TRACE nova.console.websocketproxy File "/usr/lib/python2.7/dist-packages/websockify/websocket.py", line 494, in handle_websocket
2015-10-05 23:23:42.578 108841 TRACE nova.console.websocketproxy self.new_websocket_client()
2015-10-05 23:23:42.578 108841 TRACE nova.console.websocketproxy File "/usr/lib/python2.7/dist-packages/nova/console/websocketproxy.py", line 91, in new_websocket_client
2015-10-05 23:23:42.578 108841 TRACE nova.console.websocketproxy raise Exception(_("Invalid Token"))
2015-10-05 23:23:42.578 108841 TRACE nova.console.websocketproxy Exception: Invalid Token
2015-10-05 23:23:42.578 108841 TRACE nova.console.websocketproxy

The issue happens In /usr/lib/python2.6/site-packages/nova/console/websocketproxy.py:

token = urlparse.parse_qs(query).get("token", [""]).pop()

Will return <TOKEN>/websockify, so token will be concatenated with /websockify text, and cannot be validated.

The workaround I used to fix was to add an extra line:
token = token.split('/')[0]

After this change noVNC starts working well over https.

Also found a related bug - https://bugs.launchpad.net/mos/+bug/1460070

See original description
Revision history for this message
Dmitry Sutyagin (dsutyagin) wrote :

Additionally it is necessary to uncomment 'novncproxy_base_url' on controllers and configure it to be the same as on computes (it should start with "https://") and restart nova-novncproxy services. If this is not done, the following error will be observed in logs:

ValidationError: Origin header protocol does not match this host.

This workaround was found here - https://bugzilla.redhat.com/show_bug.cgi?id=1204482#c9

Dmitry Sutyagin (dsutyagin)
description: updated
description: updated
description: updated
Revision history for this message
Roman Podoliaka (rpodolyaka) wrote :

Dmitry, this looks like a problem with configuration. And we officially support SSL only starting from 7.0.

Please go ahead and escalate this, if needed for 6.0. Otherwise, we'll take a look at this, when we have time.

Changed in mos:
milestone: none → 6.0-updates
assignee: nobody → MOS Nova (mos-nova)
importance: Undecided → Medium
status: New → Confirmed
Revision history for this message
Roman Rufanov (rrufanov) wrote :

Customer found on 6.0 - need solution. Thanks !

Revision history for this message
Denis Meltsaykin (dmeltsaykin) wrote :

Setting this as Won't fix for releases prior to 7.0, as they officially don't support SSL

Revision history for this message
Roman Podoliaka (rpodolyaka) wrote :

No longer fixing Medium bugs for 8.0

tags: added: area-nova
Timur Nurlygayanov (tnurlygayanov)
Changed in mos:
status: Confirmed → Won't Fix
Vitaly Sedelnik (vsedelnik)
tags: added: wontfix-feature
Revision history for this message
Timur Nurlygayanov (tnurlygayanov) wrote :

Hi Dev team, it is customer found issue and we have simple workaround / fix for the issue.
Let's include the fix to MOS 9.0 release.

Thank you!

tags: added: keep-in-9.0
Revision history for this message
Roman Podoliaka (rpodolyaka) wrote :

Timur, could you please elaborate here: what workaround are you talking about? This bug was filed for MOS 6.0 which *did not* support SSL officially.

novnc + SSL should work out-of-box in MOS 9.0.

Revision history for this message
Timur Nurlygayanov (tnurlygayanov) wrote :

Ok, I mean that we can copy and paste workaround from 6.x to 9.0 release :)
If it works "from the box" we need to just verify it on 9.0 and close as Invalid for 9.0

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.