python-jujuclient

mojo dependencies cause SSL: CERTIFICATE_VERIFY_FAILED failure websocket-client

Bug #1713871 reported by Ryan Beisner
24
This bug affects 5 people
Affects Status Importance Assigned to Milestone
Mojo: Continuous Delivery for Juju
Triaged
Medium
Unassigned
OpenStack Charm Test Infra
Confirmed
High
Ryan Beisner
python-jujuclient
Fix Committed
Undecided
Unassigned

Bug Description

If one creates a fresh python virtualenv and installs mojo, the version of websocket-client (0.44.0) causes mojo to fail as below.

If one restricts that version to 0.40.0, it succeeds.

Mojo should re-evaluate it's declared python dependencies.

Example:

virtualenv foo
. foo/bin/activate
pip install bzr bzr+lp:codetree#egg=codetree bzr+lp:mojo#egg=mojo websocket-client

...

(foo) jenkins@juju-0d10ff-osci-16:~/tools/z/charm-test-infra$ pip freeze
argcomplete==1.9.2
asn1crypto==0.22.0
Babel==2.5.0
bzr==2.7.0
certifi==2017.7.27.1
cffi==1.10.0
chardet==3.0.4
cryptography==2.0.3
debtcollector==1.17.0
enum34==1.1.6
funcsigs==1.0.2
futures==3.1.1
idna==2.6
ipaddress==1.0.18
iso8601==0.1.12
Jinja2==2.9.6
juju-deployer==0.10.0
jujuclient==0.54.0
keystoneauth1==3.1.0
MarkupSafe==1.0
mojo==0.4.5
monotonic==1.3
netaddr==0.7.19
netifaces==0.10.6
oslo.i18n==3.17.0
oslo.utils==3.28.0
pbr==3.1.1
pkg-resources==0.0.0
positional==1.2.1
prettytable==0.7.2
pycparser==2.18
pylxd==2.0.5
pyparsing==2.2.0
python-cinderclient==3.1.0
python-codetree==0.1.5
python-dateutil==2.6.1
pytz==2017.2
PyYAML==3.12
requests==2.18.4
requests-unixsocket==0.1.5
simplejson==3.11.1
six==1.10.0
stevedore==1.25.0
urllib3==1.22
websocket-client==0.40.0
wrapt==1.10.11
ws4py==0.4.2

...

https://pastebin.canonical.com/197065/

2017-08-29 22:08:20 [ERROR] There was an unrecognised problem with running a deploy phase
2017-08-29 22:08:20 [ERROR] Error found during deployment phase
2017-08-29 22:08:19 [DEBUG] deployer.cli: Using runtime GoEnvironment on auto-osci-sv07:admin/auto-osci-sv07
2017-08-29 22:08:19 [INFO] deployer.cli: Starting deployment of xenial-mitaka
2017-08-29 22:08:19 [DEBUG] deployer.import: Getting charms...
2017-08-29 22:08:19 [DEBUG] deployer.deploy: Resolving configuration
2017-08-29 22:08:20 [DEBUG] deployer.env: Connecting to auto-osci-sv07:admin/auto-osci-sv07...
Traceback (most recent call last):
  File "/var/lib/jenkins/tools/z/charm-test-infra/.tox/clients/bin/juju-deployer", line 11, in <module>
    sys.exit(main())
  File "/var/lib/jenkins/tools/z/charm-test-infra/.tox/clients/local/lib/python2.7/site-packages/deployer/cli.py", line 140, in main
    run()
  File "/var/lib/jenkins/tools/z/charm-test-infra/.tox/clients/local/lib/python2.7/site-packages/deployer/cli.py", line 250, in run
    importer.Importer(env, deployment, options).run()
  File "/var/lib/jenkins/tools/z/charm-test-infra/.tox/clients/local/lib/python2.7/site-packages/deployer/action/importer.py", line 289, in run
    self.env.connect()
  File "/var/lib/jenkins/tools/z/charm-test-infra/.tox/clients/local/lib/python2.7/site-packages/deployer/env/go.py", line 78, in connect
    self.client = self.client_class.connect(self.name)
  File "/var/lib/jenkins/tools/z/charm-test-infra/.tox/clients/local/lib/python2.7/site-packages/jujuclient/environment.py", line 87, in connect
    return connector().run(cls, env_name)
  File "/var/lib/jenkins/tools/z/charm-test-infra/.tox/clients/local/lib/python2.7/site-packages/jujuclient/connector.py", line 56, in run
    cert_path, data.get('environ-uuid'))
  File "/var/lib/jenkins/tools/z/charm-test-infra/.tox/clients/local/lib/python2.7/site-packages/jujuclient/connector.py", line 64, in connect_env
    env = cls(endpoint, name=name, ca_cert=cert_path, env_uuid=env_uuid)
  File "/var/lib/jenkins/tools/z/charm-test-infra/.tox/clients/local/lib/python2.7/site-packages/jujuclient/juju2/environment.py", line 15, in __init__
    super(Environment, self).__init__(*args, **kw)
  File "/var/lib/jenkins/tools/z/charm-test-infra/.tox/clients/local/lib/python2.7/site-packages/jujuclient/environment.py", line 41, in __init__
    endpoint, self._ca_cert)
  File "/var/lib/jenkins/tools/z/charm-test-infra/.tox/clients/local/lib/python2.7/site-packages/jujuclient/connector.py", line 99, in connect_socket
    endpoint, origin=endpoint, sslopt=sslopt)
  File "/var/lib/jenkins/tools/z/charm-test-infra/.tox/clients/local/lib/python2.7/site-packages/websocket/_core.py", line 487, in create_connection
    websock.connect(url, **options)
  File "/var/lib/jenkins/tools/z/charm-test-infra/.tox/clients/local/lib/python2.7/site-packages/websocket/_core.py", line 211, in connect
    options.pop('socket', None))
  File "/var/lib/jenkins/tools/z/charm-test-infra/.tox/clients/local/lib/python2.7/site-packages/websocket/_http.py", line 77, in connect
    sock = _ssl_socket(sock, options.sslopt, hostname)
  File "/var/lib/jenkins/tools/z/charm-test-infra/.tox/clients/local/lib/python2.7/site-packages/websocket/_http.py", line 182, in _ssl_socket
    sock = _wrap_sni_socket(sock, sslopt, hostname, check_hostname)
  File "/var/lib/jenkins/tools/z/charm-test-infra/.tox/clients/local/lib/python2.7/site-packages/websocket/_http.py", line 160, in _wrap_sni_socket
    server_hostname=hostname,
  File "/usr/lib/python2.7/ssl.py", line 353, in wrap_socket
    _context=self)
  File "/usr/lib/python2.7/ssl.py", line 601, in __init__
    self.do_handshake()
  File "/usr/lib/python2.7/ssl.py", line 830, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:590)

jenkins@juju-0d10ff-osci-16:~/tools/z/ubuntu-openstack-ci/run/job-parts$ juju version
2.2.2-xenial-amd64
jenkins@juju-0d10ff-osci-16:~/tools/z/ubuntu-openstack-ci/run/job-parts$ juju controllers
Use --refresh flag with this command to see the latest information.

Controller Model User Access Cloud/Region Models Machines HA Version
auto-osci-sv07* auto-osci-sv07 admin superuser serverstack/serverstack 2 1 none 2.2.2

jenkins@juju-0d10ff-osci-16:~/tools/z/ubuntu-openstack-ci/run/job-parts$ juju models
Controller: auto-osci-sv07

Model Cloud/Region Status Machines Cores Access Last connection
auto-osci-sv07* serverstack/serverstack available 0 - admin 5 seconds ago
controller serverstack/serverstack available 1 4 admin just now
default serverstack/serverstack available 0 - admin 2017-08-25

Related branches

lp://staging/~cjwatson/python-jujuclient/websocket-client-compat
David Britton (community): Approve
Diff: 14 lines (+4/-0)
1 file modified
jujuclient/connector.py (+4/-0)
Ryan Beisner (1chb1n)
Changed in charm-test-infra:
status: New → Confirmed
importance: Undecided → High
assignee: nobody → Ryan Beisner (1chb1n)
Revision history for this message
Ryan Beisner (1chb1n) wrote :

FYI, this dependency issue is coming from jujuclient:

Collecting websocket-client>=0.18.0 (from jujuclient>=0.0.7->mojo)
  Using cached websocket_client-0.44.0-py2.py3-none-any.whl

Revision history for this message
Stuart Bishop (stub) wrote :

Tagged python-jujuclient, as it appears to be incompatible with the latest websocket_client.

Revision history for this message
Felipe Reyes (freyes) wrote :

bundletester pinned websocket-client to <=0.40.0 https://github.com/juju-solutions/bundletester/commit/78b7b5b2707d1f6eaa507b38b33a1eccf715c3d9

Junien Fridrick (axino)
Changed in mojo:
status: New → Triaged
importance: Undecided → High
importance: High → Medium
Revision history for this message
Casey Marshall (cmars) wrote :

Is there a workaround for this? I seem to be unable to get mojo working on bionic, getting this same error.

Revision history for this message
Daniel Manrique (roadmr) wrote :

This is a horrible workaround, please don't use it.

If your environment allows (mine does, mojo deploying to a same-machine Juju controller on a VM inside a closed network, so it's super isolated), you can disable SSL cert checking and move ahead:

sudo vim /usr/lib/python3/dist-packages/jujuclient/connector
.py

look for def connect_socket(... and set sslopt['cert_reqs'] to ssl.CERT_NONE. The method should look like this after the surgery:

    def connect_socket(cls, endpoint, cert_path=None):
        """Return a websocket connection to an endpoint."""

        sslopt = cls.get_ssl_config(cert_path)
        sslopt['cert_reqs'] = ssl.CERT_NONE # I added this line
        return websocket.create_connection(
            endpoint, origin=endpoint, sslopt=sslopt)

Mojo should work after this.

To repeat, this is horrid because it entirely disables cert verification :( so please don't do it.

Adam Collard (adam-collard)
Changed in python-jujuclient:
status: New → Fix Committed
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.