bootstrap fails with TLS handshake timeout

Bug #1899793 reported by Heather Lanigan
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Canonical Juju
Fix Released
High
Simon Richardson

Bug Description

https://jenkins.juju.canonical.com/job/nw-deploy-client-macos/2381/console

2020-10-14 01:13:28 INFO juju --show-log bootstrap google/europe-west1 testing-osx-client1 --config /var/folders/_1/j7_4g0n94m7_pllg_373gtc80000gp/T/tmpd33Ura.yaml --constraints --default-model testing-osx-client1 --agent-version 2.8.6 --bootstrap-series xenial
01:13:29 INFO juju.cmd supercommand.go:54 running juju [2.8.6 257 bfc8c5fd6cbb91cd94ab37ae8c0e46e8f6bb325f gc go1.14.9]
01:13:29 INFO cmd authkeys.go:114 Adding contents of "/Users/jenkins/cloud-city/juju-homes/testing-osx-client1/ssh/juju_id_rsa.pub" to authorized-keys
01:13:29 INFO cmd authkeys.go:114 Adding contents of "/Users/jenkins/.ssh/id_rsa.pub" to authorized-keys
ERROR Get "https://www.googleapis.com/compute/v1/projects/gothic-list-89514?alt=json&prettyPrint=false": oauth2: cannot fetch token: Post "https://oauth2.googleapis.com/token": net/http: TLS handshake timeout
2020-10-14 01:13:39 ERROR Command '('juju', '--show-log', 'bootstrap', 'google/europe-west1', 'testing-osx-client1', '--config', '/var/folders/_1/j7_4g0n94m7_pllg_373gtc80000gp/T/tmpd33Ura.yaml', '--constraints', '', '--default-model', 'testing-osx-client1', '--agent-version', u'2.8.6', '--bootstrap-series', 'xenial')' returned non-zero exit status 1
Traceback (most recent call last):
  File "/Users/jenkins/workspace/nw-deploy-client-macos/acceptancetests/utility.py", line 426, in logged_exception
    yield
  File "/Users/jenkins/workspace/nw-deploy-client-macos/acceptancetests/deploy_stack.py", line 1012, in handle_bootstrap_exceptions
    yield
  File "/Users/jenkins/workspace/nw-deploy-client-macos/acceptancetests/deploy_stack.py", line 968, in bootstrap_context
    yield
  File "/Users/jenkins/workspace/nw-deploy-client-macos/acceptancetests/deploy_stack.py", line 1166, in new_bootstrap
    upload_tools, self.series, kwargs,
  File "/Users/jenkins/workspace/nw-deploy-client-macos/acceptancetests/deploy_stack.py", line 595, in create_initial_model
    **boot_kwargs)
  File "/Users/jenkins/workspace/nw-deploy-client-macos/acceptancetests/jujupy/client.py", line 1046, in bootstrap
    retvar, ct = self.juju('bootstrap', args, include_e=False)
  File "/Users/jenkins/workspace/nw-deploy-client-macos/acceptancetests/jujupy/client.py", line 1336, in juju
    model, check, timeout, extra_env, suppress_err=suppress_err)
  File "/Users/jenkins/workspace/nw-deploy-client-macos/acceptancetests/jujupy/backend.py", line 226, in juju
    rval = call_func(args, stderr=stderr)
  File "/usr/local/Cellar/python@2/2.7.15_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 190, in check_call
    raise CalledProcessError(retcode, cmd)
CalledProcessError: Command '('juju', '--show-log', 'bootstrap', 'google/europe-west1', 'testing-osx-client1', '--config', '/var/folders/_1/j7_4g0n94m7_pllg_373gtc80000gp/T/tmpd33Ura.yaml', '--constraints', '', '--default-model', 'testing-osx-client1', '--agent-version', u'2.8.6', '--bootstrap-series', 'xenial')' returned non-zero exit status 1
2020-10-14 01:13:39 INFO /usr/local/opt/python@2/bin/python2.7 /Users/jenkins/workspace/nw-deploy-client-macos/acceptancetests/jujupy/timeout.py 1200.00 -- juju --show-log kill-controller testing-osx-client1 -y
01:13:40 INFO juju.cmd supercommand.go:54 running juju [2.8.6 257 bfc8c5fd6cbb91cd94ab37ae8c0e46e8f6bb325f gc go1.14.9]
ERROR controller testing-osx-client1 not found

Revision history for this message
Pen Gale (pengale) wrote :

Adding to 2.8.7 milestone. This is part of the work to address green in the CI.

Changed in juju:
milestone: 2.8.6 → 2.8.7
status: New → Triaged
importance: Undecided → High
Changed in juju:
status: Triaged → In Progress
assignee: nobody → Simon Richardson (simonrichardson)
Revision history for this message
John A Meinel (jameinel) wrote :

Is this something that we aren't retrying if there is a failure?

Changed in juju:
milestone: 2.8.7 → 2.8.8
Changed in juju:
milestone: 2.8.8 → 2.8.9
Changed in juju:
milestone: 2.8.9 → 2.8.10
Changed in juju:
milestone: 2.8.10 → 2.8.11
Revision history for this message
John A Meinel (jameinel) wrote :

Also, it seems like an oversight that:
01:13:29 INFO cmd authkeys.go:114 Adding contents of "/Users/jenkins/.ssh/id_rsa.pub" to authorized-keys
ERROR Get...

The ERROR line doesn't have a timestamp. So you can't tell if there is 10s between the INFO line and the ERROR, or 20min between the two.

Changed in juju:
milestone: 2.8.11 → 2.9.4
Changed in juju:
milestone: 2.9.4 → 2.9.5
Revision history for this message
Simon Richardson (simonrichardson) wrote :
Changed in juju:
milestone: 2.9.5 → 2.9.6
Changed in juju:
status: In Progress → Fix Committed
milestone: 2.9.6 → 2.9.5
Changed in juju:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.