[aws] adding a machine post-bootstrap on the controller model closes of api port in controller security group

Bug #1598164 reported by Dimiter Naydenov
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Canonical Juju
Fix Released
Critical
Dimiter Naydenov

Bug Description

Bootstrap juju 2.0 on aws, switch to the controller model, add a machine, run juju status.

Expected: success
Actual: controller instance's api port 17070 is no longer reachable, so status hangs forever.

Adding machines to other hosted models is fine.
It seems this is pre-existing for a while.

Revision history for this message
Dimiter Naydenov (dimitern) wrote :

Using the AWS web console, once the issue is reproduced you can observe the ingress rule on the controller security group for tcp port 17070 switches to port "0", effectively closing the port.

tags: added: add-machine addressability ec2-provider tech-debt
Changed in juju-core:
status: New → Triaged
importance: Undecided → Critical
milestone: none → 2.0-beta12
Changed in juju-core:
assignee: nobody → Dimiter Naydenov (dimitern)
Changed in juju-core:
status: Triaged → In Progress
Revision history for this message
Dimiter Naydenov (dimitern) wrote :

I couldn't make a lot of progress today, but at least managed to confirm the regression occurred after beta9 and before beta10. Now trying to find the commit which caused it.

Revision history for this message
Dimiter Naydenov (dimitern) wrote :

I have a strong suspicion this is the commit that introduced the EC2 issue:
https://github.com/juju/juju/pull/5683/files#diff-06274097d1cb1729d2d50ed7a575a8e3L452

It was present in both OpenStack and EC2 providers since PR https://github.com/juju/juju/pull/5683/ landed, and apparently restored with a follow-up PR https://github.com/juju/juju/pull/5700/, but only in the OpenStack provider, EC2 still needs fixing.

Will double check and confirm tomorrow.

Revision history for this message
Dimiter Naydenov (dimitern) wrote :

The commit introduced the regression, in both the EC2 and OpenStack providers is this one:
https://github.com/juju/juju/pull/5683/commits/f417397980d07fe565582d51c00176c9a1bffaf1

OpenStack provider regression was fixed with https://github.com/juju/juju/pull/5700 (see bug 1595278).

Currently testing a similar fix for EC2.

Revision history for this message
Dimiter Naydenov (dimitern) wrote :
tags: added: blocker
Changed in juju-core:
status: In Progress → Fix Committed
tags: removed: blocker
Curtis Hovey (sinzui)
Changed in juju-core:
status: Fix Committed → Fix Released
affects: juju-core → juju
Changed in juju:
milestone: 2.0-beta12 → none
milestone: none → 2.0-beta12
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.