OpenStack Heat

Autoscaling updates loadbalancer with bad IP address

Bug #1192128 reported by Steven Hardy on 2013-06-18

This bug affects 2 people

Affects		Status	Importance	Assigned to	Milestone
	OpenStack Heat	Fix Released	High	Steven Hardy	OpenStack Heat 2013.2 "havana"

Bug Description

When you launch AutoScalingMultiAZSample.template, more often than not (actually every time in my tests), we update the haproxy config with "0.0.0.0" instead of the instance IP address.

Appears to be a race between us and nova, where we assume ACTIVE state means the instance is ready to be added to the LB, but then LoadBalancer::_instance_to_ipaddress gives us "0.0.0.0", and we update the config anyway.

I'm sure we fixed a similar bug a while back.

server server1 0.0.0.0:80 check inter 30s fall 5 rise 3

Steven Hardy (shardy) on 2013-06-18

Changed in heat:
status:	New → Triaged
importance:	Undecided → High
milestone:	none → havana-2

Steven Hardy (shardy) on 2013-06-18

Changed in heat:
assignee:	nobody → Steven Hardy (shardy)
assignee:	Steven Hardy (shardy) → nobody

Vijendar Komalla (vijendar-komalla) on 2013-06-20

Changed in heat:
assignee:	nobody → Vijendar Komalla (vijendar-komalla)

Randall Burt (randall-burt) on 2013-06-26

Changed in heat:
status:	Triaged → In Progress

Steven Hardy (shardy) on 2013-07-02

Changed in heat:
assignee:	Vijendar Komalla (vijendar-komalla) → Steven Hardy (shardy)

Revision history for this message

Steven Hardy (shardy) wrote on 2013-07-03:

Looks like this is a regression caused by the following commit:

commit 4bc1bcb608379cc396113a03f2ae37ce78f9049c
Author: Andrew Plunk <email address hidden>
Date: Wed Jun 5 14:54:41 2013 -0500

Make autoscale not dependent on loadbalancer impl.

Autoscale was calling reload on a resource, which is not part of the
resource interface. Autoscale has been modified to use update.

Change-Id: Ifb17c9def7e560d96e26308db67b4ee3f93a2b9f

diff --git a/heat/engine/resources/autoscaling.py b/heat/engine/resources/autoscaling.py
index 7020aef..ec596e5 100644
--- a/heat/engine/resources/autoscaling.py
+++ b/heat/engine/resources/autoscaling.py
@@ -211,7 +211,9 @@ class InstanceGroup(resource.Resource):
id_list.append(inst.FnGetRefId())

for lb in self.properties['LoadBalancerNames']:
- self.stack[lb].reload(id_list)
+ self.stack[lb].json_snippet['Properties']['Instances'] = \
+ inst_list
+ self.stack[lb].update(self.stack[lb].json_snippet)

The loadbalancer Instances list should be id_list, not inst_list

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2013-07-03: Fix proposed to heat (master)

Fix proposed to branch: master
Review: https://review.openstack.org/35485

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2013-07-04: Fix merged to heat (master)

Reviewed: https://review.openstack.org/35485
Committed: http://github.com/openstack/heat/commit/9d13c2cbd31e5416bb6b9aa86aeefe862eef9e2f
Submitter: Jenkins
Branch: master

commit 9d13c2cbd31e5416bb6b9aa86aeefe862eef9e2f
Author: Steven Hardy <email address hidden>
Date: Wed Jul 3 14:40:32 2013 +0100

engine : autoscaling pass instance id's not names to loadbalancer

Regression caused by 4bc1bcb608379cc396113a03f2ae37ce78f9049c

    We need to pass the id_list to the loadbalancer, otherwise it
    can't lookup the instance IP adresses with nova and the loadbalancer
    config ends up with 0.0.0.0 for every instance

Fixes bug #1192128

Change-Id: I73bab70e897c0fb0b317793fa31ff8d07720a6c4