Juju-ci3 cannot upgrade to 1.21.1

juju-ci3 is in upgrade hell to 1.21.1 state server upgraded in a few minutes, after an hour it fell over. I restarted the agents and can see it, but all the other machines are down. They are trying to contact 10.0.3.1...which is not where the state server is

35 of 36 agents are wrong after buggered upgrade

2015-02-02 21:21:32 INFO juju.worker runner.go:260 start "api"
2015-02-02 21:21:32 INFO juju.state.api apiclient.go:242 dialing "wss://10.0.3.1:17070/"
2015-02-02 21:21:32 INFO juju.state.api apiclient.go:250 error dialing "wss://10.0.3.1:17070/": websocket.Dial wss://10.0.3.1:17070/: dial tcp 10.0.3.1:17070: connection refused
2015-02-02 21:21:32 ERROR juju.worker runner.go:218 exited "api": unable to connect to "wss://10.0.3.1:17070/"
2015-02-02 21:21:32 INFO juju.worker runner.go:252 restarting "api" in 3s

The address should be 172.31.9.3 or ip-172-31-9-3.ec2.internal

Setting the apiaddress to ip-172-31-9-3.ec2.internal:17070 in both machine and service agent.conf files allowed the agents to call home. They saw they needed
an upgrade...

2015-02-02 21:51:04 INFO juju.worker.upgrader upgrader.go:134 upgrade requested from 1.20.13-precise-amd64 to 1.21.1
2015-02-02 21:51:04 INFO juju.worker.upgrader upgrader.go:167 fetching tools from "https://10.0.3.1:17070/environment/e7513400-44ff-40a0-8e6a-8776784d94fa/tools/1.21.1-precise-amd64"
2015-02-02 21:51:04 INFO juju.utils http.go:66 hostname SSL verification disabled
2015-02-02 21:51:04 ERROR juju.worker.upgrader upgrader.go:157 failed to fetch tools from "https://10.0.3.1:17070/environment/e7513400-44ff-40a0-8e6a-8776784d94fa/tools/1.21.1-precise-amd64": Get https://10.0.3.1:17070/environment/e7513400-44ff-40a0-8e6a-8776784d94fa/tools/1.21.1-precise-amd64: dial tcp 10.0.3.1:17070: connection refused

Which is again the wrong address. Reseting the apiaddress to the private ip
or the private name fails because the value is rewritten to an impossible
value.

Attached is a log from machine-8 (juju-reports, the app behind reports.vapour.ws) We can see the api server was connacted after I changed the api address, but the value is immediately changed back to 10.0.3.1.