This sort of issue is not unheard of in cases where an IP address moves from interface to interface, or between hosts. Most situations that expect this type of issue (e.g., bonding link failover) already issue gratuitous ARPs in order to update L2 peers.
I think the bottom line here is that dhclient (the DHCP client typically used on Ubuntu, and presumably the one in use here) does not implement RFC 5227, "IPv4 Address Conflict Detection," which describes how gratutious ARPs must be done if the host provides that functionality (5227 2.3, "Announcing an Address").
Most network configuration tools (and the kernel itself) on linux do not issue gratuitous ARPs by default at address assignment time, so this lack isn't especially unusual. E.g., there is no option in /etc/network/interfaces to instruct ifup to issue a GARP.
I'll note that 5227 is a proposed standard, and, as such, hosts are not required to implement it, so dhclient is not violating any standards by not issuing gratuitous ARPs.
Now, none of the above actually resolves the problem here, it just explains that you've landed in a corner case that doesn't come up very often.
As far as resolving this, one obvious possibility is to add RFC 5227 functionality to dhclient through its dhclient-script facility (and in fact the man page for that is close to suggesting that: for the BOUND case, the script should "somehow" perform duplicate address detection via ARP).
I'm not too familiar with cloud-init's internals, but for 5227 compliance, the GARP would be issued on every boot, and cloud-init only runs on first boot, so an implementation within cloud-init would likely be setting up some persistent configuration.
Sam / Christian,
This sort of issue is not unheard of in cases where an IP address moves from interface to interface, or between hosts. Most situations that expect this type of issue (e.g., bonding link failover) already issue gratuitous ARPs in order to update L2 peers.
I think the bottom line here is that dhclient (the DHCP client typically used on Ubuntu, and presumably the one in use here) does not implement RFC 5227, "IPv4 Address Conflict Detection," which describes how gratutious ARPs must be done if the host provides that functionality (5227 2.3, "Announcing an Address").
Most network configuration tools (and the kernel itself) on linux do not issue gratuitous ARPs by default at address assignment time, so this lack isn't especially unusual. E.g., there is no option in /etc/network/ interfaces to instruct ifup to issue a GARP.
I'll note that 5227 is a proposed standard, and, as such, hosts are not required to implement it, so dhclient is not violating any standards by not issuing gratuitous ARPs.
Now, none of the above actually resolves the problem here, it just explains that you've landed in a corner case that doesn't come up very often.
As far as resolving this, one obvious possibility is to add RFC 5227 functionality to dhclient through its dhclient-script facility (and in fact the man page for that is close to suggesting that: for the BOUND case, the script should "somehow" perform duplicate address detection via ARP).
I'm not too familiar with cloud-init's internals, but for 5227 compliance, the GARP would be issued on every boot, and cloud-init only runs on first boot, so an implementation within cloud-init would likely be setting up some persistent configuration.