compute port lose fixed_ips on restart l3-agent if subnet is prefix delegated

I've created two networks: net04_ext and slaac-auto. First is dualstack public external network and second is tenant network with static private IPv4 subnet and prefix delegated IPv6 subnet.

I have ISC DHCPv6 prefix delegation server, that serves GUA's:
prefix6 2001:0470:739e:fe00:: 2001:0470:739e:feff:: /64;

| id | name | subnets |
| 848cb239-d423-40fb-89fb-3f22212a83f6 | net04_ext | 62a101e5-26fe-451c-a3a1-cf7ebdefc308 2001:470:739e::/64 |
| | | 1996fd93-d890-4500-a552-1602f5e1114e 172.16.0.0/24 |
| 147aa018-315d-4344-a26e-ca1bd3f22afe | slaac-auto | ebd9bb05-c180-4466-bb92-f6ab6efa58a8 2001:470:739e:fefe::/64 |
| | | 6031c529-12d8-4ba5-be4e-90b52330c4e0 192.168.114.0/24 |

I have a router named slaac-router, that connects slaac-auto to net04_ext. Prefix 2001:470:739e:fefe::/64 was automatically delegated by prefix delegation.

| id | name | external_gateway_info | distributed | ha |
| 5c758a05-5771-4d8b-a60c-62549ca9fc34 | slaac-router | {"network_id": "848cb239-d423-40fb-89fb-3f22212a83f6", "enable_snat": true, "external_fixed_ips": [{"subnet_id": "1996fd93-d890-4500-a552-1602f5e1114e", "ip_address": "172.16.0.143"}, {"subnet_id": "62a101e5-26fe-451c-a3a1-cf7ebdefc308", "ip_address": "2001:470:739e:0:f816:3eff:fefc:c99b"}]} | False | False |

I have an instance in slaac-auto network.

When I've restarted neutron-l3-agent, router has rebind its GUA address, but instance lost both IPv4 and IPv6 fixed_ips.

| id | name | mac_address | fixed_ips |
+--------------------------------------+------+-------------------+----------------------------------------------------------------------------------------------+
| be16f0fd-d185-4cc7-b4a5-129b7ed9537a | | fa:16:3e:98:69:8e | |
| c559bdcd-76f0-4914-84b5-6dca82371668 | | fa:16:3e:ba:ea:ea | {"subnet_id": "ebd9bb05-c180-4466-bb92-f6ab6efa58a8", "ip_address": "2001:470:739e:fefe::1"} |
| f839c7d2-da54-40d4-96f7-28bf0c64b48b | | fa:16:3e:df:c2:ea | {"subnet_id": "6031c529-12d8-4ba5-be4e-90b52330c4e0", "ip_address": "192.168.114.2"} |

To reproduce:
1. Setup OpenStack with prefix delegation
2. Create public dualstack network
3. Create private network with prefix delegated subnet
4. Create router and hook it up to new subnets
5. Create instance
6. Restart neutron-l3-agent
7. Check out instance fixed_ips (both in nova and neutron)

Neutron from master https://github.com/openstack/neutron/commit/b060e5b