[L3] Unexcepted HA router scheduled instance shows up after manully scheduling

ENV: stable/queens, but master branch basically has the same code.

Unexcepted HA router scheduled instance shows up after manully scheduling and admin-state down/up.

Step to reproduce:

$ openstack network agent list --router c0f96d58-5521-40fa-9536-205635facc69 --long
+--------------------------------------+------------+----------+-------------------+-------+-------+------------------+----------+
| ID | Agent Type | Host | Availability Zone | Alive | State | Binary | HA State |
+--------------------------------------+------------+----------+-------------------+-------+-------+------------------+----------+
| 581cb0b8-10e9-41cf-bc4a-01054e5303f3 | L3 agent | network3 | nova | :-) | UP | neutron-l3-agent | active |
| a195dc34-b325-4b5b-8dde-78a261ca5692 | L3 agent | network2 | nova | :-) | UP | neutron-l3-agent | standby |
+--------------------------------------+------------+----------+-------------------+-------+-------+------------------+----------+
$ openstack network agent add router --l3 0eb1d073-8b2e-4fa3-bd27-43f2ec684334 c0f96d58-5521-40fa-9536-205635facc69
$ openstack network agent list --router c0f96d58-5521-40fa-9536-205635facc69 --long
+--------------------------------------+------------+----------+-------------------+-------+-------+------------------+----------+
| ID | Agent Type | Host | Availability Zone | Alive | State | Binary | HA State |
+--------------------------------------+------------+----------+-------------------+-------+-------+------------------+----------+
| 0eb1d073-8b2e-4fa3-bd27-43f2ec684334 | L3 agent | network1 | nova | :-) | UP | neutron-l3-agent | standby |
| 581cb0b8-10e9-41cf-bc4a-01054e5303f3 | L3 agent | network3 | nova | :-) | UP | neutron-l3-agent | active |
| a195dc34-b325-4b5b-8dde-78a261ca5692 | L3 agent | network2 | nova | :-) | UP | neutron-l3-agent | standby |
+--------------------------------------+------------+----------+-------------------+-------+-------+------------------+----------+

# Now remove router from l3-agent (network2)
$ openstack network agent remove router --l3 a195dc34-b325-4b5b-8dde-78a261ca5692 c0f96d58-5521-40fa-9536-205635facc69
$ openstack network agent list --router c0f96d58-5521-40fa-9536-205635facc69 --long
+--------------------------------------+------------+----------+-------------------+-------+-------+------------------+----------+
| ID | Agent Type | Host | Availability Zone | Alive | State | Binary | HA State |
+--------------------------------------+------------+----------+-------------------+-------+-------+------------------+----------+
| 0eb1d073-8b2e-4fa3-bd27-43f2ec684334 | L3 agent | network1 | nova | :-) | UP | neutron-l3-agent | standby |
| 581cb0b8-10e9-41cf-bc4a-01054e5303f3 | L3 agent | network3 | nova | :-) | UP | neutron-l3-agent | active |
+--------------------------------------+------------+----------+-------------------+-------+-------+------------------+----------+

# Admin state down/up the router
$ openstack router set --disable c0f96d58-5521-40fa-9536-205635facc69
$ openstack router set --enable c0f96d58-5521-40fa-9536-205635facc69

# Unexcepted scheduled router instance shows up, router was scheduled to network2
$ openstack network agent list --router c0f96d58-5521-40fa-9536-205635facc69 --long
+--------------------------------------+------------+----------+-------------------+-------+-------+------------------+----------+
| ID | Agent Type | Host | Availability Zone | Alive | State | Binary | HA State |
+--------------------------------------+------------+----------+-------------------+-------+-------+------------------+----------+
| 0eb1d073-8b2e-4fa3-bd27-43f2ec684334 | L3 agent | network1 | nova | :-) | UP | neutron-l3-agent | standby |
| 581cb0b8-10e9-41cf-bc4a-01054e5303f3 | L3 agent | network3 | nova | :-) | UP | neutron-l3-agent | standby |
| a195dc34-b325-4b5b-8dde-78a261ca5692 | L3 agent | network2 | nova | :-) | UP | neutron-l3-agent | standby |
+--------------------------------------+------------+----------+-------------------+-------+-------+------------------+----------+

# checking the router HA state...
$ openstack network agent list --router c0f96d58-5521-40fa-9536-205635facc69 --long
+--------------------------------------+------------+----------+-------------------+-------+-------+------------------+----------+
| ID | Agent Type | Host | Availability Zone | Alive | State | Binary | HA State |
+--------------------------------------+------------+----------+-------------------+-------+-------+------------------+----------+
| 0eb1d073-8b2e-4fa3-bd27-43f2ec684334 | L3 agent | network1 | nova | :-) | UP | neutron-l3-agent | standby |
| 581cb0b8-10e9-41cf-bc4a-01054e5303f3 | L3 agent | network3 | nova | :-) | UP | neutron-l3-agent | standby |
| a195dc34-b325-4b5b-8dde-78a261ca5692 | L3 agent | network2 | nova | :-) | UP | neutron-l3-agent | standby |
+--------------------------------------+------------+----------+-------------------+-------+-------+------------------+----------+
$ openstack network agent list --router c0f96d58-5521-40fa-9536-205635facc69 --long
+--------------------------------------+------------+----------+-------------------+-------+-------+------------------+----------+
| ID | Agent Type | Host | Availability Zone | Alive | State | Binary | HA State |
+--------------------------------------+------------+----------+-------------------+-------+-------+------------------+----------+
| 0eb1d073-8b2e-4fa3-bd27-43f2ec684334 | L3 agent | network1 | nova | :-) | UP | neutron-l3-agent | standby |
| 581cb0b8-10e9-41cf-bc4a-01054e5303f3 | L3 agent | network3 | nova | :-) | UP | neutron-l3-agent | active |
| a195dc34-b325-4b5b-8dde-78a261ca5692 | L3 agent | network2 | nova | :-) | UP | neutron-l3-agent | standby |
+--------------------------------------+------------+----------+-------------------+-------+-------+------------------+----------+