ML2: Consider enabling arp_responder

Enabling arp_responder would allow OVS to avoid sending broadcast traffic across the whole overlay L2 for ARP requests.

Using VIPs on the overlay with allowed-address-pairs extension is something that needs to be assessed with arp_responder as, for example, Octavia Amphorae rely on an ability to use GARP broadcasts during VRRP failover to change ARP table entries for load-balancer VIPs.

arp_responder is disabled by default:
https://github.com/openstack/neutron/blob/28f3e371aa33c0288a626fe5a9d89440880d7cca/neutron/conf/plugins/ml2/drivers/ovs_conf.py#L122-L127

Using L2 population does not enable it:
https://github.com/openstack/neutron/blob/28f3e371aa33c0288a626fe5a9d89440880d7cca/neutron/plugins/ml2/drivers/openvswitch/agent/ovs_neutron_agent.py#L158
# ...
        self.arp_responder_enabled = agent_conf.arp_responder and self.l2_pop

Linux bridge-related note:
https://github.com/openstack/neutron/blob/stable/queens/releasenotes/notes/linuxbridge_vxlan_arp_responder-e9ea91552e1b62a7.yaml#L2-L13
"The responder is now disabled to improve compatibility with the allowed-address-pair extension and to match the default behavior of the ML2 OVS agent. The logical network will now utilize traditional flood and learn through the overlay."

ARP tables to make note of:

* VM ARP tables;
* qrouter (legacy, l3ha, non-distributed) namespace ARP table;
* qrouter namespace (dvr);
* snat namespace (dvr).

ARP tables in network namespaces will be used for ARP resolution when traffic arrives from a different subnet (i.e. multiple networks attached to one router or traffic coming from a provider network or fip namespace).

With DVR, PERMANENT ARP table entries are used which also needs to be considered:

https://bugs.launchpad.net/neutron/+bug/1774459
https://bugs.launchpad.net/neutron/+bug/1821357