Comment 0 for bug 2002687

Some network designs include multiple L3 gateways to:

* Share the load across different gateways;
* Provide independent network paths for the north-south direction (e.g. via
  different ISPs).

Having multi-homing implemented at the instance level imposes additional burden
on the end user of a cloud and support requirements for the guest OS, whereas
utilizing ECMP and BFD at the router side alleviates the need for instance-side
awareness of a more complex routing setup.

Adding more than one gateway port implies extending the existing data model
which was described in the multiple external gateways spec (https://specs.openstack.org/openstack/neutron-specs/specs/xena/multiple-external-gateways.html). However, it left
adding additional gateway routes out of scope leaving this to future
improvements around dynamic routing. Also the focus of neutron-dynamic-routing
has so far been around advertising routes, not accepting new ones from the
external peers - so dynamic routing support like this is a very different
subject. However, manual addition of extra routes does not utilize the default
gateway IP information available from subnets in Neutron while this could be
addressed by implementing an extra conditional behavior when adding more than
one gateway port to a router.

ECMP routes can result in black-holing of traffic should the next-hop of a
route becomes unreachable. BFD is a standard protocol adopted by IETF
for next-hop failure detection which can be used for route eviction. OVN
supports BFD as of v21.03.0 (https://github.com/ovn-org/ovn/commit/6e0a69ad4bcdf9e4cace5c73ef48ab06065e8519) with a data model that allows enabling
BFD on a per next-hop basis by associating BFD session information with routes,
however, it is not modeled at the Neutron level even if a backend supports it.

From the Neutron data model perspective, ECMP for routes is already a supported
concept since ECMP support spec got implemented (https://specs.openstack.org/openstack/neutron-specs/specs/wallaby/l3-router-support-ecmp.html) in Wallaby (albeit the
spec focused on the L3-agent based implementation only).

As for OVN and BFD, the OVN database state needs to be populated by Neutron
based on the data from the Neutron database, therefore, data model changes to
the Neutron DB are needed to represent the BFD session parameters.

---

The previous work on multiple gateway ports did not get completed and the neutron-lib changes were reverted. Likewise, the scope of this RFE is bigger with some overlap and augmentation compared to prior art. The spec will follow for this RFE with more details as to how the data model and API changes are proposed to be made.