Comment 0 for bug 1940463

Currently the os-net-config schema for routes accepts only one gateway per destination for a given route. If multiple routes are added for the same destination with different gateways on different physical interfaces, the init-scripts call out to the "ip route" utility with two separate routes, and only one gets installed by the kernel. We need to be able to specify multiple gateways across different interfaces in one route, in which case the kernel will install the route across multiple physical interfaces.

Example (two routes to destination IP 192.168.54.1 on two different interfaces):

network_config:
  -
    type: interface
    name: nic1
    addresses:
      -
        ip_netmask: 192.168.30.1/24
    routes:
      -
        ip_netmask: 192.168.54.1/32
        next_hop: 192.168.30.2
  -
    type: interface
    name: nic2
    addresses:
      -
        ip_netmask: 192.168.31.2/24
    routes:
      -
        ip_netmask: 192.168.54.1/32
        next_hop: 192.168.31.3

This results in the following routing configuration files:

File: /etc/sysconfig/network-scripts/route-eth0

192.168.54.1/32 via 192.168.30.2 dev eth0
----
File: /etc/sysconfig/network-scripts/route-eth1

192.168.54.1/32 via 192.168.31.3 dev eth1

When ifup is run on both of these interfaces, only one route will be installed after running os-net-config to configure and enable both interfaces (whichever is associated with the interface brought up first). When one of the interfaces is already active the following kernel error occurs when the second route is added: "RTNETLINK answers: File exists". Then, if the first interface is brought down the route disappears and there's no route to the destination, even though a route to the destination was present in the configuration file for the second interface. Restarting the second interface while the first is down will bring the second route up, since there is no conflict.

The kernel will add both gateways to the routing table if they are specified in one route using the "ip route" utility. Although the kernel by default will send traffic on only one gateway/interface for a given source/destination IP pair when using static routing, software may take advantage of multiple gateway/interfaces in the same route. This particular feature has been requested for use with NFV virtual routing software. Both routes may be used