Comment 2 for bug 1653708

Revision history for this message
John A Meinel (jameinel) wrote :

I do believe there was intent to support dynamic operation. And we had certainly discussed something like RIP. I think what happened in practice is that people noticed that they can configure routes for a space as long as they are careful what subnets they assign to a space.
For example, if you do:
space: internal
  - Rack1: 10.0.1.0/24
  - Rack2: 10.0.2.0/24
  - Rack3: 10.0.3.0/24
  - ...
space: ceph-data
  - Rack1: 10.100.1.0/24
  - Rack2: 10.100.2.0/24
  - Rack3: 10.100.3.0/24
  - ...

Then you can create a "route to internal" as:
 - Rack 1:
    10.0.0.0/16 via 10.0.1.1
    10.100.0.0/16 via 10.100.1.1
 - Rack 2:
    10.0.0.0/16 via 10.0.2.1
    10.100.0.0/16 via 10.100.2.1
 - Rack 3:
    10.0.0.0/16 via 10.0.3.1
    10.100.0.0/16 via 10.100.3.1

And this is 'dynamic enough', because when you add Rack 4:
  internal: 10.0.4.0/24
    10.0.0.0/16 via 10.0.4.1
  ceph-data: 10.100.4.0/24
    10.100.4.0/16 via 10.100.4.1

You don't have to go back an update any of the "static" routes, because they
already allow you the spectrum of 255 racks.

It's a pragmatic solution that does fit in with how humans looking at route
tables would like to see them. You still get "route per space", you just have
to orchestrate your subnets such that they always fit within a larger CIDR.

I think that the way we would want to actually model it would *be* to have
users declare routes for space. (In subnet 10.0.3.0/24 get to ceph-data space
via 10.0.3.1)
That would, ultimately, give you more flexibility, as then if you ever grew
large enough that your original CIDR (/16) wasn't big enough, you could grab
any other unused range and the routes would update to follow.