Hardware NICs sometimes get a different ethX label in otherwise identical hardware

Depending on the order that Ubuntu's udev system finds the NICs, they can come up in various different orderings.

Typically, for a given hardware configuration, they're always the same. Increased numbers of NICs and, specifically, mixed NIC driver types, make this more likely; I have had one specific machine where the ordering is reliably different to every other instance of the same hardware configuration, but generally this is a 'once in a blue moon' problem.

The solution is to fix the NIC name to the PCI device name. One way to do this is to, in the late commands pre-reboot, remove /etc/udev/rules.d/70-persistent-net.rules and add a 71-net.rules with a PCI binding rather than a MAC binding, such as:

# PCI slot 1
SUBSYSTEM=="net", KERNELS=="0000:02:00.0", ACTION=="add", NAME="eth1"
SUBSYSTEM=="net", KERNELS=="0000:02:00.1", ACTION=="add", NAME="eth6"
# PCI slot 2
SUBSYSTEM=="net", KERNELS=="0000:04:00.0", ACTION=="add", NAME="eth4"
SUBSYSTEM=="net", KERNELS=="0000:04:00.1", ACTION=="add", NAME="eth7"
# PCI slot 5
SUBSYSTEM=="net", KERNELS=="0000:83:00.0", ACTION=="add", NAME="eth8"
SUBSYSTEM=="net", KERNELS=="0000:83:00.1", ACTION=="add", NAME="eth9"

The PCI slots in given hardware don't always have logical PCI bus numbers, unfortunately, so I had to produce the above file by checking the boot messages and creating it by hand. A nicer solution would be to write a script using 'lspci' or sysfs to enumerate and name the NICs, forcing their names to be fixed for future reboots. The numbering would, however, be thrown by adding a NIC.

For the sake of our installer we may also optionally want to give them friendlier names based on hardware location, though that might equally confuse anyone logging in, so it's an open question how good an idea that is.

(With a fix like the above, conveniently, replacing a broken NIC will not cause a renumbering and will not necessitate a reinstall. Conversely, moving a NIC to another PCI slot will definitely throw the above off.)