netaddr cached indexes are not coherent when changes occur to OUI or IAB data
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
MAAS |
Opinion
|
Undecided
|
Unassigned | ||
netaddr |
New
|
Unknown
|
|||
python-netaddr (Ubuntu) |
Confirmed
|
Undecided
|
Unassigned |
Bug Description
For some background, see Launchpad bug #1655447.[1]
The `netaddr.eui.ieee` package is responsible for indexing IEEE data at build time, and generating index files that the `netaddr` upstream code ships with.
This works fine until you have a long-running process relying on the data. For long-running processes, the index is cached the first time the OUI class is used (or the `netaddr.eui.ieee` package is imported). That means if `netaddr` is updated to a more recent version while the process is running, undefined behavior occurs until the process is restarted. This is because the cache becomes out-of-sync with the IEEE data on disk. (For example, we have observed `netaddr` crashing with an `IndexError` in Launchpad bug #1655049.[2])
In order to use `netaddr` in a long-running process, the cache must be re-read from disk if it changes.
Ideally, `netaddr` could also employ some type of integrity check, such as metadata in the cache file to ensure it is consistent with the data on-disk. Since a thorough integrity check might be expensive enough to defeat the purpose of an on-disk index, a basic sanity check (such as file size; possibly a hash of a subset of the file) should be sufficient.
Alternatively, the on-disk cache of the index could be dropped, and it could be calculated in memory. (Then recalculated if the data file changes.) However, this may be a problem on embedded systems without much CPU power.
---
[1]: https:/
[2]: https:/
Changed in python-netaddr (Ubuntu): | |
status: | New → Confirmed |
Changed in netaddr: | |
status: | Unknown → New |
Changed in maas: | |
importance: | Critical → Undecided |
no longer affects: | maas/2.1 |
Marking 'Opinion' for MAAS, since we may need to patch this in MAAS until a fix lands upstream in `netaddr`. I'd really rather not, if the fix can get into -updates.