reference set leaks memory
Bug #250819 reported by
Benjamin Kampmann
This bug affects 1 person
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
Storm |
Confirmed
|
Medium
|
Unassigned |
Bug Description
The reference set internally makes some strong references. That leads to memory leaks as the reference sets are not cleaned up and the flush time increases rapidly.
Changed in storm: | |
importance: | Undecided → Medium |
status: | New → Confirmed |
To post a comment you must log in.
So, Key.value here is composed of two relation objects, effectively:
relation1 = Relation(Key.id, KeyValue.key, many=True, remote=True)
relation2 = Relation(Value.id, KeyValue.value, many=True, remote=True)
The "key.values. add(value) " line essentially does:
link = KeyValue() link(key, link, setting=True) link(value, link, setting=True)
relation1.
relation2.
At this point, we have a few circular references set up:
1. references to get_obj_info(link) and link are stored in get_obj_info(key) and get_obj_info(value)
2. get_obj_info(link) -> EventSystem -> callback that takes get_obj_info(key) as an argument
3. get_obj_info(link) -> EventSystem -> callback that takes get_obj_info(value) as an argument
So we've got circular references Key <-> KeyValue and KeyValue <-> Value, and nothing seems to break those loops (not even the invalidate() call done by the commit).
Furthermore, the "key" variable keeps one Key object alive. Due to the circular references, this will keep 400 KeyValue objects alive, which will in turn keep every Value object alive. Going in the opposite direction, every other KeyValue and Key object is referenced.
Inserting the statement "del key, value" before the store.commit() call seems to stabilise things, letting the gc.collect() call clean up the mess.
Ideally, these cycles would be cleaned up when the objects were invalidated.