reference set leaks memory

Bug #250819 reported by Benjamin Kampmann
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Storm
Confirmed
Medium
Unassigned

Bug Description

The reference set internally makes some strong references. That leads to memory leaks as the reference sets are not cleaned up and the flush time increases rapidly.

Revision history for this message
Benjamin Kampmann (lightyear) wrote :
Thomas Herve (therve)
Changed in storm:
importance: Undecided → Medium
status: New → Confirmed
Revision history for this message
James Henstridge (jamesh) wrote :

So, Key.value here is composed of two relation objects, effectively:

    relation1 = Relation(Key.id, KeyValue.key, many=True, remote=True)
    relation2 = Relation(Value.id, KeyValue.value, many=True, remote=True)

The "key.values.add(value)" line essentially does:

    link = KeyValue()
    relation1.link(key, link, setting=True)
    relation2.link(value, link, setting=True)

At this point, we have a few circular references set up:

 1. references to get_obj_info(link) and link are stored in get_obj_info(key) and get_obj_info(value)
 2. get_obj_info(link) -> EventSystem -> callback that takes get_obj_info(key) as an argument
 3. get_obj_info(link) -> EventSystem -> callback that takes get_obj_info(value) as an argument

So we've got circular references Key <-> KeyValue and KeyValue <-> Value, and nothing seems to break those loops (not even the invalidate() call done by the commit).

Furthermore, the "key" variable keeps one Key object alive. Due to the circular references, this will keep 400 KeyValue objects alive, which will in turn keep every Value object alive. Going in the opposite direction, every other KeyValue and Key object is referenced.

Inserting the statement "del key, value" before the store.commit() call seems to stabilise things, letting the gc.collect() call clean up the mess.

Ideally, these cycles would be cleaned up when the objects were invalidated.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.