Memory leak during file list update

Bug #351393 reported by Ben
22
This bug affects 3 people
Affects Status Importance Assigned to Milestone
DC++
Status tracked in Trunk
Trunk
Fix Released
High
Steven Sheehy
LinuxDC++
Fix Released
High
Steven Sheehy

Bug Description

Version: r1728

When I refresh my file list, memory usage steadily increases while shared directories are scanned for new or modified files, and does not decrease again after the refresh is complete. If I initiate another refresh, memory usage again increases by the same amount. Since I share a very large number of files and directories (226709 files), memory usage can surpass 1GB in less than a day.

Revision history for this message
Ben (mimenarrator) wrote :

This bug also exists in version 0.75.

Revision history for this message
Ben (mimenarrator) wrote :

Here's a strange bit of information that might help find the cause of this bug: the first refresh after I rebuilt my hash database did not cause memory usage to increase, but the bug returned on subsequent refreshes.

Revision history for this message
kwolfe (mr-kwolfe) wrote :

I've also noticed this problem since DC++ 0.75 when sharing many files and directories. After its been open for a few days I need to close and reopen DC++ because it is consuming most of the memory on my computer.

Revision history for this message
Ben (mimenarrator) wrote :

It looks to me like this leak was probably introduced in revision 1638, when some sharemanager memory was automanaged.

Revision history for this message
Ben (mimenarrator) wrote :

I'd just like to add that I've now verified that this bug continues to exist in r1765. It can be pretty bad for users with large file list so I would think a fix would be a priority.

Revision history for this message
Steven Sheehy (steven-sheehy) wrote :

I can confirm the memory leak and it is indeed introduced in revision #1638. The memory leak is due to never releasing ShareManager::Directory objects. A simple printf("Deleting Directory: %s\n", name.c_str()); in the Directory destructor proves this.

Changed in dcplusplus:
importance: Undecided → High
status: New → Confirmed
tags: added: core
Changed in linuxdcpp:
status: New → Confirmed
importance: Undecided → High
Revision history for this message
Big Muscle (bigmuscle) wrote :

Each directory object contains list of file and subdirectories. Each file and (sub)directory has a reference to its parent object. And it seems that these lists are never cleared, therefore reference count to parent objects is never decreased and such object will never be destroyed.

Revision history for this message
Big Muscle (bigmuscle) wrote :

So it seems that using common pointer to parent object (instead of intrusive pointer) solves the problem.

ShareManager::Directory and ShareManager::Directory::File >>> GETSET(Directory*, parent, Parent);
their constructors: parent(aParent.get())

Revision history for this message
Steven Sheehy (steven-sheehy) wrote :

That is the same conclusion I came to in my testing. Committed a fix to trunk.

Changed in dcplusplus:
assignee: nobody → Steven Sheehy (steven-sheehy)
status: Confirmed → Fix Committed
Revision history for this message
Razzloss (razzloss) wrote :

What about Linuxdcpp fix? You intend to wait for next core release and upgrade to that? Or are just using windows users as guinea pigs?

--RZ

Revision history for this message
Ben (mimenarrator) wrote :

To get r1815 to compile on my machine I must edit \boost\boostnoncopyable.hpp to make const noncopyable& operator=( const noncopyable& ); and noncopyable( const noncopyable& ); protected instead of private, though I imagine there is a more elegant solution.

Revision history for this message
Steven Sheehy (steven-sheehy) wrote :

Razzloss: Yeah, I was thinking to just wait until next core release since we're not planning a release anytime soon. But if you want to check in the patch, go right ahead.

Revision history for this message
Razzloss (razzloss) wrote :

Done. Apparently there's quite a few people using bzr-trunk as their primary client, so less annoying (and fixed) bugs in it means more useful bug reports for us (hopefully), when people are able to actually test it without restarting every couple of hours...

--RZ

Changed in linuxdcpp:
status: Confirmed → Fix Committed
Revision history for this message
dairinin (nowhere-hakkenden) wrote :

Running for two days. Rehashing every 30 minutes. Added some files to share, removed - no problems with memory:

$ cat /proc/`pidof linuxdcpp`/status
Name: linuxdcpp
State: S (sleeping)
Tgid: 24231
Pid: 24231
PPid: 24228
TracerPid: 0
Uid: 102 102 102 102
Gid: 100 100 100 100
FDSize: 256
Groups: 100
VmPeak: 459116 kB
VmSize: 388008 kB
VmLck: 0 kB
VmHWM: 100816 kB
VmRSS: 85288 kB
VmData: 268716 kB
VmStk: 84 kB
VmExe: 2104 kB
VmLib: 16692 kB
VmPTE: 460 kB
Threads: 8
SigQ: 0/36864
SigPnd: 0000000000000000
ShdPnd: 0000000000000000
SigBlk: 0000000000000000
SigIgn: 0000000000001006
SigCgt: 0000000180000000
CapInh: 0000000000000000
CapPrm: 0000000000000000
CapEff: 0000000000000000
CapBnd: ffffffffffffffff
voluntary_ctxt_switches: 542185
nonvoluntary_ctxt_switches: 102076

Revision history for this message
Steven Sheehy (steven-sheehy) wrote :

Reverted fix due to bug #480065.

tags: added: memoryleak
Revision history for this message
Jacek Sieka (arnetheduck) wrote :

tentative fix in bzr - items weren't being reparented correctly...

Changed in linuxdcpp:
status: Fix Committed → Confirmed
Changed in linuxdcpp:
assignee: nobody → Steven Sheehy (steven-sheehy)
milestone: none → 1.1.0
status: Confirmed → Fix Committed
Changed in linuxdcpp:
status: Fix Committed → Fix Released
tags: added: memory-leak
removed: memoryleak
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Related questions

Remote bug watches

Bug watches keep track of this bug in other bug trackers.