major memory leak on plasmashell

Hi!
I have ArchLinux with Qt 5.4, KDE Framework 5.7.0 and KDE Plasma 5.2.0 installed on laptop.
During launch plasmashell uses almost twice more, than in KDE 4.14. During use plasmashell utilize over 600MB - that is more than as for KDE.
Install was made by different way:
- by deleting KDE 4.14 and then installing Plasma 5 with KDE Framework from official repository ArchLinux
- by installing clear ArchLinux and after that installing Plasma 5 with KDE Framework
- by installing clear ArchLinux and after that compiling Plasma 5 with KDE Framework from git sources

No one of this methods does not improve situation with using RAM on Plasma 5. In this form it is not suitable for use, even with 5.2.1 version released. In 4.14 plasma uses ~60mb ram on cold start, and not more than 80mb during use whole day. At case of 5.2.1 - 600mb by half day. This is more than all system and browser.
Hope, that this very big issue will be fixed with nearest releases.

Here is my explanation screenshots:
http://res.cloudinary.com/metsys/image/upload/v1425513469/kde5/image.png
http://res.cloudinary.com/metsys/image/upload/v1425513470/kde5/image1.png
http://res.cloudinary.com/metsys/image/upload/v1425513469/kde5/image2.png
http://res.cloudinary.com/metsys/image/upload/v1425513468/kde5/image3.png
http://rghost.net/private/8Qgfs2rd4/f05d5b1e87aad02d863e9009e864c8da]plasmashell.smaps

luebking advice me post this issue here, in bug tracker

here is my first post: https://forum.kde.org/viewtopic.php?f=289&t=125251

Reproducible: Always

Steps to Reproduce:
In dedails

Expected Results:
use max 100 mb ram ))

corrupted link, here is right:
http://rghost.net/private/8Qgfs2rd4/f05d5b1e87aad02d863e9009e864c8da

Welcome into JS world! :)

More plasma Debug http://rghost.net/private/8PVkCKVKd/6c607378e1060bcaf6f2935b287335cd

I see the same exact problem on my Gentoo 64bit laptop.
Plasma 5.2.1, KF5.7, Qt 5.4.1

Since I upgraded to plasma-5 I often run into an unresponsible (because of swapping) desktop, and have to hard-reset. Please have a look memory consumption please. I cannot upgrade my RAM anymore.

I'm affected to.

Which desktop applets are you using?
I'm not using any additional desktop applets, besides the following builtins:
- K-Menu
- Virtual Desktops Switcher
- Window switcher
- Clock

I've got what I assume to be the same problem. I upgraded to Kubuntu 15.04 with Plasma 5.2.
On boot plasmashell uses ~160MB ram. I left the machine idle with nothing but a Konsole running, and a few hours later plasmashell had 3.2GB res RAM.

Sadly I'm not sure what logs I should attach to let anyone make sense of this.

memsys, how did you create that valgrind output?

most importantly, how did you close valgrind/plasmashell.
Almost all these traces are just in Qt and it seems unlikely that's the problem.

can all of you above try removing applets to see if we can pinpoint one causing a problem.

Can you all above tell me what graphic card you're using + driver versions.

It happens on two different machines for me, one is an intel GM45 Express with a recent i915 driver

The only plasmoids I've got running in my tray are notifications, volume control, network, klipper and removable devices. I don't see any memory being freed up if I stop/remove any of them.

I'm using the nvidia 346.47 driver on a nvidia 770GTX card.

David Edmundson,
Here is info about graphic card and driver version:
http://res.cloudinary.com/metsys/image/upload/v1427053941/kde5/image.png

This is valgrind launch command:
localhost% valgrind --tool=memcheck --leak-check=full --leak-resolution=high --log-file=Plasma_Debug plasmashell and file created https://yadi.sk/d/pIXnTtd6fSbqB and https://yadi.sk/d/BiviLfVWfSbyd

"how did you close valgrind/plasmashell" - what do you mean? It's not clear for me, sorry for question.

All log files was created with default installed applets, if it matter, ok, I remove them and make new one test/log files.

Regards

Here is plasma launch without applets. After ~20 minutes closed by Ctrl+C https://yadi.sk/d/Szh6GAwTfSgqv
Here is valgrind results https://yadi.sk/d/WE0wP5xgfSgyS

https://yadi.sk/d/voVYIUg2fTRMH
- Plasma launch with 1 empty panel and 3 applets:
1. org.kde.plasma.kickoff
2. org.kde.plasma.systemtray (by adding this applet plasmashell crashed (KCrash: Application 'plasmashell' crashing...)
3. org.kde.plasma.taskmanager
After attempt to add third applet valgrind shutdown ([1] + exit 253 valgrind --tool=memcheck --leak-check=full --leak-resolution=high plasmashel)

and
https://yadi.sk/d/7jH9HwSBfTRi8 result of command:
valgrind --tool=memcheck --leak-check=full --leak-resolution=high --log-file=Plasma_Debug_3 plasmashell

It happens to me mostly some time after the screen was locked, but I can't confirm it is really related to each other.

*** Bug 345711 has been marked as a duplicate of this bug. ***

Have traced this down to the qt app: transmission-qt

The memory usage continuously rises while seeding with this app, after some time, possibly as much as an hour. Then immediately stops rising once I kill the app.

Transmission-gtk is fine.

This continues through the latest beta to 5.3.

Transmission-qt makes plasmashell use more memory?

Does it have a system tray icon? Does that icon do anything "odd"?

Ok David, that's it!

When I turn off the tray icon in transmission-qt (under preferences>desktop), the memory usage stops rising.

Good to know.

Sorry, didn't answer your question, David.

It is the plasmashell using the memory, and no, the icon was not doing anything odd, when on.

I have the same problem with plasmashell. I have no plasmoids on the desktop, and only a minimal menu bar.

My suspicion is that non-native apps are the issue. I use sparkleshare (a mono app), which I haven't managed to display in the system tray. Whenever it's running, plasmashell memory usage starts going up totally out of control.

I experience the same problems on plasma 5.3 with no plasmoids, but with owncloud sync client running. I notice the problem only after I unlock the lock screen, and in my case Xorg.bin does exactly the same thing. Whereas "plasmashell" uses up to 2GB of Ram, Xorg.bin doubles the ram usage. I have attached smaps and screenshots from ksysguard.

Created attachment 92365
Plasma memory usage smap export

Created attachment 92366
ksysguard detailed view of plasmashell memory usage

Created attachment 92367
ksysguard view of plasmashell and xorg memory usage

I forgot to add that I am using Fedora packages:
Fedora 21
Linux 3.19.5-200.fc21
KDE Plasma 5.3

Also I do not have transmission-qt or transmission installed or running

bugzy, can you list what things you do have in the sysem tray?

Created attachment 96028
valgrind log of Xorg with debug symbols, from start to end

Xorg's valgrind run, with debug symbols, through shell script with following command line:
exec valgrind --error-limit=no --leak-check=full --log-file=/tmp/Xorg-valgrind.log /usr/lib/xorg-server/Xorg.valgrind "$@"
Finished with 830M+ RAM usage, even with no windows open.

Experiencing this as well for quite some time now. Went up to 2gb within 4 hours, 4gb within 8 hours. Pretty linear.

It's always happening, having a pretty fresh kubuntu 15.10 install (with updates). I can reproduce it by being afk, so let me know if I can do anything to get this backtraced.

I have got - not scientific :) - feeling the leak is correlated with workspace background: using solid color helps.

Interestingly, for me, extremely reproducable if I'm running Wine OpenGL applications. There I've like 1mb/s mem leak. Running no wine OpenGL application, it seems to be pretty stable.

But why would plasmashell freak out because of wine OpenGL applications?

Fact is, as soon as I start an OpenGL application in Wine (as WoW), plasmashell it leask 920k/s. As soon as I close it, it stops leaking. Note that normal Wine applications don't start the leak, as in example the Battle.net Launcher.

So that seems to be connected at least.

I have an idea but it is difficult to explain. Picture a compositor written in Qt5 and using opengl. Applications using libx11 directly and not Qt5 are causing leaks.

Decided to put some time on this issue tonight. Happens on the following distributions:
Arch, KUbuntu, OpenSUSE, Fedora (KDE Build), Tanglu (3.0, 4.0 Alpha), Manjaro, Mint

Happens on DEs: Only KDE. DEs Tested:
KDE, Unity, Gnome, LXQt, KDE, LXDE, Cinnamon, MATE, Xfce.

I think the leak only happens when I play CS GO (OpenGL app) and plasmashell is running. I now stop the plasmashell process while playing, and Xorg seems to have a normal memory usage.

I think so too. Especially if wine was leaking in libx11, we'd see a different pattern (other DEs would very likely leak too!).

I didn't experience this on native OpenGL applications (yet) though, but I'l do some checks on that as well the next few days.

(In reply to AnAkkk from comment #98)
> I think the leak only happens when I play CS GO (OpenGL app)...

I guess this generalization is too wide: I don't play games at all, but the leak takes place.

(In reply to Andrew Gaydenko from comment #100)
> (In reply to AnAkkk from comment #98)
> > I think the leak only happens when I play CS GO (OpenGL app)...
>
> I guess this generalization is too wide: I don't play games at all, but the
> leak takes place.

Perhaps everything not gtk3/Qt5/sdl2 is causing leaks?

Anyone on NVIDIA proprietary driver and getting memory leaks in Xorg process, can you try the following?
1) swapoff -a
2) quit all applications and anything minimized to your system tray but keep a terminal open and restart kwin (kwin_x11 or something --replace & disown).
3) close your terminal application.
4) Then ctrl+alt+F3 or F4 then ctrl+alt+f1 back to your session.
5) swapon -a
6) Check if Xorg process is now using less memory (in other terms, if some garbage collection happened).

I am having the same problem as André Vitor de Lima Matos. Here's what I know:

* Only killing the X server reclaims the memory (killing plasmashell/kwin_x11 doesn't release it); it leaks about 5GB per day
* Plasma/kwin 5.5.2 (has been occurring for a while though)
* EGL/OpenGL 2.0 are my compositing options, but it happens with xrender too
* Using Intel driver 2.99.917-r2, Xorg 1.17.4, Kernel 4.3.3
* No real widgets besides the default
* I run thunderbird from time to time (but leaks even when I don't run it), firefox (5 or so tabs, no inordinate memory usage), pidgin (a few conversations), konversation (about 35 tabs or so), and konsole (pretty much vim, htop, and basic system administration tasks).
* plasmashell is using about 200M of memory, kwin_x11 is using about 40M of memory
* xrestop shows nothing out of the ordinary
* Suspending does not worsen it (I don't hibernate)
* I run infinality
* I have only one tray icon not from KDE: pidgin, and it blinking doesn't seem to worsen the problem after a cursory look
* Closing/opening windows doesn't seem to cause it (though it changes memory usage slightly, it doesn't resolve the general leak)
* Rolling over the task manager (looking at thumbnails) doesn't cause it

I'm at a loss as to what more I can do, though, to elucidate the bug. It seems to happen under ill-defined circumstances.

(In reply to Elizabeth Myers from comment #103)
> I am having the same problem as André Vitor de Lima Matos. Here's what I
> know:
>
> * Only killing the X server reclaims the memory (killing
> plasmashell/kwin_x11 doesn't release it); it leaks about 5GB per day
Can you try my suggestion please? close all applications including system tray ones after disabling swap. restart kwin_x11/plasma and ctrl+alt+f3 or f4 and then ctrl+alt+f1 back to your X session.
I would like to know if ctrl+alt+f* operations are triggering garbage collection in Xorg.

> * I run infinality
I think infinality patches can cause issues

(In reply to Hussam Al-Tayeb from comment #104)
> Can you try my suggestion please? close all applications including system
> tray ones after disabling swap. restart kwin_x11/plasma and ctrl+alt+f3 or
> f4 and then ctrl+alt+f1 back to your X session.

I usually don't have swap enabled. The memory usage goes down when I close the apps and switch to a TTY to kill plasma and kwin_x11 (more from closing apps than anything else) but the leaked memory seems to remain.

Note though I only tried this when X was using about 300MB of memory (it starts with about 100). I will try it when the memory usage goes up again to the gigabyte range.

> I think infinality patches can cause issues

I've never had issues with them. I can try switching, but call me skeptical.

(In reply to Elizabeth Myers from comment #105)
> I usually don't have swap enabled. The memory usage goes down when I close
> the apps and switch to a TTY to kill plasma and kwin_x11 (more from closing
> apps than anything else) but the leaked memory seems to remain.
Thank you. That is what I was trying to figure out.
If I understand correctly, closing an application and then switching to a TTY and back causes Xorg to release the xserver memory used for that application. If memory leaks remain afterwards, then this indicates issues with the xorg driver and the compositor (kwin_x11 in your case).

I never had swap enabled, and didn't use infinality either. For some reason I haven't had the issue the last few days though, it looks like it fixed itself for me. I have no idea what changed, maybe this is after I updated to Plasma 5.5.2.

Confirming that at least main memory leak is gone in plasma 5.5.2. Creating and closing lots of windows (which triggered leak previously) doesn't seem to leak anymore, as Xorg recovers memory after that windows were closed.

I've tried another compositing WM (xfwm4) and I have no leaks. It definitely seems to be something KDE is doing.

problem still exists on a version 5.5.3
apparently any plasmoid refresh take next leak
just set the display seconds in clock and during the day plasmashell take 600 mB ram

That test case is not reproducible for me, clock widget with seconds displayed... plasmashell hasn't grown any over the course of the past few minutes at least.

(In reply to painless roaster from comment #110)
Did you use Breeze? https://bugs.kde.org/show_bug.cgi?id=357800
Select Oxygen LookAndFeeel, logout - login make same tests.

(In reply to Rex Dieter from comment #111)
> That test case is not reproducible for me, clock widget with seconds
> displayed... plasmashell hasn't grown any over the course of the past few
> minutes at least.

Few minutes is very short time for test with clock.

Faster test:
 - download plasmoid thermal monitor and set:
     - refresh speed - 0.1s
     - create 5 fields for temperature monitoring (physical id and 4 cores)
 - in this test is leak speed is 12MB / minute

Is any chance for use memory leak profiler please? For example compile plasma-workspace with library jemalloc and use jeprof.
Valgrind is too slow and not monitor during run. But jemalloc is ideal library for memory monitoring.

(In reply to Rex Dieter from comment #111)
> That test case is not reproducible for me, clock widget with seconds
> displayed... plasmashell hasn't grown any over the course of the past few
> minutes at least.

I tried leak with thermal monitor on second pc. Leak exists, but it is slower (500kB / minute) . But for leak need set 6 virtual workspaces.

(In reply to Anthony from comment #112)
> (In reply to painless roaster from comment #110)
> Did you use Breeze? https://bugs.kde.org/show_bug.cgi?id=357800
> Select Oxygen LookAndFeeel, logout - login make same tests.

Leak (new test with thermal monitor) exists with oxygen and breeze .

I find that the leak persists even when the screensaver kicks in (overnight), so I wake to "kquitapp5 plasmashell && sleep 10s && plasmashell &>/dev/null & disown" every day.
I have a handful of systray icons which are handled by the xembed-sni-proxy, and I wonder if that contributes.
Also, if I let plasmashell's stdout and stderr come to my terminal window, I see xcb errors every time i move my mouse pointer over the taskbar and hover from one window marker to the next.
I don't know if either of these facts are helpful.

(In reply to painless roaster from comment #113)
> Faster test:
> - download plasmoid thermal monitor and set:
> - refresh speed - 0.1s
> - create 5 fields for temperature monitoring (physical id and 4 cores)
> - in this test is leak speed is 12MB / minute
>
> Is any chance for use memory leak profiler please? For example compile
> plasma-workspace with library jemalloc and use jeprof.
> Valgrind is too slow and not monitor during run. But jemalloc is ideal
> library for memory monitoring.

I cannot confirm i use it regular since it created, not remember, not leak at all. Tell your graphic driver, mine is radeon, your nvidia?

(In reply to Anthony from comment #117)
> (In reply to painless roaster from comment #113)
> > Faster test:
> > - download plasmoid thermal monitor and set:
> > - refresh speed - 0.1s
> > - create 5 fields for temperature monitoring (physical id and 4 cores)
> > - in this test is leak speed is 12MB / minute
> >
> > Is any chance for use memory leak profiler please? For example compile
> > plasma-workspace with library jemalloc and use jeprof.
> > Valgrind is too slow and not monitor during run. But jemalloc is ideal
> > library for memory monitoring.
>
> I cannot confirm i use it regular since it created, not remember, not leak
> at all. Tell your graphic driver, mine is radeon, your nvidia?

nouveau 1.0.12

$ uname -a
Linux 4.3.3-303.fc23.x86_64 #1 SMP Tue Jan 19 18:31:55 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux

$ lspci | grep VGA
01:00.0 VGA compatible controller: NVIDIA Corporation GT216 [GeForce 210] (rev a2)

$ rpm -q xorg-x11-drv-nouveau
xorg-x11-drv-nouveau-1.0.12-1.fc23.x86_64

valgrind detect memor leak (from 15minutes run) here:

==13702== 56,137,960 bytes in 32,858 blocks are definitely lost in loss record 62,663 of 62,663
==13702== at 0x4C28C50: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==13702== by 0x7496DFA: QSGBatchRenderer::Renderer::map(QSGBatchRenderer::Buffer*, int, bool) (in /usr/lib64/libQt5Quick.so.5.5.1)
==13702== by 0x7499074: QSGBatchRenderer::Renderer::uploadBatch(QSGBatchRenderer::Batch*) (in /usr/lib64/libQt5Quick.so.5.5.1)
==13702== by 0x74A4375: QSGBatchRenderer::Renderer::render() (in /usr/lib64/libQt5Quick.so.5.5.1)
==13702== by 0x74B00AE: QSGRenderer::renderScene(QSGBindable const&) (in /usr/lib64/libQt5Quick.so.5.5.1)
==13702== by 0x74B08FA: QSGRenderer::renderScene(unsigned int) (in /usr/lib64/libQt5Quick.so.5.5.1)
==13702== by 0x74C0DDD: QSGRenderContext::renderNextFrame(QSGRenderer*, unsigned int) (in /usr/lib64/libQt5Quick.so.5.5.1)
==13702== by 0x750ADCA: QQuickWindowPrivate::renderSceneGraph(QSize const&) (in /usr/lib64/libQt5Quick.so.5.5.1)
==13702== by 0x74DB78A: ??? (in /usr/lib64/libQt5Quick.so.5.5.1)
==13702== by 0x74DC890: ??? (in /usr/lib64/libQt5Quick.so.5.5.1)
==13702== by 0xA9D041B: QApplicationPrivate::notify_helper(QObject*, QEvent*) (in /usr/lib64/libQt5Widgets.so.5.5.1)
==13702== by 0xA9D58E5: QApplication::notify(QObject*, QEvent*) (in /usr/lib64/libQt5Widgets.so.5.5.1)

(In reply to painless roaster from comment #119)
> valgrind detect memor leak (from 15minutes run) here:
>
> ==13702== 56,137,960 bytes in 32,858 blocks are definitely lost in loss
> record 62,663 of 62,663
> ==13702== at 0x4C28C50: malloc (in
> /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
> ==13702== by 0x7496DFA:
> QSGBatchRenderer::Renderer::map(QSGBatchRenderer::Buffer*, int, bool) (in
> /usr/lib64/libQt5Quick.so.5.5.1)
> ==13702== by 0x7499074:
> QSGBatchRenderer::Renderer::uploadBatch(QSGBatchRenderer::Batch*) (in
> /usr/lib64/libQt5Quick.so.5.5.1)
> ==13702== by 0x74A4375: QSGBatchRenderer::Renderer::render() (in
> /usr/lib64/libQt5Quick.so.5.5.1)
> ==13702== by 0x74B00AE: QSGRenderer::renderScene(QSGBindable const&) (in
> /usr/lib64/libQt5Quick.so.5.5.1)
> ==13702== by 0x74B08FA: QSGRenderer::renderScene(unsigned int) (in
> /usr/lib64/libQt5Quick.so.5.5.1)
> ==13702== by 0x74C0DDD: QSGRenderContext::renderNextFrame(QSGRenderer*,
> unsigned int) (in /usr/lib64/libQt5Quick.so.5.5.1)
> ==13702== by 0x750ADCA: QQuickWindowPrivate::renderSceneGraph(QSize
> const&) (in /usr/lib64/libQt5Quick.so.5.5.1)
> ==13702== by 0x74DB78A: ??? (in /usr/lib64/libQt5Quick.so.5.5.1)
> ==13702== by 0x74DC890: ??? (in /usr/lib64/libQt5Quick.so.5.5.1)
> ==13702== by 0xA9D041B: QApplicationPrivate::notify_helper(QObject*,
> QEvent*) (in /usr/lib64/libQt5Widgets.so.5.5.1)
> ==13702== by 0xA9D58E5: QApplication::notify(QObject*, QEvent*) (in
> /usr/lib64/libQt5Widgets.so.5.5.1)

probably fixed in qt-5.5.2 nad qt-5.6.0
https://bugreports.qt.io/browse/QTBUG-48799

now i compiling qt5-qtdeclarative package with this patch

SOLVED!

these patches are needed:

for plasma-breeze - https://bugsfiles.kde.org/attachment.cgi?id=96588 (from https://bugs.kde.org/show_bug.cgi?id=357800)

for qt5-qtdeclarative - https://bugreports.qt.io/secure/attachment/51678/0001-Scene-Graph-Fixed-memory-leak-in-QSGBatchRenderer-Re.patch (from https://bugreports.qt.io/browse/QTBUG-48799)

thanks Antony, wojtek

I know it's not exactly appropriate to discuss distro-specifics here, but I know Rex Dieter is subscribed to this bug.
Of the two patches listed here as resolving this (very important) problem, one is for plasma-breeze and the other is for qt5-qtdeclarative. It looks like the latter is only committed against QT 5.6.

Rex (or other Fedora-type person), how can we communicate to the relevant Fedora package maintainers that we need a backport of this for QT 5.5.1 ?
(It looks like KDE bug 357800 for breeze is already in a 5.5.x context, and not a problem in master.)

Backported patches are now included in qt5-qtdeclarative-5.5.1-4 builds (and newer), see also
https://bodhi.fedoraproject.org/updates/?packages=qt5-qtdeclarative

Are there any openSUSE packagers of the QT components that are subbed to this bug report? If so, please backport it please.

As someone who understood some of the words in this thread, thank you to everyone involved in diagnosing and producing a fix for this. I look forward to the update on Manjaro :)

Thanks, Rex!
And thanks to all who diagnosed this.

Still getting this in plasmashell 5.6.4 (ubuntu backported) with insane cpu and ram usage, and often crashes when searching. Not sure if related.

I have quite a similar issue with Archilinux and plasma 5.7.4

Still seeing this with Kubuntu 16.10. plasma-shell using 1GB of RAM by the end of the day. Plasma 5.7.5

I also have this problem on Kubuntu 16.10 and created a new ticket #372384.