Atlanteans mission 1 needs a lot of memory

What makes you think that this is a memory leak?

The rise in memory could just be due to more animations being loaded into memory. Also, how do you measure memory? If you use Activity manager or top or something external, the total will be misleading since the kernel will not take memory away from processes unless there is memory pressure - even if the program frees memory.

To be sure, this needs to be tested with MSAN or Mac Os X instruments leak detector, efence or similar.

Well I have joined the same game to ensure you can reproduce the memleak. This games uses near 200MB in all games I can launch, except the saved one. But you know the code, only you know if this is plausible.

I also think, that there is a memory leak.
I just found one small tool memleax and this is the result:

   0x0000000000e4344d widelands _ZN9Widelands15CmdLuaCoroutine7executeERNS_4Ga()+53 /var/widelands/BZR/production_statistics/src/ui_basic/panel.h:130
    0x0000000000e44901 widelands _ZN9Widelands8CmdQueue9run_queueEiRj()+427 /var/widelands/BZR/production_statistics/src/ui_basic/panel.h:130
    0x0000000000d34427 widelands _ZN9Widelands4Game5thinkEv()+187 /var/widelands/BZR/production_statistics/src/ui_basic/panel.h:130
    0x0000000000f6a105 widelands _ZN15InteractiveBase5thinkEv()+595 /usr/include/c++/6.2.1/bits/stl_iterator_base_funcs.h:103
    0x0000000000f7adba widelands _ZN17InteractivePlayer5thinkEv()+34 /usr/include/c++/6.2.1/bits/stl_iterator_base_funcs.h:103
    0x0000000000ec3fe5 widelands _ZN2UI5Panel8do_thinkEv()+51 /var/widelands/BZR/production_statistics/src/ui_basic/panel.h:130
    0x0000000000ec388b widelands _ZN2UI5Panel6do_runEv()+647 /var/widelands/BZR/production_statistics/src/ui_basic/panel.h:130
    0x0000000000c25a30 widelands _ZN2UI5Panel3runINS0_11ReturncodesEEET_v()+24 /var/widelands/BZR/production_statistics/src/base/vector.h:30
    0x0000000000d34119 widelands _ZN9Widelands4Game3runEPN2UI14ProgressWindowEN()+2225 /var/widelands/BZR/production_statistics/src/ui_basic/panel.h:130
    0x0000000000d33584 widelands _ZN9Widelands4Game13run_load_gameERKNSt7__cxx1()+1162 /var/widelands/BZR/production_statistics/src/ui_basic/panel.h:130
    0x0000000000c2054a widelands _ZN13WLApplication9load_gameEv()+238 /usr/include/c++/6.2.1/bits/hashtable_policy.h:2009
    0x0000000000c1f69e widelands _ZN13WLApplication21mainmenu_singleplayerEv()+130 /usr/include/c++/6.2.1/bits/hashtable_policy.h:2009
    0x0000000000c1f088 widelands _ZN13WLApplication8mainmenuEv()+346 /usr/include/c++/6.2.1/bits/hashtable_policy.h:2009
    0x0000000000c1bbbc widelands _ZN13WLApplication3runEv()+732 /usr/include/c++/6.2.1/bits/hashtable_policy.h:2009
    0x0000000000c1a231 widelands main()+107 /usr/include/c++/6.2.1/bits/hashtable_policy.h:2009

CallStack[49]: may-leak=497 (27832 bytes)
    expired=497 (27832 bytes), free_expired=0 (0 bytes)
    alloc=3143 (176008 bytes), free=0 (0 bytes)
    freed memory live time: min=0 max=0 average=0
    un-freed memory live time: max=11

I would say it points to ui_basic/panel.h:130 what is weird, but who knows... :)

Tibor: that seems spurious - it is unlikely that the standard library is leaking memory. I rather believe that memleax cannot deal with virtual objects. On Linux,I think MSAN and valgrind are the only reliable tools.

Looking at output of instruments in the game I can confirm that ever more memory is allocated. But it is not leaked, it is actively used by the process and all memory is still reachable. Backtraces have not been useful - it seems l_malloc is the responsible function, but there is no context in Instrument. This probably requires building widelands with a specific apple memory library to get more useful information.

Klaus, would you be interested in trying out where the memory goes?

I can confirm that memory usage raises with this savegame, just by open the game and doing nothing. Btw, it is the atlantean campaign "From Nemesis to Genesis".

I run this with valgrind (what a pain playing with 5fps :-D), but imho it shows no leaked memory responsible to this savegame... for comparison i ran widelands under valgrind and opened just the options menu and switched a bit between the tabs.

The increasing memory is annoying anyway.

Uhm, Ill try this (on OSX), first with top, then with XCode.
No idea wat I can find out, but thanks for involving me.

OK, I will try Build 19 first, but I have only 64 Bit.
Perhaps the 32 Bits result in a Problem, mhh.

Perhpas I can spare some Time at the weekend.

I can reproduce the ever rising memory with the attached savegame easily and I have 64bit Mac OS X too, so that should be fine.

Instruments (which comes with XCode) is really nice for this sort of thing: You can start widelands, loading the game, then use Instruments to 'attach' to the running process and inspeat its memory allocation/freeing behavior. I remember starting with this documentation when I first tried this:

https://developer.apple.com/library/content/documentation/Performance/Conceptual/ManagingMemory/Articles/FindingLeaks.html

Mhh, I playesd this on my selfcompiled build-19 (debug)
and leaks just reports

Process 20148: 3 leaks for 64 total leaked bytes.

Thats not 100% perfect, but it does not increase over time
wideland starts with about 300M, goes up to some 400 but now
is down to 371M (I think all the buildings are lost meanwhile).

(I use top -o mem -s 3 to monitor this)

as this is a spanish (or portuges?) game I am bit lost how to get progress :-)

with my "default" MacPort build I reach 631M+ peak and go down to 318M+ again.

0 leaks for 0 total leaked bytes.

Now checking the "old official" OSX build from SirVer (with the broken ctrl-key mapping)

Process 20600: 0 leaks for 0 total leaked bytes.

OK, here it raises up to 787M but hen goes down to 544M+ and even 366M-

I did play a bit to escape the flood in all three games.

Could you please describe in more detail what you did (or not did).

I have lua @5.3.3_0 (active)

So _I_ can not confirm this.

My best Guess is this must be some of the Libraries we use.
As this is a scripted scenario I would search th Problem somewhere around lua.

lists-jjorge can you build this with some special malloc tracing glibc?
I am sure Linux has such tools but not what exactly I can suggest for you.

Le 18/01/2017 à 19:18, Klaus Halfmann a écrit :
> Could you please describe in more detail what you did (or not did).

I did nothing : just run the saved game without any click. Now it was on
a 1GB hardware, so I could not afford leaving it run till memory
decreases, I killed it before.

Really weird that you cannot repo, Klaus. I will retest more thoroughly on my system.

I investigated more closely and can now confirm what Klaus already found: there is no leak.

But the scenario requires a lot of memory: Virtual memory (VM) peaks here on my system at 2.31 GB, all allocated by Lua. You can see in the attached screenshot how occasionally the Lua garbage collector gets some memory back.

I went ahead and hacked into the loaded savegame the following Lua coroutine:
run(function()
   while true do
      print("doing gc")
      collectgarbage()
      print("now: ", collectgarbage("count"))
      sleep(20000)
   end
end)

This forces a full Lua garbage collection through all objects every 20 in-game seconds (see picture of next post). The good news is: Memory usage stays below 150 MB at all times. The bad is that the pauses of the GC are noticeable (~1-2 seconds) and very annoying.

Looking through the code I see that I implemented a Set and a Triangle Lua 'class' and there is a class called WaterRiser which handles all of the logic of making the water rise. It keeps a Set of ocean tiles (_ocean) and a Set of shore tiles it needs to flood next (_shore). So at the end, we have for each Triangle of the map an entry in the _ocean set.

Also for each flooded tile, we check all of its neighbor triangles - there we create two tables with 3 Triangle each, i.e. ~10 or so Objects. I think this yields many, many objects and the GC does not properly investigate them and delete them.

The summary is therefore:
We create too many Lua objects, the GC of lua is not clever enough to free them on its own. Calling GC manually is too expensive.

Possible solutions:
1) first step would be: make Triangle not be a Lua class with a metatable, instead make all of its 'method' stand alone functions that get the data passed in.
2) pull some of the logic of this scenario into C. For example we could totally have a Triangle representation in C++. While we would still have as many objects, those objects would not be as wasteful as the Lua tables.
3) Investigate if Lua tables can be created smaller, i.e. give it the exact number of keys we expect it to have. This could save bytes, but it would probably not help with the GC being too stupid.

Memory over time with manually triggering GC periodically.

Another short code reading session makes me believe that Set is the main culprit: we use a Lua table t and putting something into the table is t[item._hash] = item and removing it is t[item._hash] = nil. Apparently the garbagce collector is not clever enough to collect this efficiently.

Setting to low since this has not been an issue before - so people seem to have a lot of memory these days. Also unassigning myself, since I do not have the time to work on this atm.

Same issue reported on the forum for the new Frisians scenario:

"There is one spot of bother. The stormflood consumes lots of memory; the further it progresses, the more memory is flooded. In the end, it sometimes gets so bad that Widelands no longer responds and I have to force-close it. I´m unable to find the reason for this…"

So, we should try to do something about this for Build 20.

Migrated to https://github.com/widelands/widelands/issues/1040