Allow Widelands to address more than 2GB memory.

Bug #1249836 reported by Joe
18
This bug affects 3 people
Affects Status Importance Assigned to Milestone
widelands
Fix Released
Medium
Tino

Bug Description

Use the attached save game and map. The game crashes on autosave. Not on every autosave, but approximately on one out of three.
I am using Widelands build 6820 on Windows 7.

Revision history for this message
Joe (joe-kleine-schweiz) wrote :
Revision history for this message
Joe (joe-kleine-schweiz) wrote :
Revision history for this message
Hans Joachim Desserud (hjd) wrote :

Thanks for reporting this issue.

I tried to reproduce it by setting the time between automatic save games to 3 minutes, loaded the game and let it run for ~20 minutes. I did not see a crash, but I suspect it might be related to the enormous size of the map as both saving and loading took quite some time. How much RAM does your computer have, and are you running a 64bit or 32bit OS? Also, do you get some sort of error message or does it simply crash?

tags: added: crash savegame
Revision history for this message
Joe (joe-kleine-schweiz) wrote :

It is Windows 7 64Bit SP1 6GB Memory. For details see attached file

Revision history for this message
Joe (joe-kleine-schweiz) wrote :

Looks like the Process is near the 2GB limit.
Is it possible to compile Widelands with Large Memory support? This would allow 4GB memory for a 32Bit process.

Revision history for this message
Joe (joe-kleine-schweiz) wrote :

You asked about the type of the crash. The game window disappears and the Windows Desktop is shown. No error message is displayed.

Maybe this helps. A play at game speed 3x and with "Bauhilfe" (Construction Help) enabled.

Revision history for this message
Hans Joachim Desserud (hjd) wrote :

> Looks like the Process is near the 2GB limit.

This might be the underlying problem here. Since I'm running Ubuntu I can't really tell for certain, but I know we have had similar bug reports in the past. I tested a bit more with the attached savegame, set autosave to each minute and let it run for a while. No crashes, though I saw that the process exceeded 2.0 GiB after a short while.

Could someone running Windows please check whether they can reproduce this issue?

>The game window disappears and the Windows Desktop is shown. No error message is displayed.

Could you check for a file called stderr.txt in the folder where Widelands is installed (at least I believe that's where it is stored)? Note that it is overwritten each time you start Widelands, but it might log some error message there. Otherwise, WL might print some error if you start it in a terminal (just open cmd and run widelands.exe).

tags: added: windows
_aD (ad-simplypeachy)
Changed in widelands:
status: New → Confirmed
Revision history for this message
_aD (ad-simplypeachy) wrote :

Confirmed on Windows 7 SP1 64-bit, 6GB RAM total, 1.5GB free at time of Widelands crash. I used the reporter's autosave. The Widelands window closed and no error message was shown, but stderr contains:
PANIC: unprotected error in call to Lua API (not enough memory)

Attached is detail of the process's performance data around the time it crashed. It was using 1.6GB RAM.

Revision history for this message
Joe (joe-kleine-schweiz) wrote :

I could play Widelands for more than one hour after applying a small patch to the widelands.exe. See attached file.
The patch enables the 32Bit Widelands.exe to use 4GB RAM on a Win7. Useally this done with a Linker option.
VC++ Linker option LARGEADDRESSAWARE
<http://msdn.microsoft.com/en-us/library/wz223b1z.aspx>

Another way is to use the EDITBIN Tool from VC++ Compiler:
EDITBIN /LARGEADDRESSAWARE [PATH_TO_WIDELANDS_EXE]

Revision history for this message
SirVer (sirver) wrote : Re: Widelands needs more memory than 32bit can address.

Sounds interesting - it seems a bit risky to do this for b18, but we should find a way to do this in cmake after b18.

Changed in widelands:
milestone: none → build19-rc1
summary: - Widelands crashes on autosave
+ Widelands needs more memory than 32bit can address.
Changed in widelands:
importance: Undecided → Medium
Revision history for this message
Tino (tino79) wrote :

I can reproduce it, too: Autosave every minute, run at 3x speed it will crash at about the 4th autosave.
But the memory usage at this point is only ~1,6GB, so quite a bit away from 2GB.

I think the issue is more the lua persistancy:

Autosave: interval elapsed (60 s), saving
Game: Writing Preload Data ... took 88ms
Game: Writing Game Class Data ... took 0ms
Game: Writing Player Info ... took 541ms
Game: Writing Map Data!
Writing Elemental Data ... took 1ms
 Writing Player Names And Tribe Data ... took 0ms
 Writing Port Spaces Data ... took 5ms
 Writing Heights Data ... took 68ms
 Writing Terrain Data ... took 221ms
 Writing Player Start Position Data ... took 0ms
 Writing Player Message Data ... took 126ms
 Writing Resources Data ... took 247ms
 Writing Map Extra Data ... took 0ms
 Writing Map Version ... took 1ms
 Writing Allowed Worker Types Data ... took 0ms
 Writing Flag Data ... took 37ms
 Writing Road Data ... took 7ms
 Writing Building Data ... took 19ms
 Writing Area Watchers Data ... took 276ms
 Writing Map Objects ... took 1277ms
 Writing Flagdata Data ... took 27ms
 Writing Roaddata Data ... took 104ms
 Writing Buildingdata Data ... took 71ms
 Writing Node Ownership Data ... took 26ms
 Writing Exploration Data ... took 98ms
 Writing Players Unseen Data ... took 1200ms
 Writing Scripting Data ... took 13317ms
 Writing Objective Data ... took 0ms
 Map_Saver::save() took 17128ms
Game: Writing Map Data took 17816ms
Game: Writing Player Economies Info ... took 60ms
Game: Writing Command Queue Data ... took 14910ms
Game: Writing Interactive Player Data ... took 1ms
Game_Saver::save() took 33418ms
SaveHandler::save_game() took 33430ms
Autosave: save took 33420 ms

It takes really a long time to save, and most time takes the Scripting data and the command queue.
And the crash occurs (at least here), when the scripting takes place, last log entry:

 Writing Scripting Data ...

Full log attached.

Revision history for this message
Tino (tino79) wrote :

Ok, my fault. Having a closer look at the memory stats, it does peak at over 2GB.
And compiling with the flag enabled makes it just run and save fine (the savetimes still remind me to get an SSD ;) )

I've pushed a branch with the needed cmake changes, in my opinion this could be merged before b18, it should not break anything i think.

Revision history for this message
SirVer (sirver) wrote :

Remember that one time when we changed a picture during feature freeze and it broke something for a few people (because it had a palette vs being rgb or so). I am not risking this :).

Tino (tino79)
Changed in widelands:
assignee: nobody → Tino (tino79)
milestone: build19-rc1 → build18-rc1
status: Confirmed → Fix Committed
tags: added: win32
summary: - Widelands needs more memory than 32bit can address.
+ Allow Widelands to address more than 2GB memory.
Revision history for this message
SirVer (sirver) wrote :

Released in build-18 rc1.

Changed in widelands:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.