translations are recompiled when nothing changed

Bug #535906 reported by Sigra
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
widelands
Fix Released
Low
Unassigned

Bug Description

To reproduce:
scons build=release
scons build=release

The build system compiles the translations every time even if nothing changed. So it seems like dependency handling is broken. This slows down development.

Revision history for this message
bedouin (bedouin-users) wrote :

Logged In: YES
user_id=20557
Originator: NO

No it's not broken - it was neither planned nor implemented.

Revision history for this message
bedouin (bedouin-users) wrote :

Logged In: YES
user_id=20557
Originator: NO

I'll try to see what can be done, but it'll be some time.

Timowi (timo-wingender)
Changed in widelands:
status: New → Confirmed
importance: Undecided → Low
tags: added: buildsystem locales
SirVer (sirver)
tags: added: scons
Revision history for this message
Jens Beyer (qcumber-some) wrote :

This also affects cmake, but currently only Release builds run the lang target automatically, so development should not be blocked too much.

tags: added: cmake
Changed in widelands:
assignee: nobody → Jens Beyer (Qcumber-some) (qcumber-some)
Revision history for this message
SirVer (sirver) wrote :

Is there not a way to add dependencies to a target? Locales only depend on the */*po and */*pot files, adding those to the dependencies of make lang should do it.

Revision history for this message
Jens Beyer (qcumber-some) wrote :

add_custom_target is always considered out of date...

Revision history for this message
Jens Beyer (qcumber-some) wrote :

This is probably solvable with an ugly chain of top level target and custom command + custom target per translatable file in worlds, tribes, .....
On the other hand, for development compilation, this is not run automatically. For release compilation I'll try to move this to the end in the timeline of make task.

Revision history for this message
Jens Beyer (qcumber-some) wrote :

Proof of concept in lp:~qcumber-some/widelands/spice-up-cmake
It does not make use of the buildlocale.py and buildcat.py scripts but does everything in cmake.

Changed in widelands:
status: Confirmed → In Progress
Revision history for this message
SirVer (sirver) wrote :

Jens, is this a wise way to go? With Lua landing, a lot of new scripts need to be tagged with a new version, a lot of recursive globbing must be done. This is very easy with python but rather hard with cmake (I guess?). I think the current state of just building it for Release and then always is valid and does not disturb anyone. Comments?

Revision history for this message
Jens Beyer (qcumber-some) wrote :

Well, the idea behind the proof of concept is as follows:

In build15 buildlocale.py calls buildcat.py to update the pot templates, then creates the translations using msgmerge and msgfmt.

With launchpad as translation system, builtcat.py is absolutely not necessary anymore. Remains buldlocale.py.
CMake runs through po directory and collects all files there, regardless of the templates and regardless if the languages for the templates exist. It then creates a command for every po file.
Recursive globbing is easy with CMake... file(GLOB_RECURSIVE ...)

You only need to run cmake again if there are new templates, or new translation catalogs for a certain language. Everything else is handled by make.

You can only have one thing with cmake - caching of results, or being completely independent of file structure changes.
You have the choice :-)

Revision history for this message
SirVer (sirver) wrote :

buildcat.py is necessary to get the new templates compiled that will be pushed to launchpad. But this does not needed to be called by cmake as it is not necessary for the build process. Otoh, the globbing (finding of files and template names) still needs to be done for both scripts in the same way (or maybe I am mistaken here, I am not sure): all Lua files need to be found, all conf files need to be found, all map files and so on. So wouldn't doing thins in cmake as well add another place to add all new pot domains (currently, they only need to get updated in one python file)?

Revision history for this message
Jens Beyer (qcumber-some) wrote :

For the creating of the locale folder nothing of that is necessary, only the po files (and the directory structure also containing the pot files there, as it is at the moment).
The other files (campaigns, worlds, tribes...) are only necessary for (re-)creating the pot files, so this is not superfluous, but it is not needed anymore in the build. It will stay there as a tool for pushing new templates to launchpad, no problem... :-)

so the working process looks like this:

1. Lua lands, new folders are in trunk which need to be translated
2. Someone runs buildcat.py and pushes the new stuff (meaning pot files) in po directory to launchpad.
3. People translate. Launchpad updates translation.
4. Someone pulls translation updates.
5. Run cmake again
6. Run make lang (for Debug) or make (for Release). Everything is translated.

Revision history for this message
SirVer (sirver) wrote :

sounds good to me. But notice that make lang does not work properly for me for out of source builds:

$ mkdir widelands.build
$ cd widelands.build
$ cmake ../widelands
$ make && make lang
$ cd ../widelands && ../widelands.build/src/widelands

game runs, but no locales.

Revision history for this message
Jens Beyer (qcumber-some) wrote :

Well, in your special case you need to specify WL_INSTALL_LOCALEDIR=../widelands.build/locale

Normally it is only defined that "make install" creates a correct filesystem layout, nothing is defined for using the remains of "make (ALL)", this may be different on all systems.
To make it correct, you need to define

$ mkdir widelands.build
$ cd widelands.build
$ cmake -DWL_PORTABLE=true ../widelands -DCMAKE_INSTALL_PREFIX=../widelands.install
$ make lang && make install
$ cd ../widelands.install && ./widelands

Revision history for this message
SirVer (sirver) wrote :

okay. thanks.

Revision history for this message
Jens Beyer (qcumber-some) wrote :

Available in spice-up-cmake branch.

Changed in widelands:
assignee: Jens Beyer (Qcumber-some) (qcumber-some) → nobody
SirVer (sirver)
Changed in widelands:
milestone: none → build16-rc1
Changed in widelands:
status: In Progress → Fix Committed
Revision history for this message
SirVer (sirver) wrote :

Released in build16-rc1

Changed in widelands:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.