Adding options (+=) fail with extended sections (macros)
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
Buildout |
New
|
Undecided
|
Unassigned |
Bug Description
Adding new options using the += notation to an option in a section that is an extended section (using another section as a macro with <= notation) fails if the option in question is defined only in the macro section.
For example, the following doctest
>>> write(sample_
... """
... [buildout]
... develop = recipes
... parts = mypart
... log-level = INFO
...
... [macro]
... recipe = recipes:debug
... foo =
... original_value_1
... original_value_2
... original_value_3
...
... [mypart]
... <= macro
... """)
>>> write(sample_
... """
... [buildout]
... extends = base.cfg
...
... [mypart]
... foo +=
... extended_option_1
... extended_option_2
... """)
Gives
>>> print system(buildout),
Develop: '/sample-
Installing mypart.
foo
<BLANKLINE>
extended_
extended_
recipe recipes:debug
Losing the original values from the ``foo`` option. It should print.
>>> print system(buildout),
Develop: '/sample-
Installing mypart.
foo
extended_
extended_
original_
original_
original_
recipe recipes:debug
It appears that the +=/-= processing takes place before the macro substitution which should be vice versa.
After taking a better look at the zc.buildout code I can see that the +=/-= semantics are implemented in _update_section() function which is called by _open() when configurations are read in. However, the macro expansion is currently done much later by Options. _do_extend_ raw().
I prototyped a different implementation that expands macros as part of the _open() function before it calls _update_section() which seems to partially solve the issue and make. However, the prototype introduces a regression in the ``tests. increment_ on_command_ line`` test because the command line overrides are only applied after all the config files have been read (and therefore _update_section() calls have been made and expanding macros is too late).
This has lead me to believe that at the moment it is not possible to fix this issue by simply moving the macro expansion to an earlier stage in the initialization of the buildout.
I would be willing to work on this but would appreciate some guidance as how to best approach this as some restructuring of the configuration reading logic may be needed.