Wanted: a linkkit to produce standalone applications

rtoy explains:

"Fred came up with the original idea, but at the time he only knew how to
do it by creating a custom linker script. I talked to him about it and
figured out gcc with a few special arguments would do the same thing.
That allowed me to extend the idea to work on solaris/sparc and even Mac
OS X.

The C side of the stuff is in src/lisp/elf.c and src/lisp/mach-o.c and
src/lisp/save.c, with a little bit of help from the lisp side in
src/code/save.lisp. elf.c and mach-o.c dump the three spaces into valid
elf or mach-o object files.

The script that does the linking is src/tools/linker.sh. It takes the
arguments created by save.c and does the right thing on the various
supported architectures and calls gcc in the right way to tell the
executable that an image is attached and to append the three spaces onto
the end of the C executable."

NB: the current way to checkout CMUCL, not well documented, is:
cvs -d common-lisp.net:/project/cmucl/cvsroot checkout src

i might be way off with this and/or missing key constraints...

but why not just store the .so's in the image, and upon startup write them to /tmp/ and load them from there with absolute paths?

@Attila: your "solution" is about just as much work, and not as effective in the end (not to tell about the security nightmare).

After discussing with pkhuong on #sbcl today, I find that trying to relink an executable can be tricky in several ways I didn't foresee.
http://ccl.clozure.com/irc-logs/sbcl/2011-12/sbcl-2011.12.01.txt

1- If you want to reuse existing image files, you want some essential code / data addresses used by sbcl itself to be fixed, which supposes that they be put in their own sections, separate from normal C text and data sections, and not offset by object file headers or non-text/non-data segments. On a GNU platform, a linker script can then make sure these things are in front and/or at a fixed address. On a BSD platform (at least for darwin), it seems you can't fully control what "in front" means, and that you must put things at a fixed address using -segaddr.

2- Things are actually worse, in that if you want to be compatible with an existing image, you can only *add* things, and you may have to add new code and data sections to the end of the existing object (e.g. at the end of it, assuming it doesn't conflict with the heap). For that you must either have kept all the objects with which you build the previous image, or somehow be able to uncompile the current executable into a linkable object. Back in the days, ELK used to do just that using a.out and GNU ld. I don't know how that is portable to other platforms or not. Also, whether you want contiguous zones for each of text and data, or alternate pages of text and data, that will be tricky to manage.

3- Another easier solution might be to regenerate an image using slam.sh, which supposes access to a writable SBCL source directory and all the right shell parameters. Painful, but possible.

4- Yet another solution would be to somehow maintain linking information in the lisp image, and re-link everytime the base C image changes. I have recollections of having been told CMUCL doing something like that back in the day, but that was old second hand information.

One quick-and-dirty option would be to teach save-lisp-and-die how to append shared objects to the core, and dump them in /private/tmp or similar on startup.

akovalenko told me (see same #sbcl transcript) that he already did option #4 in his Windows port of SBCL, effectively implementing a linkkit for his own SBCL porting purposes, so he could frequently recompile the runtime without having to regenerate a core.

There are implications for the FFI layer, and clients (e.g. contribs, CFFI, etc.) may have to be modified to support relinking. akovalenko would know the details.

SBCL already does the linkage-table stuff for shared objects loaded by users (based on the CMUCL code you refer to).

What Anton has done, and which enables relinking the runtime, is getting rid of /all/ static foreign addresses, and always going
through the linkage table.

This bug should probably be closed now. Bazel demonstrated a solution, but it needs to be made more portable. See https://bugs.launchpad.net/sbcl/+bug/1500628

Bug reporter says to close, so let's close.