Cross build needs rpath with xdeb

Peter, I think this is a valid bug for the default cross-compiler library search path, could you include the exact version of binutils and gcc which you use for cross-compilation?

However, the reason you're failing to cross-compile this particular package with xdeb is due to a missing -ldl in the ncurses build rules; this should be addressed as well, I've opened another task in this bug against ncurses to track this. Would you take care of this?

Offhand, current code adds a -ldl. More details (such as which patch level of ncurses) helps.

Peter, another thing which might influence the build is the linker script of libc; check /usr/lib/libc.so and check whether the one from your cross-libc was properly converted to cross-dirs.

ncurses_5.7+20100313-2ubuntu1

ncurses configure prunes -ldl from LIBS, causing the tic & (at least) background to fail.

For cross ncurses/debian/rules needs to add --with-build-cflags=_GNU_SOURCE to the configure
diff against for both ncurses fixes attached

The -D_GNU_SOURCE is necessary to sort out the headers for the build-host tools make_hash & make_keys when cross building.

Is this only needed when cross-building or should the ncurses build always set -D_GNU_SOURCE when building make_hash & make_keys? What's the error you get otherwise?

Quoting Loïc Minier <email address hidden>:

> Is this only needed when cross-building or should the ncurses build
> always set -D_GNU_SOURCE when building make_hash & make_keys? What's
> the error you get otherwise?

It's needed for cross-building because the configure script can only
do checks for one compile-environment, and the source you're building
with includes headers that rely on the extra checks. When not cross
compiling, those are built using the flags used for the target.

I simplified that last month in this change, making the extra
flags unneeded:

20100522
 + simplify include-dependencies of make_hash and make_keys, to reduce
   the need for setting BUILD_CPPFLAGS in cross-compiling when the
   build- and target-machines differ.

Thomas, I checked the NEWS from latest ncurses dev tarball and it seems a relatively harmless bunch of updates when compared to Ubuntu's current 5.7+20100313, but still quite a few.

Do you see any issue in updating to this version, such as API/ABI changes (didn't see any myself) or things which could regress packages building against ncurses? Any serious known bugs pending a fix that you would know of/would be working on?

If instead you recommend I backport the individual fixes for -ldl and -D_GNU_SOURCE, is there a public VCS for ncurses, or should I diff tarballs to extra these?

Thanks,

On Thu, 1 Jul 2010, Loïc Minier wrote:

> Thomas, I checked the NEWS from latest ncurses dev tarball and it seems
> a relatively harmless bunch of updates when compared to Ubuntu's current
> 5.7+20100313, but still quite a few.
>
> Do you see any issue in updating to this version, such as API/ABI
> changes (didn't see any myself) or things which could regress packages
> building against ncurses? Any serious known bugs pending a fix that you
> would know of/would be working on?

I don't know of any serious/unfixed bugs. Currently I'm working on the
configure script for the Ada95 tree, to incorporate shared libraries there
for the platforms that can support it. Once done with that, I have
unfinished work for the MinGW port (to get the process for making DLLs
working reliably, etc). Neither of those has any nasty problems. The
ongoing changes in working in those areas are rather complicated though.

> If instead you recommend I backport the individual fixes for -ldl and
> -D_GNU_SOURCE, is there a public VCS for ncurses, or should I diff
> tarballs to extra these?

I can supply a copy of the weekly patches, if that helps. Mostly it's the
configure script that you're interested in.

--
Thomas E. Dickey
http://invisible-island.net
ftp://invisible-island.net

Looking again at the default linker script above - the search path has =, so presumably the problem lies in the linker's sysroot configuration, rather than the script entry itself.

Perhaps we need http://sourceware.org/bugzilla/show_bug.cgi?id=10340

(@Thomas: thanks for the advice!)

FWIW, I think I'm hitting the same linker script bug as Peter is reporting with pcre3:
libtool: install: warning: relinking `libpcreposix.la'
libtool: install: (cd /home/lool/pcre3; /bin/bash /home/lool/pcre3/libtool --tag CC --mode=relink arm-linux-gnueabi-gcc -Wall -g -O2 -version-info 15:1:12 -Wl,-Bsymbolic-functions -o libpcreposix.la -rpath /usr/lib pcreposix.lo libpcre.la -inst-prefix-dir /home/lool/pcre3/debian/tmp)
libtool: relink: arm-linux-gnueabi-gcc -shared .libs/pcreposix.o -L/home/lool/pcre3/debian/tmp/usr/lib -L/usr/lib -lpcre -Wl,-Bsymbolic-functions -Wl,-soname -Wl,libpcreposix.so.3 -o .libs/libpcreposix.so.3.12.1
/usr/lib/gcc/arm-linux-gnueabi/4.4.4/../../../../arm-linux-gnueabi/bin/ld: skipping incompatible /usr/lib/libc.so when searching for -lc
/usr/lib/libc.a: could not read symbols: File format not recognized
collect2: ld returned 1 exit status
libtool: install: error: relink `libpcreposix.la' with the above command before installing it
 /bin/bash ./libtool --mode=install /usr/bin/install -c 'libpcrecpp.la' '/home/lool/pcre3/debian/tmp/usr/lib/libpcrecpp.la'
libtool: install: warning: relinking `libpcrecpp.la'
libtool: install: (cd /home/lool/pcre3; /bin/bash /home/lool/pcre3/libtool --tag CXX --mode=relink arm-linux-gnueabi-g++ -g -O2 -version-info 0:0:0 -Wl,-Bsymbolic-functions -o libpcrecpp.la -rpath /usr/lib pcrecpp.lo pcre_scanner.lo pcre_stringpiece.lo libpcre.la -inst-prefix-dir /home/lool/pcre3/debian/tmp)
libtool: relink: arm-linux-gnueabi-g++ -shared -nostdlib /usr/lib/gcc/arm-linux-gnueabi/4.4.4/../../../../arm-linux-gnueabi/lib/crti.o /usr/lib/gcc/arm-linux-gnueabi/4.4.4/crtbeginS.o .libs/pcrecpp.o .libs/pcre_scanner.o .libs/pcre_stringpiece.o -L/home/lool/pcre3/debian/tmp/usr/lib -L/usr/lib -lpcre -L/usr/lib/gcc/arm-linux-gnueabi/4.4.4 -L/usr/lib/gcc/arm-linux-gnueabi/4.4.4/../../../../arm-linux-gnueabi/lib -lstdc++ -lm -lc -lgcc_s /usr/lib/gcc/arm-linux-gnueabi/4.4.4/crtendS.o /usr/lib/gcc/arm-linux-gnueabi/4.4.4/../../../../arm-linux-gnueabi/lib/crtn.o -Wl,-Bsymbolic-functions -Wl,-soname -Wl,libpcrecpp.so.0 -o .libs/libpcrecpp.so.0.0.0
/usr/lib/libm.so: file not recognized: File format not recognized
collect2: ld returned 1 exit status
libtool: install: error: relink `libpcrecpp.la' with the above command before installing it

I've asked Marcin whether we could test that binutils patch you link to, then I shall look at ncurses again, considering the new upstream since it seems to address our cross-build issues.

Ok, my pcre3 issue is probably unrelated; just for the record, I remember I hit this a while ago while building my own cross-toolchains: libtool insists adding -L/usr/lib to the build while libs are only in /usr/$triplet/lib; this causes the linker to try to link against the libs from the build system rather than the target libs first. The linker would actually skip over the x86 /usr/lib/libm.so if it knew it is an x86 lib, but it doesn't unless configured with --enable-target=$(triplet of build system).

I used this gross hack of adding an --enable-target= for the build system back then, simply because the number of upstreams using libtool was too big and I didn't have time to identify a libtool fix. I believe the right way is to solve libtool. The issue with the hack is that it would allow doing the wrong thing when cross-building e.g. armv5 on armv6 or vice-versa, or i686 on i586 etc.

Ah so with just the --with-build-cflags=-D_GNU_SOURCE change and with a binutils patched with the bugzilla patch Peter linked to, I can cross-build maverick's ncurses, confirming that indeed the upstream binutils patch helps.

I tested with the binary cross-binutils package which Marcin built with the patch: http://people.canonical.com/~hrw/binutils-arm-linux-gnueabi_2.20.51.20100617-0ubuntu1hrw2_amd64.deb

I tested http://people.canonical.com/~hrw/binutils-arm-linux-gnueabi_2.20.51.20100617-0ubuntu1_amd64.deb which is Marcin's latest build. It has the workaround I mentioned in comment #16 but it doesn't have the binutils patch mentionned here. As expected, it fixes the pcre3 FTBFS, but it doesn't fix the ncurses issue:
/usr/lib/gcc/arm-linux-gnueabi/4.4.4/../../../../arm-linux-gnueabi/bin/ld: warning: libdl.so.2, needed by ../lib/libncurses.so, not found (try using -rpath or -rpath-link)
../lib/libncurses.so: undefined reference to `dlsym@GLIBC_2.4'

So all expected so far.

I've test the binutils with the "hrw2" version with the ncurses snapshot I mentionned earlier, 5.7+20100626, and after updating the Debian patches to apply I can confirm it cross-builds fine. It does NOT cross-build with the above binutils though, due to missing link with -ldl:
arm-linux-gnueabi-gcc -I../progs -I/home/lool/ncurses/progs -DHAVE_CONFIG_H -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -DNDEBUG -I. -I../include -I/home/lool/ncurses/progs/../include -O2 -g --param max-inline-insns-single=1200 -fPIC -c /home/lool/ncurses/progs/transform.c -o ../obj_s/transform.o
arm-linux-gnueabi-gcc ../obj_s/tic.o ../obj_s/dump_entry.o ../obj_s/transform.o -Wl,-Bsymbolic-functions -I../progs -I/home/lool/ncurses/progs -DHAVE_CONFIG_H -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -DNDEBUG -I. -I../include -I/home/lool/ncurses/progs/../include -O2 -g --param max-inline-insns-single=1200 -fPIC -L../lib -ltic -L../lib -lncurses -o tic
/usr/lib/gcc/arm-linux-gnueabi/4.4.4/../../../../arm-linux-gnueabi/bin/ld: warning: libdl.so.2, needed by ../lib/libncurses.so, not found (try using -rpath or -rpath-link)
../lib/libncurses.so: undefined reference to `dlsym@GLIBC_2.4'
../lib/libncurses.so: undefined reference to `dlclose@GLIBC_2.4'
../lib/libncurses.so: undefined reference to `dlopen@GLIBC_2.4'
collect2: ld returned 1 exit status
make[2]: *** [tic] Error 1
make[2]: Leaving directory `/home/lool/ncurses/obj/progs'

I will attach the full ncurses build log for this failure.

What's odd is that the libncurses.so in the built tree is linked to libdl.so,
Conclusion: with latest ncurses, the patch to binutils to lookup libs in sysroot is needed (hrw2) and the kludge to add x86 support for libtool-based packages doesn't help. I don't think ncurses needs further fixing upstream.

This bug was fixed in the package ncurses - 5.7+20100626-0ubuntu1

---------------
ncurses (5.7+20100626-0ubuntu1) maverick; urgency=low

  * New upstream release
    - Fixes a couple of issues preventing cross-compilation; LP: #598389.
    - Update patch 05-emdebian-wchar to apply cleanly.
    - Drop 08-pkg-config-libdir patch as dpkg >= 1.15.6 don't set
      PKG_CONFIG_LIBDIR anymore.
 -- Loic Minier <email address hidden> Mon, 05 Jul 2010 21:11:31 +0200

So Marcin notes that binutils has sysroot set to /, which is probably incorrect (should be /usr/$triplet for cross-binutils) and that also fixes the ncurses cross-build link issues, so it's probably a better fix than including the bugzilla patch.

This bug was fixed in the package binutils - 2.20.51.20100710-1ubuntu1

---------------
binutils (2.20.51.20100710-1ubuntu1) maverick; urgency=low

  * Update HJ patches from binutils 2.20.51.0.10.

binutils (2.20.51.20100710-1) experimental; urgency=low

  * Snapshot, taken from the trunk 20100710.
    - Fix PR gas/10531. LP: #599485.
  * debian/rules: Introduce PF macros (Jim Heck). Closes: #588357.

  [ Marcin Juszkiewicz ]
  * Don't install documentation files in -cross packages. Closes: #586636.
  * Set sysroot to /usr/$(TARGET) for cross builds. LP: #598389.
 -- Matthias Klose <email address hidden> Sat, 10 Jul 2010 11:42:27 +0200

needs an update in gcc-4.4 and gcc-4.5 too.

no fix needed for gcc-4.4 and gcc-4.5. the sysroot isn't set in the build files, but from the environment.

Attached patch for binutils disables sysroot (if it is not set). This allowed me to compile ncurses without problems.

pcre3 is indeed other problem - there is something need to be done with libtool to make it work.

Accepted armel-cross-toolchain-base into maverick-proposed, the package will build now and be available in a few hours. Please test and give feedback here. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you in advance!

Fixed the maverick tasks; not sure why so many were bogus

I am able to build ncurses for armel using xdeb in maverick chroot with armel-cross-toolchain-base packages in 1.52 version taken from maverick-proposed.

This bug was fixed in the package armel-cross-toolchain-base - 1.52

---------------
armel-cross-toolchain-base (1.52) maverick-proposed; urgency=low

  * build binutils without sysroot support - closes LP: #598389

armel-cross-toolchain-base (1.51) maverick; urgency=low

  * debian/rules: add epoch for libgcc1-dbg version mangling - closes
    LP: #646729
 -- Marcin Juszkiewicz <email address hidden> Wed, 06 Oct 2010 14:46:16 +0200

Do we need to update ia32-libs? Maverick ia32-libs seems to conflict with lib32ncurses5w