[Exherbo-dev] Multiarch discussion

Saleem Abdulrasool compnerd at compnerd.org
Sat Feb 8 21:54:09 UTC 2014

On Wed, Feb 5, 2014 at 8:49 PM, Michael Forney <mforney at mforney.org> wrote:

> Hi,
> While multiarch is getting closer, I think there are still a couple of
> things that need discussion.
> - PT_INTERP path of multiarch executables
> Currently, we patch gcc in order to use a special path for the PT_INTERP
> field of the resulting executables. This has the benefit of ensuring
> that none of the dynamic loaders clash (as they are in architecture
> specific lib directories).
> However, it also has the downside of breaking binary compatibility with
> other distributions (executables built on an Exherbo multiarch system
> will not run on any other non-Exherbo system). I'm strongly of the
> opinion that PT_INTERP should be left as /lib/ld-whatever.so for the
> following reasons:
>     1. The ld.so names on the architectures most people care about do
>        not conflict. From the gcc sources, I've come up with this list:
>        aarch64             /lib/ld-linux-aarch64.so.1
>        ia64                /lib/ld-linux-ia64.so.2
>        arm-hardfloat       /lib/ld-linux-armhf.so.3
>        arm-softfloat       /lib/ld-linux.so.3
>        x32                 /libx32/ld-linux-x32.so.2
>        x86_64              /lib64/ld-linux-x86-64.so.2
>        alpha               /lib/ld-linux.so.2
>        i386                /lib/ld-linux.so.2
>        sparc               /lib/ld-linux.so.2
>        sparc64             /lib64/ld-linux.so.2
>        sh                  /lib/ld-linux.so.2
>        If someone really does want to add alpha, sparc, or sh support to
>        Exherbo, these cases could be worked around in a case-by-case
>        basis; e.g. we could change PT_INTERP for sparc to
>        /lib/ld-linux-sparc.so.2
>     2. Retain binary compatibility with other systems.
>     3. We no longer need to create a /lib symlink, or manage a bunch of
>        symlinks of dynamic loaders across libdirs for different
>        architectures, or whatever is planned.
>     4. We don't have to patch gcc as much (only the
>        native_system_header_dir change is required from the current
>        multiarch patch).
> In any case, even if we do decide to continue using a special path for
> PT_INTERP, I think that it is a major enough change to warrant some more
> discussion (before it is difficult to change).

I would be strongly against changing this aspect of the implementation that
I proposed and prototyped.

The compatibility that you are after is pointless.  Any sufficiently
complex software is already unable to run cross-distribution.  There are
subtle differences between the libc used in various distributions that
prevent this already.  If you take a binary from RHEL and try to run that
on Ubuntu and you are going to run into quite a few issues, and some of
them quite atrocious to track down being caused by the differences in libc

Secondly, you didnt consider the subtleties of my proposal.  Multilib is a
horrendous hack that GCC has implemented and this change intends to repair
the damage from that.

You must first canonicalise the paths: everything goes to /lib, trim
dirname.  Second, canonicalise the basename arc: drop the DSO version

So what you are left with at the end of this exercise:

AArch64: ld-linux-aarch64.so
IA-64: ld-linux-ia64.so
ARM HF: ld-linux-armhf.so
ARM SF: ld-linux.so
x32: ld-linux-x32.so
x86-64: ld-linux-x86-64.so
Alpha64: ld-linux.so
x86: ld-linux.so
SPARC: ld-linux.so
SPARC64: ld-linux.so
SH: ld-linux.so

Now, you didnt consider MIPS.  Lets look at that: {LE, BE}, {32, 64},
{OABI, NABI} (for the sake of simplicity, Im ignoring the fact that there
is actually o32, n32, n64).  That is 2 * 2 * 2: 8 combinations.

Now, you didnt really consider ARM 32.  You listed HF, SF.  Thats
uninteresting.  Lets really look at ARM32: {LE, BE}, {EABI, GNUEABI},
{Soft, Soft Float, Hard Float} that is 2 * 2 * 3: 12 combinations.

You also glossed over PPC32 and PPC64.

At the very least the ABI is something which can be switched.  MIPS (with
the exception of the R8000) and PPC can also switch endianness!  The
current cross implementation makes handling this a breeze: just switch a
symlink and be on your way.

What you are suggesting is to forego this flexibility in order to retain
"compatibility" with other distributions.  This "compatibility" that doesnt
even allow you run all applications from another distribution or
vice-versa.  And to propagate a change that even GCC has said that they
regret.  So I restate, I would be *EXTREMELY* reluctant to give up this

- build/host/target flags/tools
> Currently there is no way to determine the correct settings for tools
> and flags for the build or target architectures. As a workaround,
> several packages assume build and target tools have a prefix of
> ${arch}-. Additionally, we don't get the variable inheritance that
> ebuild.bash provides for the host system (CFLAGS being prepended to
> CPPFLAGS). As far as how cross-compiling is currently set up in paludis,
> I'm not sure how to solve this issue, because tool_prefix is set in the
> installed repository config (and the build/target system installed
> repository is not involved in the installation to a host installed
> repository).

I agree that the assumption that ${arch}- is the tool prefix is incorrect.
 That is something that exheres should be able to query from paludis.
 However, that is something that we can do once we are certain that we have
the desired infrastructure.

The CFLAGS handling is an ugly wart in the system that is the artifact of
how the prototype was built.  I based the build infrastructure heavily
based upon the excellent work that zlin and company did when I initially
brought up multibuild (multilib) on exherbo.  cross does not follow the
same build mentality, but rather relies on paludis targets for targeting
each host.  This is obviously something that needs to be revisited.  One
thought that comes to mind is to use a target hierarchy for environments,
and introduce the concept of a default target.  You could imagine something

/etc/paludis/default -> /etc/paludis/i686-pc-linux-gnu

(I use this as an example since this is pretty close to my setup).

Implementation details aside, I think a tool that acts like this would
> be neat:
>     $(exarch build)
>     # returns x86_64-pc-linux-gnu
>     $(exarch host)
>     # returns i686-pc-linux-gnu
>     $(exarch build CC)
>     # returns ${x86_64_pc_linux_gnu_CC}
>     $(exarch build CFLAGS)
>     # returns ${x86_64_pc_linux_gnu_CFLAGS}
>     $(exarch arm-exherbo-linux-gnueabi CPPFLAGS)
>     # returns ${arm_unknown_linux_gnueabi_CPPFLAGS} prepended by
>     # ${arm_unknown_linux_gnueabi_CFLAGS}. Intended to be used by
>     # packages which support target suboptions like gcc.

This looks quite reasonable to me.

> - paludis changes
> I know that the cross-compiling branch is still a WIP, but I wonder what
> the plan is for this. It is quite inconvenient to have to manage a bunch
> of configs/environments in order to cross-compile for multiple hosts.

I have no intentions of merging this into master until cross is deemed
ready to merge into master.  The work that is on the WIP branch is specific
to cross.  The parts work that I did to enable some of this is now merged
to master.  Unless there is some other functionality that is enabled by the
current work on the WIP branch, I would rather leave it there.  This is
primarily driven by the motivation to not unnecessarily cause churn on
master until we are sure that we are happy with the infrastructure for

> Also, dependency resolution for multiarch packages doesn't take into
> account the architecture of installed packages. This makes users have to
> do most dependency resolution for cross-compiled packages by hand at the
> moment.
> I see a brief mention of some necessary dependency syntax changes in the
> TODO section of the multiarch migration guide, but I don't quite
> understand the proposal. Maybe someone can expand upon this?

There was no concrete proposal that I put forth for this, only vague
mutterings that I think are some things that we should consider to improve
the infrastructure for cross.  If there is a particular point that is
unclear to you, I can try to expound on that.

> --
> Michael Forney <mforney at mforney.org>
> _______________________________________________
> Exherbo-dev mailing list
> Exherbo-dev at lists.exherbo.org
> http://lists.exherbo.org/mailman/listinfo/exherbo-dev

Saleem Abdulrasool
compnerd (at) compnerd (dot) org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.exherbo.org/pipermail/exherbo-dev/attachments/20140208/3aa8658b/attachment.html>

More information about the Exherbo-dev mailing list