How to build 32-bit application on powerpc64(target:power8) system?

Public Discussion
#1

To build a 32-bit application on a powerpc64le system(target:power8), you need the 32-bit development libraries, such as glibc, which are not available as precompiled packages for powerpc64le. Therefore, you need to build the 32-bit glibc library from source. However, building the 32-bit glibc library depends on the 32-bit libgcc.a library. This creates a circular dependency issue.

how to solve this?

When building glibc using the command:


../configure --prefix=/usr/glibc-powerpc32 CC="gcc -m32 -mcpu=power8 -mtune=power8" --disable-werror --disable-multi-arch --host=powerpc-linux-gnu"

the final linking requires the 32-bit libgcc.a library. Therefore, i need to build the libgcc.a library from source code. However, when compiling GCC with the --enable-multilib configuration option, i encounter the error:

/usr/bin/ld: skipping incompatible //usr/lib64/libc.so when searching for -lc
/usr/bin/ld: cannot find -lc
/usr/bin/ld: cannot find crtn.o: No such file or directory

This error is caused by the absence of the 32-bit glibc library, creating a circular dependency issue.

how to resolve this?

#2

you haven’t installed a 32-bit and/or multilib version of glibc6. you will need that first.

can i suggest getting the debian gcc multilib source package and compiling it up for ppc64/ppc32?

note how that depends on gcc-10-multilib?

now note how THAT depends on BOTH lib32gcc-dev AND lib64gcc-dev, AND libc6-dev-amd64 AND libc6-dev-i386?

i.e it is sufficiently complex that i strongy recommend you just use powerpc64-gnu-linux-gcc with the “-m32” option unless there is a really compelling reason to double-compile (aka multilib).

l.

#3

I think that the simple answer is you can’t. All supported distros for P8 are LE and 64-bit only. This was the distros deciding what they were willing to do,

You might be able to find a back-level BE distro that will install and run on P8. I think RHEL7, CentOS 7, Fedora 30, (maybe) Ubuntu 14 (or old debian). any newer version will be LE 64-bit only.

Or you would have to build a full cross compiler and runtime and run it QEMU. Not for the faint of heart.

But I have to ask why 32-bit?

#4

Is your core target a 32b implementation or a real Power core? There are a couple OpenPower 32b soft-cores running bare-metal code like Litex, Micropython, etc. using crosscompiling on Intel boxes.

#5

there are? i thought there was only the one - chiselwatt. are there any others? i’d love to know in order to study them.

#6

I am building 32-bit application on a ppc64le (power8) platform running CentOS 7.9.
The gcc-xxx-multilib package is not available on this platform, so, it needs to be built multilib gcc from source code.

#7

then you can use the “-m32” option to build all software needed.

you have come to a mistaken conclusion.

the mistake that you have made is to assume that because you are on a 64-bit system it is necessary to build the program to run on both a 64-bit system and a 32-bit system.

you do not.

you can in fact compile the program for a 32-bit system and run it under a guest 32-bit operating system hosted under QEMU, set up to emulate 32-bit hardware.

or, you could get some actual 32-bit Power ISA hardware (such as a Powerbook G4 notebook - PowerPC G4 - Wikipedia) although bear in mind these are typically big-endian CPUs.

if you really really absolutely must build and run on the 64-bit hardware you will need to begin by creating the full cross-compiler and cross-compiled libraries to “thunk” between the 64-bit system and the 32-bit userspace, which is why i referred you to the debian cross-compiler and cross-libc6 software.

if you cannot move off of CentOS 8 then i recommend that you install QEMU then run debian for ppc64 under QEMU on the CentOS 8 host, and begin from there.

basically, what you are attempting to do (run 32-bit) has not been done in a long time, and it is not completely straightforward. there does exist a debian-ppc community that continues to build 32-bit PPC OSes for the older hardware (Apple G4 macbooks) but it is BE-only Debian -- PowerPC Port and they stopped “officially” supporting beyond Debian 8.

the other alternative idea you might like to consider is finding a 32-bit version of CentOS and/or Fedora and/or any other OS and actually boot the hardware that you are currently using on that OS 10 Best PowerPC Linux Distros for 2023 with Installation Links!

this will get you native 32-bit gcc (an older version) directly on the hardware that you are using and the problem of attempting to compile multilib support and multilib cross-compiled libraries will “automagically disappear”

#8

yuu i think it would be a good idea to let us know what the target 32-bit Power ISA CPU actually is. this is part of your goal but if we do not know what your goal is we cannot really help you avoid assumptions that you might have made (in a reasonable timeframe), nor really give you the correct advice because we have to guess what your goal is, and that’s not fair for you or fair for anyone trying to help you.

if you can let us know the make and model of the CPU that you are attempting to build for then we can properly and promptly give you good and experienced advice.

#9

You are running a centos 7.9 big endian kernel and 64bit runtime on a power 8 system. This implies a gcc 4.8 compiler.

It is definitely NOT little endian.

Very very old. Not fully enabled for P8. Barely supports power7.

I forget when rhel stopped supporting 32 bit.

The kernel should support concurrent 32 and 64 apps in separate processes.

You may have to build the full 32bit stack yourself.

#10

My goal is to build a 32-bit library on a ppc64le system so that other 32-bit applications dependent on that 32-bit library can run properly on the ppc64le system.
os : Linux redhat 3.10.0-1160.el7.ppc64le
cpu: power8

#11

ooo that’s fascinating :slight_smile: ok it’s very similar to the “x32” project, except ported (Power not x86) onto ppc64le-ppc32le (or ppc64-ppc32). which is why i mentioned the debian cross-libs + cross-compiler.

the first problem is:

which endian-ness? 32-bit LE applications? or 32-bit BE applications? and are they “legacy” (or pre-compiled, lost the source code) or can the 32-bit applications be fully recompiled from source?

if you have the source code and can recompile them then you can ensure that the endian-ness is the same. without that, you are going to have to… deep breath: use the same endian-ness which means if the binaries are ppc32-BE you need to download and run “Linux redhat 3.10.0-1160.el7.ppc64”, even if it is run in a Virtual Machine (KVM / Hypervisor) on the POWER8 system… last resort under qemu.

btw you might find it easier to start all this from PowerEL PowerEL - Enterprise Linux distribution for Power users
it is like CentOS used to be (i.e. not owned and controlled by Redhat). maintained by Toshaan Bharvani of Vantosh Ltd as a FOSS distro. Vantosh provides both LE and BE versions of PowerEL 7.

the second issue is: again, to reiterate: a multilib compiler won’t help you. as Steven mentions you need the full 32-bit stack built up, yourself. How to build 32-bit application on powerpc64(target:power8) system? - #9 by munroesj52

does the kernel properly / easily support 32/64 systemcalls with no extra software or messing about? so just cross-compiling a static-linked ppc32le binary, and running it on a ppc64le OS (or ppc32be-ppc64be) would that “just work”?

if so that’s an awful lot easier than having to mess about like in the x86 world with x32 / multilib / multiarch!

#12

Thank you very much for your response.

It is a 32-bit LE application, and the source code is missing. If I have the source code, I only need to rebuild it on the PPC64LE platform.

Yes, so I want to build a 32-bit stack on a ppc64le (cpu: power8) system. It includes libraries such as libc,libgcc,etc.
However, I’m facing the dependency issue as described earlier - the 32-bit glibc depends on the 32-bit libgcc.a, and the 32-bit libgcc.a depends on the 32-bit glibc. This is causing me to be unable to build the 32-bit stack.

#13

ahh okaaaay so this is a very specific problem that you’ll need to solve. you will need to run the following command:

/usr/bin/ldd {the-32-bit-binary}

(it might need to be powerpc64le-linux-gnu-ldd - ymmv)

if you run that on bash you get:

lkcl@fizzy:~$ ldd /bin/bash
        linux-vdso.so.1 (0x00007ffdaabf2000)
        libncurses.so.5 => /lib/x86_64-linux-gnu/libncurses.so.5 (0x00007f9863de1000)
        libtinfo.so.5 => /lib/x86_64-linux-gnu/libtinfo.so.5 (0x00007f9863bb7000)
        libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f9863bb2000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f98639f2000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f9864053000)

this will tell you exactly which libraries you will need to get the source code for, and their version numbers. if you know what OS the binary was running on previously, then you can find out the exact version of all libraries that way.

if it was statically-linked you’re in a whole lower world-of-pain :slight_smile:

then once built and put into a subdirectory (say ~/ppc32le_libs) you can do:

LD_LIBRARY_PATH=~/ppc32le_libs {nameofthe32bitexecutable}

and if Steven is correct about there being syscall support for 32-bit ppc32le binaries on 64-bit ppc64le linux OSes it should just work.

note that you do not need multilib gcc to do this, but given that the binary is so old you may have to build gcc-4.8 (or whatever compiler was originally used to build the executable) in order to then build the missing dynamic libraries (if any)

use “export LD_LIBRARY_PATH={32bit_library_location}” and/or add the location where the 32-bit libraries have been installed, to /etc/ld.so.conf, and re-run “/sbin/ldconfig”. check the library(s) has(have) been picked up with “/sbin/ldconfig -p”.

then when you re-run /usr/bin/ldd {the_32bit_binary} it should also show up.

#14

you can also try the file command:

$ file /bin/bash
/bin/bash: ELF 32-bit MSB executable, PowerPC or cisco 4500, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.32, BuildID[sha1]=8b357202db38023077c7e9b8739b0cf30afecfce, stripped

This is Ubuntu 14.04 LTS on a PowerMAC G5. This a ByARCH system that supports both 32-/64-bit application runtimes. Also from the G5.

$ file /lib64/libc-2.19.so
/lib64/libc-2.19.so: ELF 64-bit MSB shared object, 64-bit PowerPC or cisco 7500, version 1 (SYSV), dynamically linked (uses shared libs), BuildID[sha1]=c32e7ede499c50dc60805f32fc762d39bb2a094f, for GNU/Linux 2.6.32, stripped

This (and Fedora 27, RHEL7/Centos7.x) was the last of the big endian ByARCH disto releases. To my knowledge all the PowerMACs were Big Endian.

After that 64-bit Little Endian only, ByARCH was eliminated as cost savings (no one who cared had any 32-bit LSB applications they need to run on POWER systems):

$ file /usr/bin/bash
/usr/bin/bash: ELF 64-bit LSB pie executable, 64-bit PowerPC or cisco 7500, OpenPOWER ELF V2 ABI, version 1 (GNU/Linux), dynamically linked, interpreter /lib64/ld64.so.2, BuildID[sha1]=8b472420d76db7c9dfebbba811a9e443515e9921, for GNU/Linux 3.10.0, stripped

So I am not sure where you got a 32-bit LSB binary. It would have to be an embedded (game?) system with a custom SDK.

#15

Then you will have to build that (32-bit LSB) stack yourself, or find the SDK that was used to build this binary.

#16

Thank you both very much. I understand what you said.
My main issues are:

  1. Currently, I don’t have a 32-bit cross-compilation toolchain for the powerpc64le system. Where can I obtain a 32-bit cross-compilation toolchain for the powerpc64le platform? If I have this cross-compilation toolchain, I can follow lkcl1’s method to solve the problem.
  2. I don’t have a 32-bit powerpc system at the moment, only the powerpc64le 32-bit libraries that remain due to historical reasons. If I had a 32-bit powerpc system, I could follow munroesj52’s method to solve the problem.

If I only have a powerpc64le (CPU: Power8) system, what options do I have to solve the issues mentioned above?
(Additionally, it is crucial that the rebuilt 32-bit application runs on powerpc64le (CPU: Power8).)

#17

Actually I did not say that. My comment was specific to the older big endian distros. I have no specific data on current le distributions. Paul would know more.
But APIs evolve and old apps may use deprecated kernel calls.

#18

I understand now. Thank you very much!

#19

This makes zero sense and you must be confused. CentOS 7.9 was based on RHEL7, which only supported big-endian on Power, so you can cannot execute either 32-bit or 64-bit little-endian binaries on CentOS 7.9 (that is a distro & kernel limitation, not a cpu technical limitation). The Power cpus are endian agnostic, meaning you can run them in either big-endian or little-endian modes. In fact, we (the IBM Power Toolchain team) have Power8, Power9 and Power10 partitions running both little-endian (RHEL8/9, SUSE & Ubuntu) and big-endian (RHEL7 & Debian unstable) distros to facilitate testing the toolchain in both endian modes on those cpus.

If you’re sure you’re on a CentOS 7.9 distro (whether it’s on Power8 or not is irrelevant), then you have a big-endian system. You can check that by “cat /etc/redhat-release”). The good news is that CentOS 7.9 does support 32-bit big-endian applications, you just need to install the 32-bit libraries before you compiler your application 32-bit (ie, -m32). Note it may take a special CentOS 7.9 repo to see them. I too have access to a CentOS 7.9 system and both 32-bit and 64-bit work fine:

[bergner@gcc1-power7 ~]$ cat /etc/redhat-release 
CentOS Linux release 7.9.2009 (AltArch)
[bergner@gcc1-power7 ~]$ gcc -m64 hello.c 
[bergner@gcc1-power7 ~]$ file a.out 
a.out: ELF 64-bit MSB executable, 64-bit PowerPC or cisco 7500, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.32, BuildID[sha1]=0c8655682569ca14e1cea334af1bf0ca91df3da2, not stripped
[bergner@gcc1-power7 ~]$ ./a.out 
Hello world!
[bergner@gcc1-power7 ~]$ gcc -m32 hello.c 
[bergner@gcc1-power7 ~]$ file a.out 
a.out: ELF 32-bit MSB executable, PowerPC or cisco 4500, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.32, BuildID[sha1]=be8e69d06a55499cb0ffff6c01d55c2d9940a4e9, not stripped
[bergner@gcc1-power7 ~]$ ./a.out 
Hello world!

The 32-bit packages you’ll need to install depend on what features your application requires (C++ or Fortran or ???), but you should be looking for the packages that end with .ppc and not the .ppc64 packages. Here are all of the 32-bit packages that are installed on the CentOS 7.9 system I have access to. You probably don’t need all of them, but it gives you an idea what of you may need to install:

[bergner@gcc1-power7 ~]$ rpm -qa | grep -w ppc
libgcc-4.8.5-44.el7.ppc
gmp-6.0.0-15.el7.ppc
libmpc-1.0.1-3.el7.ppc
glibc-2.17-326.el7_9.3.ppc
mpfr-3.1.1-4.el7.ppc
zlib-1.2.7-21.el7_9.ppc
bzip2-libs-1.0.6-13.el7.ppc
pcre2-10.23-2.el7.ppc
mpfr-devel-3.1.1-4.el7.ppc
libstdc++-static-4.8.5-44.el7.ppc
glibc-static-2.17-326.el7_9.3.ppc
libmpc-devel-1.0.1-3.el7.ppc
libgfortran-4.8.5-44.el7.ppc
glibc-devel-2.17-326.el7_9.3.ppc
nss-softokn-freebl-3.90.0-6.el7_9.ppc
gmp-devel-6.0.0-15.el7.ppc
libstdc++-4.8.5-44.el7.ppc
libstdc++-devel-4.8.5-44.el7.ppc

If you’re not actually on CentOS 7.9 and are really on a little-endian distro (Use “uname -m” to find out), then you are out of luck with respect to 32-bit binary support. It’s not a technical limitation, but a distro and toolchain limitation, as no one has ever built 32-bit little-endian binaries or libraries on powerpc64le and even if you managed to do that, it has never (AFAIK) been tested to execute a powerpcle binary on the powerpc64le kernel. It could just go boom as far as I know. :slight_smile:

#20

CentOS 7.9 was based on RHEL7, which only supported big-endian on Power,

This is false. CentOS 7.x supported both ppc64 and ppc64le.