sysconfig has wrong values for ppc64le

[minrk]

Solution to issue cannot be found in the documentation.

• I checked the documentation.

Issue

sysconfig appears to not get set up correctly when cross-compiling to linux-ppc64le. This has consequences, especially for pypy builds, where e.g. an empty EXT_SUFFIX prevents modules being imported.

Attempting to run $PYTHON -m sysconfig to debug values, I see this error on the CPython builds:

ModuleNotFoundError: No module named '_sysconfigdata__linux_powerpc64le-linux-gnu'

but on pypy3.9, it produces incorrect output:

EXT_SUFFIX = ".pypy39-pp73-x86_64-linux-gnu.so"

In all ppc cases, CMake's Python_SOABI is incorrectly empty which results in building with scikit-build-core producing extmodule.so instead of the correct extmodule.pypy39-pp73-ppc_64-linux-gnu.so. This doesn't ultimately fail on CPython because module.so is a vaild import name in CPython, but not PyPy.

aarch64 builds don't have this issue, producing correctly:

EXT_SUFFIX = ".pypy39-pp73-aarch64-linux-gnu.so"

so it is specific to ppc, not cross-compiling in general.

My hunch is it has to do with the fact that ppc64le has 3 names:

• ppc64le
• powerpc64le
• ppc_64 (first in _imp.extension_suffixes(), which populates SOABI, EXT_SUFFIX on PyPy)

but I don't know if the issue is here or in crossenv itself or somewhere else.

Installed packages

_libgcc_mutex:                             0.1-conda_forge           conda-forge
    _openmp_mutex:                             4.5-2_gnu                 conda-forge
    _sysroot_linux-ppc64le_curr_repodata_hack: 4-h43410cf_14             conda-forge
    binutils_impl_linux-64:                    2.40-hf600244_0           conda-forge
    binutils_impl_linux-ppc64le:               2.40-hef43fe1_0           conda-forge
    binutils_linux-ppc64le:                    2.40-h631d6fd_2           conda-forge
    bzip2:                                     1.0.8-hd590300_5          conda-forge
    c-ares:                                    1.26.0-hd590300_0         conda-forge
    ca-certificates:                           2024.2.2-hbcca054_0       conda-forge
    cffi:                                      1.16.0-1_pypy39           conda-forge
    cmake:                                     3.28.3-hcfe8598_0         conda-forge
    cross-python_linux-ppc64le:                3.9-41_73_pypy            conda-forge
    crossenv:                                  1.4.0-pyhd8ed1ab_0        conda-forge
    expat:                                     2.5.0-hcb278e6_1          conda-forge
    gcc_impl_linux-64:                         12.3.0-he2b93b0_5         conda-forge
    gcc_impl_linux-ppc64le:                    12.3.0-h783c3ad_5         conda-forge
    gcc_linux-ppc64le:                         12.3.0-h7a1d020_2         conda-forge
    gdbm:                                      1.18-h0a1914f_2           conda-forge
    gxx_impl_linux-64:                         12.3.0-he2b93b0_5         conda-forge
    gxx_impl_linux-ppc64le:                    12.3.0-h783c3ad_5         conda-forge
    gxx_linux-ppc64le:                         12.3.0-hbee9562_2         conda-forge
    kernel-headers_linux-64:                   2.6.32-he073ed8_17        conda-forge
    kernel-headers_linux-ppc64le:              3.10.0-h23d7e6c_14        conda-forge
    keyutils:                                  1.6.1-h166bdaf_0          conda-forge
    krb5:                                      1.21.2-h659d440_0         conda-forge
    ld_impl_linux-64:                          2.40-h41732ed_0           conda-forge
    ld_impl_linux-ppc64le:                     2.40-h2f009d6_0           conda-forge
    libcurl:                                   8.5.0-hca28451_0          conda-forge
    libedit:                                   3.1.20191231-he28a2e2_2   conda-forge
    libev:                                     4.33-hd590300_2           conda-forge
    libexpat:                                  2.5.0-hcb278e6_1          conda-forge
    libffi:                                    3.4.2-h7f98852_5          conda-forge
    libgcc-devel_linux-64:                     12.3.0-h8bca6fd_105       conda-forge
    libgcc-devel_linux-ppc64le:                12.3.0-h227d28a_5         conda-forge
    libgcc-ng:                                 13.2.0-h807b86a_5         conda-forge
    libgomp:                                   13.2.0-h807b86a_5         conda-forge
    libiconv:                                  1.17-hd590300_2           conda-forge
    libnghttp2:                                1.58.0-h47da74e_1         conda-forge
    libsanitizer:                              12.3.0-h0f45ef3_5         conda-forge
    libsqlite:                                 3.45.1-h2797004_0         conda-forge
    libssh2:                                   1.11.0-h0841786_0         conda-forge
    libstdcxx-devel_linux-64:                  12.3.0-h8bca6fd_105       conda-forge
    libstdcxx-devel_linux-ppc64le:             12.3.0-h227d28a_5         conda-forge
    libstdcxx-ng:                              13.2.0-h7e041cc_5         conda-forge
    libuv:                                     1.47.0-hd590300_0         conda-forge
    libxcb:                                    1.15-h0b41bf4_0           conda-forge
    libzlib:                                   1.2.13-hd590300_5         conda-forge
    lz4-c:                                     1.9.4-hcb278e6_0          conda-forge
    make:                                      4.3-hd18ef5c_1            conda-forge
    ncurses:                                   6.4-h59595ed_2            conda-forge
    openssl:                                   3.2.1-hd590300_0          conda-forge
    pkg-config:                                0.29.2-h36c2ea0_1008      conda-forge
    popt:                                      1.16-h0b475e3_2002        conda-forge
    pthread-stubs:                             0.4-h36c2ea0_1001         conda-forge
    pypy3.9:                                   7.3.15-h9557127_0         conda-forge
    python:                                    3.9.18-1_73_pypy          conda-forge
    python_abi:                                3.9-4_pypy39_pp73         conda-forge
    readline:                                  8.2-h8228510_1            conda-forge
    rhash:                                     1.4.4-hd590300_0          conda-forge
    rsync:                                     3.2.7-h70740c4_0          conda-forge
    sed:                                       4.8-he412f7d_0            conda-forge
    sqlite:                                    3.45.1-h2c6b66d_0         conda-forge
    sysroot_linux-64:                          2.12-he073ed8_17          conda-forge
    sysroot_linux-ppc64le:                     2.17-h23d7e6c_14          conda-forge
    tk:                                        8.6.13-noxft_h4845f30_101 conda-forge
    tzdata:                                    2024a-h0c530f3_0          conda-forge
    xorg-kbproto:                              1.0.7-h7f98852_1002       conda-forge
    xorg-libx11:                               1.8.7-h8ee46fc_0          conda-forge
    xorg-libxau:                               1.0.11-hd590300_0         conda-forge
    xorg-libxdmcp:                             1.1.3-h7f98852_0          conda-forge
    xorg-xextproto:                            7.3.0-h0b41bf4_1003       conda-forge
    xorg-xproto:                               7.0.31-h7f98852_1007      conda-forge
    xxhash:                                    0.8.0-h7f98852_3          conda-forge
    xz:                                        5.2.6-h166bdaf_0          conda-forge
    zlib:                                      1.2.13-hd590300_5         conda-forge
    zstd:                                      1.5.5-hfc55251_0          conda-forge

Environment info

active environment : base
    active env location : /opt/conda
            shell level : 1
       user config file : /home/conda/.condarc
 populated config files : /opt/conda/.condarc
                          /home/conda/.condarc
          conda version : 23.11.0
    conda-build version : 3.28.4
         python version : 3.10.13.final.0
                 solver : libmamba (default)
       virtual packages : __archspec=1=broadwell
                          __conda=23.11.0=0
                          __glibc=2.17=0
                          __linux=6.2.0=0
                          __unix=0=0
       base environment : /opt/conda  (writable)
      conda av data dir : /opt/conda/etc/conda
  conda av metadata url : None
           channel URLs : https://conda.anaconda.org/conda-forge/linux-64
                          https://conda.anaconda.org/conda-forge/noarch
          package cache : /home/conda/feedstock_root/build_artifacts/pkg_cache
                          /opt/conda/pkgs
       envs directories : /opt/conda/envs
                          /home/conda/.conda/envs
               platform : linux-64
             user-agent : conda/23.11.0 requests/2.31.0 CPython/3.10.13 Linux/6.2.0-1019-azure centos/7.9.2009 glibc/2.17 solver/libmamba conda-libmamba-solver/24.1.0 libmambapy/1.5.6
                UID:GID : 1001:1001
             netrc file : None
           offline mode : False

[minrk]

I've managed to reproduce this with the following Dockerfile:

FROM quay.io/condaforge/miniforge3

ENV BUILD_PREFIX=/opt/conda/envs/build
ENV PREFIX=/opt/conda/envs/host
ENV target_platform=linux-ppc64le

RUN mamba create -y -p $BUILD_PREFIX python pypy3.9 cross-python_${target_platform}
RUN mamba create -y --override-channels -c conda-forge/noarch -c conda-forge/${target_platform} -p $PREFIX python pypy3.9

# set CONDA_BUILD
ENV CONDA_BUILD=1

Here's the bit I don't understand: cross-python -m sysconfig and cross-python -c 'import sysconfig; sysconfig._main()' (i.e. the same thing) produce different results:

(build) root@5c058377c907:/# $BUILD_PREFIX/venv/bin/cross-python -m sysconfig | grep EXT_SUFFIX
	EXT_SUFFIX = ".pypy39-pp73-x86_64-linux-gnu.so"
(build) root@5c058377c907:/# $BUILD_PREFIX/venv/bin/cross-python -c 'import sysconfig; sysconfig._main()' | grep EXT_SUFFIX
	EXT_SUFFIX = ".pypy39-pp73-ppc_64-linux-gnu.so"

and enabling isolated mode restores the correct behavior:

(build) root@5c058377c907:/# $BUILD_PREFIX/venv/bin/cross-python -I -m sysconfig | grep EXT_SUFFIX
	EXT_SUFFIX = ".pypy39-pp73-ppc_64-linux-gnu.so"