Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .claude/skills/forge-error-catalogue/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -88,6 +88,8 @@ instead of re-deriving it.
transient/scattered in CI where a rerun clears it). **A vendored lib built by
setup.py's OWN cmake call** (arg list hardcoded, so `CMAKE_ARGS` does nothing) →
green host-configured library, fix by patching in a `FORGE_CMAKE_ARGS` extend.
**`conflicting types for 'fseek'` from a project's own `compat.h`, 32-bit Android only →
the NDK toolchain pins `CMAKE_SYSTEM_VERSION` to 1; read `ANDROID_PLATFORM_LEVEL`**.
- **Runtime failures** (device/emulator/simulator) — **the Flet 0.86 Android
`sitepackages.zip` class** (its umbrella entry explains "why only now"):
`NotADirectoryError` on a bundled data file → **`extract_packages`** meta field;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -1732,6 +1732,48 @@ iOS doesn't need this — Apple's clang resolves the C++ runtime to system libc+

---

### A **cffi** ABI-mode wrapper cannot open a `flet-lib*` on iOS — `ffi.dlopen` is not `.fwork`-aware

**Symptom:** a pure-Python package that does `ffi.dlopen(...)` (cffi *ABI* mode —
`ffibuilder.set_source("_x", None)`, no compiled extension anywhere) fails at import on
iOS even though the `flet-lib*` wheel is bundled and the same recipe works on Android.
cffi's error helpfully appends `Additionally, ctypes.util.find_library() did not manage to
locate a library called '<name>.fwork'`, which sends you chasing `find_library` instead of
the real cause.

**Cause.** serious-python *moves* the real binary into an embedded framework and leaves a
`.fwork` ASCII **text** stub at the old `opt/lib/` path. The dereference of that stub lives
**only** in iOS CPython's patched `Lib/ctypes/__init__.py` — `CDLL.__init__` rewrites a
`*.fwork` name to `os.path.join(dirname(sys.executable), <file contents>)` before
`_dlopen`, then stores it in `self._name`. Nothing below ctypes knows about `.fwork`:
`_ctypes` doesn't, dyld doesn't, and `cffi`'s `ffi.dlopen` is a raw `dlopen(3)`.
(`importlib`'s `AppleFrameworkLoader` maps `.so`→`.fwork` for module **imports** only,
never for data libraries.) So on iOS the old path is gone and the `.fwork` path is not
Mach-O — both fail.

**Fix — let ctypes resolve, then hand cffi the path it settled on:**

```python
path = _ctypes_find_library(name)
if path is None and _sys.platform in ('android', 'ios'):
import ctypes as _ctypes
optlib = os.path.join(os.path.dirname(os.path.abspath(__file__)), 'opt', 'lib')
for candidate in (f'lib{name}.so', # Android jniLibs, by soname
os.path.join(optlib, f'lib{name}.fwork'), # iOS embedded framework
os.path.join(optlib, f'lib{name}.so')): # not relocated
try:
return _ctypes.CDLL(candidate)._name # resolved absolute path
except OSError:
continue
```

Wrapping `ctypes.util.find_library` (rather than rewriting the package's loader) keeps
everything downstream upstream's. Worked example: `recipes/soundfile/patches/mobile.patch`.

**Pure-ctypes wrappers need none of this** — `CDLL('lib<name>.fwork')` just works
(pyzbar, pysodium, python-magic all rely on it). The problem is specific to cffi, and to
anything else calling `dlopen` below the Python level.

### `ImportError: Unable to find <name> shared library` (ctypes wrapper, at import)

**Cause:** a pure-Python `ctypes` wrapper called `ctypes.util.find_library()`,
Expand Down Expand Up @@ -2091,6 +2133,36 @@ face3d, still counts as pure.)

---

### `error: conflicting types for 'fseek'` / `'ftell'` from a project's own `compat.h` (Android **32-bit only**)

```
sysroot/usr/include/stdio.h:216:5: error: conflicting types for 'fseek'
216 | int fseeko(FILE*, off_t, int) __RENAME(fseeko64) __INTRODUCED_IN(24);
compat.h:67:16: note: expanded from macro 'fseeko'
67 | #define fseeko fseek
```

**Cause — a CMake project reading the Android API level from `CMAKE_SYSTEM_VERSION`.**
The NDK's own `android.toolchain.cmake` pins `CMAKE_SYSTEM_VERSION` to **1** and reports
the real level in **`ANDROID_PLATFORM_LEVEL`**; only CMake's *built-in* Android support
(no toolchain file) puts the API level in `CMAKE_SYSTEM_VERSION`. Every recipe here uses
the toolchain file, so a `CMAKE_SYSTEM_VERSION VERSION_LESS 24` test is **always true**.
The project then concludes fseeko is unavailable and defines `fseeko`→`fseek`, which
rewrites bionic's own LFS *declaration* — and `off_t` is 64-bit (forge compiles with
`_FILE_OFFSET_BITS=64`) while `long` is 32-bit, so the two prototypes disagree.

**Why only armeabi-v7a (and x86):** 64-bit ABIs have `off_t == long`, and such guards are
usually also gated on `CMAKE_SYSTEM_PROCESSOR MATCHES "i686|armv7-a"`, so arm64/x86_64
escape and the recipe looks fine until the 32-bit slice.

**Fix:** patch the condition to prefer `ANDROID_PLATFORM_LEVEL` when defined. Passing
`-DCMAKE_SYSTEM_VERSION=$SDK_VERSION` does **not** work — the toolchain file overwrites
it with a normal (non-cache) `set()`, which shadows your cache entry. Worked example:
`recipes/flet-libflac/patches/android-api-level.patch`.

**Generalise the tell:** any CMake project inspecting `CMAKE_SYSTEM_VERSION` for Android
is reading `1`. Grep a new native recipe for it before trusting a green 64-bit build.

### `<pkg>: no licence file found, so this wheel would ship the library's object code with no notice`

**Cause:** a `build.sh` recipe's wheel is synthesised by forge, so nothing carries a
Expand Down
4 changes: 3 additions & 1 deletion .claude/skills/local-recipe-testing/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -100,7 +100,9 @@ for i in $(seq 1 30); do grep EXIT "$DATA/Library/Caches/console.log" 2>/dev/nul

1. **Use forge's `dist/` wheel, NOT `build/.../target/wheels/`.** The latter is maturin's raw output — **unstripped**. For polars that meant a **1.27 GB** `.so` (vs 130 MB stripped); it blows up install space and may not load. forge strips + repacks into `dist/`. Always test the `dist/` wheel.

2. **Build the recipe against the SAME Python `flet build` bundles (3.12 for flet 0.85.x).** forge's Android Rust `.so` hard-links `libpythonX.Y.so` (`DT_NEEDED`) — so the **`abi3` wheel tag is misleading**; it still needs the matching `libpython` at `dlopen`. A 3.14-built wheel in a 3.12 app fails: `dlopen … libpython3.14.so` missing → the package reports its "binary missing" (e.g. polars `NameError: PySeries`). Verify with `llvm-readelf -d <so> | grep NEEDED`. If you only have a different support tree, you can retag a wheel for flet's python with `uvx --from wheel wheel tags --python-tag cp312 --abi-tag abi3 --remove <whl>`, but the underlying `libpython` link still has to match — so really, build on the right python.
2. **Build the recipe against the SAME Python `flet build` bundles — and know which one that is.** `flet build` **0.86.5 defaults to Python 3.14** (measured 2026-09-14: the app's staged site-packages hold `_cffi_backend.cpython-314-*.so`); the snippets above pass `--python-version 3.12` to pin it, and CI's recipe-tester does the same. A **consumer** example app built the plain way (`flet build apk`, no flag) therefore gets 3.14, so build the recipe for 3.14 (`source ./setup.sh 3.14`) before that pass, or pass the flag. `flet-lib*` build.sh recipes are `py3-none-<plat>` and version-independent — only the Python package needs the extra build.

Two different failure shapes if you get it wrong. For a **compiled** package: forge's Android Rust `.so` hard-links `libpythonX.Y.so` (`DT_NEEDED`) — so the **`abi3` wheel tag is misleading**; it still needs the matching `libpython` at `dlopen`. A 3.14-built wheel in a 3.12 app fails: `dlopen … libpython3.14.so` missing → the package reports its "binary missing" (e.g. polars `NameError: PySeries`). Verify with `llvm-readelf -d <so> | grep NEEDED`. If you only have a different support tree, you can retag a wheel for flet's python with `uvx --from wheel wheel tags --python-tag cp312 --abi-tag abi3 --remove <whl>`, but the underlying `libpython` link still has to match — so really, build on the right python. For a **pure-Python** package with a mobile patch there is no link error at all — pip simply finds no `cp3XX` match, silently installs PyPI's `py3-none-any` wheel, and the app fails on device with the unpatched loader's own error. Tell: the app's `build/site-packages/<abi>/<pkg>-*.dist-info/METADATA` is missing the `flet-lib*` `Requires-Dist` the recipe promotes.

3. **Clear `tests/recipe-tester/build/site-packages` + `build/.hash` between rebuilds.** `flet build` keys its skip-site-packages cache on the requirement *string*, not wheel content — so swapping a same-version wheel is silently ignored and it re-bundles the old `.so`. Tell-tale: the APK size doesn't change after you changed the wheel.

Expand Down
33 changes: 33 additions & 0 deletions .claude/skills/new-mobile-recipe/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -77,6 +77,7 @@ Match the package to one of these shapes. Each maps to a template in `templates/
| C-ext consuming an existing flet-lib | Already-built `flet-libX` covers the C dep (libxml2, libcurl, libssl via openssl, etc.) | Adapt `templates/meta-with-patches.yaml` + add `requirements.host` |
| Native library itself (flet-lib*), **static** | New C library a Python C-extension links at build time (libxml2, libcurl, libgeos…) | `templates/meta-flet-lib.yaml` + `templates/build-flet-lib.sh` |
| Native library, **ctypes-loaded (shared)** | A pure-Python wrapper `dlopen`s the lib at runtime via `ctypes` (pyzbar→libzbar, python-magic→libmagic) | `templates/meta-flet-lib.yaml` + `templates/build-flet-lib-shared.sh`; see Pattern H |
| Native library, **cffi-loaded (shared)** | Same, but the wrapper is **cffi ABI mode** (`ffibuilder.set_source(name, None)` → `ffi.dlopen`, no compiled extension anywhere) (soundfile→libsndfile) | As Pattern H, plus a loader patch that resolves through ctypes — `ffi.dlopen` is a raw `dlopen(3)` and cannot follow an iOS `.fwork`; copy `recipes/soundfile/` |
| Cython-accelerated pure-Python (poetry-core build script) | `build-backend = "poetry.core.masonry.api"` + `[tool.poetry.build] script` that cythonizes the runtime `.py` files themselves (zeroconf; the Home-Assistant-ecosystem idiom). Forge's PEP 517 path handles poetry-core unchanged | No template — copy `recipes/zeroconf/` (branch `zeroconf`): `script_env REQUIRE_CYTHON: "1"` + a fail-loud patch (upstream swallows compile errors → silent pure-py wheel), test asserts the modules are real extensions |
| C-ext that links a lib via a `*-config` tool | Compiled C-ext whose `setup.py` shells out to `pg_config`/`mysql_config`/… (psycopg2→libpq, mysqlclient→libmysqlclient) | A **static+PIC** `flet-lib*` (`build-flet-lib.sh` + `-fPIC`) shipping a config-shim, + consumer `script_env`/patch; see Pattern I |
| **setup.py that drives CMake itself** for a vendored native lib | An sdist that vendors a C library and builds it with its own `subprocess` CMake call inside `build_ext`, then links the static result via `extra_objects` (pycares→c-ares). Not scikit-build-core — the arg list is hardcoded in `setup.py` | No template — copy `recipes/pycares/`: one patch appends `shlex.split(os.environ['FORGE_CMAKE_ARGS'])` to the arg list, `requirements.build: [cmake]`; see "vendored-CMake" deep-dive below |
Expand Down Expand Up @@ -156,6 +157,38 @@ The shape that works — a `flet-lib*` prebuilt-repackage dep + a small opt-in p

**Real example:** branch `curl-cffi` — `recipes/flet-libcurl-impersonate/` (build.sh, `source.strip: 0`) + `recipes/curl-cffi/` (`patches/mobile.patch`, 3 hunks). World-first curl-cffi iOS wheels; on-device 4/4 both platforms, and a real impersonated HTTPS request returns 200 on each. Needed one forge-core change: exposing `source.strip` in `src/forge/schema/meta-schema.yaml` (the code already honored it).

### Shape deep-dive: cffi ABI-mode wrapper of a shared flet-lib (soundfile -> libsndfile)

**When:** the package has **no compiled extension at all** — `soundfile_build.py` does
`ffibuilder.set_source("_soundfile", None)`, so cffi emits a pure-Python `_soundfile.py`
and the library is reached entirely through `ffi.dlopen(...)`. PyPI ships a
`py2.py3-none-any` wheel plus platform wheels that differ only by a bundled `.so`.

It looks like Pattern H and mostly is, with three differences that each cost a cycle:

1. **`ffi.dlopen` is a raw `dlopen(3)`.** It cannot dereference the iOS `.fwork` stub —
that lives only in iOS CPython's patched `ctypes/__init__.py`. Resolve through ctypes
and hand cffi `CDLL(candidate)._name`. Full mechanism and patch shape in the
`forge-error-catalogue`, "A **cffi** ABI-mode wrapper cannot open a `flet-lib*` on iOS".
2. **A recipe is still required even though nothing compiles**, because the loader's
`sys.platform` dispatch (darwin/win32/linux) ends in a bare `raise` on `android`/`ios`.
forge's `fix_wheel` retags the pure wheel `cp3XX-cp3XX-<platform>`, which is what makes
the patched copy outrank PyPI's `any` wheel at the same version — do not "fix" upstream's
own tagging in a patch.
3. **Build it for the Python `flet build` actually bundles.** A cp312-only wheel loses to
PyPI's `py2.py3-none-any` on a 3.14 app with no resolution error at all; the tell is the
app's staged `METADATA` missing the `flet-lib*` `Requires-Dist`. See `local-recipe-testing`.

**Optional sub-libraries:** where the native library has them (libsndfile's
FLAC/Ogg/Vorbis/Opus/mpg123/LAME), build each as a **static PIC** `flet-lib*` and declare
them `requirements.host_build` of the shared one, which absorbs them into a single `.so`.
`host` would promote six wheels into every consuming app whose contents already ride inside
that `.so`. Restrict the iOS export list to the public prefix
(`-Wl,-exported_symbol,'_sf_*'`) so the absorbed symbols stay hidden the way libsndfile's
own version script already keeps them on Android — jniLibs is a flat namespace shared with
every other native wheel. And keep `-Wl,-headerpad_max_install_names` on any hand-linked
iOS shared image: CMake adds it automatically, a hand `$CC -shared` does not.

### Naming

- Recipe directory name and `package.name` in meta.yaml must match the PyPI sdist filename exactly (case-sensitive). `MarkupSafe`, not `markupsafe`. `cffi`, not `CFFI`. Look at `<pypi-name>-<version>.tar.gz` filename on PyPI for the ground truth.
Expand Down
43 changes: 43 additions & 0 deletions recipes/flet-libflac/build.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
#!/bin/bash
set -eu

# ENABLE_MULTITHREADING puts Threads::Threads in libFLAC's exported CMake
# target, which libsndfile (no find_package(Threads)) then cannot resolve.
# libsndfile drives the single-threaded encoder API regardless.
common_args="\
-DCMAKE_BUILD_TYPE=Release \
-DCMAKE_POLICY_VERSION_MINIMUM=3.5 \
-DCMAKE_INSTALL_PREFIX=$PREFIX \
-DCMAKE_POSITION_INDEPENDENT_CODE=ON \
-DCMAKE_PREFIX_PATH=$PLATLIB/opt \
-DBUILD_SHARED_LIBS=OFF \
-DBUILD_CXXLIBS=OFF \
-DBUILD_PROGRAMS=OFF \
-DBUILD_EXAMPLES=OFF \
-DBUILD_TESTING=OFF \
-DBUILD_DOCS=OFF \
-DINSTALL_MANPAGES=OFF \
-DWITH_OGG=ON \
-DENABLE_MULTITHREADING=OFF \
-DOGG_INCLUDE_DIR=$PLATLIB/opt/include \
-DOGG_LIBRARY=$PLATLIB/opt/lib/libogg.a"

if [ "$CROSS_VENV_SDK" = "android" ]; then
cmake -B build \
-DCMAKE_SYSTEM_NAME=Android \
-DANDROID_PLATFORM=$SDK_VERSION \
-DANDROID_ABI=$ANDROID_ABI \
-DCMAKE_TOOLCHAIN_FILE=$NDK_ROOT/build/cmake/android.toolchain.cmake \
$common_args
else
cmake -B build \
-DCMAKE_SYSTEM_NAME=iOS \
-DCMAKE_OSX_SYSROOT=$SDK \
-DCMAKE_OSX_ARCHITECTURES=$HOST_ARCH \
$common_args
fi

cmake --build build -j "$CPU_COUNT"
cmake --install build

rm -rf "$PREFIX/share"
29 changes: 29 additions & 0 deletions recipes/flet-libflac/meta.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
{% set version = "1.5.0" %}

# Static PIC libFLAC for flet-libsndfile's FLAC support (native .flac and
# Ogg-FLAC). Link-time only -- nothing ships to the device.
package:
name: flet-libflac
version: '{{ version }}'

build:
number: 1

source:
url: https://github.com/xiph/flac/releases/download/{{ version }}/flac-{{ version }}.tar.xz

requirements:
build:
- cmake
host_build:
- flet-libogg 1.3.6

patches:
- android-api-level.patch

about:
# Only libFLAC is built here. The GPL-2.0 arm of FLAC's split licence covers
# the command-line tools (BUILD_PROGRAMS=OFF), which this wheel does not
# contain; the library itself is BSD-3-Clause (COPYING.Xiph).
license_file: COPYING.Xiph
license: BSD-3-Clause
37 changes: 37 additions & 0 deletions recipes/flet-libflac/patches/android-api-level.patch
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
Read the Android API level from the variable the NDK toolchain actually sets.

FLAC disables fseeko/ftello on 32-bit Android "before API 24" and reads the API
level from CMAKE_SYSTEM_VERSION. That only works under CMake's built-in Android
support; the NDK's own android.toolchain.cmake -- which is what every recipe
here uses -- pins CMAKE_SYSTEM_VERSION to 1 and reports the level in
ANDROID_PLATFORM_LEVEL instead. So the test fires at every API level, and on
armeabi-v7a compat.h's `#define fseeko fseek` then collides with bionic's own
64-bit-off_t declaration (forge compiles with _FILE_OFFSET_BITS=64):

stdio.h:216:5: error: conflicting types for 'fseek'
compat.h:67:16: note: expanded from macro 'fseeko'

arm64-v8a and x86_64 escape only because the processor test does not match them.

diff -ruN fa/CMakeLists.txt fb/CMakeLists.txt
--- fa/CMakeLists.txt 2025-02-10 15:20:26
+++ fb/CMakeLists.txt 2026-09-14 21:42:44
@@ -122,7 +122,17 @@
endif()


-if(ANDROID AND CMAKE_SYSTEM_VERSION VERSION_LESS 24 AND (CMAKE_SYSTEM_PROCESSOR MATCHES "i686" OR CMAKE_SYSTEM_PROCESSOR MATCHES "armv7-a"))
+if(ANDROID AND DEFINED ANDROID_PLATFORM_LEVEL)
+ # The NDK's own android.toolchain.cmake pins CMAKE_SYSTEM_VERSION to 1 and
+ # reports the real API level in ANDROID_PLATFORM_LEVEL; only CMake's built-in
+ # Android support puts the API level in CMAKE_SYSTEM_VERSION. Read whichever
+ # is meaningful, or the test below disables fseeko at every API level.
+ set(FLAC_ANDROID_API_LEVEL ${ANDROID_PLATFORM_LEVEL})
+else()
+ set(FLAC_ANDROID_API_LEVEL ${CMAKE_SYSTEM_VERSION})
+endif()
+
+if(ANDROID AND FLAC_ANDROID_API_LEVEL VERSION_LESS 24 AND (CMAKE_SYSTEM_PROCESSOR MATCHES "i686" OR CMAKE_SYSTEM_PROCESSOR MATCHES "armv7-a"))
# fseeko/ftello may link, but it's not usable before Android API 24 on 32-bit Android
# https://android.googlesource.com/platform/bionic/+/main/docs/32-bit-abi.md
message(STATUS "Disabling fseeko/ftello for 32-bit Android before API 24")
39 changes: 39 additions & 0 deletions recipes/flet-libmp3lame/build.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
#!/bin/bash
set -eu

# PIC: the archive is linked into the shared libsndfile.so.
export CFLAGS="${CFLAGS:-} -fPIC"

# Only libmp3lame is wanted; the frontend drags in a terminal UI and, on 3.100,
# an frontend/get_audio.c that does not cross-compile.
common_args="\
--disable-dependency-tracking \
--enable-static --disable-shared \
--disable-frontend --disable-decoder --disable-gtktest \
--disable-analyzer-hooks --disable-nasm"

if [ "$CROSS_VENV_SDK" = "android" ]; then
host=$HOST_TRIPLET
else
# lame 3.100's config.sub (2016) rejects arm64-apple-ios -- feed it the
# equivalent Darwin triplet; CC/CFLAGS do the real targeting.
case $HOST_TRIPLET in
arm64-apple-ios) host=aarch64-apple-darwin23 ;;
arm64-apple-ios-simulator) host=aarch64-apple-darwin23 ;;
x86_64-apple-ios-simulator) host=x86_64-apple-darwin23 ;;
*) echo "Unknown iOS host triplet: $HOST_TRIPLET"; exit 1 ;;
esac
# forge's iOS flags carry a quoted -F path; the quotes reach the compiler
# literally and break preprocessing. The forge paths have no spaces.
export CFLAGS="$(printf '%s' "$CFLAGS" | tr -d '"')"
export CPPFLAGS="$(printf '%s' "${CPPFLAGS:-}" | tr -d '"')"
export LDFLAGS="$(printf '%s' "${LDFLAGS:-}" | tr -d '"')"
fi

./configure --host=$host --prefix=$PREFIX $common_args
make -j "$CPU_COUNT"
make install

shopt -s nullglob
rm -rf "$PREFIX/share" "$PREFIX/bin"
rm -f "$PREFIX"/lib/*.la
Loading
Loading