Skip to content

Test and ship free-threaded Python support deliberately - #58

Merged
hardbyte merged 3 commits into
mainfrom
brian/jolly-franklin-75fkzx-free-threading
Sep 15, 2026
Merged

hardbyte merged 3 commits into
mainfrom
brian/jolly-franklin-75fkzx-free-threading

Conversation

@hardbyte

Copy link
Copy Markdown
Owner

Why

PR 1 of the plan in #45. Free-threaded support is already shipping by accident: PyO3 0.28 made gil_used = false the default, so this module has declared it runs without the GIL since 0.7.0, and 0.10.0 already publishes cp314t/cp315t wheels for every Linux architecture because the manylinux images carry those interpreters. Nothing had ever run the tests on a free-threaded build. This PR makes the support deliberate, tested, and complete across platforms.

What

CI

  • New 3.14t entry in the test matrix, running the whole suite with PYTHON_GIL=0 so a dependency lacking the declaration cannot quietly re-enable the GIL for the run.
  • UV_PYTHON pinned to the matrix interpreter at job level. Without it uv run re-resolves to whichever compatible Python it prefers; while validating this locally, an unpinned uv run silently replaced the 3.14t venv with 3.11.
  • macOS (both arches) and Windows x64 wheel jobs install 3.14t via setup-python and build the free-threaded wheel explicitly (-i <python-path>), after the --find-interpreter build so it is not built twice. Linux needs nothing: the manylinux images already provide it.

Rust

  • #[pymodule(gil_used = false)] spelled out, with the reasoning next to it.
  • Compile-time Send + Sync assertions for cel-rust's Program, Context<'static>, Value and Env, so an upstream bump that loses a bound fails the build instead of shipping a data race in a free-threaded wheel.
  • Program and OptionalValue are frozen: no mutating methods existed, and sharing them between threads now involves no borrow tracking.
  • pyo3_log::init()try_init(): init panics if the module is initialised twice in one process.
  • The Context cache mutex is taken with MutexExt::lock_py_attached, since clone_ref runs under it.
  • Bug found by the new leg: prepare_environment extracted a PyRef<Context> and fell through to "evaluation_context must be a Context object or a dict" when the extract failed. On 3.14t that happens whenever another thread is inside a mutator, so a reader racing an add_variable got a ValueError claiming its Context was not a Context. Now: type check first, short yield-retry for the borrow (a mutator holds it for one call), then a RuntimeError naming the concurrent modification.

Tests (tests/test_free_threading.py)

  • In a fresh interpreter without PYTHON_GIL, importing cel leaves sys._is_gil_enabled() false and emits no RuntimeWarning (warnings as errors). This is the test that would catch losing the declaration.
  • The suite's own leg is really GIL-free (_is_gil_enabled() is false under PYTHON_GIL=0).
  • 4 writers × 500 add_variable racing 4 readers × 500 execute on one Context: every observed value is one a writer stored; the only permitted failure is a borrow RuntimeError.
  • Program/OptionalValue immutability; 8 threads sharing one frozen Program and one Context with a Python callback agree on every result.

Docs: the threading contract for Context and Program in the API reference and the class docstring; CHANGELOG. Maturin floor raised to 1.14 (first release to discover free-threaded interpreters).

Verification

  • Python 3.11: 564 passed, 3 skipped (the two free-threaded-only tests plus one pre-existing), 5 xfailed. clippy -D warnings, cargo fmt --check, cargo test, ruff, mypy clean.
  • Python 3.14.0rc2 free-threading build, PYTHON_GIL=0: 566 passed, 1 skipped, 5 xfailed, three consecutive runs. Import without PYTHON_GIL leaves the GIL disabled with warnings-as-errors.

The macOS/Windows free-threaded wheel steps cannot be run locally; if setup-python or maturin behaves differently there I'll fix it up from the CI logs.

Related: #46 (abi3 would remove the free-threaded wheels on 3.14; decide together), #45 (PR 2, releasing the GIL around the parse, follows separately).

https://claude.ai/code/session_019WbvXZFm8Nb2LXF2kiWoWW


Generated by Claude Code

PyO3 0.28 made gil_used = false the default, so this module has declared that
it runs without the GIL since the 0.7.0 upgrade, and 0.10.0 already published
cp314t and cp315t wheels for Linux because the manylinux images carry the
free-threaded interpreters. Nothing had ever run the tests there.

- CI gains a python3.14t leg that runs the whole suite with PYTHON_GIL=0, and
  pins uv to the matrix interpreter (UV_PYTHON) so it cannot fall back to a GIL
  build of the same version. The macOS and Windows x64 wheel jobs install
  3.14t and build the free-threaded wheel explicitly, since those runners have
  no such interpreter for --find-interpreter to discover.
- The gil_used = false declaration is now explicit, and compile-time
  assertions pin cel-rust's Program, Context, Value and Env as Send + Sync so
  an upstream change cannot silently reintroduce a data race into a
  free-threaded wheel.
- Program and OptionalValue are frozen: they have no mutating methods, and
  sharing them between threads now involves no borrow tracking.
- pyo3_log::init() panicked if the module was initialised twice in one
  process; try_init() tolerates the second logger installation.
- The Context environment cache is locked with lock_py_attached, so a thread
  waiting for it cannot stall a free-threaded interpreter's stop-the-world
  pause while the holder runs Python API under the lock.
- tests/test_free_threading.py checks, in a fresh interpreter without
  PYTHON_GIL, that importing cel does not make CPython re-enable the GIL, and
  pins the concurrency contract for Context: concurrent evaluation is safe and
  consistent; concurrent mutation raises "Already borrowed" rather than racing.
- Documented the threading contract; maturin floor raised to 1.14, the first
  release that discovers free-threaded interpreters.

Claude-Session: https://claude.ai/code/session_019WbvXZFm8Nb2LXF2kiWoWW
Found by the new free-threaded CI leg. prepare_environment extracted a
PyRef<Context> and, when the extract failed, fell through to the dict check
and finally to "evaluation_context must be a Context object or a dict". On a
free-threaded interpreter the extract fails whenever another thread is inside
a mutator holding the exclusive borrow, so a reader racing an add_variable got
a ValueError claiming its Context was not a Context.

Check the type first, then take the borrow with a short yield-retry (a mutator
holds it only for one call), and if it still cannot be taken raise a
RuntimeError that names the concurrent modification.

Verified on python3.14t with PYTHON_GIL=0: 566 passed across three runs.

Claude-Session: https://claude.ai/code/session_019WbvXZFm8Nb2LXF2kiWoWW
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

@hardbyte
hardbyte merged commit 5de865d into main Sep 15, 2026
19 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants