Skip to content

TypedPython 2: compile @compiled functions through Cython (first part) - #39

Closed
b-re-w wants to merge 7 commits into
developfrom
work/typedpython-compile
Closed

b-re-w wants to merge 7 commits into
developfrom
work/typedpython-compile

Conversation

@b-re-w

@b-re-w b-re-w commented Oct 2, 2026

Copy link
Copy Markdown
Member

Refs #23. This is the first part. #23 also requires nbody/spectral-norm ≥ 10× measured alone, and a list of what Cython rejects; both are still open.

Stacked on #19 → #33. The change in this PR is the last commit, ada2fe8.

Feature added

  • typedpython.compiled decorator: the identity in CPython. A first-line # typedpython: compiled marks every module-level function.
  • compiler.compile_module(path, out) compiles through Cython pure-Python mode to a CPython extension. It refuses any module that fails the gate in compiled mode. The output must give the same results as CPython:
    • float becomes C double.
    • int becomes long long only after an entry range check. Pure functions run with overflow checking, and on overflow the call is redone interpreted, so the result is promoted to an arbitrary-precision int.
    • An argument of a different runtime type sends the call to the interpreted version.
    • list[float] elements are unboxed only when the function cannot turn them into non-floats.
    • math.sqrt keeps its ValueError.
  • python -m typedpython demo <module> [args] builds the module, runs main() both interpreted and compiled, prints the timings, and fails unless the output is identical.
  • The compile extra adds cython>=3.1 and setuptools.

Test added

  • test_compile.py, 10 tests. Before implementation, 6 failed with NotImplementedError (the decorator-identity test passed, since the decorator already existed). The 3 float-list tests were added later; one of them was red until len(xs) was allowed.
  • test_cli.py demo tests: 2 red, then 2 green.
  • test_compile_measurement.py measures the scalar loop: 136× (277 ms → 2.0 ms) at load 109. Only the ratio means anything.
  • Mutation checks: turning overflow checking off makes the promotion test fail; turning the entry guard off makes the big-int test fail.
  • A new workflow, .github/workflows/typedpython.yml, runs the Python tests on ubuntu and macOS. Gradle CI never ran them.

Demo (by hand, not a measurement)

I added the module marker to benchmarks/typedpython/py/nbody.py (from #28) and ran demo with 200000 steps. Output was identical: -0.169075164 -0.169083713. Interpreted took 44.8 s and compiled 11.3 s, about 4× faster, at load average 197.

Documentation corrected

  • SPEC N-7 (partial)
  • design §6.2

Not done

  • Methods, default arguments and *args compile as untyped Python.
  • List elements are still PyFloat, so every store boxes. The next step is to lower fixed-size arrays to C arrays.
  • No measurement on an idle machine yet.

b-re-w added 7 commits October 3, 2026 07:04
Step 1 of docs/design/typedpython.md: python -m typedpython check runs Pyrefly
with its Any-leak kinds raised to errors, reports user-code expressions of type
Any outside cast() (any-flow), and reports the forbidden list as warnings in
checked mode and errors in compiled mode. Lives in python-multiplatform-ksp.

INTENT gains TypedPython as purpose 4 (maintainer statements quoted); SPEC
gains N-6 (partial: not yet run by the build).

Refs #16
toolchain installs the gate with uv at a pinned version (1b, #20), so the
package needs a build backend and a console script. Wheel built and smoke-
tested: installed from --find-links, typedpython check exits 0 on clean code
and 1 on an unannotated function.

Refs #16
Pyrefly 1.3.2 reports neither and records no types for assignment targets,
but it does record the type of every name load. The gate now writes a probed
copy in which each candidate assignment is followed on the same line by a
load of its targets (line numbers unchanged), runs Pyrefly on it, and judges
the types at the probe positions. The second run happens only when a file
assigns a name twice or has a container display.

Refs #22
The gate's Pyrefly config lives in a temporary directory, which Pyrefly took
as the import root, so relative and absolute self-imports of user packages did
not resolve and every value from them failed as any-flow. Each checked file's
import root (above its outermost package) now goes on the search path.

Red before the fix: 3 failed in test_imports.py.

Refs #16
A probed copy written alone into a temporary directory lost its package, so
relative imports in it did not resolve, their values were Unknown, and
rebinding went unreported in exactly the code that lives in packages. The
probe pass now copies each file's package tree with the same layout.

Red before the fix: test_rebinding_inside_a_package_with_relative_imports.

Refs #22
… CPython's results

typedpython.compiled is the identity in CPython. Marked functions (or every
function under '# typedpython: compiled') build into a CPython extension:
float as C double; int as long long only behind an entry range check, and in
pure functions with overflow checking that redoes the call interpreted, so
int keeps arbitrary precision; list[float] elements unboxed only when the
function cannot make them non-floats; math.sqrt keeps its ValueError.
python -m typedpython demo runs main() both ways and fails on any difference.

A TypedPython CI workflow runs these tests (Gradle does not).

Red before implementation: 6 NotImplementedError in test_compile.py, 2 demo
tests failing. Mutations: overflow check off -> promotion test fails; entry
guard off -> big-int test fails.

Refs #23
@b-re-w

b-re-w commented Oct 2, 2026

Copy link
Copy Markdown
Member Author

Closed: the maintainer dropped the Cython backend (2026-10-03). The reusable parts move to the stage-2 typed IR → C back end in #41: the semantic rules (int promotion through an interpreted redo, entry guards before any effect, exact runtime types, math errors), the demo command, the differential tests, and the TypedPython CI workflow. The branch is kept for reference.

@b-re-w b-re-w closed this Oct 2, 2026
@b-re-w
b-re-w deleted the work/typedpython-compile branch October 3, 2026 10:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant