Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -160,7 +160,8 @@ interoperability in both directions:
| `binary/` | Source archives of per-platform CPython distributions |
| `python_for_kotlin_binding.mermaid` | User-authored sketch of the object model (see rule 14) |
| `docs/INTENT.md`, `docs/SPEC.md` | Intent and behavioural contract (rule 5) |
| `docs/design/`, `docs/platforms/`, `docs/investigations/`, `docs/roadmap/` | Design records, platform notes, investigations, progress log |
| `docs/design/`, `docs/platforms/`, `docs/investigations/`, `docs/roadmap/` | Current design records (one per topic), platform notes, investigation conclusions, the work left to do |
| `docs/archive/` | Superseded designs and investigation narratives, kept for history (see its `README.md`) |

### Source-set hierarchy

Expand Down
4 changes: 2 additions & 2 deletions PROJECT.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ Kotlin Multiplatform 에 CPython 을 임베딩해 **Kotlin 과 Python 이 서로
쓰게 한다. 모든 지원 플랫폼에서 같은 방식으로 동작하고, 경계를 넘는 비용을 측정하며 줄인다.

- 의도(경계): [`docs/INTENT.md`](docs/INTENT.md) · 스펙(동작 계약과 상태): [`docs/SPEC.md`](docs/SPEC.md)
- 에이전트 규정: [`AGENTS.md`](AGENTS.md) · 진척 기록: [`docs/roadmap/ROADMAP.md`](docs/roadmap/ROADMAP.md)
- 에이전트 규정: [`AGENTS.md`](AGENTS.md) · 남은 작업: [`docs/roadmap/ROADMAP.md`](docs/roadmap/ROADMAP.md)

---

Expand All @@ -13,7 +13,7 @@ Kotlin Multiplatform 에 CPython 을 임베딩해 **Kotlin 과 Python 이 서로
| 방향 | 이름 | 메커니즘 | 상세 |
|---|---|---|---|
| Kotlin → Python | **다운콜** | CPython Stable ABI 함수 호출 (`expect` 약 330개) | [`docs/design/downcall-design.md`](docs/design/downcall-design.md) |
| Python → Kotlin | **업콜** | 빌드 타임 생성 함수 테이블 + 플랫폼별 진입점 1개 | [`docs/design/upcall-design.md`](docs/design/upcall-design.md) |
| Python → Kotlin | **업콜** | 빌드 타임 생성 함수 테이블 + 플랫폼별 진입점 1개 | [`docs/design/upcall.md`](docs/design/upcall.md) |

```
┌──────────────────────────────────────────┐
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -185,7 +185,7 @@ no per-platform Python → Kotlin binding code. Details: [design notes](docs/des

## 📊 Status

Honest summary — see the [roadmap](docs/roadmap/ROADMAP.md) for the full record.
Honest summary — see the [roadmap](docs/roadmap/ROADMAP.md) for what is left to do.

| Area | State |
|---|---|
Expand Down
2 changes: 1 addition & 1 deletion benchmarks/cost_table.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@

Why this exists
---------------
`docs/upcall-design.md` and `docs/downcall-design.md` both carry cost tables, and both have been
`docs/design/upcall.md` and `docs/design/downcall-design.md` both carry cost tables, and both have been
invalidated at least once by numbers that were transcribed by hand from a run whose conditions were
not written down next to them. The two failures were:

Expand Down
12 changes: 6 additions & 6 deletions benchmarks/rn-benchmark/COMPARISON.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# React Native's boundary against PythonMultiplatform's, measured the same way

Reference project: `/Volumes/macMini/thisisthepy/PythonMultiplatform`, whose figures below are
quoted from `docs/upcall-design.md`, `docs/downcall-design.md` and `ROADMAP.md` at the commit
quoted from `docs/design/upcall.md`, `docs/downcall-design.md` and `ROADMAP.md` at the commit
**`73c40da4`** (`develop`, 2026-08-14). **Nothing in the reference project was modified or re-run**;
its numbers are read, not reproduced.

Expand All @@ -13,7 +13,7 @@ its numbers are read, not reproduced.
**One fourth source, added for the iOS comparison and cited separately everywhere it appears:**
`docs/cost-table.md`, whose `iosSimulatorArm64` row was captured at **`958c0082b294`** at load
[1.97, 2.29, 5.92], marked uncontaminated, from three runs. §4b and §1c use it rather than
`upcall-design.md` because it is the only reference-side source that publishes an iOS upcall, its
`docs/design/upcall.md` because it is the only reference-side source that publishes an iOS upcall, its
same-run pure-Python callee control and its `withPython` scope figure together — the three cells the
comparison needs from one execution. Its upcall figure (2253.20–2312.97 ns) and §4a's quoted
2266–2301 ns are close but **are not the same measurement**; neither is derived from the other.
Expand Down Expand Up @@ -62,7 +62,7 @@ by reading generated code rather than documentation:

| | React Native | PythonMultiplatform |
|---|---|---|
| **iOS** | JSI (C++) → Objective-C message send. One crossing. The generated `RNBenchSpec.h` builds `NativeBenchSpecJSI` over `ObjCTurboModule::InitParams` and calls the ObjC method directly. | No runtime boundary at all: a `PyMethodDef` whose `ml_meth` is a `staticCFunction` in the same binary (`docs/upcall-design.md`, "cheapest of the three"). |
| **iOS** | JSI (C++) → Objective-C message send. One crossing. The generated `RNBenchSpec.h` builds `NativeBenchSpecJSI` over `ObjCTurboModule::InitParams` and calls the ObjC method directly. | No runtime boundary at all: a `PyMethodDef` whose `ml_meth` is a `staticCFunction` in the same binary (`docs/design/upcall.md` §7.4). |
| **Android** | JSI (C++) → **JNI** → Kotlin. Two crossings. Confirmed with `nm` on the built `libappmodules.so`: `NativeBenchSpecJSI::NativeBenchSpecJSI(JavaTurboModule::InitParams const&)`. | C entry point in `artMain/cinterop/jni_onload.def` → **JNI** → Kotlin. Two crossings, "one shim per shape, plus a JNI upcall per call". |

So on both projects Android is structurally the expensive side of the same asymmetry, for the same
Expand Down Expand Up @@ -107,7 +107,7 @@ all. Getting this wrong is the failure mode that matters most: two tables side b
comparison whether or not the cells correspond.

Reference-side sources: `UpcallBoundaryCostTest` (`commonTest`, quoted through
`docs/upcall-design.md` §"One upcall, across all five platforms") and `overhead/BenchmarkTest`
`docs/design/upcall.md` §"One upcall, across all five platforms") and `overhead/BenchmarkTest`
(quoted through `docs/downcall-design.md`).

| row here | what it charges | reference-project counterpart | same direction? | comparable? |
Expand Down Expand Up @@ -265,7 +265,7 @@ JS (Hermes, no JIT) --JSI--> C++ --JNI--> Kotlin, running on ART

The callee of every boundary row is `BenchModule.kt` executing on ART. **ART is precisely the
runtime whose tier-up the reference project measured, and it produced the largest warmup requirement
in its entire table** (`docs/upcall-design.md`, "Is 100 000 enough on a device?"):
in its entire table** (`docs/design/upcall.md` §7.2):

| reference target | first repetition ÷ plateau | flat from | margin at a 100 000 warmup |
|---|---|---|---|
Expand Down Expand Up @@ -350,7 +350,7 @@ row on iOS**, which is the opposite of what §3a predicts for Android.

### 4a. Ours, quoted (not re-run)

`UpcallBoundaryCostTest`, `docs/upcall-design.md`. **Re-quoted 2026-08-14 at `73c40da4`.** The
`UpcallBoundaryCostTest`, `docs/design/upcall.md`. **Re-quoted 2026-08-14 at `73c40da4`.** The
previous version of this table was taken at `bd0029d1` and is superseded: it carried four rows as
*italic* 3 000-warmup readings and two ART rows with three columns reading *not recorded*. Upstream
has since re-measured every device row at the 100 000 warmup and filled all five columns from one
Expand Down
2 changes: 1 addition & 1 deletion benchmarks/rn-benchmark/README.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# rn-benchmark

A React Native app whose only purpose is to price the **JS ↔ native boundary** with the same
measurement method the [PythonMultiplatform](../thisisthepy/PythonMultiplatform) repository uses to
measurement method the [PythonMultiplatform](../..) repository uses to
price its **Python ↔ Kotlin** boundary, so that the two can be read next to each other.

The comparison itself, including what corresponds to what and what is still unmeasured, is in
Expand Down
2 changes: 1 addition & 1 deletion benchmarks/rn-benchmark/RNBench/bench/appRun.js
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ const REPS = 5;
* JS (Hermes, no JIT) --JSI--> C++ --JNI--> **Kotlin, running on ART**
*
* The callee of every boundary row is `BenchModule.kt` executing on ART, and ART is precisely the
* runtime whose tier-up the reference project measured: its `docs/upcall-design.md` sweep found ART
* runtime whose tier-up the reference project measured: its `docs/design/upcall.md` sweep found ART
* on `pmp_api36` still 9% above its plateau at 70 000 calls and only arriving at ~90 000-100 000 --
* the largest warmup requirement of any configuration it measured, and the one that set its
* constant. So the RN Android boundary rows sit on the same curve, for the same reason, on the same
Expand Down
2 changes: 1 addition & 1 deletion benchmarks/rn-benchmark/RNBench/bench/harness.js
Original file line number Diff line number Diff line change
Expand Up @@ -134,7 +134,7 @@

/**
* Where a sweep settles, in the same terms the reference project's device sweep table uses
* (`docs/upcall-design.md`, "Is 100 000 enough on a device?"): first repetition over the plateau,
* (`docs/design/upcall.md` §7.2): first repetition over the plateau,
* the call count from which the series stays flat, and the margin a given warmup has over it.
*
* The plateau is the mean of the second half, so it cannot be dragged by the early readings the
Expand Down
4 changes: 2 additions & 2 deletions docs/INTENT.md
Original file line number Diff line number Diff line change
Expand Up @@ -78,7 +78,7 @@ Behind the per-platform entry point, every platform uses the same generated func
no per-platform Python → Kotlin binding code to maintain.

> Inferred — confirm with the maintainer: this "one mechanism everywhere" property is a goal, not
> only an implementation detail. It is stated as a decision in `PROJECT.md` and `docs/design/upcall-design.md`.
> only an implementation detail. It is stated as a decision in `PROJECT.md` and `docs/design/upcall.md`.

## 3. What it deliberately is NOT

Expand All @@ -87,7 +87,7 @@ no per-platform Python → Kotlin binding code to maintain.
| A binder that renames namespaces (`androidx` → `pythonx`, or any other) | §2.2 |
| A replacement for the `pythonx` packages | §2.3 — they are real packages in their own repository |
| Runtime-reflection or name-lookup based binding | §2.1 — impossible under Kotlin/Native and native image |
| A hand-written wrapper per Kotlin function or class | Generic adaptation plus generated tables; per-declaration wrappers were tried in 2024 and rotted (`docs/design/pythonx-adapter-design.md`) |
| A hand-written wrapper per Kotlin function or class | Generic adaptation plus generated tables; per-declaration wrappers were tried in 2024 and rotted (`docs/archive/pythonx-adapter-design.md`) |
| A Python re-implementation, transpiler, or JVM-hosted Python | It embeds CPython itself, so C extensions keep working |
| A build tool, dependency resolver or packager | That is pypackpack (the work) and toolchain (the Gradle vocabulary) |
| A sub-interpreter parallelism model | Rejected: extensions that do not declare multi-interpreter support fail to import; parallelism comes from free-threaded CPython |
Expand Down
46 changes: 46 additions & 0 deletions docs/archive/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
# Archive

Superseded design documents and the long narratives of investigations, kept for history.
**Nothing here is current.** Every file starts with a "Superseded by … on <date>" header that names
the document that replaced it. When an archived file disagrees with [`../INTENT.md`](../INTENT.md),
[`../SPEC.md`](../SPEC.md), a current document in `docs/design/`, `docs/platforms/` or
`docs/investigations/`, or the code, the archived file is the one that is wrong.

Archived on 2026-10-03 (issue #15). Documents that were merged rather than archived, such as the
three upcall documents now in [`../design/upcall.md`](../design/upcall.md), are not copied here. Their
full text is in git (`git log -- docs/design/upcall-design.md`).

## Superseded designs

| File | Replaced by | Why it is here |
|---|---|---|
| [`pythonx-adapter-design.md`](pythonx-adapter-design.md) | SPEC U-7/U-8, `KotlinSurface.kt`, [`../design/kotlin-extensions-in-python.md`](../design/kotlin-extensions-in-python.md) | The design in which the binder synthesised `pythonx.*` modules and renamed to snake_case. AGENTS.md §12.1–12.2 forbid both: Kotlin names stay Kotlin names, and `pythonx` is a real package in pythonx-compose. Its measurements of Compose (synthetic parameters, `pointerInput`, `draggable`, `layoutId`) are history; the behaviour is asserted by the `ksp-fixtures/compose` tests. |
| [`pyi-generation-pythonic-stubs.md`](pyi-generation-pythonic-stubs.md) | [`../design/pyi-generation-design.md`](../design/pyi-generation-design.md), SPEC B-7 | The Pythonic stub product: snake_case names, `pythonx` stubs, the stub manifest, `@overload` sets, and the `Modifier` metaclass/Protocol analysis. The plugin now emits Kotlin-name stubs only, and the Pythonic product belongs to pythonx-compose. |
| [`kotlin-extensions-in-python-pyi.md`](kotlin-extensions-in-python-pyi.md) | [`../design/pyi-generation-design.md`](../design/pyi-generation-design.md), SPEC B-7 | The original §4.5 of `kotlin-extensions-in-python.md`, which proposed Pythonic stubs. |
| [`android-ffm-design.md`](android-ffm-design.md) | [`../design/downcall-design.md`](../design/downcall-design.md), `python-multiplatform/src/androidMain/README.md` | The PanamaPort-based Android FFM plan. It was never built: AGENTS.md §12.9 forbids PanamaPort, and Android binds through JNI `RegisterNatives` (SPEC C-3). |
| [`threading-and-abi-3.15t-gate.md`](threading-and-abi-3.15t-gate.md) | [`../design/threading-and-abi.md`](../design/threading-and-abi.md) | The reasoning that gated free-threading on 3.15t. Free-threaded 3.14t now runs the desktop suite as an opt-in (`-PpythonFreeThreaded=true`, SPEC T-2). |
| [`downcall-design-android-first-diagnosis.md`](downcall-design-android-first-diagnosis.md) | [`../design/downcall-design.md`](../design/downcall-design.md) | The first diagnosis of the Android JNI argument-shift bug. The bug is fixed. |
| [`downcall-design-benchmark-drift.md`](downcall-design-benchmark-drift.md) | [`../design/downcall-design.md`](../design/downcall-design.md) ("Warmup"), [`../design/upcall.md`](../design/upcall.md) §7.2 | The investigation into benchmark figures that turned out to measure the benchmark itself (missing warmup). |
| [`wasm-design-experiment-log.md`](wasm-design-experiment-log.md) | [`../platforms/wasm-design.md`](../platforms/wasm-design.md) | The chronological lab notebook of the wasm experiment, including its "parked" phase and the conclusions that were wrong before they were tested. wasm is now integrated (`tools/wasm/build-cpython.sh`, `wasmJsTest`). |
| [`wasm-youtrack-issue.md`](wasm-youtrack-issue.md) | [`../platforms/wasm-design.md`](../platforms/wasm-design.md) | A draft Kotlin/Wasm compiler issue. Do not file it: Kotlin 2.4.20-Beta2 imports the shared memory, so the problem it describes no longer applies. |

## Investigation narratives — `investigations/`

The current file of the same name in [`../investigations/`](../investigations/) keeps the conclusion,
the numbers with their conditions, and how to reproduce them. The archived copy is the full original,
with its step-by-step reasoning, the hypotheses it rejected, and its corrections.

| File | Current |
|---|---|
| [`investigations/android-unregistered-surface.md`](investigations/android-unregistered-surface.md) | [`../investigations/android-unregistered-surface.md`](../investigations/android-unregistered-surface.md). The migration is done; this copy keeps the call-graph traces. |
| [`investigations/gc-scheduling-investigation.md`](investigations/gc-scheduling-investigation.md) | [`../investigations/gc-scheduling-investigation.md`](../investigations/gc-scheduling-investigation.md) |
| [`investigations/gil-parking-investigation.md`](investigations/gil-parking-investigation.md) | [`../investigations/gil-parking-investigation.md`](../investigations/gil-parking-investigation.md). This copy ends with the cause open; the current file names it: a borrowed pointer wrapped as owned, causing a double free. |
| [`investigations/jni-call-convention-audit.md`](investigations/jni-call-convention-audit.md) | [`../investigations/jni-call-convention-audit.md`](../investigations/jni-call-convention-audit.md). This copy keeps the 71-row classification table. |

## Not archived, for reference

- [`../design/typedpython.md`](../design/typedpython.md) is a live design proposal from another session
and stays in `docs/design/`. A note at its top says that the stub manifest it relies on moved to
pythonx-compose.
- [`../roadmap/ROADMAP.md`](../roadmap/ROADMAP.md) now lists only the work that is left. The old
progress log is in git: `git show 4ed98143:docs/roadmap/ROADMAP.md`.
Original file line number Diff line number Diff line change
@@ -1,3 +1,16 @@
> **Superseded** by [`../design/downcall-design.md`](../design/downcall-design.md) and
> [`python-multiplatform/src/androidMain/README.md`](../../python-multiplatform/src/androidMain/README.md) on 2026-10-03; kept for history.
> This was a plan to replace the Kotlin/Native JNI bridge on Android with a PanamaPort-style FFM path
> (ART method patching, hidden-API bypass, 14 NDK trampolines). It was **not** built: PanamaPort is
> not a dependency (AGENTS.md §12.9 — licence, and dependence on ART-internal struct layouts, see §2–§3
> below), and Android binds CPython through JNI `RegisterNatives` with the calling convention chosen
> per function and per API level (measured; SPEC C-3). The Kotlin/Native Android library
> (`libmultiplatform_python3.14.so`) is still the bridge. The 14-shape census in §6 is still
> accurate and became `jvmMain/.../ShapeDowncalls.kt`. The closing "Open constraint: GraalVM Native
> Image" section is also resolved: native-image upcalls are verified manually
> (`docs/platforms/graal-native-image-verification.md`, SPEC U-6) and the desktop FFM reachability
> metadata is guarded by `ReachabilityMetadataTest.kt` (SPEC C-2).

# Android FFM Design Document (PanamaPort Analysis)

This document provides an implementation design for replacing the existing Kotlin/Native JNI bridge on Android with a Panama-style FFM approach, based on an analysis of the PanamaPort reference implementation.
Expand Down
46 changes: 46 additions & 0 deletions docs/archive/downcall-design-android-first-diagnosis.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
# 다운콜 설계 초안의 Android 진단 (2026-08 초)

> **Superseded** by [`../design/downcall-design.md`](../design/downcall-design.md) on 2026-10-03; kept for history.
> 아래 "확인된 버그" 는 **수정되었다** — Android 는 `RegisterNatives` 로 전부 바인딩한다
> (`python-multiplatform/src/androidMain/README.md`, ROADMAP §2). 호출 규약 표의 전환 비용(115/35/25 ns)은
> 외부 벤치마크에서 가져온 값이며, 이 프로젝트가 실측한 값(API 26-36)과 반대 경향이다 — 실측은 본 문서의
> "Measured" 절과 `androidMain/README.md` 에 있다.

### 확인된 버그: 현재 Android JNI 배선이 어긋나 있다

`@CName` 이 만드는 C 함수는 선언한 Kotlin 인자만 받는다:

```
Java_python_native_ffi_bindings_PyList_1Size(jlong list)
```

그런데 `androidMain/bindings.kt` 는 이를 **일반 JNI 메서드**로 선언한다. 일반 JNI 메서드를 ART 는
이렇게 호출한다:

```
Java_..._PyList_1Size(JNIEnv* env, jobject thiz, jlong list)
```

**인자가 두 칸 밀린다.** `list` 자리에 `JNIEnv*` 가 들어간다. 확인 결과 `@CName` 익스포트 중
`JNIEnv` 를 받는 것은 0개이고, `@CriticalNative`/`@FastNative` 도 어디에도 없다.

즉 **Android JVM 경로는 컴파일만 되고 런타임에 동작하지 않는다.** 실행된 적이 없어 드러나지 않았다.

### ART 의 세 가지 호출 규약

| | 일반 JNI | `@FastNative` | `@CriticalNative` |
|---|---|---|---|
| 전환 오버헤드 | 115 ns | 35 ns | 25 ns |
| C 시그니처 | `(JNIEnv*, jobject, args…)` | 동일 | **`(args…)` 만** |
| 객체 인자·반환 | 가능 | 가능 | **원시 타입만** |
| 정적/인스턴스 | 둘 다 | 둘 다 | **정적만** |
| JVM 으로 콜백 | 가능 | 가능 | **불가** |
| 실행 중 GC | 허용 | **차단** | **차단** |

shape 트램폴린은 정적이고 `Long`/`Double` 만 주고받으므로 `@CriticalNative` 에 정확히 맞는다. 그리고
우리 `@CName` 익스포트가 `JNIEnv` 를 받지 않는다는 사실이 곧 `@CriticalNative` 규약과 일치하므로,
**애노테이션을 붙이는 것이 성능 개선인 동시에 위 버그의 수정**이다.

`@CriticalNative` 는 API 34 부터 공식 SDK 에 포함된다. minSdk 26 구간에서는 `RegisterNatives` 로
등록해야 한다 — `RegisterNatives` 는 호출 규약이 아니라 바인딩 수단이므로 둘은 대안 관계가 아니다.

Loading
Loading