diff --git a/docs/SPEC.md b/docs/SPEC.md index 19ac3529f..ed5118bc1 100644 --- a/docs/SPEC.md +++ b/docs/SPEC.md @@ -294,6 +294,12 @@ All in `PM/commonTest`, so they run wherever the interpreter loads. consumable `*Elements` configuration, and the resource `META-INF/python-multiplatform/python.properties` in the jar and AAR (see `docs/platforms/python-version-acquisition.md` "Published version") — `GP/EmbeddedPythonVersionTest.kt`. +- **T-5** Every acquired CPython reports the pinned version: `PY_VERSION` in each target's `patchlevel.h` + equals `pythonVersion`, and a `downloadPython_*` task fails the build, naming the target and both + versions, when it does not (issue #47: iOS shipped 3.14.6 under `pythonVersion=3.14.7`). The iOS + extraction is stamped with its lock key so bumping the pinned archive discards the old tree. + `Status: implemented` — `GP/AcquiredHeaderVersionTest.kt` (the real-tree case needs the trees from + `downloadAllPythonBuilds`; it skips when nothing was acquired). ## 8. Measurement diff --git a/docs/platforms/python-version-acquisition.md b/docs/platforms/python-version-acquisition.md index 8e005a87b..9ae64b95f 100644 --- a/docs/platforms/python-version-acquisition.md +++ b/docs/platforms/python-version-acquisition.md @@ -20,6 +20,7 @@ The project currently uses `python-build-standalone` (maintained by Astral). For ## 3. Android and iOS Support * **Android:** Prebuilt artifacts for Android are provided natively by python.org. These archives are plain NDK cross-compilations (e.g., CPython's own build tree). They can be found at `https://www.python.org/ftp/python//python---linux-android.tar.gz`. +* **iOS (3.14):** pinned to BeeWare `3.14-b11` (Python 3.14.7). `3.14-b10`, pinned earlier, shipped 3.14.6 while every other target shipped 3.14.7 (issue #47); b11 is the first 3.14 build with 3.14.7 (release 2026-09-04; its `VERSIONS` file and all four `patchlevel.h` say 3.14.7). The build now checks `PY_VERSION` in the extracted headers against `pythonVersion` for every target and fails if they differ. python.org's own iOS XCframework starts at 3.15 (section 5), so there is no alternative 3.14 source. * **iOS:** Prebuilt artifacts for iOS are provided by the BeeWare project via their `Python-Apple-support` repository, which are CPython's own iOS build layouts (e.g. `prefix: iOS/Frameworks/...`). They are located at `https://github.com/beeware/Python-Apple-support/releases`. The kivy toolchains are **not** the source of these artifacts. * **python-build-standalone Status:** `python-build-standalone` has moved from indygreg to Astral (`https://github.com/astral-sh/python-build-standalone`). It **does not** publish prebuilt binaries for `android` or `ios`. * **Prebuilt-cpython Status:** The `python/prebuilt-cpython` repository is currently a planning repository containing no artifacts, so it is not a source. @@ -47,7 +48,7 @@ The project currently uses `python-build-standalone` (maintained by Astral). For * **The signer identity is pinned, and has to be version-keyed.** Verifying a bundle *without* pinning an identity proves only that somebody holding a Sigstore certificate signed the bytes, which anyone can arrange. The build pins the Fulcio SAN and OIDC issuer of the actual CPython release manager, per . That must be a map rather than a constant, because it changes per release series: 3.14/3.15 are `hugo@python.org` via `https://github.com/login/oauth`, whereas 3.12/3.13 are `thomas@python.org` via `https://accounts.google.com`. An unrecorded series is a hard build failure, never a silent skip. * **Opt-in — `-PverifyPythonSignatures=true`.** Off by default because `sigstore-java` pulls in grpc-netty-shaded, protobuf, bouncycastle and guava, and because fetching the TUF trust root needs network access, which would turn an offline build from working into failing. Gradle resolves the configuration lazily, so a default build downloads none of it (measured: a default `downloadPython_android_*` run mentions Sigstore zero times). * **Desktop (`python-build-standalone`) — Sigstore verified via GitHub Attestations.** The release carries 853 assets and the only non-archive among them is `SHA256SUMS`: there is no `.sigstore` file. Provenance is published through GitHub's *attestations* API instead, keyed by artifact **digest** rather than filename, and that endpoint is rate-limited to 60 requests/hour unauthenticated. Because of this rate limit and the bundle size (~212KB each), the build does not check these bundles into the repository. Instead, when `-PverifyPythonSignatures=true` is set, it fetches the DSSE bundle from the API, caches it locally as `.sigstore`, and uses `sigstore-java` to verify it against the pinned `astral-sh/python-build-standalone` workflow identity. Desktop is therefore protected by the lockfile, the release's own `SHA256SUMS`, and Sigstore. -* **iOS (BeeWare, ≤ 3.14) — nothing exists to verify against.** The `Python-Apple-support` releases publish five `tar.gz` assets and nothing else: no checksums, no signatures, and no GitHub attestations. The lockfile pin is the only honest instrument available here, and no amount of build wiring changes that. +* **iOS (BeeWare, ≤ 3.14; `3.14-b11` pin `b591f330...ab7b`, taken from GitHub's asset digest and recomputed locally) — nothing exists to verify against.** The `Python-Apple-support` releases publish five `tar.gz` assets and nothing else: no checksums, no signatures, and no GitHub attestations. The lockfile pin is the only honest instrument available here, and no amount of build wiring changes that. * **The lockfile is not superseded by any of this.** The two gates prove different things — the lockfile says "these are the exact bytes this repository reviewed and pinned", Sigstore says "these are the bytes the release manager actually signed". The lockfile is also the only check that works offline and the only one covering every source, so it stays unconditional and Sigstore is layered on top of it. * **The verification is known to fail when it should**, which is the only thing that makes it worth having. Two negative controls were run against the real 3.14.7 Android bundle: flipping one base64 character in `messageSignature.signature` gave `KeylessVerificationException: Artifact signature was not valid`, and pointing the identity map at the 3.13 release manager gave `No provided certificate identities matched values in certificate`. Both failed the build with a non-zero exit. diff --git a/gradle.properties b/gradle.properties index 6e29d8712..a0b3f7c09 100644 --- a/gradle.properties +++ b/gradle.properties @@ -26,7 +26,7 @@ org.jetbrains.compose.experimental.uikit.enabled=true pythonVersion=3.14.7 pythonFreeThreaded=false pythonBuildStandaloneRelease=20260807 -pythonAppleSupportBuild=b10 +pythonAppleSupportBuild=b11 # GraalVM native-image (sample's nativeCompile task). Liberica NIK is not in a location Gradle's # toolchain auto-detection scans, so it is registered explicitly here. Adding a path only makes a diff --git a/python-checksums.properties b/python-checksums.properties index 82c0a553a..47988ce2d 100644 --- a/python-checksums.properties +++ b/python-checksums.properties @@ -3,7 +3,7 @@ android-aarch64-3.14.7=6d50cc3aa66e414a439594089bcdfb5f1264358155c70c1f00471c24c android-aarch64-3.15.0rc1=7d6fb28a69368841656f919a08141b0e3720837ac67987388037ee9d2a5c5492 android-x86_64-3.14.7=2c16ce2359565cd8c24f86cfb75630768ba6607e732946b294b969797f583b60 android-x86_64-3.15.0rc1=f048f5c60b538efc64e4c26399a0a21202ce430949d5b2c10369d624548a5dc3 -ios-3.14-b10=200ef60eb67be0483ceb638daa9048f84f41a9a952707a5ad4c3198037c7b583 +ios-3.14-b11=b591f3301bd22a4f423c49c746cac9e55558b909fd14d6eb8327ccc62234ab7b ios-3.15.0rc1-pythonorg=178bf7bef9cd0f18b27cacb98b14332a5ba44c9427543ffb71f8809bc1f11c3c linux-x86_64-3.14.7-20260807=3d1705fee7747c491d774e26fa91fad67e25d1eb3ede4124dc88501279f2e7d4 linux-x86_64-3.14.7-20260807-freethreaded=f9150ecd2080336500508f56b960400239b16ce9d3e0d6905f1dc93e41d28759 diff --git a/python-multiplatform-gradle-plugin/src/main/kotlin/python/multiplatform/gradle/AcquiredHeaderVersion.kt b/python-multiplatform-gradle-plugin/src/main/kotlin/python/multiplatform/gradle/AcquiredHeaderVersion.kt new file mode 100644 index 000000000..a95951aa5 --- /dev/null +++ b/python-multiplatform-gradle-plugin/src/main/kotlin/python/multiplatform/gradle/AcquiredHeaderVersion.kt @@ -0,0 +1,32 @@ +package python.multiplatform.gradle + +import java.io.File + +/** + * Reads `PY_VERSION` out of the acquired CPython headers (`patchlevel.h`) and compares it with the + * pinned `pythonVersion` (issue #47). The archive name says what a download was meant to be; the + * header says what it is. + */ +object AcquiredHeaderVersion { + private val pyVersion = Regex("""^\s*#\s*define\s+PY_VERSION\s+"([^"]+)"""", RegexOption.MULTILINE) + + fun parse(patchlevelText: String): String = + requireNotNull(pyVersion.find(patchlevelText)) { "no PY_VERSION in patchlevel.h" }.groupValues[1] + + fun patchlevelFile(extractedRoot: File, target: String, pythonVersion: String): File = + extractedRoot.resolve(CPythonIncludeLayout.relativePath(target, CPythonFlavour.GIL, pythonVersion)).resolve("patchlevel.h") + + /** target -> description, for every GIL target whose headers exist and report a different version. */ + fun mismatches(extractedRoot: File, pythonVersion: String): Map = + CPythonIncludeLayout.allTargets.mapNotNull { t -> + val f = patchlevelFile(extractedRoot, t, pythonVersion) + if (!f.isFile) return@mapNotNull null + val actual = parse(f.readText()) + if (actual == pythonVersion) null else t to "headers say PY_VERSION $actual, expected $pythonVersion ($f)" + }.toMap() + + fun requireAll(extractedRoot: File, pythonVersion: String) { + val bad = mismatches(extractedRoot, pythonVersion) + check(bad.isEmpty()) { "Acquired CPython does not match pythonVersion=$pythonVersion:\n" + bad.entries.joinToString("\n") { " ${it.key}: ${it.value}" } } + } +} diff --git a/python-multiplatform-gradle-plugin/src/test/kotlin/python/multiplatform/gradle/AcquiredHeaderVersionTest.kt b/python-multiplatform-gradle-plugin/src/test/kotlin/python/multiplatform/gradle/AcquiredHeaderVersionTest.kt new file mode 100644 index 000000000..c0dc4a075 --- /dev/null +++ b/python-multiplatform-gradle-plugin/src/test/kotlin/python/multiplatform/gradle/AcquiredHeaderVersionTest.kt @@ -0,0 +1,71 @@ +package python.multiplatform.gradle + +import java.io.File +import kotlin.test.Test +import kotlin.test.assertEquals +import kotlin.test.assertFailsWith +import kotlin.test.assertTrue + +/** + * Issue #47: every acquired CPython must be the version `pythonVersion` names. iOS shipped 3.14.6 + * (BeeWare `3.14-b10`) while every other target shipped 3.14.7, and nothing noticed. + */ +class AcquiredHeaderVersionTest { + + private fun patchlevel(version: String) = """ + /* Version parsed out into numeric values */ + #define PY_MAJOR_VERSION 3 + #define PY_VERSION "$version" + #define PY_VERSION_HEX 0x030e07f0 + """.trimIndent() + + @Test + fun parsesPyVersionOutOfPatchlevelH() { + assertEquals("3.14.7", AcquiredHeaderVersion.parse(patchlevel("3.14.7"))) + assertEquals("3.15.0rc1", AcquiredHeaderVersion.parse(patchlevel("3.15.0rc1"))) + } + + @Test + fun aHeaderWithoutPyVersionIsAnError() { + assertFailsWith { AcquiredHeaderVersion.parse("#define PY_MAJOR_VERSION 3") } + } + + @Test + fun matchingTreePassesAndMismatchNamesTheTargetAndBothVersions() { + val root = kotlin.io.path.createTempDirectory("acquired").toFile() + try { + for (t in CPythonIncludeLayout.allTargets) { + val v = if (t.startsWith("ios")) "3.14.6" else "3.14.7" + root.resolve(CPythonIncludeLayout.relativePath(t, CPythonFlavour.GIL, "3.14.7")) + .also { it.mkdirs() }.resolve("patchlevel.h").writeText(patchlevel(v)) + } + val failures = AcquiredHeaderVersion.mismatches(root, "3.14.7") + assertEquals(CPythonIncludeLayout.iosTargets.toSet(), failures.keys) + assertTrue(failures.getValue("ios-arm64").contains("3.14.6")) + val e = assertFailsWith { AcquiredHeaderVersion.requireAll(root, "3.14.7") } + assertTrue(e.message!!.contains("ios-arm64") && e.message!!.contains("3.14.6") && e.message!!.contains("3.14.7")) + } finally { + root.deleteRecursively() + } + } + + /** + * The real acquired trees (`python-multiplatform/build/python-standalone/extracted//`), + * as extracted by `downloadAllPythonBuilds`. Skipped only when nothing was acquired at all; a + * target that is present must match, and iOS must be present once anything is. + */ + @Test + fun everyAcquiredTargetReportsThePinnedPythonVersion() { + val gradleProps = File("../gradle.properties").readLines() + val pinned = gradleProps.first { it.startsWith("pythonVersion=") }.substringAfter('=').trim() + val root = File("../python-multiplatform/build/python-standalone/extracted/$pinned") + if (!root.isDirectory) { + println("SKIPPED: nothing acquired under $root; run :python-multiplatform:downloadAllPythonBuilds") + return + } + val present = CPythonIncludeLayout.allTargets.filter { AcquiredHeaderVersion.patchlevelFile(root, it, pinned).isFile } + assertTrue("ios-arm64" in present, "iOS headers missing under $root") + val failures = AcquiredHeaderVersion.mismatches(root, pinned) + assertTrue(failures.isEmpty(), "acquired headers disagree with pythonVersion=$pinned: $failures") + } +} diff --git a/python-multiplatform/build.gradle.kts b/python-multiplatform/build.gradle.kts index 6f8a0ac0d..9dfefdc63 100644 --- a/python-multiplatform/build.gradle.kts +++ b/python-multiplatform/build.gradle.kts @@ -39,7 +39,7 @@ plugins { val configuredPythonVersion = project.findProperty("pythonVersion")?.toString() ?: project.rootProject.version.toString() val pythonFreeThreaded = project.findProperty("pythonFreeThreaded")?.toString()?.toBoolean() ?: false val pbsRelease = project.findProperty("pythonBuildStandaloneRelease")?.toString() ?: "20260807" -val pythonAppleSupportBuild = project.findProperty("pythonAppleSupportBuild")?.toString() ?: "b10" +val pythonAppleSupportBuild = project.findProperty("pythonAppleSupportBuild")?.toString() ?: "b11" val pythonVersion = configuredPythonVersion val libraryVersion = "$pythonVersion-alpha01" @@ -113,6 +113,26 @@ val extractedDir = file("$downloadDir/extracted/$configuredPythonVersion") */ val desktopFlavourSuffix = if (pythonFreeThreaded) "-freethreaded" else "" +/** + * Fails the build when the extracted CPython's own headers report a version other than + * `pythonVersion` (issue #47). The archive name and the lockfile say what was meant to be + * downloaded; `PY_VERSION` in `patchlevel.h` says what it is. iOS once shipped 3.14.6 (BeeWare + * `3.14-b10`) under `pythonVersion=3.14.7` and nothing noticed. + */ +fun requireHeaderVersion(label: String, extracted: File) { + val headers = extracted.walkTopDown().filter { it.name == "patchlevel.h" }.toList() + if (headers.isEmpty()) throw GradleException("$label: no patchlevel.h under $extracted, cannot verify the CPython version") + val re = Regex("""^\s*#\s*define\s+PY_VERSION\s+"([^"]+)"""", RegexOption.MULTILINE) + for (h in headers) { + val actual = re.find(h.readText())?.groupValues?.get(1) + ?: throw GradleException("$label: no PY_VERSION in $h") + if (actual != configuredPythonVersion) { + throw GradleException("$label: $h says PY_VERSION $actual but pythonVersion=$configuredPythonVersion. " + + "Pin an archive that ships $configuredPythonVersion (see docs/platforms/python-version-acquisition.md).") + } + } +} + val checksumsFile = rootProject.file("python-checksums.properties") val pythonArchiveKeys = mutableMapOf() @@ -482,6 +502,7 @@ val downloadTasks = desktopTargets.map { (platform, pbsTarget) -> into(extractDir) } } + requireHeaderVersion(platform, extractDir) } } } @@ -531,6 +552,7 @@ val androidDownloadTasks = androidTargets.map { (platform, arch) -> into(extractDir) } } + requireHeaderVersion(platform, extractDir) } } } @@ -540,7 +562,7 @@ val androidDownloadTasks = androidTargets.map { (platform, arch) -> * * python.org began publishing an official iOS XCframework with 3.15 (the first entries in * `ftp/python/3.15.0/` are the 3.15.0b1 betas). BeeWare's Python-Apple-support, which was the - * only source before that, stops at `3.14-b10` and has no 3.15 tag. The two do not overlap: + * only source before that, stops at `3.14-b11` (3.14.7) and has no 3.15 tag. The two do not overlap: * 3.14 and earlier can only come from BeeWare, 3.15 and later only from python.org. So this is a * hard switch on the version, not a preference. * @@ -601,6 +623,15 @@ val downloadPython_ios = tasks.register("downloadPython_ios") { verifyChecksum(iosLockKey, iosArchive) if (iosFromPythonOrg) maybeVerifySigstore(iosArchive, iosUrl) + // The iOS extract dir is keyed by pythonVersion, not by archive, so bumping the pinned + // archive (b10 -> b11) would otherwise keep the old tree: "not empty, skip". A stamp of the + // archive's lock key + digest tells a stale tree from a current one (issue #47). + val iosStamp = file("$iosExtractDir/.archive-stamp") + val iosWanted = "$iosLockKey ${iosArchive.name}" + if (iosExtractDir.exists() && (!iosStamp.exists() || iosStamp.readText().trim() != iosWanted)) { + println("Discarding stale iOS extraction in $iosExtractDir") + iosExtractDir.deleteRecursively() + } val isEmpty = iosExtractDir.list()?.isEmpty() ?: true if (isEmpty) { println("Extracting $iosArchive to $iosExtractDir") @@ -608,7 +639,9 @@ val downloadPython_ios = tasks.register("downloadPython_ios") { from(tarTree(resources.gzip(iosArchive))) into(iosExtractDir) } + iosStamp.writeText(iosWanted + "\n") } + requireHeaderVersion("ios", iosExtractDir) } }