Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions docs/SPEC.md
Original file line number Diff line number Diff line change
Expand Up @@ -294,6 +294,12 @@ All in `PM/commonTest`, so they run wherever the interpreter loads.
consumable `*Elements` configuration, and the resource `META-INF/python-multiplatform/python.properties`
in the jar and AAR (see `docs/platforms/python-version-acquisition.md` "Published version") —
`GP/EmbeddedPythonVersionTest.kt`.
- **T-5** Every acquired CPython reports the pinned version: `PY_VERSION` in each target's `patchlevel.h`
equals `pythonVersion`, and a `downloadPython_*` task fails the build, naming the target and both
versions, when it does not (issue #47: iOS shipped 3.14.6 under `pythonVersion=3.14.7`). The iOS
extraction is stamped with its lock key so bumping the pinned archive discards the old tree.
`Status: implemented` — `GP/AcquiredHeaderVersionTest.kt` (the real-tree case needs the trees from
`downloadAllPythonBuilds`; it skips when nothing was acquired).

## 8. Measurement

Expand Down
3 changes: 2 additions & 1 deletion docs/platforms/python-version-acquisition.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ The project currently uses `python-build-standalone` (maintained by Astral). For
## 3. Android and iOS Support

* **Android:** Prebuilt artifacts for Android are provided natively by python.org. These archives are plain NDK cross-compilations (e.g., CPython's own build tree). They can be found at `https://www.python.org/ftp/python/<version>/python-<version>-<arch>-linux-android.tar.gz`.
* **iOS (3.14):** pinned to BeeWare `3.14-b11` (Python 3.14.7). `3.14-b10`, pinned earlier, shipped 3.14.6 while every other target shipped 3.14.7 (issue #47); b11 is the first 3.14 build with 3.14.7 (release 2026-09-04; its `VERSIONS` file and all four `patchlevel.h` say 3.14.7). The build now checks `PY_VERSION` in the extracted headers against `pythonVersion` for every target and fails if they differ. python.org's own iOS XCframework starts at 3.15 (section 5), so there is no alternative 3.14 source.
* **iOS:** Prebuilt artifacts for iOS are provided by the BeeWare project via their `Python-Apple-support` repository, which are CPython's own iOS build layouts (e.g. `prefix: iOS/Frameworks/...`). They are located at `https://github.com/beeware/Python-Apple-support/releases`. The kivy toolchains are **not** the source of these artifacts.
* **python-build-standalone Status:** `python-build-standalone` has moved from indygreg to Astral (`https://github.com/astral-sh/python-build-standalone`). It **does not** publish prebuilt binaries for `android` or `ios`.
* **Prebuilt-cpython Status:** The `python/prebuilt-cpython` repository is currently a planning repository containing no artifacts, so it is not a source.
Expand Down Expand Up @@ -47,7 +48,7 @@ The project currently uses `python-build-standalone` (maintained by Astral). For
* **The signer identity is pinned, and has to be version-keyed.** Verifying a bundle *without* pinning an identity proves only that somebody holding a Sigstore certificate signed the bytes, which anyone can arrange. The build pins the Fulcio SAN and OIDC issuer of the actual CPython release manager, per <https://www.python.org/download/sigstore/>. That must be a map rather than a constant, because it changes per release series: 3.14/3.15 are `hugo@python.org` via `https://github.com/login/oauth`, whereas 3.12/3.13 are `thomas@python.org` via `https://accounts.google.com`. An unrecorded series is a hard build failure, never a silent skip.
* **Opt-in — `-PverifyPythonSignatures=true`.** Off by default because `sigstore-java` pulls in grpc-netty-shaded, protobuf, bouncycastle and guava, and because fetching the TUF trust root needs network access, which would turn an offline build from working into failing. Gradle resolves the configuration lazily, so a default build downloads none of it (measured: a default `downloadPython_android_*` run mentions Sigstore zero times).
* **Desktop (`python-build-standalone`) — Sigstore verified via GitHub Attestations.** The release carries 853 assets and the only non-archive among them is `SHA256SUMS`: there is no `.sigstore` file. Provenance is published through GitHub's *attestations* API instead, keyed by artifact **digest** rather than filename, and that endpoint is rate-limited to 60 requests/hour unauthenticated. Because of this rate limit and the bundle size (~212KB each), the build does not check these bundles into the repository. Instead, when `-PverifyPythonSignatures=true` is set, it fetches the DSSE bundle from the API, caches it locally as `.sigstore`, and uses `sigstore-java` to verify it against the pinned `astral-sh/python-build-standalone` workflow identity. Desktop is therefore protected by the lockfile, the release's own `SHA256SUMS`, and Sigstore.
* **iOS (BeeWare, ≤ 3.14) — nothing exists to verify against.** The `Python-Apple-support` releases publish five `tar.gz` assets and nothing else: no checksums, no signatures, and no GitHub attestations. The lockfile pin is the only honest instrument available here, and no amount of build wiring changes that.
* **iOS (BeeWare, ≤ 3.14; `3.14-b11` pin `b591f330...ab7b`, taken from GitHub's asset digest and recomputed locally) — nothing exists to verify against.** The `Python-Apple-support` releases publish five `tar.gz` assets and nothing else: no checksums, no signatures, and no GitHub attestations. The lockfile pin is the only honest instrument available here, and no amount of build wiring changes that.
* **The lockfile is not superseded by any of this.** The two gates prove different things — the lockfile says "these are the exact bytes this repository reviewed and pinned", Sigstore says "these are the bytes the release manager actually signed". The lockfile is also the only check that works offline and the only one covering every source, so it stays unconditional and Sigstore is layered on top of it.
* **The verification is known to fail when it should**, which is the only thing that makes it worth having. Two negative controls were run against the real 3.14.7 Android bundle: flipping one base64 character in `messageSignature.signature` gave `KeylessVerificationException: Artifact signature was not valid`, and pointing the identity map at the 3.13 release manager gave `No provided certificate identities matched values in certificate`. Both failed the build with a non-zero exit.

Expand Down
2 changes: 1 addition & 1 deletion gradle.properties
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ org.jetbrains.compose.experimental.uikit.enabled=true
pythonVersion=3.14.7
pythonFreeThreaded=false
pythonBuildStandaloneRelease=20260807
pythonAppleSupportBuild=b10
pythonAppleSupportBuild=b11

# GraalVM native-image (sample's nativeCompile task). Liberica NIK is not in a location Gradle's
# toolchain auto-detection scans, so it is registered explicitly here. Adding a path only makes a
Expand Down
2 changes: 1 addition & 1 deletion python-checksums.properties
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ android-aarch64-3.14.7=6d50cc3aa66e414a439594089bcdfb5f1264358155c70c1f00471c24c
android-aarch64-3.15.0rc1=7d6fb28a69368841656f919a08141b0e3720837ac67987388037ee9d2a5c5492
android-x86_64-3.14.7=2c16ce2359565cd8c24f86cfb75630768ba6607e732946b294b969797f583b60
android-x86_64-3.15.0rc1=f048f5c60b538efc64e4c26399a0a21202ce430949d5b2c10369d624548a5dc3
ios-3.14-b10=200ef60eb67be0483ceb638daa9048f84f41a9a952707a5ad4c3198037c7b583
ios-3.14-b11=b591f3301bd22a4f423c49c746cac9e55558b909fd14d6eb8327ccc62234ab7b
ios-3.15.0rc1-pythonorg=178bf7bef9cd0f18b27cacb98b14332a5ba44c9427543ffb71f8809bc1f11c3c
linux-x86_64-3.14.7-20260807=3d1705fee7747c491d774e26fa91fad67e25d1eb3ede4124dc88501279f2e7d4
linux-x86_64-3.14.7-20260807-freethreaded=f9150ecd2080336500508f56b960400239b16ce9d3e0d6905f1dc93e41d28759
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
package python.multiplatform.gradle

import java.io.File

/**
* Reads `PY_VERSION` out of the acquired CPython headers (`patchlevel.h`) and compares it with the
* pinned `pythonVersion` (issue #47). The archive name says what a download was meant to be; the
* header says what it is.
*/
object AcquiredHeaderVersion {
private val pyVersion = Regex("""^\s*#\s*define\s+PY_VERSION\s+"([^"]+)"""", RegexOption.MULTILINE)

fun parse(patchlevelText: String): String =
requireNotNull(pyVersion.find(patchlevelText)) { "no PY_VERSION in patchlevel.h" }.groupValues[1]

fun patchlevelFile(extractedRoot: File, target: String, pythonVersion: String): File =
extractedRoot.resolve(CPythonIncludeLayout.relativePath(target, CPythonFlavour.GIL, pythonVersion)).resolve("patchlevel.h")

/** target -> description, for every GIL target whose headers exist and report a different version. */
fun mismatches(extractedRoot: File, pythonVersion: String): Map<String, String> =
CPythonIncludeLayout.allTargets.mapNotNull { t ->
val f = patchlevelFile(extractedRoot, t, pythonVersion)
if (!f.isFile) return@mapNotNull null
val actual = parse(f.readText())
if (actual == pythonVersion) null else t to "headers say PY_VERSION $actual, expected $pythonVersion ($f)"
}.toMap()

fun requireAll(extractedRoot: File, pythonVersion: String) {
val bad = mismatches(extractedRoot, pythonVersion)
check(bad.isEmpty()) { "Acquired CPython does not match pythonVersion=$pythonVersion:\n" + bad.entries.joinToString("\n") { " ${it.key}: ${it.value}" } }
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
package python.multiplatform.gradle

import java.io.File
import kotlin.test.Test
import kotlin.test.assertEquals
import kotlin.test.assertFailsWith
import kotlin.test.assertTrue

/**
* Issue #47: every acquired CPython must be the version `pythonVersion` names. iOS shipped 3.14.6
* (BeeWare `3.14-b10`) while every other target shipped 3.14.7, and nothing noticed.
*/
class AcquiredHeaderVersionTest {

private fun patchlevel(version: String) = """
/* Version parsed out into numeric values */
#define PY_MAJOR_VERSION 3
#define PY_VERSION "$version"
#define PY_VERSION_HEX 0x030e07f0
""".trimIndent()

@Test
fun parsesPyVersionOutOfPatchlevelH() {
assertEquals("3.14.7", AcquiredHeaderVersion.parse(patchlevel("3.14.7")))
assertEquals("3.15.0rc1", AcquiredHeaderVersion.parse(patchlevel("3.15.0rc1")))
}

@Test
fun aHeaderWithoutPyVersionIsAnError() {
assertFailsWith<IllegalArgumentException> { AcquiredHeaderVersion.parse("#define PY_MAJOR_VERSION 3") }
}

@Test
fun matchingTreePassesAndMismatchNamesTheTargetAndBothVersions() {
val root = kotlin.io.path.createTempDirectory("acquired").toFile()
try {
for (t in CPythonIncludeLayout.allTargets) {
val v = if (t.startsWith("ios")) "3.14.6" else "3.14.7"
root.resolve(CPythonIncludeLayout.relativePath(t, CPythonFlavour.GIL, "3.14.7"))
.also { it.mkdirs() }.resolve("patchlevel.h").writeText(patchlevel(v))
}
val failures = AcquiredHeaderVersion.mismatches(root, "3.14.7")
assertEquals(CPythonIncludeLayout.iosTargets.toSet(), failures.keys)
assertTrue(failures.getValue("ios-arm64").contains("3.14.6"))
val e = assertFailsWith<IllegalStateException> { AcquiredHeaderVersion.requireAll(root, "3.14.7") }
assertTrue(e.message!!.contains("ios-arm64") && e.message!!.contains("3.14.6") && e.message!!.contains("3.14.7"))
} finally {
root.deleteRecursively()
}
}

/**
* The real acquired trees (`python-multiplatform/build/python-standalone/extracted/<version>/`),
* as extracted by `downloadAllPythonBuilds`. Skipped only when nothing was acquired at all; a
* target that is present must match, and iOS must be present once anything is.
*/
@Test
fun everyAcquiredTargetReportsThePinnedPythonVersion() {
val gradleProps = File("../gradle.properties").readLines()
val pinned = gradleProps.first { it.startsWith("pythonVersion=") }.substringAfter('=').trim()
val root = File("../python-multiplatform/build/python-standalone/extracted/$pinned")
if (!root.isDirectory) {
println("SKIPPED: nothing acquired under $root; run :python-multiplatform:downloadAllPythonBuilds")
return
}
val present = CPythonIncludeLayout.allTargets.filter { AcquiredHeaderVersion.patchlevelFile(root, it, pinned).isFile }
assertTrue("ios-arm64" in present, "iOS headers missing under $root")
val failures = AcquiredHeaderVersion.mismatches(root, pinned)
assertTrue(failures.isEmpty(), "acquired headers disagree with pythonVersion=$pinned: $failures")
}
}
37 changes: 35 additions & 2 deletions python-multiplatform/build.gradle.kts
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ plugins {
val configuredPythonVersion = project.findProperty("pythonVersion")?.toString() ?: project.rootProject.version.toString()
val pythonFreeThreaded = project.findProperty("pythonFreeThreaded")?.toString()?.toBoolean() ?: false
val pbsRelease = project.findProperty("pythonBuildStandaloneRelease")?.toString() ?: "20260807"
val pythonAppleSupportBuild = project.findProperty("pythonAppleSupportBuild")?.toString() ?: "b10"
val pythonAppleSupportBuild = project.findProperty("pythonAppleSupportBuild")?.toString() ?: "b11"

val pythonVersion = configuredPythonVersion
val libraryVersion = "$pythonVersion-alpha01"
Expand Down Expand Up @@ -113,6 +113,26 @@ val extractedDir = file("$downloadDir/extracted/$configuredPythonVersion")
*/
val desktopFlavourSuffix = if (pythonFreeThreaded) "-freethreaded" else ""

/**
* Fails the build when the extracted CPython's own headers report a version other than
* `pythonVersion` (issue #47). The archive name and the lockfile say what was meant to be
* downloaded; `PY_VERSION` in `patchlevel.h` says what it is. iOS once shipped 3.14.6 (BeeWare
* `3.14-b10`) under `pythonVersion=3.14.7` and nothing noticed.
*/
fun requireHeaderVersion(label: String, extracted: File) {
val headers = extracted.walkTopDown().filter { it.name == "patchlevel.h" }.toList()
if (headers.isEmpty()) throw GradleException("$label: no patchlevel.h under $extracted, cannot verify the CPython version")
val re = Regex("""^\s*#\s*define\s+PY_VERSION\s+"([^"]+)"""", RegexOption.MULTILINE)
for (h in headers) {
val actual = re.find(h.readText())?.groupValues?.get(1)
?: throw GradleException("$label: no PY_VERSION in $h")
if (actual != configuredPythonVersion) {
throw GradleException("$label: $h says PY_VERSION $actual but pythonVersion=$configuredPythonVersion. " +
"Pin an archive that ships $configuredPythonVersion (see docs/platforms/python-version-acquisition.md).")
}
}
}

val checksumsFile = rootProject.file("python-checksums.properties")
val pythonArchiveKeys = mutableMapOf<String, File>()

Expand Down Expand Up @@ -482,6 +502,7 @@ val downloadTasks = desktopTargets.map { (platform, pbsTarget) ->
into(extractDir)
}
}
requireHeaderVersion(platform, extractDir)
}
}
}
Expand Down Expand Up @@ -531,6 +552,7 @@ val androidDownloadTasks = androidTargets.map { (platform, arch) ->
into(extractDir)
}
}
requireHeaderVersion(platform, extractDir)
}
}
}
Expand All @@ -540,7 +562,7 @@ val androidDownloadTasks = androidTargets.map { (platform, arch) ->
*
* python.org began publishing an official iOS XCframework with 3.15 (the first entries in
* `ftp/python/3.15.0/` are the 3.15.0b1 betas). BeeWare's Python-Apple-support, which was the
* only source before that, stops at `3.14-b10` and has no 3.15 tag. The two do not overlap:
* only source before that, stops at `3.14-b11` (3.14.7) and has no 3.15 tag. The two do not overlap:
* 3.14 and earlier can only come from BeeWare, 3.15 and later only from python.org. So this is a
* hard switch on the version, not a preference.
*
Expand Down Expand Up @@ -601,14 +623,25 @@ val downloadPython_ios = tasks.register("downloadPython_ios") {
verifyChecksum(iosLockKey, iosArchive)
if (iosFromPythonOrg) maybeVerifySigstore(iosArchive, iosUrl)

// The iOS extract dir is keyed by pythonVersion, not by archive, so bumping the pinned
// archive (b10 -> b11) would otherwise keep the old tree: "not empty, skip". A stamp of the
// archive's lock key + digest tells a stale tree from a current one (issue #47).
val iosStamp = file("$iosExtractDir/.archive-stamp")
val iosWanted = "$iosLockKey ${iosArchive.name}"
if (iosExtractDir.exists() && (!iosStamp.exists() || iosStamp.readText().trim() != iosWanted)) {
println("Discarding stale iOS extraction in $iosExtractDir")
iosExtractDir.deleteRecursively()
}
val isEmpty = iosExtractDir.list()?.isEmpty() ?: true
if (isEmpty) {
println("Extracting $iosArchive to $iosExtractDir")
copy {
from(tarTree(resources.gzip(iosArchive)))
into(iosExtractDir)
}
iosStamp.writeText(iosWanted + "\n")
}
requireHeaderVersion("ios", iosExtractDir)
}
}

Expand Down
Loading