Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 4 additions & 2 deletions docs/SPEC.md
Original file line number Diff line number Diff line change
Expand Up @@ -91,8 +91,10 @@ fail, then implement.
`docs/platforms/desktop-packaged-app.md`.
- **L-9a** (issue #74) Desktop packaging and loading of `libpython`: every `lib/**` entry of the desktop jar is a real
file (never 0 bytes -- Gradle extracts the python-build-standalone symlink `libpython3.14.so` as an empty file, so
symlinks are resolved and the real library is packed under the `System.mapLibraryName` name, no `.so.1.0`); linux
libraries are stripped of debug info when an ELF-capable `strip` exists on the build host (best-effort). The
symlinks are resolved and the real library is packed under the `System.mapLibraryName` name, no `.so.1.0`); desktop
archives are python-build-standalone's `install_only_stripped` flavour (issue #86; URL and SHA-256 pinned in
`python-checksums.properties`), so no linux `libpython*.so` in the jar has a `.debug_*` section (a stripped
3.14.7 x86_64 library is 33,106,704 bytes against 251,884,016 unstripped); no host `strip` is involved. The
classpath library is extracted to `<user cache>/python-multiplatform/<python version>/<platform>/` (macOS
`~/Library/Caches`, Windows `%LOCALAPPDATA%`, else `$XDG_CACHE_HOME` or `~/.cache`; fallback
`java.io.tmpdir/python-multiplatform-<user>`; override `-Dpython.multiplatform.cache`), never the working
Expand Down
7 changes: 7 additions & 0 deletions docs/platforms/python-version-acquisition.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,13 @@ The project currently uses `python-build-standalone` (maintained by Astral). For
* **Asset Naming Scheme:** The asset naming follows the pattern: `cpython-<python-version>+<build-date>-<target>-<flavour>.<ext>`.
* Targets include: `aarch64-apple-darwin`, `x86_64-apple-darwin`, `x86_64-unknown-linux-gnu`, `x86_64-pc-windows-msvc`, `aarch64-pc-windows-msvc`.
* Flavours include: `install_only`, `install_only_stripped`, and `full` (which also includes debug symbols, pgo/lto variants). The archives are available in `.tar.gz` and `.tar.zst` formats.
* **Flavour acquired: `install_only_stripped`** (issue #86). The plain `install_only` linux `libpython3.14.so.1.0`
carries debug info (251,884,016 bytes); the `install_only_stripped` one is 33,106,704 bytes with no `.debug_*`
section (it keeps `.symtab`, so `file` still says "not stripped"). The stripped flavour exists for every desktop
target and both GIL and free-threaded builds, for 3.14.7 and 3.15.0rc1 in release `20260807`, so no platform falls
back. The archives are pinned in `python-checksums.properties` and agree with the release's `SHA256SUMS`; the
desktop jar's build check (`DesktopJarLibrariesTest`) fails if a linux library in it has `.debug_info`. A host
`strip` (absent or ELF-blind on macOS) is not used.
* **URL Pattern:** `https://github.com/astral-sh/python-build-standalone/releases/download/<release-tag>/<asset-name>`

## 2. Free-threaded Variants
Expand Down
32 changes: 16 additions & 16 deletions python-checksums.properties
Original file line number Diff line number Diff line change
Expand Up @@ -5,19 +5,19 @@ android-x86_64-3.14.7=2c16ce2359565cd8c24f86cfb75630768ba6607e732946b294b969797f
android-x86_64-3.15.0rc1=f048f5c60b538efc64e4c26399a0a21202ce430949d5b2c10369d624548a5dc3
ios-3.14-b11=b591f3301bd22a4f423c49c746cac9e55558b909fd14d6eb8327ccc62234ab7b
ios-3.15.0rc1-pythonorg=178bf7bef9cd0f18b27cacb98b14332a5ba44c9427543ffb71f8809bc1f11c3c
linux-x86_64-3.14.7-20260807=3d1705fee7747c491d774e26fa91fad67e25d1eb3ede4124dc88501279f2e7d4
linux-x86_64-3.14.7-20260807-freethreaded=f9150ecd2080336500508f56b960400239b16ce9d3e0d6905f1dc93e41d28759
linux-x86_64-3.15.0rc1-20260807=6c4d2bf1ef2972203e9da61402a3c1f95a259c023ed154802f91317f963f5e1a
linux-x86_64-3.15.0rc1-20260807-freethreaded=46a2aed3c5037ee28a049e3b65d83f34c8a3738999709bddfdf6861a0217a3f5
macos-aarch64-3.14.7-20260807=433c2c61f5edfb2beda061d3b1f226bbe3493fa926cec68068843dd49110e539
macos-aarch64-3.14.7-20260807-freethreaded=1a2223bbcf11884be31c77ea5cb62702f9e12be5dbbd25118c8e00a7b0db33fa
macos-aarch64-3.15.0rc1-20260807=23b0ba386bfb7fe1274dfb6271f82c6286d3de02ebaccebd369fe08477f36759
macos-aarch64-3.15.0rc1-20260807-freethreaded=3dc0426570ebcbe90f5fafadf99e5851b78edca043a86d6f569dbfdf8be59756
macos-x86_64-3.14.7-20260807=2551ac222fbf0ec16ad8bd575bf94614d4da0877e74adb0e155cef019bf3007d
macos-x86_64-3.14.7-20260807-freethreaded=c6479d66d5451df756fb5beae0f8a866d55d42394c9ec6163e645791a745a619
macos-x86_64-3.15.0rc1-20260807=71b24ed7a775f2256e0cc7561a2754b22e188bee608a7962439861f57c60699d
macos-x86_64-3.15.0rc1-20260807-freethreaded=47ca20dec1c6f3070de39ad17057d12d97a9219223d5673c7757662cd923bb71
windows-x86_64-3.14.7-20260807=b06367695ce177e4abb2e02a8ff00fa60903c29fb381e40affaee2f51ef6115a
windows-x86_64-3.14.7-20260807-freethreaded=f7b293333186600624f0c68a0cbe766f372ea22dd20e26b09abe979894baaf0c
windows-x86_64-3.15.0rc1-20260807=bfed9c6b94f2bac9e266161fbc9460ece45f245de4a5dd465f64e58e93d54573
windows-x86_64-3.15.0rc1-20260807-freethreaded=d6745efcb2dad907ffe8826c6a9566dc21f8b1713335c7311c9307af985dc2f2
linux-x86_64-3.14.7-20260807=a2478d654ed51d443bae21ec20ad927f116b4f5aae4094ab74918a6aa38f0575
linux-x86_64-3.14.7-20260807-freethreaded=13f80b7fc242814c0be4e2a4d8283f39c7e1c0f3f0d8f6d6a15cedb5b62eb6d3
linux-x86_64-3.15.0rc1-20260807=aeb0d2cfe5e7028170039a95b96d74aa6830efe2fc49808122aa8f835d6e9f31
linux-x86_64-3.15.0rc1-20260807-freethreaded=c04c940dbf8eca39c66c73d8c51330341a5a7f9b3b13f689ec14521c6357a44d
macos-aarch64-3.14.7-20260807=532645be202d3df8510ab318ca5faed92060a694c8327144e14fa81f31af0f6d
macos-aarch64-3.14.7-20260807-freethreaded=618fe8d34b0704ce4a8aa2cfa6fa96063d6aa48970a131f19e1591d9785d1111
macos-aarch64-3.15.0rc1-20260807=bfd0fa8be7d7dfb5c75f6848757feb4bb9e4274da634e97e77aa00c8e521a6a0
macos-aarch64-3.15.0rc1-20260807-freethreaded=7dc7e3758b6a77358b24d795e0a7eee3b5f3d9bc5fa0e3e3c65db52898f07815
macos-x86_64-3.14.7-20260807=9964e2d618ebea03be8ea3e65ab0ecc0f2b030ce203345b8f92654641fd4de66
macos-x86_64-3.14.7-20260807-freethreaded=746207f0d2644c5c98619f9d0ed37824d9871ea8d27640edc779a46d85038b3b
macos-x86_64-3.15.0rc1-20260807=f36c0921d03468747ed74c434e4b04749a35593718cd641d0f08fa01e9c7885e
macos-x86_64-3.15.0rc1-20260807-freethreaded=69beed4659c78b3683f1ea56c12956c7234f16e0417edf24f7f69c5a9f7b09ec
windows-x86_64-3.14.7-20260807=840e368318d1630bd8efbd3f7de8fab32cb4bbd735baef733f9034f2676c6a92
windows-x86_64-3.14.7-20260807-freethreaded=c8323293c9e76e8b244a0e533894512f067dac2a6aa2dfcb3494cbe74b3af58e
windows-x86_64-3.15.0rc1-20260807=bc0d0f3c7b88873688f30bc78da8387a6653a5493424951d3a84c658c973198e
windows-x86_64-3.15.0rc1-20260807-freethreaded=e5998e014f16074ac9f6d61eb2a31ff8ac8bc331b61da2c5acf7d15abf4a2650
Original file line number Diff line number Diff line change
Expand Up @@ -111,7 +111,7 @@ internal fun hostDesktopPlatform(osName: String, osArch: String): String? {
name.startsWith("linux") -> "linux"
else -> return null
}
// Upstream publishes no aarch64 Windows `install_only` build, and this library has never
// Upstream publishes no aarch64 Windows `install_only_stripped` build, and this library has never
// shipped a `libpython` for one either.
if (os == "windows" && arch == "aarch64") return null
return "$os-$arch"
Expand Down Expand Up @@ -141,7 +141,7 @@ internal fun pbsAssetName(
platform: String,
freeThreaded: Boolean,
): String {
val flavour = if (freeThreaded) "freethreaded-install_only" else "install_only"
val flavour = if (freeThreaded) "freethreaded-install_only_stripped" else "install_only_stripped"
return "cpython-$pythonVersion+$pbsRelease-${pbsTripleFor(platform)}-$flavour.tar.gz"
}

Expand All @@ -152,8 +152,8 @@ internal fun pbsAssetUrl(pbsRelease: String, assetName: String): String =
* The SHA-256 upstream records for [assetName], or null when the release does not list it.
*
* Matched on the whole filename rather than with `endsWith`, which the root build uses and which
* is wrong in principle: `...-install_only.tar.gz` is a suffix of
* `...-freethreaded-install_only.tar.gz`, so asking for the default asset could return the
* is wrong in principle: `...-install_only_stripped.tar.gz` is a suffix of
* `...-freethreaded-install_only_stripped.tar.gz`, so asking for the default asset could return the
* free-threaded hash and fail verification on a file that is perfectly intact.
*/
internal fun sha256Of(sha256sums: String, assetName: String): String? =
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -62,15 +62,15 @@ class PythonHomeStagingTest {
// *different* CPython build than the `libpython` in `desktopJar` was taken from, and the
// two can disagree about ABI while both being version 3.14.7.
assertEquals(
"cpython-3.14.7+20260807-aarch64-apple-darwin-install_only.tar.gz",
"cpython-3.14.7+20260807-aarch64-apple-darwin-install_only_stripped.tar.gz",
pbsAssetName("3.14.7", "20260807", "macos-aarch64", freeThreaded = false),
)
assertEquals(
"cpython-3.14.7+20260807-x86_64-unknown-linux-gnu-install_only.tar.gz",
"cpython-3.14.7+20260807-x86_64-unknown-linux-gnu-install_only_stripped.tar.gz",
pbsAssetName("3.14.7", "20260807", "linux-x86_64", freeThreaded = false),
)
assertEquals(
"cpython-3.14.7+20260807-x86_64-pc-windows-msvc-install_only.tar.gz",
"cpython-3.14.7+20260807-x86_64-pc-windows-msvc-install_only_stripped.tar.gz",
pbsAssetName("3.14.7", "20260807", "windows-x86_64", freeThreaded = false),
)
}
Expand All @@ -81,7 +81,7 @@ class PythonHomeStagingTest {
// prefix holds `lib/python3.14t/` rather than `lib/python3.14/`. Staging the default
// tarball for a free-threaded consumer produces a prefix `PythonHomeCheck` rejects.
assertEquals(
"cpython-3.14.7+20260807-aarch64-apple-darwin-freethreaded-install_only.tar.gz",
"cpython-3.14.7+20260807-aarch64-apple-darwin-freethreaded-install_only_stripped.tar.gz",
pbsAssetName("3.14.7", "20260807", "macos-aarch64", freeThreaded = true),
)
}
Expand All @@ -90,8 +90,8 @@ class PythonHomeStagingTest {
fun theUrlIsTheAstralReleaseTheRepoPins() {
assertEquals(
"https://github.com/astral-sh/python-build-standalone/releases/download/20260807/" +
"cpython-3.14.7+20260807-aarch64-apple-darwin-install_only.tar.gz",
pbsAssetUrl("20260807", "cpython-3.14.7+20260807-aarch64-apple-darwin-install_only.tar.gz"),
"cpython-3.14.7+20260807-aarch64-apple-darwin-install_only_stripped.tar.gz",
pbsAssetUrl("20260807", "cpython-3.14.7+20260807-aarch64-apple-darwin-install_only_stripped.tar.gz"),
)
}

Expand All @@ -103,29 +103,29 @@ class PythonHomeStagingTest {
// than on a line index matters: the file lists every asset of the release, hundreds of
// them, in no order this code should depend on.
val sums = """
1111111111111111111111111111111111111111111111111111111111111111 cpython-3.14.7+20260807-x86_64-pc-windows-msvc-install_only.tar.gz
2222222222222222222222222222222222222222222222222222222222222222 cpython-3.14.7+20260807-aarch64-apple-darwin-install_only.tar.gz
1111111111111111111111111111111111111111111111111111111111111111 cpython-3.14.7+20260807-x86_64-pc-windows-msvc-install_only_stripped.tar.gz
2222222222222222222222222222222222222222222222222222222222222222 cpython-3.14.7+20260807-aarch64-apple-darwin-install_only_stripped.tar.gz
""".trimIndent()
assertEquals(
"2222222222222222222222222222222222222222222222222222222222222222",
sha256Of(sums, "cpython-3.14.7+20260807-aarch64-apple-darwin-install_only.tar.gz"),
sha256Of(sums, "cpython-3.14.7+20260807-aarch64-apple-darwin-install_only_stripped.tar.gz"),
)
}

@Test
fun aFilenameThatIsOnlyASuffixOfAnotherIsNotMatched() {
// `endsWith` is what the root build uses, and it is wrong in principle here: the
// free-threaded asset name *contains* nothing of the default one, but
// `...-install_only.tar.gz` is a suffix of `...-freethreaded-install_only.tar.gz`. Asking
// `...-install_only_stripped.tar.gz` is a suffix of `...-freethreaded-install_only_stripped.tar.gz`. Asking
// for the default asset must not return the free-threaded hash, or verification fails on
// a file that is actually intact.
val sums = """
3333333333333333333333333333333333333333333333333333333333333333 cpython-3.14.7+20260807-aarch64-apple-darwin-freethreaded-install_only.tar.gz
4444444444444444444444444444444444444444444444444444444444444444 cpython-3.14.7+20260807-aarch64-apple-darwin-install_only.tar.gz
3333333333333333333333333333333333333333333333333333333333333333 cpython-3.14.7+20260807-aarch64-apple-darwin-freethreaded-install_only_stripped.tar.gz
4444444444444444444444444444444444444444444444444444444444444444 cpython-3.14.7+20260807-aarch64-apple-darwin-install_only_stripped.tar.gz
""".trimIndent()
assertEquals(
"4444444444444444444444444444444444444444444444444444444444444444",
sha256Of(sums, "cpython-3.14.7+20260807-aarch64-apple-darwin-install_only.tar.gz"),
sha256Of(sums, "cpython-3.14.7+20260807-aarch64-apple-darwin-install_only_stripped.tar.gz"),
)
}

Expand Down
28 changes: 4 additions & 24 deletions python-multiplatform/build.gradle.kts
Original file line number Diff line number Diff line change
Expand Up @@ -437,7 +437,7 @@ val desktopTargets = mapOf(
)

val downloadTasks = desktopTargets.map { (platform, pbsTarget) ->
val flavour = if (pythonFreeThreaded) "freethreaded-install_only" else "install_only"
val flavour = if (pythonFreeThreaded) "freethreaded-install_only_stripped" else "install_only_stripped"
val assetName = "cpython-$configuredPythonVersion+$pbsRelease-$pbsTarget-$flavour.tar.gz"
val url = "https://github.com/astral-sh/python-build-standalone/releases/download/$pbsRelease/$assetName"
val archive = file("$downloadDir/$assetName")
Expand Down Expand Up @@ -1137,7 +1137,9 @@ kotlin {
// 1. drops every zero-length library (a symlink stub is never a library),
// 2. renames `libX.so.1.0` to `libX.so`, so the jar carries the real file under the name
// `System.mapLibraryName` produces,
// 3. strips debug info from linux libraries, best-effort (see below).
// Debug info is not stripped here (issue #86): the archives acquired above are python-build-
// standalone's `install_only_stripped` flavour, so the linux library already has none, and
// a host `strip` (absent or ELF-blind on macOS) would only be a second, unreliable mechanism.
val stageDesktopLibraries = tasks.register<Sync>("stageDesktopLibraries") {
dependsOn(downloadAllPythonBuilds)
val stageDir = layout.buildDirectory.dir("desktop-libs")
Expand All @@ -1157,28 +1159,6 @@ kotlin {
}
includeEmptyDirs = false
}
doLast {
// Best-effort: GNU strip / llvm-strip are not on every build host (Apple's strip
// cannot read ELF). When none works the unstripped library ships, and says so.
val candidates = listOf("llvm-strip", "x86_64-linux-gnu-strip", "strip")
stageDir.get().asFile.walkTopDown()
.filter { it.isFile && it.parentFile.name.startsWith("linux-") && it.name.endsWith(".so") }
.forEach { lib ->
val before = lib.length()
val done = candidates.any { tool ->
try {
val proc = ProcessBuilder(tool, "--strip-debug", lib.absolutePath)
.redirectErrorStream(true).start()
proc.inputStream.readBytes()
proc.waitFor() == 0
} catch (_: java.io.IOException) { false }
}
logger.lifecycle(
if (done) "stripped ${lib.name}: $before -> ${lib.length()} bytes"
else "no ELF-capable strip found; ${lib.name} ships unstripped ($before bytes)"
)
}
}
}
tasks.withType<Jar>().matching { it.name == "desktopJar" }.configureEach {
if (configuredPythonVersion == "3.13.0" && !pythonFreeThreaded) {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -40,4 +40,69 @@ class DesktopJarLibrariesTest {
assertTrue(names.none { Regex("""\.so\.\d""").containsMatchIn(it) }, "versioned .so.N.M names must not ship: $names")
}
}

/**
* Issue #86. python-build-standalone's plain `install_only` linux library carries debug info
* (252 MB); the `install_only_stripped` flavour does not. Expected red before the switch: the
* `.debug_info` section of `lib/linux-x86_64/libpython3.14.so`.
*/
@Test
fun linuxLibrariesHaveNoDebugInfo() {
jar().use { zip ->
val libs = zip.entries().asSequence()
.filter { !it.isDirectory && it.name.startsWith("lib/linux-") && Regex("""libpython.*\.so$""").containsMatchIn(it.name) }
.toList()
val offenders = libs.filter { e -> elfSectionNames(zip.getInputStream(e).use { it.readBytes() }).any { it.startsWith(".debug_info") } }
.map { "${it.name} (${it.size} bytes)" }
assertTrue(offenders.isEmpty(), "linux libpython with a .debug_info section: $offenders")
}
}

@Test
fun elfParserSeesDebugInfoInItsOwnFixture() {
// Guards the parser: a check that cannot find a section proves nothing.
val names = elfSectionNames(buildElf64(listOf(".text", ".debug_info")))
assertTrue(".debug_info" in names, "parsed $names")
assertTrue(".debug_info" !in elfSectionNames(buildElf64(listOf(".text"))))
}

/** Section names of a little-endian ELF64 image, read from its section header table. */
private fun elfSectionNames(d: ByteArray): List<String> {
require(d.size > 0x40 && d[0] == 0x7f.toByte() && d[1] == 'E'.code.toByte() && d[4].toInt() == 2 && d[5].toInt() == 1) { "not a little-endian ELF64" }
val bb = java.nio.ByteBuffer.wrap(d).order(java.nio.ByteOrder.LITTLE_ENDIAN)
val shoff = bb.getLong(0x28).toInt()
val shentsize = bb.getShort(0x3A).toInt() and 0xFFFF
val shnum = bb.getShort(0x3C).toInt() and 0xFFFF
val shstrndx = bb.getShort(0x3E).toInt() and 0xFFFF
if (shoff == 0 || shnum == 0) return emptyList()
val strOff = bb.getLong(shoff + shstrndx * shentsize + 0x18).toInt()
return (0 until shnum).map { i ->
var p = strOff + bb.getInt(shoff + i * shentsize)
val sb = StringBuilder()
while (d[p].toInt() != 0) sb.append(d[p++].toInt().toChar())
sb.toString()
}
}

/** A minimal ELF64 image: header, a string table, and section headers named [names] (index 0 is null). */
private fun buildElf64(names: List<String>): ByteArray {
val strtab = java.io.ByteArrayOutputStream().also { it.write(0) }
val nameOffsets = names.map { n -> strtab.size().also { strtab.write(n.toByteArray()); strtab.write(0) } }
val shstrName = strtab.size().also { strtab.write(".shstrtab".toByteArray()); strtab.write(0) }
val str = strtab.toByteArray()
val shnum = names.size + 2 // null + names + .shstrtab
val shoff = 0x40 + str.size
val bb = java.nio.ByteBuffer.allocate(shoff + shnum * 64).order(java.nio.ByteOrder.LITTLE_ENDIAN)
bb.put(byteArrayOf(0x7f, 'E'.code.toByte(), 'L'.code.toByte(), 'F'.code.toByte(), 2, 1, 1))
bb.putLong(0x28, shoff.toLong()); bb.putShort(0x3A, 64); bb.putShort(0x3C, shnum.toShort()); bb.putShort(0x3E, (shnum - 1).toShort())
bb.position(0x40); bb.put(str)
fun header(index: Int, nameOff: Int, off: Int, size: Int) {
val b = shoff + index * 64
bb.putInt(b, nameOff); bb.putInt(b + 4, if (index == shnum - 1) 3 else 1)
bb.putLong(b + 0x18, off.toLong()); bb.putLong(b + 0x20, size.toLong())
}
names.forEachIndexed { i, _ -> header(i + 1, nameOffsets[i], 0, 0) }
header(shnum - 1, shstrName, 0x40, str.size)
return bb.array()
}
}
Loading