From 2b8269a834c37b7885250418ff3b297dbea20c87 Mon Sep 17 00:00:00 2001 From: Raman Gupta Date: Mon, 19 Oct 2026 22:40:21 -0400 Subject: [PATCH 1/3] docs: document the CI signing subkey in releasing.adoc Releases still sign locally with the YubiKey primary, but a dedicated ed25519 signing subkey now exists for CI, stored as SIGNING_KEY and SIGNING_KEY_PASSPHRASE in the release environment. - Record both fingerprints, why the primary stays [SC], and why CI must sign with gpg rather than KOTLIN_TOOLCHAIN_SIGNING_KEY. - Add the expiry, rotation, revocation and key publishing procedures. --- docs/releasing.adoc | 74 ++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 73 insertions(+), 1 deletion(-) diff --git a/docs/releasing.adoc b/docs/releasing.adoc index bd7f3d2..f37987c 100644 --- a/docs/releasing.adoc +++ b/docs/releasing.adoc @@ -1,7 +1,8 @@ = Releasing Bootable Releases are published to Maven Central from a maintainer's machine, not from CI. -The GPG signing key never leaves local hardware; the GitHub Actions workflow only builds and tests. +They are signed with the maintainer's YubiKey-held primary key; the GitHub Actions workflow only builds and tests. +A separate signing subkey is set up for signing in CI (see <>), but no workflow uses it yet. The published version is derived from the release tag — a `vX.Y.Z` tag on the commit being released. `build.gradle.kts` only holds `developmentVersion` (a `-SNAPSHOT`), used for every untagged build. @@ -24,3 +25,74 @@ Central Portal credentials (`mavenCentralUsername` / `mavenCentralPassword`) are NOTE: This publishes every module, including the per-flavor coordinates (`boot-metro`, `boot-kodein`, …). Confirm they all appear in the Central deployment before releasing. + +[[signing]] +== Signing keys + +Artifacts are signed with OpenPGP key `1F3C E06C 51CE 20F7 C440 5181 943F 2875 3338 2C9D`. +Its primary key is on a YubiKey, and local releases sign with it through gpg-agent (`useGpgCmd()`). +It can both certify and sign (`[SC]`). + +The primary keeps its signing capability on purpose. +gpg checks a key's current usage flags when it verifies, so a certify-only primary would make gpg reject every signature the primary has already made ("No public key"). + +=== CI signing subkey + +A dedicated ed25519 signing subkey, `2B2E CF97 500A BAB4 E08C E247 008A FAD1 792E A3BB`, exists so that CI can sign releases. +The card primary certifies it, so its signatures verify as the same identity, but its secret is in software and it can be revoked on its own. + +Its secret is on no maintainer's machine. +It was exported with `gpg --armor --export-secret-subkeys '!'`, which yields a stub primary (`gnu-dummy`, no secret material) plus only this subkey, protected by a passphrase. +It was then removed from the local keyring with `gpg --delete-secret-keys '!'`, so local signing stays on the card. +The `!` limits both commands to exactly that subkey. + +It is stored as secrets in the `release` GitHub environment: + +`SIGNING_KEY`:: The ASCII-armored export. +`SIGNING_KEY_PASSPHRASE`:: Its passphrase. + +A CI job should import `SIGNING_KEY` into gpg and sign with gpg. +Don't hand it to the Kotlin Toolchain as `KOTLIN_TOOLCHAIN_SIGNING_KEY`. +Its BouncyCastle signer picks the signing-capable stub primary instead of the subkey and fails (see link:kotlin-toolchain-evaluation.adoc[the Kotlin Toolchain evaluation]). + +=== Expiry + +The primary and its card subkeys expire on 2028-07-08, and the CI subkey on 2028-10-18. +A subkey stops working when its primary expires, so extend them together a few months ahead, with the card plugged in: + +[source] +---- +gpg --quick-set-expire 1F3CE06C51CE20F7C4405181943F287533382C9D 2y +gpg --quick-set-expire 1F3CE06C51CE20F7C4405181943F287533382C9D 2y '*' +---- + +Then publish the key again (see <>). +`SIGNING_KEY` still carries the old expiry, and gpg in CI refuses to sign with an expired subkey. +Either have the job import the current public key after `SIGNING_KEY`, or rebuild the export: import the old export and the updated public key into a temporary `GNUPGHOME`, run the export command above again, and replace the secret. + +=== Rotation and revocation + +To rotate, create a new subkey with `gpg --quick-add-key 1F3CE06C51CE20F7C4405181943F287533382C9D ed25519 sign 2y`. +Export it, remove it locally, publish the key, and replace `SIGNING_KEY` and `SIGNING_KEY_PASSPHRASE`, as for the current subkey. +Never move it to the card with `keytocard`, or it can't be exported. +Then revoke the old subkey. + +To revoke a subkey, run `gpg --edit-key 1F3CE06C51CE20F7C4405181943F287533382C9D`, select it with `key N` (check that the `*` marks the right subkey), then `revkey` and `save`. +Choose the reason "Key is compromised" only if the secret leaked, because verifiers then distrust everything it ever signed. +For a planned rotation choose "Key is superseded", which keeps past signatures valid. +Publish the key again afterwards. + +[[publishing-the-key]] +=== Publishing the key + +Maven Central looks signing keys up on public keyservers, so publish the key after any change to it: + +[source] +---- +gpg --armor --export --export-options export-clean 1F3CE06C51CE20F7C4405181943F287533382C9D > pub.asc +curl --data-urlencode keytext@pub.asc https://keyserver.ubuntu.com/pks/add +curl -T - https://keys.openpgp.org < pub.asc +---- + +keyserver.ubuntu.com never deletes data, so export only the user IDs that should be public, with `--export-filter 'keep-uid=mbox = … || mbox = …'`. +keys.openpgp.org shows only user IDs whose email address was verified, but it always serves the subkeys. From 7ec4902180ef60390a9d0d2d0cbd63ef484445a0 Mon Sep 17 00:00:00 2001 From: Raman Gupta Date: Mon, 19 Oct 2026 22:51:22 -0400 Subject: [PATCH 2/3] docs: CI signing subkey in the Kotlin Toolchain evaluation The built-in signer can't use the new CI signing subkey either: with an [SC] primary, BouncyCastle's addSigningKey picks the stub primary in the export and fails, while gpg signs fine with the same export. - Add a ci-subkey section with the failure, why the primary stays [SC], and that CI must sign through the signForCentral plugin too. - Replace the stale releasing.adoc quote and update the verdict, the upstream note and the next steps; cross-link both docs. --- docs/kotlin-toolchain-evaluation.adoc | 33 +++++++++++++++++++++------ docs/releasing.adoc | 2 +- 2 files changed, 27 insertions(+), 8 deletions(-) diff --git a/docs/kotlin-toolchain-evaluation.adoc b/docs/kotlin-toolchain-evaluation.adoc index 73bfdd0..0870fc4 100644 --- a/docs/kotlin-toolchain-evaluation.adoc +++ b/docs/kotlin-toolchain-evaluation.adoc @@ -2,7 +2,7 @@ :toc: Evaluated 2026-10-16 against https://github.com/JetBrains/kotlin-toolchain[Kotlin Toolchain] 0.13.0 (the project formerly known as Amper), by converting a scratch copy of this repository end to end. -Updated 2026-10-19 with feedback from the Kotlin Toolchain maintainers (see <>) and prototype plugins for signing with gpg-agent (see <>) and building GraalVM native images (see <>). +Updated 2026-10-19 with feedback from the Kotlin Toolchain maintainers (see <>) and prototype plugins for signing with gpg-agent (see <>) and building GraalVM native images (see <>), and with a test of the built-in signer against our CI signing subkey (see <>). == Verdict @@ -11,8 +11,9 @@ Building, testing and running all work today, and the build definition gets much Two things the Gradle build does have no built-in equivalent, and each needed a small plugin: * *Signing.* -The toolchain signs only with an exported private key, while our key lives on a YubiKey behind gpg-agent. -A plugin that signs a staged publication with `gpg` works around it (see <>). +The toolchain signs only with an exported private key, while our primary key lives on a YubiKey behind gpg-agent. +Our exportable CI signing subkey doesn't help, because the built-in signer picks the export's stub primary instead of the subkey (see <>). +A plugin that signs a staged publication with `gpg` works around both, locally and in CI (see <>). * *GraalVM native image.* There is no native-image support, so the `server-ktor` smoke test needs a plugin that runs `native-image` and applies the GraalVM reachability metadata repository the way the Gradle plugin does (see <>). @@ -80,7 +81,8 @@ Signatures and checksums verified locally; nothing was uploaded. . *Signing (the one blocker; workaround in <>).* Signing works only through `KOTLIN_TOOLCHAIN_SIGNING_KEY`, an ASCII-armored private key in an environment variable. There is no gpg-agent / `useGpgCmd()` equivalent or external-signer hook. -That contradicts `docs/releasing.adoc` ("the GPG signing key never leaves local hardware"), and a key on a YubiKey can't be exported at all. +Our primary key is on a YubiKey and can't be exported at all. +The dedicated CI signing subkey (see link:releasing.adoc#signing[Releasing]) can be exported, but the built-in signer fails with it too (see <>). . *GraalVM native image (workaround in <>).* The toolchain has no equivalent of the `org.graalvm.buildtools.native` Gradle plugin, which the `server-ktor` native smoke test relies on. . *Version from git tag (not a blocker).* @@ -125,7 +127,7 @@ It registers a `signForCentral` command that, for each module: * finds the module's staged directory from `${module.settings.publishing.group}`, `.artifactId` and `.version`; * deletes the local-install bookkeeping files (`_remote.repositories`, `maven-metadata-local.xml`); -* runs `gpg --armor --detach-sign` on every artifact, so signing goes through gpg-agent and works with a YubiKey; +* runs `gpg --armor --detach-sign` on every artifact, so signing goes through gpg-agent and works with a YubiKey, or in CI with the imported signing subkey; * writes the `.md5` and `.sha1` checksums Maven Central requires. The release flow then becomes: @@ -156,6 +158,22 @@ A plugin task can't depend on `kotlin publish`, and an upload per module would c * A plain shell script (or JReleaser, which supports signing through the `gpg` command and Central Portal uploads) could do the signing step too. The plugin's advantage is that it reads each module's coordinates from the build instead of repeating them. +[[ci-subkey]] +=== The built-in signer and the CI signing subkey + +To sign in CI, we created a dedicated ed25519 signing subkey, certified by the YubiKey primary, whose secret stays in software (see link:releasing.adoc#signing[Releasing]). +It is exported with `gpg --export-secret-subkeys '!'`, which yields a stub primary with no secret material (`gnu-dummy`) plus only that subkey. + +The built-in signer can't use that export. +BouncyCastle's `addSigningKey` picks the first signing-capable key, and our primary is `[SC]`, so it picks the stub primary and can't unlock it. +`kotlin publish` fails with "PGP signing failed", caused by a `NullPointerException` because `OpenPGPSecretKey.unlock(char[])` returns null. +The same export signs fine with `gpg`, which picks the subkey because it is the only key with a secret, and the signatures verify as the primary's identity. + +An earlier throwaway test passed only because its primary was certify-only (`[C]`). +Making our primary certify-only isn't an option: gpg checks a key's current usage flags when it verifies, so gpg would then reject every signature the primary has already made. + +So in CI, too, signing has to go through the `signForCentral` plugin, with the export imported into gpg, until the toolchain either skips keys without secret material or lets us pick the signing key, as Gradle's `signing.keyId` does. + [[native]] == GraalVM native image through a custom plugin @@ -294,7 +312,7 @@ The maintainers confirmed most of them and linked these issues (all open as of 2 | Dokka Javadoc-format output for JVM libraries |=== -No issue was linked for gpg-agent / external signing. +No issue was linked for gpg-agent / external signing, or for choosing which key in an export signs (see <>). On Dokka, the maintainers explained why the toolchain ships a placeholder javadoc jar rather than Dokka HTML (summarized under the javadoc gap above). @@ -310,7 +328,8 @@ The template-conflict point was a misreading on our side: nested templates resol The spike shows they work together. . *Now, in Gradle:* fix the `server-ktor` native smoke test, which builds a shared library instead of an executable (see <>). . *If we migrate:* release through the staged-publication flow and signing plugin in <>, until the toolchain supports external or agent-based signing. -Ask upstream for that, since no issue tracks it yet. +This applies in CI too, where the built-in signer can't use our signing subkey (see <>). +Ask upstream for agent-based signing and for a way to choose the signing key, since no issue tracks either yet. . *If we migrate:* build the native smoke test with the `graalvm-native` plugin in <>, and ask upstream about native-image support. . *If we migrate:* split the DI flavors into modules nested inside their base module (see <>), and move to the toolchain's default layout. diff --git a/docs/releasing.adoc b/docs/releasing.adoc index f37987c..b089428 100644 --- a/docs/releasing.adoc +++ b/docs/releasing.adoc @@ -53,7 +53,7 @@ It is stored as secrets in the `release` GitHub environment: A CI job should import `SIGNING_KEY` into gpg and sign with gpg. Don't hand it to the Kotlin Toolchain as `KOTLIN_TOOLCHAIN_SIGNING_KEY`. -Its BouncyCastle signer picks the signing-capable stub primary instead of the subkey and fails (see link:kotlin-toolchain-evaluation.adoc[the Kotlin Toolchain evaluation]). +Its BouncyCastle signer picks the signing-capable stub primary instead of the subkey and fails (see link:kotlin-toolchain-evaluation.adoc#ci-subkey[the Kotlin Toolchain evaluation]). === Expiry From c50a4a5de6081c043e84f43a12d682524cd3c973 Mon Sep 17 00:00:00 2001 From: Raman Gupta Date: Mon, 19 Oct 2026 22:55:22 -0400 Subject: [PATCH 3/3] docs: reference KTC-6065 for signing-subkey support KTC-6065 (allow configuring a signing subkey) was filed from our report. Once fixed, CI can sign with the built-in signer, so the staged-publication workaround can go if releases move to CI. - Mention it in the ci-subkey section, the upstream issue table and the next steps; agent-based signing remains untracked. --- docs/kotlin-toolchain-evaluation.adoc | 14 +++++++++++--- 1 file changed, 11 insertions(+), 3 deletions(-) diff --git a/docs/kotlin-toolchain-evaluation.adoc b/docs/kotlin-toolchain-evaluation.adoc index 0870fc4..68c118e 100644 --- a/docs/kotlin-toolchain-evaluation.adoc +++ b/docs/kotlin-toolchain-evaluation.adoc @@ -172,7 +172,9 @@ The same export signs fine with `gpg`, which picks the subkey because it is the An earlier throwaway test passed only because its primary was certify-only (`[C]`). Making our primary certify-only isn't an option: gpg checks a key's current usage flags when it verifies, so gpg would then reject every signature the primary has already made. -So in CI, too, signing has to go through the `signForCentral` plugin, with the export imported into gpg, until the toolchain either skips keys without secret material or lets us pick the signing key, as Gradle's `signing.keyId` does. +So in CI, too, signing has to go through the `signForCentral` plugin, with the export imported into gpg. +https://youtrack.jetbrains.com/issue/KTC-6065[KTC-6065] (allow configuring a signing subkey) tracks support for subkeys in the built-in signer. +Once it is fixed, CI can pass the export as `KOTLIN_TOOLCHAIN_SIGNING_KEY`, and if releases move to CI, we can drop the staged-publication flow and the `signForCentral` plugin. [[native]] == GraalVM native image through a custom plugin @@ -310,9 +312,14 @@ The maintainers confirmed most of them and linked these issues (all open as of 2 | https://youtrack.jetbrains.com/issue/KTC-472[KTC-472] | Support javadoc jar generation (for publication) | Dokka Javadoc-format output for JVM libraries + +| https://youtrack.jetbrains.com/issue/KTC-6065[KTC-6065] +| Allow configuring a signing subkey +| Built-in signer can't use our CI signing subkey (see <>) |=== -No issue was linked for gpg-agent / external signing, or for choosing which key in an export signs (see <>). +No issue was linked for gpg-agent / external signing. +KTC-6065 was filed from our report on the signing subkey. On Dokka, the maintainers explained why the toolchain ships a placeholder javadoc jar rather than Dokka HTML (summarized under the javadoc gap above). @@ -329,7 +336,8 @@ The spike shows they work together. . *Now, in Gradle:* fix the `server-ktor` native smoke test, which builds a shared library instead of an executable (see <>). . *If we migrate:* release through the staged-publication flow and signing plugin in <>, until the toolchain supports external or agent-based signing. This applies in CI too, where the built-in signer can't use our signing subkey (see <>). -Ask upstream for agent-based signing and for a way to choose the signing key, since no issue tracks either yet. +Once https://youtrack.jetbrains.com/issue/KTC-6065[KTC-6065] lets the built-in signer use the subkey, CI releases can drop the workaround. +Local releases with the YubiKey would still need it, so ask upstream for agent-based signing, which no issue tracks yet. . *If we migrate:* build the native smoke test with the `graalvm-native` plugin in <>, and ask upstream about native-image support. . *If we migrate:* split the DI flavors into modules nested inside their base module (see <>), and move to the toolchain's default layout.