From 27ab818daa661629f7ae103958fb69858152b031 Mon Sep 17 00:00:00 2001 From: Andreas Trawoeger Date: Thu, 29 Oct 2026 07:36:23 +0100 Subject: [PATCH] docs(corpus): advance the corpus submodule to the corpus's served head, and regenerate the site MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The `marketplace` submodule advances from 5feb9c8b8350849fd2d46d252b486844c18bd0a4 to 197f42a57ab88fcaed6a044bf312d753a72062b2 — the head opencharly/marketplace's `main` serves, one commit above its tag v2026.302.0622 because tag-on-merge writes CHANGELOG/ after the tag. The four regenerated pages are the projection of exactly the corpus bodies that the refresh changed (opencharly/marketplace#599), so the bump and the pages are one change. RCA. Mechanism: the corpus repo's daily refresh advances `main`, and this repo's submodule gitlink did not move with it — the pinned corpus was 8 commits behind. Missed control: the pin-currency comparison against the corpus's `main`. This repo REPORTS each pinned input's currency and never fails on it, because the remedy for a stale pin is a regeneration PR that bumps the pin — and the drift gate cannot see a stale pin at all, by construction: the PINNED sources regenerate their own (old) pages perfectly, so the gate stays green over a corpus that is no longer the published one. Blast radius: the site's recipe pages project a corpus that is behind the published corpus, and nothing in the tree says so. Root fix: bump the submodule and carry the REGENERATED pages in the same change — the regeneration is what proves the bump is coherent, since a stale submodule plus pages generated from the new corpus would drift. One symptom was visible in the tree: the published page still taught the corpus's drift gate's OLD instrument — "including UNTRACKED regenerated files (`git status --porcelain`, not just `git diff --exit-code`)" — which the corpus's own projection had already corrected (opencharly/plugin-marketplace#56 tracks that sentence). The regenerated page now carries the corpus's corrected text. The baseline was measured first: a regeneration at the OLD pin is a byte-exact no-op against the committed tree (diff -rq = 0 over the whole content tree, 1726 generated pages cleared and rewritten), so this diff is attributable to the corpus bump alone. Closes #201 Assisted-by: Claude Code minimax/minimax-m3.1-flash:cloud (fully tested and validated) --- marketplace | 2 +- .../docs/recipes/internals/marketplace.md | 25 ++++++-- .../internals/plugin/authoring-recipes.md | 3 +- src/content/docs/recipes/kubevirt/kubevirt.md | 39 +++++++++++- src/content/docs/recipes/tools/docs-site.md | 60 ++++++++++++++++--- 5 files changed, 112 insertions(+), 17 deletions(-) diff --git a/marketplace b/marketplace index 5feb9c8b..197f42a5 160000 --- a/marketplace +++ b/marketplace @@ -1 +1 @@ -Subproject commit 5feb9c8b8350849fd2d46d252b486844c18bd0a4 +Subproject commit 197f42a57ab88fcaed6a044bf312d753a72062b2 diff --git a/src/content/docs/recipes/internals/marketplace.md b/src/content/docs/recipes/internals/marketplace.md index 63fd2a53..bcde6945 100644 --- a/src/content/docs/recipes/internals/marketplace.md +++ b/src/content/docs/recipes/internals/marketplace.md @@ -59,13 +59,28 @@ not restated here. - Regeneration on a clean tree is a NO-OP for the committed corpus (hand-authored files sit outside the known paths); a diff means a refs/source change is in flight. -## The drift gate (deploy.yml is the SOLE OWNER of generation) +## The drift gate (the marketplace repo's `scripts/drift-gate.sh` — ONE implementation, TWO callers) The marketplace repo's deploy workflow builds the pinned charly, regenerates the ENTIRE -corpus, and FAILS if the committed corpus does not match — including UNTRACKED -regenerated files (`git status --porcelain`, not just `git diff --exit-code`). A refs -bump must therefore ship its regenerated corpus in the same PR, or the gate trips on the -next run. +corpus, and FAILS if the committed corpus does not match. The check lives in ONE script, +the marketplace repo's `scripts/drift-gate.sh`, and BOTH of that repo's workflows run it: +`deploy.yml` on every PR, and the daily refresh on the tree its own PR would commit. + +The gate asks "did the tree move since the index was set", in two halves: the WORKING +TREE against the INDEX (`git diff --quiet`), and the files in NO index at all +(`git ls-files --others --exclude-standard`) — the second half is what a diff alone +misses, because a generator that emits NEW artifacts creates them as untracked files. It +deliberately does NOT use `git status --porcelain`: that command reports the index column +as well as the worktree column, and the refresh stages its regenerated corpus on purpose, +so a gate built on it fails on the refresh job's tree shape, every time it runs there. +Measured 2026-10-29 on the refresh's own tree: a check built on it aborted with DRIFT +while the tree had not moved at all (no unstaged change, no untracked file). That is a +property of the INSTRUMENT on that tree shape — the refresh ran NO gate before +2026-10-29 (it pasted the gate's SOURCE into its body), so no refresh ever aborted on +it. + +A refs bump must therefore ship its regenerated corpus in the same PR, or the gate trips +on the next run. ## The daily refresh (refresh.yml) diff --git a/src/content/docs/recipes/internals/plugin/authoring-recipes.md b/src/content/docs/recipes/internals/plugin/authoring-recipes.md index 001c936c..137eca61 100644 --- a/src/content/docs/recipes/internals/plugin/authoring-recipes.md +++ b/src/content/docs/recipes/internals/plugin/authoring-recipes.md @@ -302,7 +302,8 @@ placements, ZERO authoring change. require, so a stock binary runs the PUBLISHED candy and no bed can fail on your edit. `scripts/bootstrap-charly.sh --dev-plugin =` (repeatable) makes `pluginsgen` read that repo from the checkout instead and emits a SEPARATE `go.work.dev` selected only under an - explicit `GOWORK=` (the committed `go.work`/`go.work.sum` stay byte-identical; a later build WITHOUT the + explicit `GOWORK=` (the committed `go.work` stays byte-identical — it has ONE member, whose `go.sum` + carries every hash, so the workspace has no checksum lock of its own; a later build WITHOUT the flag deletes it, and `--install` is refused). It validates the name is in `compiled_plugins:` and that `candy//go.mod` declares module `github.com/opencharly//candy/`. Then prove the changed path EXECUTED — `go version -m` (`(devel)` vs the module-proxy tag) is necessary but not sufficient; a diff --git a/src/content/docs/recipes/kubevirt/kubevirt.md b/src/content/docs/recipes/kubevirt/kubevirt.md index ef8ed8f7..358c1a5d 100644 --- a/src/content/docs/recipes/kubevirt/kubevirt.md +++ b/src/content/docs/recipes/kubevirt/kubevirt.md @@ -1,6 +1,6 @@ --- title: "kubevirt" -description: "The `kubevirt:` deploy SUBSTRATE (7th): `kind: kubevirt` templates + `kubevirt:`/`target: kubevirt` deploys, the VirtualMachine CR lifecycle (DataVolume, runStrategy, VMI Ready + AgentConnected, managed `virtctl port-forward` SSH), instancetype, snapshots, live migration, and GPU host-device passthrough." +description: "The `kubevirt:` deploy SUBSTRATE (7th): `kind: kubevirt` templates + `kubevirt:`/`target: kubevirt` deploys, the VirtualMachine CR lifecycle (DataVolume, runStrategy, VMI Ready + AgentConnected, managed `virtctl port-forward` SSH), instancetype, snapshots, live migration, GPU host-device passthrough, and nested virtualisation via a host-passthrough CPU." --- @@ -75,6 +75,43 @@ runner: from: runner-vmi ``` +## Host devices and nested virtualisation + +These are two DIFFERENT things, and the first is not the route to the second. + +**`devices.host_devices`** is KubeVirt host-device passthrough +(`spec.domain.devices.hostDevices`) for PCI/MDEV/USB devices a cluster device +plugin advertises. The cluster must allowlist the resource in the KubeVirt CR's +`permittedHostDevices` and enable the Alpha `HostDevices` feature gate; a VMI +carrying the field without the gate is rejected at admission +(`spec.HostDevices: Host Devices feature gate is not enabled in kubevirt-config`, +MEASURED). It cannot carry a character device. + +```yaml +vmi: + kubevirt: + devices: + host_devices: + - { device_name: intel.com/qat, name: quickaccess1 } +``` + +`device_name` is the resource the cluster's device plugin advertises; `name` is +the guest-visible label and defaults to `device_name`. + +**Nested virtualisation** — a guest that itself runs VMs — needs a guest +`/dev/kvm`, and that comes from a **`host-passthrough` CPU**, not from host +devices: the guest's own kernel then loads `kvm_amd`/`kvm_intel` and creates the +device. MEASURED on an AMD host: `host-model` gave neither `svm` nor `/dev/kvm`; +after `cpu.model: host-passthrough` the guest had `svm`, `/dev/kvm` +(`root:kvm 0660`), `kvm_amd` loaded, and a nested +`qemu-system-x86_64 -accel kvm` ran with exit 0. + +```yaml +runner-vmi: + kubevirt: + cpu: { cores: 2, model: host-passthrough } # the guest then has svm + /dev/kvm +``` + ## The guest identity and the cluster it lands on A `container_disk` source keeps no guest facts of its own: `distro` and `ssh_user` diff --git a/src/content/docs/recipes/tools/docs-site.md b/src/content/docs/recipes/tools/docs-site.md index 12e893aa..f38d402d 100644 --- a/src/content/docs/recipes/tools/docs-site.md +++ b/src/content/docs/recipes/tools/docs-site.md @@ -46,23 +46,31 @@ exact source Cloudflare builds. **Ordering consequence.** The candy fetches an IMMUTABLE COMMIT, never a branch, so `DOCS_REF` must be re-pinned to each new docs merge before this box — and the `check-docs` bed above it — -prove the new content. `task docs:pin` fails when `DOCS_REF` is not the docs repo's current -`main` head (it fetches `ls-remote` and compares — charly has no `docs` gitlink anymore: the -docs repo is standalone, and this candy's pin is the only thing that fixes what the bed tests). +prove the new content. `charly task docs-pin` REPORTS that state: it fetches `ls-remote` and +compares `DOCS_REF` against the docs repo's current `main` head, and a pin that is **behind** +is reported and stays green — charly has no `docs` gitlink anymore (the docs repo is +standalone, and this candy's pin is the only thing that fixes what the bed tests), so the +advance is available NOW in `opencharly/docs` and this repo's pin moves when someone takes +it. A report that reddened the gate on every docs merge that has not been re-pinned here +would be a nuisance that gets bypassed; the org pin contract draws the same line — a pin +that is behind is REPORTED, a pin the workflow cannot JUDGE is a FAILURE. That re-pin is a CONFIG edit and changes the landing's gate and attribution tier — see [`/charly-internals:git-workflow`](/recipes/internals/git-workflow/) (B6a step 4, and B2 "the derivation stops at the gitlink" for the general rule), which owns landing mechanics. -**The pin is TWO occurrences, and both are asserted.** The sha appears once as `DOCS_REF` +**The pin is TWO occurrences, and both are checked.** The sha appears once as `DOCS_REF` (the clone's cache key) and once as a LITERAL in the `docs-site-pinned-commit` check's `contains:` matcher. The duplication is deliberate and cannot be collapsed: a check step's substitution does not resolve a candy `var:`, so writing `${DOCS_REF}` in the matcher makes charly SKIP the step with "unresolved variables: DOCS_REF" — turning the anti-staleness guard into a non-event while the bed still reports PASS, which is the exact failure the guard exists -to catch. `task docs:pin` therefore asserts BOTH: it compares `DOCS_REF` against the docs -repo's current `main` head AND requires the sha to occur at least twice in the file, so a -re-pin that updates only one of them fails the gate instead of leaving the check asserting a -commit nobody builds. Re-pinning means editing both. +to catch. `charly task docs-pin` therefore checks BOTH, and it draws the line between them by +what each one MEANS. The **currency** of the pin — is `DOCS_REF` the docs repo's current +`main` head — is REPORTED, because a pin that is behind is a state and the advance lives in +another repo. The **agreement** of the two occurrences — does the sha occur at least twice in +`charly.yml` — FAILS, because that is this tree contradicting itself, and a re-pin that +updated only one of them would leave the check asserting a commit nobody builds. Re-pinning +means editing both. A branch name here is a correctness bug, not a convenience: `var:` values are emitted as `ENV` above the steps, so with `main` the clone layer's cache key never moves and the bed silently @@ -74,6 +82,39 @@ proving stale content until review caught it. | `DOCS_REPO` | `https://github.com/opencharly/docs.git` | | `DOCS_REF` | the docs repo commit this bed tests — an immutable commit, **never a branch** (see above) | +## The pin report — `charly task docs-pin` + +The bed proves the IMAGE holds the pin; nothing in the org proved the pin is the docs repo's +CURRENT head, and nothing proved the two occurrences agree. That control lives HERE — this +repo owns the pin, and the control that reads it is a **task**, not a candy check, so it runs +over this repo's own sources rather than inside the image. + +```bash +charly task docs-pin +``` + +It reads `DOCS_REF` and `DOCS_REPO` from this repo's `charly.yml`, fetches the docs repo's +`main` head with `git ls-remote`, and reports one of four states: + +| State | Meaning | Exit | +|---|---|---| +| **current** | `DOCS_REF` IS the docs repo's `main` head, and both occurrences are present | 0 | +| **BEHIND** | the pin is not the current head — the advance is available NOW in `opencharly/docs` | 0 | +| **DISAGREE** | the sha does not occur at least twice in `charly.yml` — this tree contradicts itself | 1 | +| **UNJUDGEABLE** | the head could not be fetched, or no `DOCS_REF` is declared | 1 | + +`scripts/docs-pin-report.sh` owns the logic, and `--self-test` proves each state +discriminates against fixtures. Each case asserts the state's own REPORT, not only an +exit status: `current` and `BEHIND` both exit 0, so only the message tells them apart. +The task runs both. + +Every report NAMES the state it is in, so the state is greppable in the output — and the +self-test proves this table stays the report's own list: the count the sentence above +claims, the rows above, and the states the script declares are asserted equal, and four +mutations (a wrong count, a missing row, a row this script never declares, and a state +this script declares that this table does not name) each turn the check red. A fifth +state therefore cannot be added silently, in either direction. + ## Verification Every check is build-context, so `charly check box docs-site-app` proves the whole site builds @@ -136,7 +177,8 @@ pinned charly is gated by the docs repo's own deploy workflow, not here. | Symptom | Cause | |---|---| | `docs-site-pinned-commit` FAILS | the clone layer is stale, or `DOCS_REF` was re-pinned without rebuilding | -| `task docs:pin` fails | `DOCS_REF` (or the check's literal sha) is not the docs repo's current `main` head — bump it in a charly PR | +| `charly task docs-pin` REPORTS "DOCS_REF is BEHIND" | the pin is not the docs repo's current `main` head — the advance is available NOW in `opencharly/docs`; re-pin BOTH occurrences in `charly.yml` to take it. The report is green on purpose: a pin that is behind is a state, not a defect | +| `charly task docs-pin` FAILS | the two occurrences of the sha in `charly.yml` do not AGREE (the `docs-site-pinned-commit` matcher still names an older commit), or the docs repo's head could not be fetched, so the pin's currency is UNJUDGEABLE. The first is this tree contradicting itself — fix the matcher; the second is a gate that cannot judge, never a pass | | fetch fails `upload-pack: not our ref ` | `DOCS_REF` names a commit not reachable from ANY ref in the docs repo — never pushed, or garbage-collected | | fetch fails `couldn't find remote ref ` | `DOCS_REF` is not a FULL 40-hex sha — either a ref name (which the pinned-commit contract forbids) or an abbreviated sha, which `git fetch` rejects even when the commit exists | | `npm ci` fails on a lockfile mismatch | `package.json` and `package-lock.json` disagree — regenerate the lockfile in the docs repo |