feat: close the audit's reference and content blind spots, ship lint.yml in the kit - #31
Merged
mcarter-astronautdev merged 3 commits intoAug 2, 2026
Conversation
…yml in the kit The pin audit reported the fleet uniform through the whole v1.9.0 gap, because comparing deployed pin lines to the latest tag cannot see three things. Two of them are now checked. tools/fleet-pin-audit.sh gains: - A REFERENCE check — every uses: pin in templates/github/*.yml must equal the latest tag's SHA. Checks 2 and 3 measure the fleet against templates/, so a stale reference makes both of them lie. That is the v1.9.0 failure exactly: the wave repinned the fleet to a54c91e while the kit's own stubs still said 80c35fe, and an audit that only compared deployed pins to the latest tag called the fleet uniform throughout. - A CONTENT check — the whole waved file against its templates/github/ source, not just the pin line. This closes the other two holes: a file with no uses: line at all (an unconverted 190-line copy of what is now a 66-line stub) is no longer invisible, and DRIVER_AGENTS_REF — a raw SHA in an env: block that no bot can bump — is now compared like any other line. Only SHOPIFY_STORE_NAME is normalized away, since it is the one difference a correctly-waved repo is supposed to have. DriverDigital/workflows itself is skipped: its reusables share basenames with the stubs that call them. - A non-zero exit on any drift, so a wave can gate on it. Verified against the live fleet: 21 repo@branch pairs, 108 pin rows all at 90f0d06, 127 files byte-identical to templates/, zero drift. That run also shows the v1.11.0 wave has in fact landed, which the README still said it had not — corrected. Still unchecked, and now stated as such in both docs: the tripwire parity between templates/github/claude.yml and driver-agents' canonical instructions. The audit proves the fleet matches templates/; only the by-hand release step proves templates/ matches canonical. Also adds templates/github/lint.yml — actionlint and shellcheck over the installing repo's own .github/workflows/. This repo's own lint.yml is a superset and stays where it is; the kit version drops the three steps that reference templates/. Installed into driver-agents and driver-agents-app alongside pr-first-review.yml, which the kit README now documents as a supported partial install.
…admins call Set via the narrow required_status_checks sub-resource, not a whole-object PUT — that endpoint replaces everything and silently drops any field left out of the body, the 1-approval rule included. Diffing the full protection object before and after confirms only contexts/checks moved. enforce_admins stays false, which means an admin can still merge past a red actionlint. Requiring the check makes it binding for everyone else and puts a red X in front of an admin who previously had nothing to override. Flipping the flag was considered and rejected for fleet uniformity; the reasoning is written down rather than left implicit.
…aims Adversarial review over the branch surfaced ten findings that survived refutation. All are fixed here. tools/fleet-pin-audit.sh: - The content check could not see EOF-only drift. Both sides went through command substitution, which strips ALL trailing newlines, so a file differing only at the end compared equal and reported ok — the exact class of invisible drift this check was added to catch. Raw content now goes straight to a temp file, never a variable. - Having fixed that, nine files started reporting drift for a missing final newline: the three stub-rails-only pairs were waved without one and are otherwise identical. Nine permanent red rows is how a detector stops being read, so trailing blank lines and the final newline are now normalized DELIBERATELY, alongside the store handle, and both are documented as the only two normalizations. Internal blank lines are still compared. - A failed `gh repo list` produced an empty report, which fell through to "(converged)" and exit 0 — a drift detector reporting clean because it saw nothing at all. Enumeration now happens outside the report subshell so it can abort the run, and a second guard fails if zero pins are found fleet-wide, since every kit pair carries at least one. - Raised the repo-list limit past the silent truncation cliff. - An apostrophe in a comment inside the $( ) broke the parse: bash opens a quote on one even in a comment, and reports the error at EOF rather than the line. Noted in place. templates/github/lint.yml keeps SHELLCHECK_OPTS=--exclude=SC2015. actionlint treats an info-level shellcheck finding as a hard failure and this job is meant to be a required check, so dropping the exclusion would let ordinary defensive shell block every PR. Upstream removed the rule in shellcheck 0.11, so it is a false positive by its author's judgement, not a suppressed bug. Four doc claims were wrong and are corrected: - "a skipped required check counts as not-passed" is backwards. GitHub accepts a check run whose conclusion is skipped. The real reason not to require `review / review` is its trigger list omitting `synchronize`, so a follow-up commit produces no check run for that head SHA and a MISSING required context blocks forever. Both places that taught the wrong model now teach this one. - "every commit on main is a PR merge" is false. Six commits have no associated PR, including a54c91e, the v1.9.0 release, pushed yesterday. That is a stronger reason to leave enforce_admins false than the uniformity argument it replaces: enforce_admins true would have blocked every one of them. - "six phantom drifts" is now seven — this change adds lint.yml as a shared basename between the reusables and the kit. - "Every other kit branch is protected" understated the exposure. Surveyed all 21 pairs: 8 Palmers branches require zero approving reviews, so 10 pairs lack a human-approver rule, not 2. The README no longer says the two agent repos run the partial install — the PRs are open, not merged, and they should land only after this one ships templates/github/lint.yml.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Three to-dos from
CLAUDE.local.md, all dated 2026-08-02.1. Teach
fleet-pin-audit.shto see its own reference driftThe audit reported the fleet uniform through the whole v1.9.0 gap.
docs/fleet-operations.mdlisted three reasons; two are now checked. The script runs them in order and exits non-zero if any fires.uses:pin intemplates/github/*.ymlequals the latest tag's SHAtemplates/, so a stale reference makes both of them lie. This is the v1.9.0 failure exactly: the wave repinned the fleet toa54c91ewhile the kit's own stubs still said80c35fe.uses:SHA vs that tagtemplates/github/sourceuses:line at all (an unconverted 190-line copy of what is now a 66-line stub) is no longer invisible, andDRIVER_AGENTS_REF— a raw SHA in anenv:block no bot can bump — is compared like any other line.Two judgement calls worth reviewing:
SHOPIFY_STORE_NAMEis normalized away. It is the one difference a correctly-waved repo is supposed to have. Third-party action pins are deliberately not normalized: a repo whose Dependabot bumpedactions/checkoutpast the kit's pin is drift worth seeing — it means the kit is behind, not that the repo is wrong.DriverDigital/workflowsitself is skipped. Its.github/workflows/holds the reusables, which share basenames with the stubs that call them (pr-first-review.ymlis ~200 lines here and 25 in the kit), so comparing it totemplates/would report six phantom drifts.Verified against the live fleet — 21 repo@branch pairs, 108 pin rows all at
90f0d066, 127 files byte-identical totemplates/, zero drift, exit 0. The reference check was negative-tested by feeding it a stale tag: all six stubs reported. The store-handle normalization is load-bearing rather than vacuous — Avara carries"avara"where the kit ships"", and itsclaude.ymlreportsok.That run also turned up a stale doc claim: the README said "The fleet has not been waved yet — consumer repos still run the 190-line copy." It has been waved. Corrected.
Still unchecked, and now said plainly in both docs: the tripwire parity between
templates/github/claude.ymland driver-agents' canonicaldocs/agent-instructions-shopify.md. The audit proves the fleet matchestemplates/; only the by-hand release step provestemplates/matches canonical.2. Install the kit in
driver-agents+driver-agents-appBoth had no
.github/workflowsat all — driver-agents holds the production cron runner and merged PR #3 with zero automated review.pr-first-review.yml(pinnedv1.11.0) +lint.yml, copied verbatim.actionlintis green on both in under 10s, andreview / reviewfired from the PR head — which also confirms a first-install PR triggers its own workflows.This needed a new kit file:
templates/github/lint.yml. This repo's ownlint.ymlcan't be copied — ityaml.safe_loadstemplates/github/claude.ymland globstemplates/github/*.yml, neither of which exists in a consumer repo, so a verbatim copy would fail loudly on first run. The kit version keeps the actionlint install + run and drops the three repo-specific steps. Registered in both README tables and the copy script.templates/github/README.mdnow documents partial install as a supported shape, including what you give up:pr-first-reviewexcludesdependabot[bot]authors and self-skips on ticketed PRs, so with nothing else installed those get zero review rather than a different one.Also corrected the kit's Pin the required check step, which only described the
validate / validateshape. A local job reports its bare job id (actionlint); a reusable job reports<caller> / <reusable>. Plus a warning: don't requirereview / reviewon its own — it skips drafts, forks, bots and ticketed PRs by design, and a skipped required check counts as not-passed, so it would block exactly the PRs it means to leave alone.3.
actionlintas a required status check onmainApplied after the above landed green, via the narrow
required_status_checks/contextsendpoint rather than a full-objectPUT, so nothing else in the protection object could be dropped by omission.enforce_adminsstaysfalse— deliberately. It isfalsefleet-wide because direct-push repin waves depend on it, and although no wave has ever pushed directly to this repo (every commit onmainis a PR merge), diverging from the fleet default would make this the one exception to remember. The 1-approval review rule is what actually gates merges. That reasoning now lives in the README instead of only in my head.