From 8145841ef2cc39c7c844609a2009a18e6c41c528 Mon Sep 17 00:00:00 2001 From: Daniel Lam Date: Sun, 16 Aug 2026 04:55:19 +0700 Subject: [PATCH] feat(skills): name cicd composer traps --- docs/ai/skills/cicd.md | 24 ++++++++---- skills/cicd/SKILL.md | 32 ++++++++++------ skills/cicd/references/ci-structure.md | 8 ++++ skills/cicd/references/host-runnable.md | 46 +++++++++++++++++++++-- skills/cicd/references/policy-and-auth.md | 8 +++- 5 files changed, 95 insertions(+), 23 deletions(-) diff --git a/docs/ai/skills/cicd.md b/docs/ai/skills/cicd.md index bbb8e38..dbaf470 100644 --- a/docs/ai/skills/cicd.md +++ b/docs/ai/skills/cicd.md @@ -22,9 +22,11 @@ where pushes must trigger CI, and OIDC over static cloud keys. Specifies the GitHub ↔ IaC 1:1:1 mapping (stack ↔ GitHub Environment ↔ deploy workflow) that `cmk:infra` names but leaves to this skill to wire. Names speedup misattribution as a named trap alongside cold-cache poisoning, -skipped-job-reports-success, and workflow-token-doesn't-trigger-CI — a -multi-part change to CI's wall-clock gets every part credited for the total -unless the win is traced to the specific part that produced it. Ends with a +skipped-job-reports-success, workflow-token-doesn't-trigger-CI, +evidence-floor negation, and composer-contract drift. Host-runnable +scripts prefer TypeScript (or the repo's existing runtime); shell is +host bootstrap only; a shared mutating CLI refuses without `--confirm` +or a documented CI env. Ends with a `## Verify` section for report-only checks a caller can run against a target repo. @@ -33,19 +35,25 @@ repo. `GitHub ↔ IaC mapping is this skill's contract`, `` `workflows/README.md` is the operating doc ``, `What this teaches vs. what a project owns`, `Verify`. +- `references/host-runnable.md` — scripts-are-the-workflow, language + (TypeScript default, shell bootstrap only, no third ops language), + composer-contract drift, mutate gate, host matrix, JIT slot count, + one-fix-per-push. - `references/ci-structure.md` — the `changes` job/path-filter shape, concurrency and caching, config-selected runner tiers, deliberate - cold-cache isolation, CI self-contract tests, label-gated diagnostics. + cold-cache isolation, CI self-contract tests (implementation site + and path-filter lockstep), label-gated diagnostics. - `references/deploy-and-release.md` — validation-vs-deployment split, dispatch-against-ref inputs and what they buy (canary test deploy, rollback, ancestry-verified promotion), the GitHub ↔ IaC contract in full, orchestrator-plus-legs with two-layer concurrency, release integrity (pinned artifact reuse, attestation, `always()` cleanup, read-back verification). -- `references/policy-and-auth.md` — traceability and test-evidence gates, - branch-protection ruleset contract, automation auth (pinned actions, - non-persisted credentials, GitHub App vs. workflow token, OIDC), and - critical-invariant verification as its own required job class. +- `references/policy-and-auth.md` — traceability and test-evidence gates + (evidence-floor negation), branch-protection ruleset contract, + automation auth (pinned actions, non-persisted credentials, GitHub App + vs. workflow token, OIDC), and critical-invariant verification as its + own required job class. ## Links - `cmk:infra` — names the environment vocabulary and requires a deploy path diff --git a/skills/cicd/SKILL.md b/skills/cicd/SKILL.md index 8f83d96..8343945 100644 --- a/skills/cicd/SKILL.md +++ b/skills/cicd/SKILL.md @@ -1,7 +1,7 @@ --- name: cmk:cicd description: This skill should be used when the user asks to "set up CI", "speed up CI", "add a deploy workflow", "structure GitHub Actions", "self-hosted runners", "run CI locally", "JIT runner", "protect the main branch", or needs to structure CI, deployment, and policy automation as composable host-runnable scripts that GitHub Actions only automates. -version: 0.3.2 +version: 0.3.3 --- # CI/CD @@ -41,13 +41,15 @@ accidentally run on every push that happens to touch the workflow file. ## Scripts are the workflow GitHub Actions is a composer. The steps themselves are independently -invocable scripts (prefer TypeScript + bun) that run on a development -machine, a JIT self-hosted runner, or hosted compute with the same -behavior. Read `references/host-runnable.md` when adding a job, a -runner, or reproducing a CI failure. This is not a local-only mindset: +invocable scripts (TypeScript + bun, or the repo's existing runtime) +that run on a development machine, a JIT self-hosted runner, or hosted +compute with the same behavior. Read `references/host-runnable.md` when +adding a job, a runner, choosing a script language, extracting a fat +`run:`, or reproducing a CI failure. This is not a local-only mindset: local, AWS, GCP, and production are profiles over one production-ready -path (`cmk:infra`, `cmk:local-stack`). JIT concurrent-job count is -`references/host-runnable.md`; attested packing is `cmk:enclave`. +path (`cmk:infra`, `cmk:local-stack`). JIT concurrent-job count, +language, extract, and mutate-gate rules are `references/host-runnable.md`; +attested packing is `cmk:enclave`. ## GitHub ↔ IaC mapping is this skill's contract @@ -85,10 +87,14 @@ silently leaving a rewritten ref unverified); **speedup misattribution** (a multi-part change to CI's wall-clock cuts the total, and every part gets credited — but tracing the win to its actual cause can reveal that one part did all of it and a sibling part is silently inert, contributing nothing -while looking identical to the part that worked). Isolate which specific -change moved the number before crediting any of them; `cmk:test-resources` -covers the same trap one layer down, inside a test suite's own resource -model rather than CI's orchestration. +while looking identical to the part that worked — isolate which change +moved the number before crediting any of them; `cmk:test-resources` +covers that trap one layer down); **evidence-floor negation** (a +test-evidence gate that scans for `fail` treats `0 fail` and +`fail-closed` as a non-passing claim); **composer-contract drift** +(extracting a fat `run:` into a script leaves YAML-body pins and path +filters on the old site — raising the line-count baseline is not the +fix). Projects own: which area jobs exist and their path filters; runner labels and pool sizing; which policy gates are enabled; deploy-leg composition; label @@ -118,3 +124,7 @@ Report-only — never mutate: - Those scripts support Linux and macOS on amd64 and arm64, or fail closed with a stated reason. - A JIT host registration names how many concurrent jobs it will run. +- Programmable workflow steps are TypeScript (or the repo's chosen + runtime); remaining shell is host bootstrap only. +- A mutating host-runnable script refuses without an explicit confirm + or a documented CI environment variable. diff --git a/skills/cicd/references/ci-structure.md b/skills/cicd/references/ci-structure.md index 216644e..bb386ed 100644 --- a/skills/cicd/references/ci-structure.md +++ b/skills/cicd/references/ci-structure.md @@ -113,6 +113,14 @@ moving it to a different runner class without updating the ruleset (see `policy-and-auth.md`) becomes a CI failure instead of a silently broken required check. +Pin the **implementation site**, not a copy of the step body. A +policy-bearing job is asserted to invoke a named script; the script +carries the policy assertions. A path-filter union (orchestrator +`on.push.paths`, a derived list, a frozen snapshot) is generated or +asserted from one source of truth. Extracting a fat `run:` without +retargeting those pins is **composer-contract drift** — +`host-runnable.md`. + ## Label-gated diagnostic jobs Expensive, rarely-needed diagnostics (a heavyweight capacity benchmark, a diff --git a/skills/cicd/references/host-runnable.md b/skills/cicd/references/host-runnable.md index 17dad02..6471d91 100644 --- a/skills/cicd/references/host-runnable.md +++ b/skills/cicd/references/host-runnable.md @@ -5,9 +5,8 @@ any host can run. The YAML (or other remote CI file) only composes them. ## Scripts are the workflow -Prefer TypeScript run with bun (or the repo's existing automation -runtime) under `scripts/`. Each step is independently invocable on a -machine. A step that only works inside GitHub Actions is unfinished. +Each step is independently invocable on a machine. A step that only +works inside GitHub Actions is unfinished. Manual is the default: a human or agent can run the same entry point the remote composer calls. Automation is that composition with a @@ -20,6 +19,47 @@ cannot run the path, extend the script or the local materialization (`cmk:local-stack`, `cmk:infra`) — do not invent a host-specific dialect beside the real one. +## Language + +Programmable workflow and operator steps are TypeScript run with bun +(or the repo's existing automation runtime) under `scripts/`. That is +the default, not a suggestion next to a second ops language. + +Shell stays for host bootstrap: installing the runtime, runner hooks, +and thin wrappers that exec the script. A host that cannot run bun +yet is a reason for a `.sh` bootstrap, not for rewriting the step. + +Do not add a third ops language (Python, Ruby, a one-off DSL) for the +same class of step. Convert by replace, not shim: one implementation +site, the old file gone, callers and contract tests pointed at the +new path in the same change. + +## Composer-contract drift + +Extracting a fat inline `run:` into a script is unfinished until three +things move together: + +1. The YAML step is a one-line invocation of the script. +2. Every contract that grepped the old YAML body retargets to the + script. YAML is pinned only as "it calls that script"; policy + lives in the script. +3. Every path-filter list that named the old body (the workflow's + `on.push.paths`, a derived union, a frozen snapshot test) adds + the new script. One source of truth; the others are asserted + from it. + +Do not raise a line-count or runner-class baseline to accept leftover +YAML. The baseline exists to force the extract. + +## Mutate gate + +A mutating script the laptop and GitHub both invoke refuses unless +the caller passes an explicit confirm (`--confirm` locally) or a +documented environment variable on CI. Absence is refuse, not a +soft skip. A protected GitHub Environment is not a substitute: the +same entry point still has to refuse on a laptop or JIT host that +has no Environment in the picture. + ## Host matrix Scripts that humans or agents run on a development machine support at diff --git a/skills/cicd/references/policy-and-auth.md b/skills/cicd/references/policy-and-auth.md index 682a285..cb7f9a0 100644 --- a/skills/cicd/references/policy-and-auth.md +++ b/skills/cicd/references/policy-and-auth.md @@ -14,7 +14,13 @@ remember to check by hand: - **Test-evidence floor**: a workflow checks the PR description for a clearly labeled testing section containing a reproducible command or evidence link and a stated passing result — it judges only this one objective floor, not - the rest of the description's structure. + the rest of the description's structure. Match a **verdict** (`passed`, + `N tests passing`, `did not pass`), not the substring `fail`. A count + written as `0 fail` and a policy name such as `fail-closed` are not + non-passing claims; a gate that treats them as such is + **evidence-floor negation**. Write `N tests passing`. Keep policy names + that contain `fail` out of the testing section, or teach the checker to + ignore `0 fail` / `fail-closed`. - **Dangerous automation ships disabled by default.** Anything that mutates repository or delivery state on its own (auto-merge, auto-retarget, a conflict-resolution bot) is gated behind an explicit repository variable