Close the action contract: one registry, no prose fallback (#241 follow-up)

[jeremymanning]

Narrowly bounded follow-up to #241, as requested, before the Result API.

What was measured first

A census of 130 pipeline documents, counting only steps without a tool: key — steps with one route by tool and never touch the action vocabulary:

	steps
dispatched by action name	430
already a registered action	247 (57%)
relying on the prose → model fallback	167 (53 distinct)
<AUTO> instructions	14
declared prose families (echo …)	2

My first census was wrong and counted write/execute/read as prose — they are tool operations in the two-field form. The numbers above are from parsed steps, not raw action: lines.

Among the 167: generate-text, used six times, had never dispatched. A hyphen slip of generate_text — the same defect class as generate_structured in #437, which is exactly the drift this PR closes.

The registry

ActionSpec carries canonical name, aliases, handler, whether a model or tool is required, required parameters, result schema and v1 membership. Everything else is derived:

Consumer	Derived as
Executor dispatch	resolve_action(...).handler
Validator recognition	is_known_action(...)
supported_actions	SUPPORTED_ACTIONS
Alias normalisation	canonical_action(...), applied by the compiler
Documentation	docs/actions.md, generated + drift-tested

The ten advertised-but-unimplemented names are gone. Anything advertised is now executable, with a test asserting exactly that.

Unknown actions are refused twice

At compile time, and again at dispatch — a caller can build a Task and reach execution without going through YAML, so compile-time validation is not the only boundary. Both are tested.

<AUTO>...</AUTO> stays supported and is now a declared ActionFamily rather than an accident of the fallback: an author explicitly asking the model to interpret an instruction is not the same as a typo falling through. echo … and the prose filesystem forms are declared the same way; their regexes moved into the registry and the two now-unused predicate methods are deleted.

Three real bugs fell out of writing the tests

1. Registered actions handled one level down (generate_structured lives on ModelBasedControlSystem) hit the new raise instead of delegating.
2. The filesystem handler keys off a hardcoded list containing "file" but not "filesystem". Normalising the alias sent canonical steps past that branch into a default that tried to write to the read-only root.
3. Every failed step recorded the error handler's recovery decision — {"action": "retry", "delay": 5.0, ...} — as its error. The reason a step failed was being discarded system-wide, and Task.error was not even an exception. Now the original error is recorded with the recovery decision beside it, and a compile failure leads with the actual messages instead of Issues: 1 errors.

Number 3 is the one worth noting for the Result API: structured errors were not merely missing, they were actively overwritten.

Also in here

• A pre-existing SyntaxError in scripts/utilities/organization_maintenance.py — extend([ closed with ]. That file could not be imported at all. Found by extending the correctness lint to scripts/.
• The cleanliness guard now catches untracked files, per your point 3: git status --porcelain --untracked-files=all, since git diff cannot see newly created ones. legacy-results.txt is excluded as the single expected artifact.
• The exit-code boundary is documented, per your point 4: compile-time failures exit 2; runtime template and execution failures exit 1, with the reasoning about multi-pass resolution written down.
• tests/test_builtin_actions.py folded into tests/test_action_contract.py (53 tests); its assertions were subsumed once the registry became authoritative.

Numbers

	before	after
Blocking suite	456 passed	476 passed, 13 skipped, 0 failed, 0 xfailed
Collection	3251	3271
Examples validating	3 / 111	3 / 111 — tightening cost no valid pipeline
ruff correctness (src + scripts)	1 SyntaxError	clean
uv lock --check, git diff --check, ci.yml parse	—	clean

Deliberately not here

Parameter schemas, only required-parameter lists. ActionSpec has the field, and the validator checks required names, but full JSON Schema per action means deciding the parameter vocabulary for every action — that belongs with the canonical YAML schema work, not smuggled into a closure PR.

Result schemas are declared for generate and generate_structured only — the two whose envelope shape is actually fixed today. Asserting a schema for handlers that return ad-hoc dicts would mean inventing their contracts here rather than in the Result API, where the per-step envelope gets designed once.

[jeremymanning]

CI 9/9 green, and the legacy backlog is unchanged — exactly, not approximately.

	main @ 18d18c0	this PR
failed	475	475
passed	1845	1845
skipped	200	200
errors	189	189
deselected	555	575

Every count identical. The only movement is deselected +20 — the new contract-marked tests, which the legacy selection excludes by construction.

Notably zero UnknownActionErrors appear anywhere in the legacy failure list, so removing the prose fallback did not break a single legacy test that was previously passing or failing for another reason.

How this number was obtained, because the last two attempts were wrong

First I inferred main's baseline (474/1846/189) by projecting #440's effect onto #439's numbers. That produced an apparent +1 failed / −1 passed regression, and I went looking for a test that had broken. It had not — the inference was simply wrong.

gh run view --log returned nothing for main's legacy job, so log scraping was not available either. The reliable source turned out to be the workflow annotation:

gh api repos/OWNER/REPO/check-runs/<job-id>/annotations --jq '.[] | .message'
475 failed, 1845 passed, 200 skipped, 555 deselected, 9 warnings, 189 errors in 176.50s

The job publishes its tally as an annotation precisely so the backlog stays visible past continue-on-error, and that annotation is queryable per job. That is the route to use for future comparisons — not inference, and not log scraping.

Unrelated, but visible in the same annotations

Node.js 20 is deprecated. The following actions target Node.js 20 but are being
forced to run on Node.js 24: actions/checkout@v4, actions/setup-python@v5,
actions/upload-artifact@v4, astral-sh/setup-uv@v5

Four actions need bumping. Not touched here — it belongs with the release-engineering item that also wants actions pinned by commit SHA.