fix(prose-tests): the asserter judges which of prose or walker was at fault

[leeovery]

Summary

An epic walk came back FLAKY over a step the prose got right.

The walker read confirm-trigger's route to Step 7, talked itself into Step 6 on the grounds that the file numbers it earlier, read the reference, ran one manifest read — then caught itself: "my own misreading of the routing, not a prose deviation." It corrected and finished correctly. Both runs landed the right world and passed every deterministic check.

The asserter had that sentence in front of it and marked the step failed anyway, because its rules told it to. Never explain a failure was written against asserters that diagnosed causes and proposed fixes — and it over-reached, also forbidding the decision of whether the prose or the walker went wrong. That attribution isn't an explanation of a failure; it's what makes something a failure in the first place, and it's the one judgement only this agent can make. The deterministic checks already catch everything a script can. A reasoning model barred from reasoning is just a slower script.

So the rules now say what they should have:

• A path step fails when the prose failed. A walker that wandered off its own initiative and came back has shown nothing about the prose, and the step passes on the corrected path — quoting the correction, since a pass still needs evidence.
• The record settles that a call was made, never why. A call made during an abandoned detour looks identical to one the prose asked for; only the walk says which it was.

Every guard that made the original rule necessary stays: no invented reading that rescues a step, no assuming an unrecorded action happened, no correct end state excusing a wrong path, genuine ambiguity fails. Judgement is not generosity.

The wander is still reported, under markers — one walker wandering is noise; the same wander recurring on the same step is prose that reads misleadingly however correct it is, and that's only visible if the wobbles are recorded rather than swallowed.

Second commit: a run reports, it never repairs

Two rules the same run earned, added to the /prose-test skill:

• Nothing is fixed during or after a run — prose, case, snapshot or harness. On this run the first fix reached for was a prose change to correct prose; the second was deleting a claim doing real work. The actual fault was a third thing, in the judging. Neither repair would have found it.
• A pass with notes is not a clean pass. Markers always warrant a look.

Follow-up surfaced, not actioned

Discovery's Step 5 is the only step in the corpus that loads a reference and carries no routing of its own. Siblings with the same shape all signpost at the loading step anyway — workflow-discussion-process Step 6 (→ On return, proceed to **Step 7**) and workflow-implementation-process (conditional → Return to **Step 6**), both loading references that route back independently. Not a bug; the routing is complete if you read the reference. But it's the sole departure from a pattern the reader meets everywhere else. Left for a decision on whether a convention should cover it.

Test plan

Requires a plugin reload to take effect, then a re-run of discovery-commits-new-epic unchanged — claim 4 intact, so it's a real test: it should pass on the corrected path with the wander under markers, and still fail if the prose ever genuinely routes the epic arm into Step 6.

🤖 Generated with Claude Code

Stack

1. docs(design): prose-tests programme design log #544
2. feat(prose-tests): the framework — cases, worlds, runner, skill #545
3. test(prose): feature happy-path corpus — five worlds, seven cases #546
4. test(prose): bugfix corpus — the investigation-centric surfaces #548
5. test: retry recursive teardown removals — kill a class of phantom failures #549
6. fix(entry-skills): close the handoff fences — six files render their arms wrong #550
7. docs: a contributing page for working on the system #551
8. fix(entry-skills): every handoff arm says to invoke the skill #552
9. fix(implementation): environment setup belongs to the setup reference alone #553
10. fix(prose-tests): the asserter is told which substitutions were armed #554
11. feat(prose-tests): the mid-flow substitution, and a world only prose can describe #555
12. test(prose): claims assert consequences, not what was displayed #556
13. feat(prose-tests): record everything the agents do, results included #557
14. fix(discussion-entry): the handoff reports the source it actually had #558
15. fix(prose-tests): the stop hook records, and names the model that walked #559
16. fix(prose-tests): command output was never actually recorded #560
17. feat(prose-tests): judge the walk as told, not the summary returned #561
18. feat(prose-tests): decide in code what an agent should not be deciding #562
19. test(prose): a case starts where a session starts #563
20. feat(prose-tests): walk on Sonnet, judge on Opus, escalate a failure #564
21. test(prose): give the eight read-only cases something that can fail #565
22. test(prose): only walks that can be observed, and checks that survive the trip #566
23. fix(prose-tests): the verdict names only the model the record names #567
24. test(prose): discovery, walked to the point where work first exists #568
25. fix(prose-tests): the asserter judges which of prose or walker was at fault #569 👈 current
26. docs(conventions): a step whose reference routes every exit still signposts #570
27. test(prose): discovery's epic arm, to the same durability boundary #571