Skip to content

fix(prose-tests): each prescribed command runs as written - #582

Merged
leeovery merged 1 commit into
mainfrom
fix/prose-walker-run-as-written
Jul 27, 2026
Merged

fix(prose-tests): each prescribed command runs as written#582
leeovery merged 1 commit into
mainfrom
fix/prose-walker-run-as-written

Conversation

@leeovery

@leeovery leeovery commented Jul 27, 2026

Copy link
Copy Markdown
Owner

Summary

The quick-fix confirmation walk skipped two commands the prose prescribes unconditionally — the project-default format set and the scoping start/complete pair — and still landed a world the asserter passed, having converged on the same state by other means. calls_include caught it, which is what it's for; but the walker rule it slipped through was real: "follow the prose literally" never said what literal means for a fenced command.

Now it does — one rule added:

Run each prescribed command as written — one call per fence. Never batch adjacent commands into one invocation, merge them, reorder them, or substitute an equivalent that lands the same state. A walk that reaches the right end by different calls has not tested the calls the prose prescribes, and the record it leaves says the prose does something it does not.

Test plan

  • Agent-prose change; takes effect on reload, proven by the case re-run above the stack

🤖 Generated with Claude Code

Stack

  1. docs(design): prose-tests programme design log #544
  2. feat(prose-tests): the framework — cases, worlds, runner, skill #545
  3. test(prose): feature happy-path corpus — five worlds, seven cases #546
  4. test(prose): bugfix corpus — the investigation-centric surfaces #548
  5. test: retry recursive teardown removals — kill a class of phantom failures #549
  6. fix(entry-skills): close the handoff fences — six files render their arms wrong #550
  7. docs: a contributing page for working on the system #551
  8. fix(entry-skills): every handoff arm says to invoke the skill #552
  9. fix(implementation): environment setup belongs to the setup reference alone #553
  10. fix(prose-tests): the asserter is told which substitutions were armed #554
  11. feat(prose-tests): the mid-flow substitution, and a world only prose can describe #555
  12. test(prose): claims assert consequences, not what was displayed #556
  13. feat(prose-tests): record everything the agents do, results included #557
  14. fix(discussion-entry): the handoff reports the source it actually had #558
  15. fix(prose-tests): the stop hook records, and names the model that walked #559
  16. fix(prose-tests): command output was never actually recorded #560
  17. feat(prose-tests): judge the walk as told, not the summary returned #561
  18. feat(prose-tests): decide in code what an agent should not be deciding #562
  19. test(prose): a case starts where a session starts #563
  20. feat(prose-tests): walk on Sonnet, judge on Opus, escalate a failure #564
  21. test(prose): give the eight read-only cases something that can fail #565
  22. test(prose): only walks that can be observed, and checks that survive the trip #566
  23. fix(prose-tests): the verdict names only the model the record names #567
  24. test(prose): discovery, walked to the point where work first exists #568
  25. fix(prose-tests): the asserter judges which of prose or walker was at fault #569
  26. docs(conventions): a step whose reference routes every exit still signposts #570
  27. test(prose): discovery's epic arm, to the same durability boundary #571
  28. fix(prose-tests): keep the walk's closing turn, and name the one live difference #572
  29. fix(prose-tests): prescribe the cross-check, drop a check that cannot fire, lint the scope #573
  30. test(prose): declare the prose each case actually walks #574
  31. fix(investigation): the symptom interview never ran for work shaped in discovery #575
  32. feat(prose-tests): conduct — playing a person where a script cannot reach #576
  33. fix(prose-tests): what a write put in a file is evidence, not incident #577
  34. fix(investigation): the interview starts from the carrier, not over it #578
  35. test(prose): quick-fix scoping, the whole definition stage in one walk #579
  36. fix(prose-tests): the world log is the walker's record #580
  37. fix(prose-tests): a confirmed failure reports both runs' checks #581
  38. fix(prose-tests): each prescribed command runs as written #582 👈 current
  39. fix(scoping): context gathering reads the whole carrier, asks only the gaps #583

This was referenced Jul 27, 2026
@leeovery
leeovery force-pushed the fix/prose-walker-run-as-written branch from 4420d72 to 9725f1e Compare July 27, 2026 15:57
@leeovery
leeovery force-pushed the fix/prose-orchestrator-per-run-checks branch from 15c7654 to cdb4039 Compare July 27, 2026 15:58
@leeovery
leeovery force-pushed the fix/prose-walker-run-as-written branch from 9725f1e to b41c414 Compare July 27, 2026 15:58
@leeovery
leeovery force-pushed the fix/prose-orchestrator-per-run-checks branch from cdb4039 to 788b27a Compare July 27, 2026 15:59
@leeovery
leeovery force-pushed the fix/prose-walker-run-as-written branch from b41c414 to 40009f9 Compare July 27, 2026 15:59
@leeovery
leeovery force-pushed the fix/prose-orchestrator-per-run-checks branch from 788b27a to 220247c Compare July 27, 2026 16:00
@leeovery
leeovery force-pushed the fix/prose-walker-run-as-written branch from 40009f9 to 964711d Compare July 27, 2026 16:00
@leeovery
leeovery force-pushed the fix/prose-orchestrator-per-run-checks branch from 220247c to b36e00f Compare July 27, 2026 16:01
@leeovery
leeovery force-pushed the fix/prose-walker-run-as-written branch from 964711d to d431692 Compare July 27, 2026 16:01
@leeovery
leeovery force-pushed the fix/prose-orchestrator-per-run-checks branch from b36e00f to 36a43a5 Compare July 27, 2026 16:02
@leeovery
leeovery force-pushed the fix/prose-walker-run-as-written branch from d431692 to a77e9ff Compare July 27, 2026 16:02
@leeovery
leeovery force-pushed the fix/prose-orchestrator-per-run-checks branch from 36a43a5 to 0c91077 Compare July 27, 2026 17:05
@leeovery
leeovery force-pushed the fix/prose-walker-run-as-written branch from a77e9ff to fb6b748 Compare July 27, 2026 17:05
@leeovery
leeovery force-pushed the fix/prose-orchestrator-per-run-checks branch from 0c91077 to ef246f0 Compare July 27, 2026 17:06
@leeovery
leeovery force-pushed the fix/prose-walker-run-as-written branch from fb6b748 to a8f2f19 Compare July 27, 2026 17:06
@leeovery
leeovery force-pushed the fix/prose-orchestrator-per-run-checks branch from ef246f0 to f3f73ca Compare July 27, 2026 17:06
@leeovery
leeovery force-pushed the fix/prose-walker-run-as-written branch from a8f2f19 to b684124 Compare July 27, 2026 17:06
@leeovery
leeovery force-pushed the fix/prose-orchestrator-per-run-checks branch from f3f73ca to 5ed559d Compare July 27, 2026 17:07
@leeovery
leeovery force-pushed the fix/prose-walker-run-as-written branch from b684124 to 967b9ee Compare July 27, 2026 17:07
@leeovery
leeovery force-pushed the fix/prose-orchestrator-per-run-checks branch from 5ed559d to 1a12bc8 Compare July 27, 2026 17:08
@leeovery
leeovery force-pushed the fix/prose-walker-run-as-written branch from 967b9ee to 34347e9 Compare July 27, 2026 17:08
@leeovery
leeovery force-pushed the fix/prose-orchestrator-per-run-checks branch from 1a12bc8 to 3224e10 Compare July 27, 2026 17:08
@leeovery
leeovery force-pushed the fix/prose-walker-run-as-written branch from 34347e9 to cf257dd Compare July 27, 2026 17:08
@leeovery
leeovery force-pushed the fix/prose-orchestrator-per-run-checks branch from 3224e10 to bbe8844 Compare July 27, 2026 17:09
@leeovery
leeovery force-pushed the fix/prose-walker-run-as-written branch from cf257dd to 2898cfe Compare July 27, 2026 17:09
@leeovery
leeovery force-pushed the fix/prose-orchestrator-per-run-checks branch from bbe8844 to b8a86ad Compare July 27, 2026 17:09
@leeovery
leeovery force-pushed the fix/prose-walker-run-as-written branch from 2898cfe to 85fec01 Compare July 27, 2026 17:09
@leeovery
leeovery force-pushed the fix/prose-orchestrator-per-run-checks branch from b8a86ad to 29ce889 Compare July 27, 2026 17:09
@leeovery
leeovery force-pushed the fix/prose-walker-run-as-written branch from 85fec01 to 4045d5c Compare July 27, 2026 17:09
The quick-fix confirmation walk skipped two commands the prose
prescribes unconditionally — the project-default format set and the
scoping start/complete pair — and still landed a world the asserter
passed, having converged on the same state by other means. The
deterministic checks caught it, which is what they are for; but the
walker rule it slipped through was real. "Follow the prose literally"
never said what literal means for a fenced command.

Now it does: one call per fence, never batched, merged, reordered, or
replaced by an equivalent that lands the same state. A walk that reaches
the right end by different calls has not tested the calls the prose
prescribes, and its record claims the prose does something it does not.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant