Skip to content

test(prose): declare the prose each case actually walks#574

Open
leeovery wants to merge 3 commits into
fix/prose-prescribe-what-we-observedfrom
test/prose-declare-what-walks
Open

test(prose): declare the prose each case actually walks#574
leeovery wants to merge 3 commits into
fix/prose-prescribe-what-we-observedfrom
test/prose-declare-what-walks

Conversation

@leeovery

@leeovery leeovery commented Jul 27, 2026

Copy link
Copy Markdown
Owner

Summary

A case's files[] decides whether editing a file selects that case for a run. A file the walk goes through and the case omits is prose that can be changed without the test covering it ever running again.

Seven of ten had gaps — ten declarations missing.

Case Added
spec-entry-from-investigation validate-phase.md
review-entry-fresh invoke-skill.md
planning-entry-asks-for-late-context invoke-skill.md
discussion-entry-seeds-from-carrier invoke-skill.md
start-lists-active-work casing-conventions.md, active-work.md
discovery-commits-new-feature casing-conventions.md, detection-core.md
discovery-commits-new-epic casing-conventions.md, detection-core.md

investigation-entry, spec-entry-from-discussion and implementation-picks-first-task had no gap.

Measured, not reasoned about

Each case was walked with its own act.md and the list taken from the Read calls in the record — ten walkers, one per case.

An earlier attempt derived them by following Load links from the scoped files, and I repeated its output as fact before checking. It reports everything reachable: for the discovery cases that meant session-loop.md, task-loop.md, conclude-discovery.md — branches every case deliberately stops before. Measuring found three cases already correct that the crawl had condemned.

The omissions share one cause

  • invoke-skill.md — missing from 3 of 4 entry cases. Where every entry skill builds its handoff.
  • casing-conventions.md — missing from all 3 cases starting at workflow-start. It's Step 0.1.
  • detection-core.md — missing from both discovery cases. It's Step 2.

Each list named the files the case is about and omitted the ones it merely passes through — the exact distinction the mechanism turns on.

One result deliberately not applied

discussion-entry opens gather-context.md, but the prose only routes there when no carrier exists — and that case has one. The file was read while deciding the branch, not walked. The lint sees opened, not walked, so every result it produces needs that check before it lands.

Also

Restores literal characters in every case.json. Earlier scripted edits wrote them through a JSON encoder that escaped em-dashes to , and the escapes were committed.

Test plan

  • Prose gate 86/86; corpus valid
  • All ten worlds destroyed, tree clean

🤖 Generated with Claude Code

Stack

  1. docs(design): prose-tests programme design log #544
  2. feat(prose-tests): the framework — cases, worlds, runner, skill #545
  3. test(prose): feature happy-path corpus — five worlds, seven cases #546
  4. test(prose): bugfix corpus — the investigation-centric surfaces #548
  5. test: retry recursive teardown removals — kill a class of phantom failures #549
  6. fix(entry-skills): close the handoff fences — six files render their arms wrong #550
  7. docs: a contributing page for working on the system #551
  8. fix(entry-skills): every handoff arm says to invoke the skill #552
  9. fix(implementation): environment setup belongs to the setup reference alone #553
  10. fix(prose-tests): the asserter is told which substitutions were armed #554
  11. feat(prose-tests): the mid-flow substitution, and a world only prose can describe #555
  12. test(prose): claims assert consequences, not what was displayed #556
  13. feat(prose-tests): record everything the agents do, results included #557
  14. fix(discussion-entry): the handoff reports the source it actually had #558
  15. fix(prose-tests): the stop hook records, and names the model that walked #559
  16. fix(prose-tests): command output was never actually recorded #560
  17. feat(prose-tests): judge the walk as told, not the summary returned #561
  18. feat(prose-tests): decide in code what an agent should not be deciding #562
  19. test(prose): a case starts where a session starts #563
  20. feat(prose-tests): walk on Sonnet, judge on Opus, escalate a failure #564
  21. test(prose): give the eight read-only cases something that can fail #565
  22. test(prose): only walks that can be observed, and checks that survive the trip #566
  23. fix(prose-tests): the verdict names only the model the record names #567
  24. test(prose): discovery, walked to the point where work first exists #568
  25. fix(prose-tests): the asserter judges which of prose or walker was at fault #569
  26. docs(conventions): a step whose reference routes every exit still signposts #570
  27. test(prose): discovery's epic arm, to the same durability boundary #571
  28. fix(prose-tests): keep the walk's closing turn, and name the one live difference #572
  29. fix(prose-tests): prescribe the cross-check, drop a check that cannot fire, lint the scope #573
  30. test(prose): declare the prose each case actually walks #574 👈 current
  31. fix(investigation): the symptom interview never ran for work shaped in discovery #575
  32. feat(prose-tests): conduct — playing a person where a script cannot reach #576
  33. fix(prose-tests): what a write put in a file is evidence, not incident #577
  34. fix(investigation): the interview starts from the carrier, not over it #578
  35. test(prose): quick-fix scoping, the whole definition stage in one walk #579
  36. fix(prose-tests): the world log is the walker's record #580
  37. fix(prose-tests): a confirmed failure reports both runs' checks #581
  38. fix(prose-tests): each prescribed command runs as written #582
  39. fix(scoping): context gathering reads the whole carrier, asks only the gaps #583

This was referenced Jul 27, 2026
@leeovery
leeovery force-pushed the fix/prose-prescribe-what-we-observed branch from f141d41 to 57c2940 Compare July 27, 2026 14:33
@leeovery
leeovery force-pushed the test/prose-declare-what-walks branch from 2d4c6b8 to 7ce288f Compare July 27, 2026 14:33
@leeovery
leeovery force-pushed the fix/prose-prescribe-what-we-observed branch from 57c2940 to 334973a Compare July 27, 2026 14:35
@leeovery
leeovery force-pushed the test/prose-declare-what-walks branch from 7ce288f to 6bf9437 Compare July 27, 2026 14:35
@leeovery
leeovery force-pushed the fix/prose-prescribe-what-we-observed branch from 334973a to 995afd9 Compare July 27, 2026 14:37
@leeovery
leeovery force-pushed the test/prose-declare-what-walks branch from 6bf9437 to 16ba569 Compare July 27, 2026 14:37
@leeovery
leeovery force-pushed the fix/prose-prescribe-what-we-observed branch from 995afd9 to 963d371 Compare July 27, 2026 14:40
@leeovery
leeovery force-pushed the test/prose-declare-what-walks branch from 16ba569 to 2f7f55c Compare July 27, 2026 14:40
@leeovery
leeovery force-pushed the fix/prose-prescribe-what-we-observed branch from 963d371 to fd67ef4 Compare July 27, 2026 14:42
@leeovery
leeovery force-pushed the test/prose-declare-what-walks branch from 2f7f55c to c61e980 Compare July 27, 2026 14:42
@leeovery
leeovery force-pushed the fix/prose-prescribe-what-we-observed branch from fd67ef4 to 8e11bc3 Compare July 27, 2026 14:44
@leeovery
leeovery force-pushed the test/prose-declare-what-walks branch from c61e980 to 90030d3 Compare July 27, 2026 14:44
@leeovery
leeovery force-pushed the fix/prose-prescribe-what-we-observed branch from 8e11bc3 to 85650d2 Compare July 27, 2026 14:46
@leeovery
leeovery force-pushed the test/prose-declare-what-walks branch from 90030d3 to 2c987ae Compare July 27, 2026 14:46
@leeovery
leeovery force-pushed the fix/prose-prescribe-what-we-observed branch from 85650d2 to f7a5638 Compare July 27, 2026 14:48
@leeovery
leeovery force-pushed the test/prose-declare-what-walks branch from 2c987ae to 2562ad4 Compare July 27, 2026 14:48
@leeovery
leeovery force-pushed the fix/prose-prescribe-what-we-observed branch from f7a5638 to e990814 Compare July 27, 2026 14:50
@leeovery
leeovery force-pushed the test/prose-declare-what-walks branch from 2562ad4 to f3e17fb Compare July 27, 2026 14:50
leeovery and others added 2 commits July 27, 2026 15:51
A case's file list decides whether editing a file selects that case for a
run, so a file the walk goes through and the case omits is prose that can
be changed without the test covering it ever running again. Seven of the
ten had such gaps; ten declarations were missing in total.

They were measured, not reasoned about. Each case was walked with its own
act, and the list taken from the Read calls in the record. An earlier
attempt to derive them by following Load links from the scoped files gave
the wrong answer, and I repeated its output as fact before checking: it
reports everything reachable, which for the discovery cases meant
session-loop, task-loop and conclude-discovery — branches every case
deliberately stops before. Three cases turned out to have no gap at all.

The omissions are systematic rather than scattered, and they share one
cause. invoke-skill.md is absent from three of the four entry cases; it
is where every entry skill builds its handoff. casing-conventions.md is
absent from all three cases that begin at workflow-start; it is Step 0.1.
detection-core.md is absent from both discovery cases; it is Step 2. Each
time the list named the files the case is *about* and left out the ones
it merely passes through — which is the distinction the whole mechanism
turns on.

One measured result is deliberately not applied. discussion-entry opens
gather-context.md, but the prose only routes there when no carrier
exists, and that case has one: the file was read while deciding the
branch, not walked. The lint sees opened, not walked, so every result it
gives needs that check before it lands.

Also restores the literal characters in every case.json. Earlier scripted
edits wrote them through a JSON encoder that escaped em-dashes, and the
escapes were committed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…ough

A full run over the freshly-filled file lists reported two cases still
opening prose they do not declare: implementation-picks-first-task reads
invoke-skill.md, and discovery-commits-new-epic reads resume-detection.md.

Both were measured clean during the harvest, by a walk that genuinely did
not open them. The harvest walked each case once, and once is a sample: a
walk touches a slightly different set of files each time it runs, so a
single measurement floors the answer rather than settling it.

That is worth knowing about the mechanism itself. A file list converges
across runs rather than being audited in one pass, and the lint is the
ratchet that closes the gap each time a new file surfaces — not a
one-shot check that can be declared finished.

The implementation case is the sharper illustration. During the harvest
its walker claimed to have read invoke-skill.md, the record showed it had
not, and the divergence was recorded as a walker misreporting itself. It
was — and the file it invented reading is genuinely on the path, which
the next run took. A false claim landed on a true fact.

invoke-skill.md is now declared by all four entry cases, which is what
the path shape always implied: it is where every entry skill builds its
handoff.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@leeovery
leeovery force-pushed the fix/prose-prescribe-what-we-observed branch from e990814 to 6325906 Compare July 27, 2026 14:51
Every case so far stops where a skill hands off, which is where the phase
begins doing work. Seven processing skills had no coverage at all. This
carries on through the handoff into workflow-investigation-process and
stops at that phase's durability boundary: the file created, the topic
registered, the commit landed, immediately before the symptom interview.

The spine is an ordering claim that crosses the skill boundary — the
entry's status check must precede the processing skill's topic start,
because a topic cannot be started before anything has looked to see
whether it already exists. Alongside it, initialisation must read the
work's seed before writing: the prose opens the phase from the work's
origin, and a walk that wrote first would build the file from nothing.

Scoped to Steps 0 through 2 after reading Step 3, which is a question
bank rather than a gate — Core, Reproduction, Environment, Impact,
Follow-up. That is the organic loop `conduct` is for, and it is better to
shape that field against a case that demands it than to guess now.

Passed first run: 5/5 with the further claim, all four checks, world
matched, and no undeclared prose — the file list was right without a
harvest pass.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@leeovery
leeovery force-pushed the test/prose-declare-what-walks branch from f3e17fb to 3792c26 Compare July 27, 2026 14:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant