test(prose): claims assert consequences, not what was displayed#556
Open
leeovery wants to merge 1 commit into
Open
test(prose): claims assert consequences, not what was displayed#556leeovery wants to merge 1 commit into
leeovery wants to merge 1 commit into
Conversation
leeovery
force-pushed
the
prose-tests/discussion-conversation-stub
branch
from
July 26, 2026 13:43
f83df8e to
9d3d1b3
Compare
P4c was written and not applied. The runs then demonstrated why it matters: root-cause-validation failed only on "offers the choice and waits" and "announces that validation is running", and implementation-picks-first-task came back FLAKY on "asks the question, gathering the answer" — evidenced in one run's narrative, absent from the other, with an identical world both times. Display claims don't merely fail; they fail intermittently, which is worse than failing. Each is rewritten as something the record can prove: - "offers the choice and waits" → the scripted yes was consumed, so the run arm was taken; a skip would have returned with no dispatch - "announces running, dispatches exactly one agent" → one dispatch recorded, one report written, no second of either - "asks the question, gathering without acting" → the answer was consumed there, and no setup document was written or committed by the entry - "renders the phase note and emits it as produced" → renders it through the engine with the verb Starting; the render is an engine call, the emission is not No prose touched — this is test authoring only. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
leeovery
force-pushed
the
prose-tests/consequence-claims
branch
from
July 26, 2026 13:44
b55f4f4 to
4790742
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
root-cause-validation-clean-verdictfailed only on "offers the choice and waits" and "announces that validation is running".implementation-picks-first-taskcame back FLAKY on "asks the question, gathering the answer" — evidenced in one run's narrative, absent from the other, with an identical world both times. Display claims don't merely fail, they fail intermittently, which is worse than failing.yeswas consumed, so the run arm was taken — askipwould have returned with no dispatchNo prose touched — test authoring only.
Test plan
npm testgreen: 1699 tests, 0 fail. Corpus valid.🤖 Generated with Claude Code