From c99658cbe4b4646e802b51f856109dde4d8dcf9a Mon Sep 17 00:00:00 2001 From: Lee Overy Date: Sat, 25 Jul 2026 11:36:00 +0100 Subject: [PATCH 01/11] =?UTF-8?q?test(prose):=20bugfix=20corpus=20?= =?UTF-8?q?=E2=80=94=20the=20investigation-centric=20surfaces?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Stage 3b of design/prose-tests.md. Three `crash-fix` worlds and four cases covering only what bugfix does differently from the linear pipeline: continue-bugfix routing a fresh bug to investigation, investigation entry seeding from the carrier without re-asking, the root-cause validation agent's dispatch → scan → incorporate lifecycle, and a spec sourced from the investigation rather than a discussion. Two framework extensions the corpus demanded: a free-text `stub:` section (agent-dispatching prose is walked with the walker playing the agent — the lifecycle is under test, not the agent's judgment) and a `json` state assertion (the agent row's closing status is checked in the cache store, not read out of a transcript). Both were proven by hand-walking the lifecycle in a live world first. Co-Authored-By: Claude Opus 5 --- design/prose-tests.md | 16 ++ tests/prose/README.md | 8 + tests/prose/bugfix/cases.md | 126 +++++++++++ .../fixtures/_shared/bugfix-mainline.cjs | 200 ++++++++++++++++++ .../prose/fixtures/bugfix-created/recipe.cjs | 11 + .../snapshot/.claude/settings.json | 9 + .../snapshot/.workflows/.state/migrations | 52 +++++ .../snapshot/.workflows/_gitignore.fixture | 2 + .../discovery/sessions/session-001.md | 40 ++++ .../.workflows/crash-fix/manifest.json | 8 + .../snapshot/.workflows/manifest.json | 7 + .../fixtures/bugfix-investigated/recipe.cjs | 11 + .../snapshot/.claude/settings.json | 9 + .../snapshot/.workflows/.state/migrations | 52 +++++ .../snapshot/.workflows/_gitignore.fixture | 2 + .../discovery/sessions/session-001.md | 40 ++++ .../crash-fix/investigation/crash-fix.md | 90 ++++++++ .../.workflows/crash-fix/manifest.json | 16 ++ .../snapshot/.workflows/manifest.json | 7 + .../fixtures/bugfix-investigating/recipe.cjs | 11 + .../snapshot/.claude/settings.json | 9 + .../snapshot/.workflows/.state/migrations | 52 +++++ .../snapshot/.workflows/_gitignore.fixture | 2 + .../discovery/sessions/session-001.md | 40 ++++ .../crash-fix/investigation/crash-fix.md | 88 ++++++++ .../.workflows/crash-fix/manifest.json | 16 ++ .../snapshot/.workflows/manifest.json | 7 + tests/prose/lib/cases.cjs | 22 +- tests/prose/run.cjs | 21 ++ tests/scripts/test-prose-cases.cjs | 14 ++ 30 files changed, 986 insertions(+), 2 deletions(-) create mode 100644 tests/prose/bugfix/cases.md create mode 100644 tests/prose/fixtures/_shared/bugfix-mainline.cjs create mode 100644 tests/prose/fixtures/bugfix-created/recipe.cjs create mode 100644 tests/prose/fixtures/bugfix-created/snapshot/.claude/settings.json create mode 100644 tests/prose/fixtures/bugfix-created/snapshot/.workflows/.state/migrations create mode 100644 tests/prose/fixtures/bugfix-created/snapshot/.workflows/_gitignore.fixture create mode 100644 tests/prose/fixtures/bugfix-created/snapshot/.workflows/crash-fix/discovery/sessions/session-001.md create mode 100644 tests/prose/fixtures/bugfix-created/snapshot/.workflows/crash-fix/manifest.json create mode 100644 tests/prose/fixtures/bugfix-created/snapshot/.workflows/manifest.json create mode 100644 tests/prose/fixtures/bugfix-investigated/recipe.cjs create mode 100644 tests/prose/fixtures/bugfix-investigated/snapshot/.claude/settings.json create mode 100644 tests/prose/fixtures/bugfix-investigated/snapshot/.workflows/.state/migrations create mode 100644 tests/prose/fixtures/bugfix-investigated/snapshot/.workflows/_gitignore.fixture create mode 100644 tests/prose/fixtures/bugfix-investigated/snapshot/.workflows/crash-fix/discovery/sessions/session-001.md create mode 100644 tests/prose/fixtures/bugfix-investigated/snapshot/.workflows/crash-fix/investigation/crash-fix.md create mode 100644 tests/prose/fixtures/bugfix-investigated/snapshot/.workflows/crash-fix/manifest.json create mode 100644 tests/prose/fixtures/bugfix-investigated/snapshot/.workflows/manifest.json create mode 100644 tests/prose/fixtures/bugfix-investigating/recipe.cjs create mode 100644 tests/prose/fixtures/bugfix-investigating/snapshot/.claude/settings.json create mode 100644 tests/prose/fixtures/bugfix-investigating/snapshot/.workflows/.state/migrations create mode 100644 tests/prose/fixtures/bugfix-investigating/snapshot/.workflows/_gitignore.fixture create mode 100644 tests/prose/fixtures/bugfix-investigating/snapshot/.workflows/crash-fix/discovery/sessions/session-001.md create mode 100644 tests/prose/fixtures/bugfix-investigating/snapshot/.workflows/crash-fix/investigation/crash-fix.md create mode 100644 tests/prose/fixtures/bugfix-investigating/snapshot/.workflows/crash-fix/manifest.json create mode 100644 tests/prose/fixtures/bugfix-investigating/snapshot/.workflows/manifest.json diff --git a/design/prose-tests.md b/design/prose-tests.md index 16ef1970d..fbb97a112 100644 --- a/design/prose-tests.md +++ b/design/prose-tests.md @@ -109,6 +109,22 @@ testing. ## Log +- 2026-07-25 — Stage 3b, the bugfix corpus. Three fixtures along + `crash-fix` (created / investigating / investigated) and four cases + covering only what bugfix does differently: continue-bugfix routing a + fresh bug to investigation, investigation entry seeding from the + carrier without re-asking, the root-cause validation agent's full + lifecycle, and a specification sourced from the investigation rather + than a discussion. Two framework extensions the corpus demanded, + both small: a free-text **`stub:` section** — prose that dispatches a + background agent is walked with the walker playing the agent (writes + the report at the engine-allocated path, returns the stated result), + because the case tests the lifecycle around the agent, never the + agent's judgment; and a **`json` state assertion** so the agent row's + closing status is checked deterministically in the cache store rather + than read out of a transcript. Both proven by hand-walking the + lifecycle in a live world before the cases shipped. + - 2026-07-25 — Stage 3a, the feature corpus. Five stop-point fixtures along the canonical feature mainline (`pay` — created / discussed / specified / planned / implemented), composed from staged builders in diff --git a/tests/prose/README.md b/tests/prose/README.md index 93c6b87fa..240a049d6 100644 --- a/tests/prose/README.md +++ b/tests/prose/README.md @@ -22,6 +22,14 @@ walk). Scope `files:` tightly — diff-selection and the PR-end suggestion run off it. `routing:` expects are agent-graded against the walk transcript; `state:` expects are asserted in code and require a `world:`. +**Stub the agents, walk the lifecycle.** Prose that dispatches a +background agent gets a `### stub` section: free text telling the walker +to play the agent itself — write the report at the path the engine's +dispatch returned, return the stated result — instead of spending a real +sub-agent analysing fixture content. What the case tests is the +lifecycle around the agent (dispatch, scan, incorporate, how the verdict +routes), never the agent's own judgment. + **Name behaviour, never coordinates.** Step numbers, arm letters, and heading numbering in walk or expect text break on cosmetic renumbering — failure for the wrong reason. Write "casing conventions load before the diff --git a/tests/prose/bugfix/cases.md b/tests/prose/bugfix/cases.md new file mode 100644 index 000000000..a8b4edf78 --- /dev/null +++ b/tests/prose/bugfix/cases.md @@ -0,0 +1,126 @@ +# Bugfix mainline — happy-path corpus + +The canonical bugfix `crash-fix` walked through what makes bugfix +distinct from the linear pipeline: investigation entry seeded from the +bug's carrier, the root-cause validation agent's full lifecycle, and a +specification sourced from an investigation rather than a discussion. +The delivery phases it shares with feature are covered by the feature +corpus and not repeated here. + +## case: bugfix-continue-offers-investigation +- world: bugfix-created +- origin: bugfix mainline — continue-bugfix routes a fresh bug to investigation +- files: + - skills/workflow-continue-bugfix/SKILL.md + - skills/workflow-continue-bugfix/references/select-bugfix.md + - skills/workflow-continue-bugfix/references/bugfix-display-and-menu.md + +### walk + +Execute skills/workflow-continue-bugfix/SKILL.md with no arguments (as +routed from workflow-start with no pre-selected work unit). Follow +selection, then the bugfix state display and its menu. Record the state +display and menu verbatim, then stop — do not route into any phase +entry skill. + +### user + +1. Select the bugfix `crash-fix` (its number in the selection menu) + +### expect + +- routing: the state display shows no phase started — the pipeline is fresh +- routing: the menu's continue action routes to investigation entry (workflow-investigation-entry with bugfix and crash-fix), not to discussion or specification +- state: manifest absent crash-fix.investigation.crash-fix status + +## case: bugfix-investigation-entry-seeds +- world: bugfix-created +- origin: bugfix mainline — investigation entry seeds from the carrier, never re-asks +- files: + - skills/workflow-investigation-entry/SKILL.md + - skills/workflow-investigation-entry/references/invoke-skill.md + +### walk + +Execute skills/workflow-investigation-entry/SKILL.md with arguments +$0=bugfix, $1=crash-fix. Follow it up to the point where the handoff to +the processing skill is constructed — record the handoff arguments +block — and stop there. Do not execute the processing skill's +instructions. + +### expect + +- routing: with no investigation item in the manifest, source is new — the phase-validation path is not taken +- routing: because a discovery session log exists, the bug context is seeded from the manifest description and that log — the gather-context questioning path is not taken, and the walk asks the user nothing +- routing: a phase note is rendered by the engine for the investigation phase, with the verb Starting +- routing: the handoff invokes workflow-investigation-process for crash-fix +- state: manifest absent crash-fix.investigation.crash-fix status + +## case: bugfix-root-cause-validation-lifecycle +- world: bugfix-investigating +- origin: bugfix mainline — the validation agent's dispatch → scan → incorporate lifecycle +- files: + - skills/workflow-investigation-process/references/root-cause-validation.md + +### walk + +The investigation is open with its root cause documented, and the +process has reached its root-cause validation step. Execute +skills/workflow-investigation-process/references/root-cause-validation.md +from the top, as the processing skill would. Stop when the reference +returns to its caller. + +### user + +1. yes — run root cause validation + +### stub + +The prose directs a validation agent to be dispatched via the Task +tool. Do not dispatch one. Instead, after recording the dispatch, write +the report yourself to the exact `file` path the dispatch response +returned, containing: + + # Root Cause Validation + + The tax context is built before the payment intent and reads the + shipping address unconditionally. Traced fresh: an address-less + order fails at that read, matching the documented root cause. + + STATUS: validated + CONFIDENCE: high + GAPS_COUNT: 0 + SUMMARY: Root cause confirmed by an independent trace. + +Then continue the prose as if the agent had returned that result. + +### expect + +- routing: the dispatch is recorded through the engine, and the report path used is the one the dispatch response returned — the prose does not invent or pre-create the file +- routing: after the report lands, the row is promoted by a scan and then closed by an incorporate — the verdict is consumed inline, never surfaced finding by finding +- routing: on a validated verdict the reference reports validation with its confidence and returns to the caller — the gaps-handling menu is not reached +- state: file exists .workflows/.cache/crash-fix/investigation/crash-fix/root-cause-validation-001.md +- state: json .workflows/.cache/crash-fix/investigation/crash-fix/state.json agents.root-cause-validation-001.status incorporated + +## case: bugfix-spec-entry-sources-investigation +- world: bugfix-investigated +- origin: bugfix mainline — the spec's source is the investigation, not a discussion +- files: + - skills/workflow-specification-entry/SKILL.md + - skills/workflow-specification-entry/references/validate-source.md + - skills/workflow-specification-entry/references/invoke-skill.md + +### walk + +Execute skills/workflow-specification-entry/SKILL.md with arguments +$0=bugfix, $1=crash-fix. Follow it up to the point where the handoff to +the processing skill is constructed — record the handoff arguments +block — and stop there. Do not execute the processing skill's +instructions. + +### expect + +- routing: the source-material entry gate passes (empty render) — the completed investigation satisfies it for a bugfix +- routing: with no specification item in the manifest, the verb is Creating +- routing: the handoff names the investigation file .workflows/crash-fix/investigation/crash-fix.md as source material — no discussion file is named +- state: manifest absent crash-fix.specification.crash-fix status diff --git a/tests/prose/fixtures/_shared/bugfix-mainline.cjs b/tests/prose/fixtures/_shared/bugfix-mainline.cjs new file mode 100644 index 000000000..2ad47219f --- /dev/null +++ b/tests/prose/fixtures/_shared/bugfix-mainline.cjs @@ -0,0 +1,200 @@ +'use strict'; + +// The canonical bugfix mainline — `crash-fix`, a checkout crash — as +// staged builders each fixture recipe composes a prefix of. Engine calls +// follow the sequences the skill prose prescribes; content files are the +// artifacts a real run leaves, in the shape downstream prose reads +// (investigation file per workflow-investigation-process/references/ +// template.md). +// +// Dates are literals matching the frozen recipe clock (2026-01-01). + +const WU = 'crash-fix'; + +function init(h) { + h.knowledge('setup', '--keyword-only'); + h.engine('boot'); +} + +function create(h) { + // Shape per workflow-discovery/references/template.md: bugfix discovery + // captures brief intent; Exploration is backfilled at creation. + const log = `.workflows/${WU}/discovery/sessions/session-001.md`; + h.write(log, [ + '# Discovery Session 001', + '', + 'Date: 2026-01-01', + `Work unit: ${WU}`, + '', + '## Description (as of session)', + '', + 'Checkout crashes when an order has no shipping address.', + '', + '## Seed', + '', + '(none)', + '', + '## Imports', + '', + '(none)', + '', + '## Map State at Start', + '', + '(n/a — single-topic work)', + '', + '## Exploration', + '', + 'Reported by two users this week: the checkout page 500s at the', + 'payment step for orders with no shipping address. Reproducible on', + 'staging with a digital-only basket. No error surfaces to the user —', + 'the page just fails. Confirmed as a bugfix; no design question here,', + 'so it routes straight to investigation.', + '', + '## Edits', + '', + '(none)', + '', + '## Topics Identified', + '', + '(none)', + '', + '## Conclusion', + '', + '(none)', + '', + ].join('\n')); + h.engine('workunit', 'create', WU, 'bugfix', + '--description', 'Checkout crashes when an order has no shipping address', + '--session-log-file', log); +} + +/** The investigation file as it stands with the root cause documented. */ +function investigationFile() { + return [ + '# Investigation: Checkout Crash On Missing Shipping Address', + '', + '## Symptoms', + '', + '### Problem Description', + '', + 'Checkout returns a 500 at the payment step when the order has no', + 'shipping address. No error is surfaced to the user.', + '', + '### Manifestation', + '', + 'Server error on POST to the payment step; the page fails blank.', + '', + '### Reproduction Steps', + '', + '1. Add a digital-only product to the basket.', + '2. Proceed to checkout without entering a shipping address.', + '3. Continue to the payment step — the request 500s.', + '', + '### Environment', + '', + 'Reproduced on staging; reported twice in production this week.', + '', + '### Impact', + '', + 'Digital-only orders cannot complete checkout.', + '', + '### References', + '', + '(none)', + '', + '## Analysis', + '', + '### Hypotheses', + '', + '- The payment step assumes a shipping address is always present.', + '- Tax calculation may be the first consumer of the missing address.', + '', + '### Code Trace', + '', + 'The payment step builds a tax context from the order before creating', + 'the payment intent. The tax context reads the shipping address', + 'unconditionally; for a digital-only order the address is absent, so', + 'the read fails and the request aborts before any handler catches it.', + '', + '### Root Cause', + '', + 'The tax context treats the shipping address as mandatory. Digital-only', + 'orders legitimately have none, so the assumption is wrong rather than', + 'the data being wrong.', + '', + '### Contributing Factors', + '', + 'Digital-only baskets were introduced after the tax context was written.', + '', + "### Why It Wasn't Caught", + '', + 'No test covers a basket with no shippable line items.', + '', + '### Blast Radius', + '', + 'Any flow building a tax context from an address-less order.', + '', + '## Fix Direction', + '', + '### Chosen Approach', + '', + '(pending)', + '', + '### Options Explored', + '', + '(none yet)', + '', + '### Discussion', + '', + '(none yet)', + '', + '### Testing Recommendations', + '', + '(pending)', + '', + '### Risk Assessment', + '', + '(pending)', + '', + '## Notes', + '', + '(none)', + '', + ].join('\n'); +} + +/** + * Investigation open with the root cause documented — the state the + * process reaches just before its root-cause validation step. + */ +function investigateToRootCause(h) { + h.write(`.workflows/${WU}/investigation/${WU}.md`, investigationFile()); + h.engine('topic', 'start', WU, 'investigation', WU); + h.engine('commit', WU, '-m', `investigation(${WU}): symptoms and root cause`); +} + +/** Fix direction settled and the phase concluded. */ +function concludeInvestigation(h) { + h.write(`.workflows/${WU}/investigation/${WU}.md`, investigationFile() + .replace('### Chosen Approach\n\n(pending)', + ['### Chosen Approach', + '', + 'Make the tax context treat the shipping address as optional: when', + 'an order has no shippable items, build the context from the billing', + 'address instead.', + ].join('\n')) + .replace('### Testing Recommendations\n\n(pending)', + ['### Testing Recommendations', + '', + 'Cover a digital-only basket end to end through the payment step.', + ].join('\n')) + .replace('### Risk Assessment\n\n(pending)', + ['### Risk Assessment', + '', + 'Low — the change is confined to tax-context construction.', + ].join('\n'))); + h.engine('commit', WU, '-m', `investigation(${WU}): fix direction`); + h.engine('topic', 'complete', WU, 'investigation', WU); +} + +module.exports = { WU, init, create, investigateToRootCause, concludeInvestigation }; diff --git a/tests/prose/fixtures/bugfix-created/recipe.cjs b/tests/prose/fixtures/bugfix-created/recipe.cjs new file mode 100644 index 000000000..012ec0df2 --- /dev/null +++ b/tests/prose/fixtures/bugfix-created/recipe.cjs @@ -0,0 +1,11 @@ +'use strict'; + +// Bugfix mainline stop point — see _shared/bugfix-mainline.cjs. + +const m = require('../_shared/bugfix-mainline.cjs'); + +module.exports = { + build(h) { + m.init(h); m.create(h); + }, +}; diff --git a/tests/prose/fixtures/bugfix-created/snapshot/.claude/settings.json b/tests/prose/fixtures/bugfix-created/snapshot/.claude/settings.json new file mode 100644 index 000000000..ae019ede5 --- /dev/null +++ b/tests/prose/fixtures/bugfix-created/snapshot/.claude/settings.json @@ -0,0 +1,9 @@ +{ + "showClearContextOnPlanAccept": true, + "permissions": { + "allow": [ + "Edit(.workflows/**)", + "Bash(mv .workflows/:*)" + ] + } +} diff --git a/tests/prose/fixtures/bugfix-created/snapshot/.workflows/.state/migrations b/tests/prose/fixtures/bugfix-created/snapshot/.workflows/.state/migrations new file mode 100644 index 000000000..3c7162acc --- /dev/null +++ b/tests/prose/fixtures/bugfix-created/snapshot/.workflows/.state/migrations @@ -0,0 +1,52 @@ +001 +002 +003 +004 +005 +006 +007 +008 +009 +010 +011 +012 +013 +014 +015 +016 +017 +018 +019 +020 +021 +022 +023 +024 +025 +026 +027 +028 +029 +030 +031 +032 +033 +034 +035 +036 +037 +038 +039 +040 +041 +042 +043 +044 +045 +046 +047 +048 +049 +050 +051 +052 diff --git a/tests/prose/fixtures/bugfix-created/snapshot/.workflows/_gitignore.fixture b/tests/prose/fixtures/bugfix-created/snapshot/.workflows/_gitignore.fixture new file mode 100644 index 000000000..779cbf28a --- /dev/null +++ b/tests/prose/fixtures/bugfix-created/snapshot/.workflows/_gitignore.fixture @@ -0,0 +1,2 @@ +.cache/ +.manifest.json.*.tmp diff --git a/tests/prose/fixtures/bugfix-created/snapshot/.workflows/crash-fix/discovery/sessions/session-001.md b/tests/prose/fixtures/bugfix-created/snapshot/.workflows/crash-fix/discovery/sessions/session-001.md new file mode 100644 index 000000000..6dc56d4e3 --- /dev/null +++ b/tests/prose/fixtures/bugfix-created/snapshot/.workflows/crash-fix/discovery/sessions/session-001.md @@ -0,0 +1,40 @@ +# Discovery Session 001 + +Date: 2026-01-01 +Work unit: crash-fix + +## Description (as of session) + +Checkout crashes when an order has no shipping address. + +## Seed + +(none) + +## Imports + +(none) + +## Map State at Start + +(n/a — single-topic work) + +## Exploration + +Reported by two users this week: the checkout page 500s at the +payment step for orders with no shipping address. Reproducible on +staging with a digital-only basket. No error surfaces to the user — +the page just fails. Confirmed as a bugfix; no design question here, +so it routes straight to investigation. + +## Edits + +(none) + +## Topics Identified + +(none) + +## Conclusion + +(none) diff --git a/tests/prose/fixtures/bugfix-created/snapshot/.workflows/crash-fix/manifest.json b/tests/prose/fixtures/bugfix-created/snapshot/.workflows/crash-fix/manifest.json new file mode 100644 index 000000000..eaaa56199 --- /dev/null +++ b/tests/prose/fixtures/bugfix-created/snapshot/.workflows/crash-fix/manifest.json @@ -0,0 +1,8 @@ +{ + "name": "crash-fix", + "work_type": "bugfix", + "status": "in-progress", + "created": "2026-01-01", + "description": "Checkout crashes when an order has no shipping address", + "phases": {} +} diff --git a/tests/prose/fixtures/bugfix-created/snapshot/.workflows/manifest.json b/tests/prose/fixtures/bugfix-created/snapshot/.workflows/manifest.json new file mode 100644 index 000000000..1cd498ed5 --- /dev/null +++ b/tests/prose/fixtures/bugfix-created/snapshot/.workflows/manifest.json @@ -0,0 +1,7 @@ +{ + "work_units": { + "crash-fix": { + "work_type": "bugfix" + } + } +} diff --git a/tests/prose/fixtures/bugfix-investigated/recipe.cjs b/tests/prose/fixtures/bugfix-investigated/recipe.cjs new file mode 100644 index 000000000..85cf787f4 --- /dev/null +++ b/tests/prose/fixtures/bugfix-investigated/recipe.cjs @@ -0,0 +1,11 @@ +'use strict'; + +// Bugfix mainline stop point — see _shared/bugfix-mainline.cjs. + +const m = require('../_shared/bugfix-mainline.cjs'); + +module.exports = { + build(h) { + m.init(h); m.create(h); m.investigateToRootCause(h); m.concludeInvestigation(h); + }, +}; diff --git a/tests/prose/fixtures/bugfix-investigated/snapshot/.claude/settings.json b/tests/prose/fixtures/bugfix-investigated/snapshot/.claude/settings.json new file mode 100644 index 000000000..ae019ede5 --- /dev/null +++ b/tests/prose/fixtures/bugfix-investigated/snapshot/.claude/settings.json @@ -0,0 +1,9 @@ +{ + "showClearContextOnPlanAccept": true, + "permissions": { + "allow": [ + "Edit(.workflows/**)", + "Bash(mv .workflows/:*)" + ] + } +} diff --git a/tests/prose/fixtures/bugfix-investigated/snapshot/.workflows/.state/migrations b/tests/prose/fixtures/bugfix-investigated/snapshot/.workflows/.state/migrations new file mode 100644 index 000000000..3c7162acc --- /dev/null +++ b/tests/prose/fixtures/bugfix-investigated/snapshot/.workflows/.state/migrations @@ -0,0 +1,52 @@ +001 +002 +003 +004 +005 +006 +007 +008 +009 +010 +011 +012 +013 +014 +015 +016 +017 +018 +019 +020 +021 +022 +023 +024 +025 +026 +027 +028 +029 +030 +031 +032 +033 +034 +035 +036 +037 +038 +039 +040 +041 +042 +043 +044 +045 +046 +047 +048 +049 +050 +051 +052 diff --git a/tests/prose/fixtures/bugfix-investigated/snapshot/.workflows/_gitignore.fixture b/tests/prose/fixtures/bugfix-investigated/snapshot/.workflows/_gitignore.fixture new file mode 100644 index 000000000..779cbf28a --- /dev/null +++ b/tests/prose/fixtures/bugfix-investigated/snapshot/.workflows/_gitignore.fixture @@ -0,0 +1,2 @@ +.cache/ +.manifest.json.*.tmp diff --git a/tests/prose/fixtures/bugfix-investigated/snapshot/.workflows/crash-fix/discovery/sessions/session-001.md b/tests/prose/fixtures/bugfix-investigated/snapshot/.workflows/crash-fix/discovery/sessions/session-001.md new file mode 100644 index 000000000..6dc56d4e3 --- /dev/null +++ b/tests/prose/fixtures/bugfix-investigated/snapshot/.workflows/crash-fix/discovery/sessions/session-001.md @@ -0,0 +1,40 @@ +# Discovery Session 001 + +Date: 2026-01-01 +Work unit: crash-fix + +## Description (as of session) + +Checkout crashes when an order has no shipping address. + +## Seed + +(none) + +## Imports + +(none) + +## Map State at Start + +(n/a — single-topic work) + +## Exploration + +Reported by two users this week: the checkout page 500s at the +payment step for orders with no shipping address. Reproducible on +staging with a digital-only basket. No error surfaces to the user — +the page just fails. Confirmed as a bugfix; no design question here, +so it routes straight to investigation. + +## Edits + +(none) + +## Topics Identified + +(none) + +## Conclusion + +(none) diff --git a/tests/prose/fixtures/bugfix-investigated/snapshot/.workflows/crash-fix/investigation/crash-fix.md b/tests/prose/fixtures/bugfix-investigated/snapshot/.workflows/crash-fix/investigation/crash-fix.md new file mode 100644 index 000000000..d09b4daba --- /dev/null +++ b/tests/prose/fixtures/bugfix-investigated/snapshot/.workflows/crash-fix/investigation/crash-fix.md @@ -0,0 +1,90 @@ +# Investigation: Checkout Crash On Missing Shipping Address + +## Symptoms + +### Problem Description + +Checkout returns a 500 at the payment step when the order has no +shipping address. No error is surfaced to the user. + +### Manifestation + +Server error on POST to the payment step; the page fails blank. + +### Reproduction Steps + +1. Add a digital-only product to the basket. +2. Proceed to checkout without entering a shipping address. +3. Continue to the payment step — the request 500s. + +### Environment + +Reproduced on staging; reported twice in production this week. + +### Impact + +Digital-only orders cannot complete checkout. + +### References + +(none) + +## Analysis + +### Hypotheses + +- The payment step assumes a shipping address is always present. +- Tax calculation may be the first consumer of the missing address. + +### Code Trace + +The payment step builds a tax context from the order before creating +the payment intent. The tax context reads the shipping address +unconditionally; for a digital-only order the address is absent, so +the read fails and the request aborts before any handler catches it. + +### Root Cause + +The tax context treats the shipping address as mandatory. Digital-only +orders legitimately have none, so the assumption is wrong rather than +the data being wrong. + +### Contributing Factors + +Digital-only baskets were introduced after the tax context was written. + +### Why It Wasn't Caught + +No test covers a basket with no shippable line items. + +### Blast Radius + +Any flow building a tax context from an address-less order. + +## Fix Direction + +### Chosen Approach + +Make the tax context treat the shipping address as optional: when +an order has no shippable items, build the context from the billing +address instead. + +### Options Explored + +(none yet) + +### Discussion + +(none yet) + +### Testing Recommendations + +Cover a digital-only basket end to end through the payment step. + +### Risk Assessment + +Low — the change is confined to tax-context construction. + +## Notes + +(none) diff --git a/tests/prose/fixtures/bugfix-investigated/snapshot/.workflows/crash-fix/manifest.json b/tests/prose/fixtures/bugfix-investigated/snapshot/.workflows/crash-fix/manifest.json new file mode 100644 index 000000000..4c5ece2e7 --- /dev/null +++ b/tests/prose/fixtures/bugfix-investigated/snapshot/.workflows/crash-fix/manifest.json @@ -0,0 +1,16 @@ +{ + "name": "crash-fix", + "work_type": "bugfix", + "status": "in-progress", + "created": "2026-01-01", + "description": "Checkout crashes when an order has no shipping address", + "phases": { + "investigation": { + "items": { + "crash-fix": { + "status": "completed" + } + } + } + } +} diff --git a/tests/prose/fixtures/bugfix-investigated/snapshot/.workflows/manifest.json b/tests/prose/fixtures/bugfix-investigated/snapshot/.workflows/manifest.json new file mode 100644 index 000000000..1cd498ed5 --- /dev/null +++ b/tests/prose/fixtures/bugfix-investigated/snapshot/.workflows/manifest.json @@ -0,0 +1,7 @@ +{ + "work_units": { + "crash-fix": { + "work_type": "bugfix" + } + } +} diff --git a/tests/prose/fixtures/bugfix-investigating/recipe.cjs b/tests/prose/fixtures/bugfix-investigating/recipe.cjs new file mode 100644 index 000000000..a0e1bea2c --- /dev/null +++ b/tests/prose/fixtures/bugfix-investigating/recipe.cjs @@ -0,0 +1,11 @@ +'use strict'; + +// Bugfix mainline stop point — see _shared/bugfix-mainline.cjs. + +const m = require('../_shared/bugfix-mainline.cjs'); + +module.exports = { + build(h) { + m.init(h); m.create(h); m.investigateToRootCause(h); + }, +}; diff --git a/tests/prose/fixtures/bugfix-investigating/snapshot/.claude/settings.json b/tests/prose/fixtures/bugfix-investigating/snapshot/.claude/settings.json new file mode 100644 index 000000000..ae019ede5 --- /dev/null +++ b/tests/prose/fixtures/bugfix-investigating/snapshot/.claude/settings.json @@ -0,0 +1,9 @@ +{ + "showClearContextOnPlanAccept": true, + "permissions": { + "allow": [ + "Edit(.workflows/**)", + "Bash(mv .workflows/:*)" + ] + } +} diff --git a/tests/prose/fixtures/bugfix-investigating/snapshot/.workflows/.state/migrations b/tests/prose/fixtures/bugfix-investigating/snapshot/.workflows/.state/migrations new file mode 100644 index 000000000..3c7162acc --- /dev/null +++ b/tests/prose/fixtures/bugfix-investigating/snapshot/.workflows/.state/migrations @@ -0,0 +1,52 @@ +001 +002 +003 +004 +005 +006 +007 +008 +009 +010 +011 +012 +013 +014 +015 +016 +017 +018 +019 +020 +021 +022 +023 +024 +025 +026 +027 +028 +029 +030 +031 +032 +033 +034 +035 +036 +037 +038 +039 +040 +041 +042 +043 +044 +045 +046 +047 +048 +049 +050 +051 +052 diff --git a/tests/prose/fixtures/bugfix-investigating/snapshot/.workflows/_gitignore.fixture b/tests/prose/fixtures/bugfix-investigating/snapshot/.workflows/_gitignore.fixture new file mode 100644 index 000000000..779cbf28a --- /dev/null +++ b/tests/prose/fixtures/bugfix-investigating/snapshot/.workflows/_gitignore.fixture @@ -0,0 +1,2 @@ +.cache/ +.manifest.json.*.tmp diff --git a/tests/prose/fixtures/bugfix-investigating/snapshot/.workflows/crash-fix/discovery/sessions/session-001.md b/tests/prose/fixtures/bugfix-investigating/snapshot/.workflows/crash-fix/discovery/sessions/session-001.md new file mode 100644 index 000000000..6dc56d4e3 --- /dev/null +++ b/tests/prose/fixtures/bugfix-investigating/snapshot/.workflows/crash-fix/discovery/sessions/session-001.md @@ -0,0 +1,40 @@ +# Discovery Session 001 + +Date: 2026-01-01 +Work unit: crash-fix + +## Description (as of session) + +Checkout crashes when an order has no shipping address. + +## Seed + +(none) + +## Imports + +(none) + +## Map State at Start + +(n/a — single-topic work) + +## Exploration + +Reported by two users this week: the checkout page 500s at the +payment step for orders with no shipping address. Reproducible on +staging with a digital-only basket. No error surfaces to the user — +the page just fails. Confirmed as a bugfix; no design question here, +so it routes straight to investigation. + +## Edits + +(none) + +## Topics Identified + +(none) + +## Conclusion + +(none) diff --git a/tests/prose/fixtures/bugfix-investigating/snapshot/.workflows/crash-fix/investigation/crash-fix.md b/tests/prose/fixtures/bugfix-investigating/snapshot/.workflows/crash-fix/investigation/crash-fix.md new file mode 100644 index 000000000..d5fc263a9 --- /dev/null +++ b/tests/prose/fixtures/bugfix-investigating/snapshot/.workflows/crash-fix/investigation/crash-fix.md @@ -0,0 +1,88 @@ +# Investigation: Checkout Crash On Missing Shipping Address + +## Symptoms + +### Problem Description + +Checkout returns a 500 at the payment step when the order has no +shipping address. No error is surfaced to the user. + +### Manifestation + +Server error on POST to the payment step; the page fails blank. + +### Reproduction Steps + +1. Add a digital-only product to the basket. +2. Proceed to checkout without entering a shipping address. +3. Continue to the payment step — the request 500s. + +### Environment + +Reproduced on staging; reported twice in production this week. + +### Impact + +Digital-only orders cannot complete checkout. + +### References + +(none) + +## Analysis + +### Hypotheses + +- The payment step assumes a shipping address is always present. +- Tax calculation may be the first consumer of the missing address. + +### Code Trace + +The payment step builds a tax context from the order before creating +the payment intent. The tax context reads the shipping address +unconditionally; for a digital-only order the address is absent, so +the read fails and the request aborts before any handler catches it. + +### Root Cause + +The tax context treats the shipping address as mandatory. Digital-only +orders legitimately have none, so the assumption is wrong rather than +the data being wrong. + +### Contributing Factors + +Digital-only baskets were introduced after the tax context was written. + +### Why It Wasn't Caught + +No test covers a basket with no shippable line items. + +### Blast Radius + +Any flow building a tax context from an address-less order. + +## Fix Direction + +### Chosen Approach + +(pending) + +### Options Explored + +(none yet) + +### Discussion + +(none yet) + +### Testing Recommendations + +(pending) + +### Risk Assessment + +(pending) + +## Notes + +(none) diff --git a/tests/prose/fixtures/bugfix-investigating/snapshot/.workflows/crash-fix/manifest.json b/tests/prose/fixtures/bugfix-investigating/snapshot/.workflows/crash-fix/manifest.json new file mode 100644 index 000000000..88bf183ff --- /dev/null +++ b/tests/prose/fixtures/bugfix-investigating/snapshot/.workflows/crash-fix/manifest.json @@ -0,0 +1,16 @@ +{ + "name": "crash-fix", + "work_type": "bugfix", + "status": "in-progress", + "created": "2026-01-01", + "description": "Checkout crashes when an order has no shipping address", + "phases": { + "investigation": { + "items": { + "crash-fix": { + "status": "in-progress" + } + } + } + } +} diff --git a/tests/prose/fixtures/bugfix-investigating/snapshot/.workflows/manifest.json b/tests/prose/fixtures/bugfix-investigating/snapshot/.workflows/manifest.json new file mode 100644 index 000000000..1cd498ed5 --- /dev/null +++ b/tests/prose/fixtures/bugfix-investigating/snapshot/.workflows/manifest.json @@ -0,0 +1,7 @@ +{ + "work_units": { + "crash-fix": { + "work_type": "bugfix" + } + } +} diff --git a/tests/prose/lib/cases.cjs b/tests/prose/lib/cases.cjs index 1ff5705fd..e8458820f 100644 --- a/tests/prose/lib/cases.cjs +++ b/tests/prose/lib/cases.cjs @@ -21,16 +21,27 @@ // {free text: entry point, what to do, stop condition} // ### user // 1. {scripted answer, consumed in order} +// ### stub +// {free text: what to do INSTEAD of dispatching a real sub-agent} // ### expect // - routing: {claim graded by an agent against the walk transcript} // - state: {deterministic assertion — see parseStateAssertion} // +// `stub` is optional and free-form. Prose that dispatches background +// agents is walked with those agents stubbed: the walker plays the +// agent (writes the report the prose expects at the path the engine +// allocated, returns the stated result) instead of spending a real +// sub-agent on analysis of fixture content. The lifecycle around the +// agent — dispatch, scan, incorporate, the verdict's routing — is what +// the case tests; the agent's own judgment is not. +// // State assertion grammar (executed by the runner, zero model judgment): // file exists {rel} // file absent {rel} // manifest exists {dotpath} {field} // manifest absent {dotpath} {field} // manifest equals {dotpath} {field} {expected printed value} +// json {rel} {dot.path} {expected value} const fs = require('fs'); const path = require('path'); @@ -68,6 +79,10 @@ function parseStateAssertion(text) { if (mEquals) { return { kind: 'manifest-equals', dotpath: mEquals[1], field: mEquals[2], value: mEquals[3] }; } + const json = text.match(/^json (\S+) (\S+) (.+)$/); + if (json) { + return { kind: 'json-equals', path: json[1], pointer: json[2], value: json[3] }; + } return { error: `unparseable state assertion: "${text}"` }; } @@ -81,6 +96,7 @@ function parseCaseFile(file, flow) { const push = () => { if (current) { current.walk = current.walk.join('\n').trim(); + current.stub = current.stub.join('\n').trim(); cases.push(current); } }; @@ -92,7 +108,7 @@ function parseCaseFile(file, flow) { current = { id: caseStart[1], flow, file: path.relative(ROOT, file), world: null, origin: null, files: [], - walk: [], user: [], expect: [], + walk: [], user: [], stub: [], expect: [], }; section = null; inFiles = false; @@ -100,7 +116,7 @@ function parseCaseFile(file, flow) { } if (!current) continue; - const sub = line.match(/^### (walk|user|expect)\s*$/); + const sub = line.match(/^### (walk|user|stub|expect)\s*$/); if (sub) { section = sub[1]; inFiles = false; @@ -135,6 +151,8 @@ function parseCaseFile(file, flow) { if (line.trim() !== '') inFiles = false; } else if (section === 'walk') { current.walk.push(line); + } else if (section === 'stub') { + current.stub.push(line); } else if (section === 'user') { const answer = line.match(/^\d+\.\s+(.*)$/); if (answer) current.user.push(answer[1].trim()); diff --git a/tests/prose/run.cjs b/tests/prose/run.cjs index b03753456..b3ab7e9fa 100644 --- a/tests/prose/run.cjs +++ b/tests/prose/run.cjs @@ -132,6 +132,11 @@ function cmdPrompt(argv) { `Repository root: ${ROOT}`, ]; + const stubs = c.stub + ? ['', 'STUBS — where the prose dispatches a background agent, do NOT dispatch one.', + 'Play the agent yourself exactly as described, then continue the prose:', '', c.stub] + : []; + const prompt = [ ...setting, '', @@ -140,6 +145,7 @@ function cmdPrompt(argv) { '', 'SCOPE — the prose under walk:', ...c.files.map((f) => ` - ${f.path}${f.anchor ? ` (start at the heading containing "${f.anchor}")` : ''}`), + ...stubs, '', 'RULES', '- Follow the prose literally, step by step, arm by arm. Where it names an', @@ -185,6 +191,21 @@ function runStateAssertion(worldDir, assertion) { const want = a.kind === 'file-exists'; return { assertion, pass: present === want, actual: present ? 'present' : 'absent' }; } + if (a.kind === 'json-equals') { + const full = path.join(worldDir, a.path); + if (!fs.existsSync(full)) return { assertion, pass: false, actual: 'file absent' }; + let doc; + try { + doc = JSON.parse(fs.readFileSync(full, 'utf8')); + } catch (e) { + return { assertion, pass: false, actual: `unparseable JSON: ${e.message}` }; + } + const found = a.pointer.split('.').reduce((node, key) => ( + node === undefined || node === null ? undefined : node[key] + ), doc); + const actual = found === undefined ? '' : String(found); + return { assertion, pass: actual === a.value, actual }; + } if (a.kind === 'manifest-exists' || a.kind === 'manifest-absent') { const res = engineInWorld(worldDir, ['manifest', 'exists', a.dotpath, a.field]); if (res.status !== 0) return { assertion, pass: false, actual: `engine refused: ${res.stderr.trim()}` }; diff --git a/tests/scripts/test-prose-cases.cjs b/tests/scripts/test-prose-cases.cjs index 1cb83129b..4d2656aac 100644 --- a/tests/scripts/test-prose-cases.cjs +++ b/tests/scripts/test-prose-cases.cjs @@ -24,7 +24,21 @@ describe('prose-test corpus', () => { it('state assertion grammar rejects garbage', () => { assert.ok(cases.parseStateAssertion('file exists .workflows/x').kind === 'file-exists'); assert.ok(cases.parseStateAssertion('manifest equals wu.planning.t review_cycle 1').kind === 'manifest-equals'); + assert.ok(cases.parseStateAssertion('json .workflows/.cache/a/b/c/state.json agents.x-001.status incorporated').kind === 'json-equals'); assert.ok(cases.parseStateAssertion('the manifest should look right').error); assert.ok(cases.parseStateAssertion('file exists').error); + assert.ok(cases.parseStateAssertion('json only.json pointer').error); + }); + + it('parses the optional stub section as free text, defaulting empty', () => { + const all = cases.loadAllCases(); + const stubbed = all.filter((c) => c.stub); + assert.ok(stubbed.length > 0, 'expected at least one stubbed case in the corpus'); + for (const c of stubbed) { + assert.ok(c.world !== null, `${c.id}: a stub without a world has nothing to stub`); + } + for (const c of all.filter((c) => !c.stub)) { + assert.strictEqual(c.stub, '', `${c.id}: unstubbed cases carry an empty stub`); + } }); }); From 3f328c3ee2f2c7ad893ebd2e9b63a3bdb2506abc Mon Sep 17 00:00:00 2001 From: Lee Overy Date: Sat, 25 Jul 2026 13:02:33 +0100 Subject: [PATCH 02/11] refactor(prose-tests): one case per file, filename is the id MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Lee's call, restored: cases were bundled several-per-file against what he'd asked for. Each case now lives at tests/prose/{flow}/{case-id}.md with the filename validated against the heading id, so the convention can't drift; a flow's intro prose moves to its README.md, which the loader ignores. 13 cases split out verbatim — no grammar, fixture, or claim changes. Co-Authored-By: Claude Opus 5 --- design/prose-tests.md | 7 +- tests/prose/README.md | 9 +- tests/prose/bugfix/README.md | 8 + .../bugfix-continue-offers-investigation.md | 25 +++ .../bugfix-investigation-entry-seeds.md | 22 +++ .../bugfix-root-cause-validation-lifecycle.md | 45 +++++ ...bugfix-spec-entry-sources-investigation.md | 22 +++ tests/prose/bugfix/cases.md | 126 ------------ tests/prose/feature/README.md | 7 + tests/prose/feature/cases.md | 180 ------------------ .../feature-continue-offers-discussion.md | 26 +++ .../feature/feature-discussion-entry-seeds.md | 21 ++ .../feature-implementation-first-task.md | 31 +++ .../feature-planning-entry-context-gate.md | 25 +++ .../feature/feature-review-entry-fresh.md | 21 ++ .../feature/feature-spec-entry-handoff.md | 22 +++ .../feature/feature-start-lists-fresh.md | 20 ++ tests/prose/lib/cases.cjs | 16 +- tests/prose/smoke/README.md | 6 + tests/prose/smoke/cases.md | 44 ----- .../prose/smoke/smoke-structure-start-boot.md | 17 ++ tests/prose/smoke/smoke-world-boot.md | 19 ++ tests/scripts/test-prose-cases.cjs | 12 ++ 23 files changed, 373 insertions(+), 358 deletions(-) create mode 100644 tests/prose/bugfix/README.md create mode 100644 tests/prose/bugfix/bugfix-continue-offers-investigation.md create mode 100644 tests/prose/bugfix/bugfix-investigation-entry-seeds.md create mode 100644 tests/prose/bugfix/bugfix-root-cause-validation-lifecycle.md create mode 100644 tests/prose/bugfix/bugfix-spec-entry-sources-investigation.md delete mode 100644 tests/prose/bugfix/cases.md create mode 100644 tests/prose/feature/README.md delete mode 100644 tests/prose/feature/cases.md create mode 100644 tests/prose/feature/feature-continue-offers-discussion.md create mode 100644 tests/prose/feature/feature-discussion-entry-seeds.md create mode 100644 tests/prose/feature/feature-implementation-first-task.md create mode 100644 tests/prose/feature/feature-planning-entry-context-gate.md create mode 100644 tests/prose/feature/feature-review-entry-fresh.md create mode 100644 tests/prose/feature/feature-spec-entry-handoff.md create mode 100644 tests/prose/feature/feature-start-lists-fresh.md create mode 100644 tests/prose/smoke/README.md delete mode 100644 tests/prose/smoke/cases.md create mode 100644 tests/prose/smoke/smoke-structure-start-boot.md create mode 100644 tests/prose/smoke/smoke-world-boot.md diff --git a/design/prose-tests.md b/design/prose-tests.md index fbb97a112..1c146b02d 100644 --- a/design/prose-tests.md +++ b/design/prose-tests.md @@ -37,9 +37,10 @@ testing. prose asked something the case didn't predict), `expect:` (concrete checkable claims: routing assertions graded by the agent with quoted line evidence, end-state assertions diffed by code), `origin:` (the - finding or flow it pins). Granularity is whatever the case needs — - compact routing cases live several-per-file, long end-to-end walks - get their own file, under `tests/prose/{flow}/`. + finding or flow it pins). **One case per file**: + `tests/prose/{flow}/{case-id}.md`, filename equal to the id and + validated as such. A case runs as long or short as it needs to — + length never justifies bundling two into one file. - **P2 — worlds are real, never imagined.** A fixture is authored by a **recipe** (a node script running real engine calls + perturbations against a scratch project) and committed as a **golden snapshot** diff --git a/tests/prose/README.md b/tests/prose/README.md index 240a049d6..8d32596dc 100644 --- a/tests/prose/README.md +++ b/tests/prose/README.md @@ -8,7 +8,8 @@ Run via the `/prose-test` skill; everything deterministic lives in ## Layout -- `{flow}/*.md` — case files (grammar: header comment in `lib/cases.cjs`) +- `{flow}/{case-id}.md` — **one case per file**, filename = case id + (validated; a flow's `README.md` holds its intro and is ignored) - `fixtures/{name}/recipe.cjs` — builds the world with real engine calls under a frozen clock (`lib/fake-clock.cjs`) - `fixtures/{name}/snapshot/` — the recipe's committed golden output; @@ -17,9 +18,9 @@ Run via the `/prose-test` skill; everything deterministic lives in ## Adding a case -Add a `## case:` block to the flow's file (or a new file for a long -walk). Scope `files:` tightly — diff-selection and the PR-end suggestion -run off it. `routing:` expects are agent-graded against the walk +Create `{flow}/{case-id}.md` holding a single `## case:` block — the +filename must equal the id. Scope `files:` tightly — diff-selection and +the PR-end suggestion run off it. `routing:` expects are agent-graded against the walk transcript; `state:` expects are asserted in code and require a `world:`. **Stub the agents, walk the lifecycle.** Prose that dispatches a diff --git a/tests/prose/bugfix/README.md b/tests/prose/bugfix/README.md new file mode 100644 index 000000000..dde030be9 --- /dev/null +++ b/tests/prose/bugfix/README.md @@ -0,0 +1,8 @@ +# Bugfix mainline — happy-path corpus + +The canonical bugfix `crash-fix` walked through what makes bugfix +distinct from the linear pipeline: investigation entry seeded from the +bug's carrier, the root-cause validation agent's full lifecycle, and a +specification sourced from an investigation rather than a discussion. +The delivery phases it shares with feature are covered by the feature +corpus and not repeated here. diff --git a/tests/prose/bugfix/bugfix-continue-offers-investigation.md b/tests/prose/bugfix/bugfix-continue-offers-investigation.md new file mode 100644 index 000000000..453ffce1e --- /dev/null +++ b/tests/prose/bugfix/bugfix-continue-offers-investigation.md @@ -0,0 +1,25 @@ +## case: bugfix-continue-offers-investigation +- world: bugfix-created +- origin: bugfix mainline — continue-bugfix routes a fresh bug to investigation +- files: + - skills/workflow-continue-bugfix/SKILL.md + - skills/workflow-continue-bugfix/references/select-bugfix.md + - skills/workflow-continue-bugfix/references/bugfix-display-and-menu.md + +### walk + +Execute skills/workflow-continue-bugfix/SKILL.md with no arguments (as +routed from workflow-start with no pre-selected work unit). Follow +selection, then the bugfix state display and its menu. Record the state +display and menu verbatim, then stop — do not route into any phase +entry skill. + +### user + +1. Select the bugfix `crash-fix` (its number in the selection menu) + +### expect + +- routing: the state display shows no phase started — the pipeline is fresh +- routing: the menu's continue action routes to investigation entry (workflow-investigation-entry with bugfix and crash-fix), not to discussion or specification +- state: manifest absent crash-fix.investigation.crash-fix status diff --git a/tests/prose/bugfix/bugfix-investigation-entry-seeds.md b/tests/prose/bugfix/bugfix-investigation-entry-seeds.md new file mode 100644 index 000000000..dacac27ba --- /dev/null +++ b/tests/prose/bugfix/bugfix-investigation-entry-seeds.md @@ -0,0 +1,22 @@ +## case: bugfix-investigation-entry-seeds +- world: bugfix-created +- origin: bugfix mainline — investigation entry seeds from the carrier, never re-asks +- files: + - skills/workflow-investigation-entry/SKILL.md + - skills/workflow-investigation-entry/references/invoke-skill.md + +### walk + +Execute skills/workflow-investigation-entry/SKILL.md with arguments +$0=bugfix, $1=crash-fix. Follow it up to the point where the handoff to +the processing skill is constructed — record the handoff arguments +block — and stop there. Do not execute the processing skill's +instructions. + +### expect + +- routing: with no investigation item in the manifest, source is new — the phase-validation path is not taken +- routing: because a discovery session log exists, the bug context is seeded from the manifest description and that log — the gather-context questioning path is not taken, and the walk asks the user nothing +- routing: a phase note is rendered by the engine for the investigation phase, with the verb Starting +- routing: the handoff invokes workflow-investigation-process for crash-fix +- state: manifest absent crash-fix.investigation.crash-fix status diff --git a/tests/prose/bugfix/bugfix-root-cause-validation-lifecycle.md b/tests/prose/bugfix/bugfix-root-cause-validation-lifecycle.md new file mode 100644 index 000000000..8fda1bec2 --- /dev/null +++ b/tests/prose/bugfix/bugfix-root-cause-validation-lifecycle.md @@ -0,0 +1,45 @@ +## case: bugfix-root-cause-validation-lifecycle +- world: bugfix-investigating +- origin: bugfix mainline — the validation agent's dispatch → scan → incorporate lifecycle +- files: + - skills/workflow-investigation-process/references/root-cause-validation.md + +### walk + +The investigation is open with its root cause documented, and the +process has reached its root-cause validation step. Execute +skills/workflow-investigation-process/references/root-cause-validation.md +from the top, as the processing skill would. Stop when the reference +returns to its caller. + +### user + +1. yes — run root cause validation + +### stub + +The prose directs a validation agent to be dispatched via the Task +tool. Do not dispatch one. Instead, after recording the dispatch, write +the report yourself to the exact `file` path the dispatch response +returned, containing: + + # Root Cause Validation + + The tax context is built before the payment intent and reads the + shipping address unconditionally. Traced fresh: an address-less + order fails at that read, matching the documented root cause. + + STATUS: validated + CONFIDENCE: high + GAPS_COUNT: 0 + SUMMARY: Root cause confirmed by an independent trace. + +Then continue the prose as if the agent had returned that result. + +### expect + +- routing: the dispatch is recorded through the engine, and the report path used is the one the dispatch response returned — the prose does not invent or pre-create the file +- routing: after the report lands, the row is promoted by a scan and then closed by an incorporate — the verdict is consumed inline, never surfaced finding by finding +- routing: on a validated verdict the reference reports validation with its confidence and returns to the caller — the gaps-handling menu is not reached +- state: file exists .workflows/.cache/crash-fix/investigation/crash-fix/root-cause-validation-001.md +- state: json .workflows/.cache/crash-fix/investigation/crash-fix/state.json agents.root-cause-validation-001.status incorporated diff --git a/tests/prose/bugfix/bugfix-spec-entry-sources-investigation.md b/tests/prose/bugfix/bugfix-spec-entry-sources-investigation.md new file mode 100644 index 000000000..1a7fd5007 --- /dev/null +++ b/tests/prose/bugfix/bugfix-spec-entry-sources-investigation.md @@ -0,0 +1,22 @@ +## case: bugfix-spec-entry-sources-investigation +- world: bugfix-investigated +- origin: bugfix mainline — the spec's source is the investigation, not a discussion +- files: + - skills/workflow-specification-entry/SKILL.md + - skills/workflow-specification-entry/references/validate-source.md + - skills/workflow-specification-entry/references/invoke-skill.md + +### walk + +Execute skills/workflow-specification-entry/SKILL.md with arguments +$0=bugfix, $1=crash-fix. Follow it up to the point where the handoff to +the processing skill is constructed — record the handoff arguments +block — and stop there. Do not execute the processing skill's +instructions. + +### expect + +- routing: the source-material entry gate passes (empty render) — the completed investigation satisfies it for a bugfix +- routing: with no specification item in the manifest, the verb is Creating +- routing: the handoff names the investigation file .workflows/crash-fix/investigation/crash-fix.md as source material — no discussion file is named +- state: manifest absent crash-fix.specification.crash-fix status diff --git a/tests/prose/bugfix/cases.md b/tests/prose/bugfix/cases.md deleted file mode 100644 index a8b4edf78..000000000 --- a/tests/prose/bugfix/cases.md +++ /dev/null @@ -1,126 +0,0 @@ -# Bugfix mainline — happy-path corpus - -The canonical bugfix `crash-fix` walked through what makes bugfix -distinct from the linear pipeline: investigation entry seeded from the -bug's carrier, the root-cause validation agent's full lifecycle, and a -specification sourced from an investigation rather than a discussion. -The delivery phases it shares with feature are covered by the feature -corpus and not repeated here. - -## case: bugfix-continue-offers-investigation -- world: bugfix-created -- origin: bugfix mainline — continue-bugfix routes a fresh bug to investigation -- files: - - skills/workflow-continue-bugfix/SKILL.md - - skills/workflow-continue-bugfix/references/select-bugfix.md - - skills/workflow-continue-bugfix/references/bugfix-display-and-menu.md - -### walk - -Execute skills/workflow-continue-bugfix/SKILL.md with no arguments (as -routed from workflow-start with no pre-selected work unit). Follow -selection, then the bugfix state display and its menu. Record the state -display and menu verbatim, then stop — do not route into any phase -entry skill. - -### user - -1. Select the bugfix `crash-fix` (its number in the selection menu) - -### expect - -- routing: the state display shows no phase started — the pipeline is fresh -- routing: the menu's continue action routes to investigation entry (workflow-investigation-entry with bugfix and crash-fix), not to discussion or specification -- state: manifest absent crash-fix.investigation.crash-fix status - -## case: bugfix-investigation-entry-seeds -- world: bugfix-created -- origin: bugfix mainline — investigation entry seeds from the carrier, never re-asks -- files: - - skills/workflow-investigation-entry/SKILL.md - - skills/workflow-investigation-entry/references/invoke-skill.md - -### walk - -Execute skills/workflow-investigation-entry/SKILL.md with arguments -$0=bugfix, $1=crash-fix. Follow it up to the point where the handoff to -the processing skill is constructed — record the handoff arguments -block — and stop there. Do not execute the processing skill's -instructions. - -### expect - -- routing: with no investigation item in the manifest, source is new — the phase-validation path is not taken -- routing: because a discovery session log exists, the bug context is seeded from the manifest description and that log — the gather-context questioning path is not taken, and the walk asks the user nothing -- routing: a phase note is rendered by the engine for the investigation phase, with the verb Starting -- routing: the handoff invokes workflow-investigation-process for crash-fix -- state: manifest absent crash-fix.investigation.crash-fix status - -## case: bugfix-root-cause-validation-lifecycle -- world: bugfix-investigating -- origin: bugfix mainline — the validation agent's dispatch → scan → incorporate lifecycle -- files: - - skills/workflow-investigation-process/references/root-cause-validation.md - -### walk - -The investigation is open with its root cause documented, and the -process has reached its root-cause validation step. Execute -skills/workflow-investigation-process/references/root-cause-validation.md -from the top, as the processing skill would. Stop when the reference -returns to its caller. - -### user - -1. yes — run root cause validation - -### stub - -The prose directs a validation agent to be dispatched via the Task -tool. Do not dispatch one. Instead, after recording the dispatch, write -the report yourself to the exact `file` path the dispatch response -returned, containing: - - # Root Cause Validation - - The tax context is built before the payment intent and reads the - shipping address unconditionally. Traced fresh: an address-less - order fails at that read, matching the documented root cause. - - STATUS: validated - CONFIDENCE: high - GAPS_COUNT: 0 - SUMMARY: Root cause confirmed by an independent trace. - -Then continue the prose as if the agent had returned that result. - -### expect - -- routing: the dispatch is recorded through the engine, and the report path used is the one the dispatch response returned — the prose does not invent or pre-create the file -- routing: after the report lands, the row is promoted by a scan and then closed by an incorporate — the verdict is consumed inline, never surfaced finding by finding -- routing: on a validated verdict the reference reports validation with its confidence and returns to the caller — the gaps-handling menu is not reached -- state: file exists .workflows/.cache/crash-fix/investigation/crash-fix/root-cause-validation-001.md -- state: json .workflows/.cache/crash-fix/investigation/crash-fix/state.json agents.root-cause-validation-001.status incorporated - -## case: bugfix-spec-entry-sources-investigation -- world: bugfix-investigated -- origin: bugfix mainline — the spec's source is the investigation, not a discussion -- files: - - skills/workflow-specification-entry/SKILL.md - - skills/workflow-specification-entry/references/validate-source.md - - skills/workflow-specification-entry/references/invoke-skill.md - -### walk - -Execute skills/workflow-specification-entry/SKILL.md with arguments -$0=bugfix, $1=crash-fix. Follow it up to the point where the handoff to -the processing skill is constructed — record the handoff arguments -block — and stop there. Do not execute the processing skill's -instructions. - -### expect - -- routing: the source-material entry gate passes (empty render) — the completed investigation satisfies it for a bugfix -- routing: with no specification item in the manifest, the verb is Creating -- routing: the handoff names the investigation file .workflows/crash-fix/investigation/crash-fix.md as source material — no discussion file is named -- state: manifest absent crash-fix.specification.crash-fix status diff --git a/tests/prose/feature/README.md b/tests/prose/feature/README.md new file mode 100644 index 000000000..86f290628 --- /dev/null +++ b/tests/prose/feature/README.md @@ -0,0 +1,7 @@ +# Feature mainline — happy-path corpus + +The canonical feature `pay` walked through its pipeline: navigation, +each phase entry, and the implementation pickup. Worlds are the +`feature-*` fixtures (stop points along `_shared/feature-mainline.cjs`). +Every claim here was verified against the prose at authoring time — +a failure means the behaviour moved, not the numbering. diff --git a/tests/prose/feature/cases.md b/tests/prose/feature/cases.md deleted file mode 100644 index 0480096d7..000000000 --- a/tests/prose/feature/cases.md +++ /dev/null @@ -1,180 +0,0 @@ -# Feature mainline — happy-path corpus - -The canonical feature `pay` walked through its pipeline: navigation, -each phase entry, and the implementation pickup. Worlds are the -`feature-*` fixtures (stop points along `_shared/feature-mainline.cjs`). -Every claim here was verified against the prose at authoring time — -a failure means the behaviour moved, not the numbering. - -## case: feature-start-lists-fresh -- world: feature-created -- origin: feature mainline — workflow-start surfaces active work -- files: - - skills/workflow-start/SKILL.md - -### walk - -Execute skills/workflow-start/SKILL.md from the top. Initialisation -will find the system already migrated and the knowledge base ready, so -no migration summary or setup conversation applies. Continue until the -work dashboard and its menu are shown. Record both verbatim and stop — -select nothing. - -### expect - -- routing: initialisation runs the boot pipeline before any work is shown, and with `migrations.changed` false no confirmation gate is raised -- routing: the dashboard lists `pay` as active work -- routing: the menu offers continuing existing work (the path that routes feature work to workflow-continue-feature) -- state: manifest absent pay.discussion.pay status - -## case: feature-continue-offers-discussion -- world: feature-created -- origin: feature mainline — continue-feature shows state and routes forward -- files: - - skills/workflow-continue-feature/SKILL.md - - skills/workflow-continue-feature/references/select-feature.md - - skills/workflow-continue-feature/references/feature-display-and-menu.md - -### walk - -Execute skills/workflow-continue-feature/SKILL.md with no arguments -(as routed from workflow-start with no pre-selected work unit). Follow -selection, then the feature state display and its menu. Record the -state display and menu verbatim, then stop — do not route into any -phase entry skill. - -### user - -1. Select the feature `pay` (its number in the selection menu) - -### expect - -- routing: the selection menu is shown even though only one feature exists — no auto-select -- routing: the feature state shows no phase started — the pipeline is fresh -- routing: the menu's forward action for the current state routes to discussion entry (workflow-discussion-entry with feature and pay) -- state: manifest absent pay.discussion.pay status - -## case: feature-discussion-entry-seeds -- world: feature-created -- origin: feature mainline — discussion entry seeds from the durable carrier -- files: - - skills/workflow-discussion-entry/SKILL.md - - skills/workflow-shared/references/ensure-discovery-item.md - -### walk - -Execute skills/workflow-discussion-entry/SKILL.md with arguments -$0=feature, $1=pay. Follow it up to the point where the handoff to the -processing skill is constructed — record the handoff arguments block — -and stop there. Do not execute the processing skill's instructions. - -### expect - -- routing: with no discussion item in the manifest, the new-entry arm is taken — no phase-status validation menu appears -- routing: ensure-discovery-item returns without creating anything — the discovery map is epic-only, and this is a feature -- routing: context is seeded from the manifest description and the session log's Exploration section — the gather-context questioning path is not taken -- routing: the handoff invokes workflow-discussion-process for pay -- state: manifest absent pay.discussion.pay status - -## case: feature-spec-entry-handoff -- world: feature-discussed -- origin: feature mainline — specification entry validates and hands off the discussion -- files: - - skills/workflow-specification-entry/SKILL.md - - skills/workflow-specification-entry/references/validate-source.md - - skills/workflow-specification-entry/references/validate-phase.md - - skills/workflow-specification-entry/references/invoke-skill.md - -### walk - -Execute skills/workflow-specification-entry/SKILL.md with arguments -$0=feature, $1=pay. Follow it up to the point where the handoff to the -processing skill is constructed — record the handoff arguments block — -and stop there. Do not execute the processing skill's instructions. - -### expect - -- routing: the source-material entry gate passes (empty render) — the discussion is complete -- routing: with no specification item in the manifest, the verb is Creating — no resume note, no reopen -- routing: the handoff names the discussion file .workflows/pay/discussion/pay.md as source material -- state: manifest absent pay.specification.pay status - -## case: feature-planning-entry-context-gate -- world: feature-specified -- origin: feature mainline — planning entry validates the spec and asks for late context -- files: - - skills/workflow-planning-entry/SKILL.md - - skills/workflow-planning-entry/references/validate-spec.md - - skills/workflow-planning-entry/references/validate-phase.md - -### walk - -Execute skills/workflow-planning-entry/SKILL.md with arguments -$0=feature, $1=pay. Follow it up to the point where the handoff to the -processing skill is constructed — record the handoff arguments block — -and stop there. Do not execute the processing skill's instructions. - -### user - -1. continue — no additional context since the specification - -### expect - -- routing: the specification entry gate passes (empty render) — clear to plan -- routing: with no plan in the manifest, the fresh-start arm asks whether any context has changed since the specification (answered continue) -- routing: the handoff invokes workflow-planning-process for pay -- state: manifest absent pay.planning.pay status - -## case: feature-implementation-first-task -- world: feature-planned -- origin: feature mainline — implementation entry through to the first task pickup -- files: - - skills/workflow-implementation-entry/SKILL.md - - skills/workflow-implementation-entry/references/validate-phase.md - - skills/workflow-implementation-entry/references/environment-check.md - - skills/workflow-implementation-process/SKILL.md - - skills/workflow-implementation-process/references/task-loop.md - - skills/workflow-planning-process/references/output-formats/local-markdown/reading.md - -### walk - -Execute skills/workflow-implementation-entry/SKILL.md with arguments -$0=feature, $1=pay, then continue into the processing skill it invokes. -Follow the task loop far enough to determine and start the next -available task. Stop the moment the prose directs you to begin actually -implementing that task (writing code or tests) — implement nothing. - -### user - -1. none — no special environment setup is needed - -### expect - -- routing: the plan entry gate passes (empty render) and the implementation is a new entry -- routing: with no environment-setup file recorded, the entry asks the environment question (answered none) -- routing: the processing skill's resume detection reports mode created — the fresh path, with its start-implementation commit — never the resuming-from-previous-session note -- routing: the next available task per the plan format's reading procedure is pay-1-1 — phase 1, first task, nothing completed yet -- state: manifest equals pay.implementation.pay status in-progress -- state: manifest equals pay.implementation.pay current_task pay-1-1 - -## case: feature-review-entry-fresh -- world: feature-implemented -- origin: feature mainline — review entry passes both prerequisites, fresh review -- files: - - skills/workflow-review-entry/SKILL.md - - skills/workflow-review-entry/references/validate-phase.md - -### walk - -Execute skills/workflow-review-entry/SKILL.md with arguments -$0=feature, $1=pay. Follow it up to the point where the handoff to the -processing skill is constructed — record the handoff arguments block — -and stop there. Do not execute the processing skill's instructions, and -never dispatch any agent. - -### expect - -- routing: the prerequisite entry gate passes (empty render) — plan and implementation are both complete -- routing: with no review item in the manifest, the fresh path is taken — no reopen, no resume -- routing: the handoff invokes workflow-review-process for pay -- state: manifest absent pay.review.pay status diff --git a/tests/prose/feature/feature-continue-offers-discussion.md b/tests/prose/feature/feature-continue-offers-discussion.md new file mode 100644 index 000000000..c7e2127cf --- /dev/null +++ b/tests/prose/feature/feature-continue-offers-discussion.md @@ -0,0 +1,26 @@ +## case: feature-continue-offers-discussion +- world: feature-created +- origin: feature mainline — continue-feature shows state and routes forward +- files: + - skills/workflow-continue-feature/SKILL.md + - skills/workflow-continue-feature/references/select-feature.md + - skills/workflow-continue-feature/references/feature-display-and-menu.md + +### walk + +Execute skills/workflow-continue-feature/SKILL.md with no arguments +(as routed from workflow-start with no pre-selected work unit). Follow +selection, then the feature state display and its menu. Record the +state display and menu verbatim, then stop — do not route into any +phase entry skill. + +### user + +1. Select the feature `pay` (its number in the selection menu) + +### expect + +- routing: the selection menu is shown even though only one feature exists — no auto-select +- routing: the feature state shows no phase started — the pipeline is fresh +- routing: the menu's forward action for the current state routes to discussion entry (workflow-discussion-entry with feature and pay) +- state: manifest absent pay.discussion.pay status diff --git a/tests/prose/feature/feature-discussion-entry-seeds.md b/tests/prose/feature/feature-discussion-entry-seeds.md new file mode 100644 index 000000000..57408c959 --- /dev/null +++ b/tests/prose/feature/feature-discussion-entry-seeds.md @@ -0,0 +1,21 @@ +## case: feature-discussion-entry-seeds +- world: feature-created +- origin: feature mainline — discussion entry seeds from the durable carrier +- files: + - skills/workflow-discussion-entry/SKILL.md + - skills/workflow-shared/references/ensure-discovery-item.md + +### walk + +Execute skills/workflow-discussion-entry/SKILL.md with arguments +$0=feature, $1=pay. Follow it up to the point where the handoff to the +processing skill is constructed — record the handoff arguments block — +and stop there. Do not execute the processing skill's instructions. + +### expect + +- routing: with no discussion item in the manifest, the new-entry arm is taken — no phase-status validation menu appears +- routing: ensure-discovery-item returns without creating anything — the discovery map is epic-only, and this is a feature +- routing: context is seeded from the manifest description and the session log's Exploration section — the gather-context questioning path is not taken +- routing: the handoff invokes workflow-discussion-process for pay +- state: manifest absent pay.discussion.pay status diff --git a/tests/prose/feature/feature-implementation-first-task.md b/tests/prose/feature/feature-implementation-first-task.md new file mode 100644 index 000000000..5bad38f40 --- /dev/null +++ b/tests/prose/feature/feature-implementation-first-task.md @@ -0,0 +1,31 @@ +## case: feature-implementation-first-task +- world: feature-planned +- origin: feature mainline — implementation entry through to the first task pickup +- files: + - skills/workflow-implementation-entry/SKILL.md + - skills/workflow-implementation-entry/references/validate-phase.md + - skills/workflow-implementation-entry/references/environment-check.md + - skills/workflow-implementation-process/SKILL.md + - skills/workflow-implementation-process/references/task-loop.md + - skills/workflow-planning-process/references/output-formats/local-markdown/reading.md + +### walk + +Execute skills/workflow-implementation-entry/SKILL.md with arguments +$0=feature, $1=pay, then continue into the processing skill it invokes. +Follow the task loop far enough to determine and start the next +available task. Stop the moment the prose directs you to begin actually +implementing that task (writing code or tests) — implement nothing. + +### user + +1. none — no special environment setup is needed + +### expect + +- routing: the plan entry gate passes (empty render) and the implementation is a new entry +- routing: with no environment-setup file recorded, the entry asks the environment question (answered none) +- routing: the processing skill's resume detection reports mode created — the fresh path, with its start-implementation commit — never the resuming-from-previous-session note +- routing: the next available task per the plan format's reading procedure is pay-1-1 — phase 1, first task, nothing completed yet +- state: manifest equals pay.implementation.pay status in-progress +- state: manifest equals pay.implementation.pay current_task pay-1-1 diff --git a/tests/prose/feature/feature-planning-entry-context-gate.md b/tests/prose/feature/feature-planning-entry-context-gate.md new file mode 100644 index 000000000..16976dc63 --- /dev/null +++ b/tests/prose/feature/feature-planning-entry-context-gate.md @@ -0,0 +1,25 @@ +## case: feature-planning-entry-context-gate +- world: feature-specified +- origin: feature mainline — planning entry validates the spec and asks for late context +- files: + - skills/workflow-planning-entry/SKILL.md + - skills/workflow-planning-entry/references/validate-spec.md + - skills/workflow-planning-entry/references/validate-phase.md + +### walk + +Execute skills/workflow-planning-entry/SKILL.md with arguments +$0=feature, $1=pay. Follow it up to the point where the handoff to the +processing skill is constructed — record the handoff arguments block — +and stop there. Do not execute the processing skill's instructions. + +### user + +1. continue — no additional context since the specification + +### expect + +- routing: the specification entry gate passes (empty render) — clear to plan +- routing: with no plan in the manifest, the fresh-start arm asks whether any context has changed since the specification (answered continue) +- routing: the handoff invokes workflow-planning-process for pay +- state: manifest absent pay.planning.pay status diff --git a/tests/prose/feature/feature-review-entry-fresh.md b/tests/prose/feature/feature-review-entry-fresh.md new file mode 100644 index 000000000..8891ef2e8 --- /dev/null +++ b/tests/prose/feature/feature-review-entry-fresh.md @@ -0,0 +1,21 @@ +## case: feature-review-entry-fresh +- world: feature-implemented +- origin: feature mainline — review entry passes both prerequisites, fresh review +- files: + - skills/workflow-review-entry/SKILL.md + - skills/workflow-review-entry/references/validate-phase.md + +### walk + +Execute skills/workflow-review-entry/SKILL.md with arguments +$0=feature, $1=pay. Follow it up to the point where the handoff to the +processing skill is constructed — record the handoff arguments block — +and stop there. Do not execute the processing skill's instructions, and +never dispatch any agent. + +### expect + +- routing: the prerequisite entry gate passes (empty render) — plan and implementation are both complete +- routing: with no review item in the manifest, the fresh path is taken — no reopen, no resume +- routing: the handoff invokes workflow-review-process for pay +- state: manifest absent pay.review.pay status diff --git a/tests/prose/feature/feature-spec-entry-handoff.md b/tests/prose/feature/feature-spec-entry-handoff.md new file mode 100644 index 000000000..d6ded024b --- /dev/null +++ b/tests/prose/feature/feature-spec-entry-handoff.md @@ -0,0 +1,22 @@ +## case: feature-spec-entry-handoff +- world: feature-discussed +- origin: feature mainline — specification entry validates and hands off the discussion +- files: + - skills/workflow-specification-entry/SKILL.md + - skills/workflow-specification-entry/references/validate-source.md + - skills/workflow-specification-entry/references/validate-phase.md + - skills/workflow-specification-entry/references/invoke-skill.md + +### walk + +Execute skills/workflow-specification-entry/SKILL.md with arguments +$0=feature, $1=pay. Follow it up to the point where the handoff to the +processing skill is constructed — record the handoff arguments block — +and stop there. Do not execute the processing skill's instructions. + +### expect + +- routing: the source-material entry gate passes (empty render) — the discussion is complete +- routing: with no specification item in the manifest, the verb is Creating — no resume note, no reopen +- routing: the handoff names the discussion file .workflows/pay/discussion/pay.md as source material +- state: manifest absent pay.specification.pay status diff --git a/tests/prose/feature/feature-start-lists-fresh.md b/tests/prose/feature/feature-start-lists-fresh.md new file mode 100644 index 000000000..6add0ffdf --- /dev/null +++ b/tests/prose/feature/feature-start-lists-fresh.md @@ -0,0 +1,20 @@ +## case: feature-start-lists-fresh +- world: feature-created +- origin: feature mainline — workflow-start surfaces active work +- files: + - skills/workflow-start/SKILL.md + +### walk + +Execute skills/workflow-start/SKILL.md from the top. Initialisation +will find the system already migrated and the knowledge base ready, so +no migration summary or setup conversation applies. Continue until the +work dashboard and its menu are shown. Record both verbatim and stop — +select nothing. + +### expect + +- routing: initialisation runs the boot pipeline before any work is shown, and with `migrations.changed` false no confirmation gate is raised +- routing: the dashboard lists `pay` as active work +- routing: the menu offers continuing existing work (the path that routes feature work to workflow-continue-feature) +- state: manifest absent pay.discussion.pay status diff --git a/tests/prose/lib/cases.cjs b/tests/prose/lib/cases.cjs index e8458820f..8bd390985 100644 --- a/tests/prose/lib/cases.cjs +++ b/tests/prose/lib/cases.cjs @@ -2,7 +2,9 @@ // Case corpus: parsing and validation. // -// A case file is markdown under tests/prose/{flow}/ (any name, .md). +// One case per file: tests/prose/{flow}/{case-id}.md, where the +// filename IS the id (validated, so a rename can never drift from the +// heading). A flow's README.md carries its prose intro and is ignored. // Grammar, deterministic to parse: // // ## case: {kebab-id} @@ -180,12 +182,24 @@ function headingExists(absPath, anchor) { function validateCorpus(cases) { const errors = []; const seen = new Set(); + const perFile = new Map(); + for (const c of cases) perFile.set(c.file, (perFile.get(c.file) || 0) + 1); + + for (const [file, count] of perFile) { + if (count > 1) { + errors.push(`${file}: holds ${count} cases — one case per file, named {case-id}.md`); + } + } + for (const c of cases) { const at = `${c.file} :: ${c.id}`; if (!/^[a-z0-9][a-z0-9-]*$/.test(c.id)) errors.push(`${at}: id is not kebab-case`); if (seen.has(c.id)) errors.push(`${at}: duplicate case id`); seen.add(c.id); + const stem = path.basename(c.file, '.md'); + if (stem !== c.id) errors.push(`${at}: filename must equal the case id (expected ${c.id}.md)`); + if (c.files.length === 0) errors.push(`${at}: no files: scope`); for (const f of c.files) { const abs = path.join(ROOT, f.path); diff --git a/tests/prose/smoke/README.md b/tests/prose/smoke/README.md new file mode 100644 index 000000000..2296c99a9 --- /dev/null +++ b/tests/prose/smoke/README.md @@ -0,0 +1,6 @@ +# Smoke cases — the framework proving itself + +Not corpus: these two cases exercise every moving part of the machinery +(parser, anchor checks, world builder, engine-in-world, state assertions, +walker protocol) against prose stable enough to survive. If a smoke case +fails, suspect the framework before the prose. diff --git a/tests/prose/smoke/cases.md b/tests/prose/smoke/cases.md deleted file mode 100644 index 34018bfda..000000000 --- a/tests/prose/smoke/cases.md +++ /dev/null @@ -1,44 +0,0 @@ -# Smoke cases — the framework proving itself - -Not corpus: these two cases exercise every moving part of the machinery -(parser, anchor checks, world builder, engine-in-world, state assertions, -walker protocol) against prose stable enough to survive. If a smoke case -fails, suspect the framework before the prose. - -## case: smoke-structure-start-boot -- origin: framework smoke — structure-only walk, no world -- files: - - skills/workflow-start/SKILL.md#Boot - -### walk - -Structure-only: read the initialisation portion of -skills/workflow-start/SKILL.md — everything the skill does before any -work is shown or routed. Establish what the initialisation consists of -and in what order its parts run. Stop there — do not trace the rest of -the skill. - -### expect - -- routing: casing conventions are loaded before the boot pipeline runs -- routing: the boot pipeline is declared mandatory — the skill must complete it before proceeding - -## case: smoke-world-boot -- world: base -- origin: framework smoke — world build, engine-in-world, state assertions -- files: - - skills/workflow-start/SKILL.md#Boot - -### walk - -In the project, run the boot command exactly as the initialisation prose -of skills/workflow-start/SKILL.md prescribes (sandbox concerns do not -apply in this world — run it directly). Capture the JSON response. Stop -immediately after recording it — do not continue into any confirmation -prompt or later step. - -### expect - -- routing: the boot response reports the knowledge base ready (keyword-only mode) -- routing: the boot response reports no migrations applied on this run (the world is already migrated) -- state: file exists .workflows/.state/migrations diff --git a/tests/prose/smoke/smoke-structure-start-boot.md b/tests/prose/smoke/smoke-structure-start-boot.md new file mode 100644 index 000000000..541813e81 --- /dev/null +++ b/tests/prose/smoke/smoke-structure-start-boot.md @@ -0,0 +1,17 @@ +## case: smoke-structure-start-boot +- origin: framework smoke — structure-only walk, no world +- files: + - skills/workflow-start/SKILL.md#Boot + +### walk + +Structure-only: read the initialisation portion of +skills/workflow-start/SKILL.md — everything the skill does before any +work is shown or routed. Establish what the initialisation consists of +and in what order its parts run. Stop there — do not trace the rest of +the skill. + +### expect + +- routing: casing conventions are loaded before the boot pipeline runs +- routing: the boot pipeline is declared mandatory — the skill must complete it before proceeding diff --git a/tests/prose/smoke/smoke-world-boot.md b/tests/prose/smoke/smoke-world-boot.md new file mode 100644 index 000000000..fb7be3b9d --- /dev/null +++ b/tests/prose/smoke/smoke-world-boot.md @@ -0,0 +1,19 @@ +## case: smoke-world-boot +- world: base +- origin: framework smoke — world build, engine-in-world, state assertions +- files: + - skills/workflow-start/SKILL.md#Boot + +### walk + +In the project, run the boot command exactly as the initialisation prose +of skills/workflow-start/SKILL.md prescribes (sandbox concerns do not +apply in this world — run it directly). Capture the JSON response. Stop +immediately after recording it — do not continue into any confirmation +prompt or later step. + +### expect + +- routing: the boot response reports the knowledge base ready (keyword-only mode) +- routing: the boot response reports no migrations applied on this run (the world is already migrated) +- state: file exists .workflows/.state/migrations diff --git a/tests/scripts/test-prose-cases.cjs b/tests/scripts/test-prose-cases.cjs index 4d2656aac..5a0e24d10 100644 --- a/tests/scripts/test-prose-cases.cjs +++ b/tests/scripts/test-prose-cases.cjs @@ -7,6 +7,7 @@ const { describe, it } = require('node:test'); const assert = require('node:assert'); +const path = require('path'); const cases = require('../prose/lib/cases.cjs'); @@ -30,6 +31,17 @@ describe('prose-test corpus', () => { assert.ok(cases.parseStateAssertion('json only.json pointer').error); }); + it('holds exactly one case per file, filename equal to the id', () => { + const all = cases.loadAllCases(); + const seen = new Map(); + for (const c of all) { + assert.ok(!seen.has(c.file), `${c.file}: more than one case in a file`); + seen.set(c.file, c.id); + assert.strictEqual(path.basename(c.file, '.md'), c.id, + `${c.file}: filename must equal the case id`); + } + }); + it('parses the optional stub section as free text, defaulting empty', () => { const all = cases.loadAllCases(); const stubbed = all.filter((c) => c.stub); From b91d87f1d09a2417aae081922353decc9b063693 Mon Sep 17 00:00:00 2001 From: Lee Overy Date: Sat, 25 Jul 2026 16:53:08 +0100 Subject: [PATCH 03/11] refactor(prose-tests): Given-When-Then, whole-world assertion, case-owned stub triggers MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Lee's review of the corpus, four corrections: - Cases go flat — tests/prose/{case-id}.md, filename = id. Grouping by work type organised the corpus around the build sequence, not the prose under test, and no single-parent taxonomy fits cases spanning skills. Selection always ran off `files:`, never the directory. - The assertion DSL is deleted. The expectation is a whole committed world; code diffs acted-vs-expected and the asserting agent classifies each difference volatile or material. Nothing goes unchecked because nobody thought to assert it, and there is no normalisation table or engine-vs-model file split to maintain. - Stub triggers move to the case. Binding a substitution to a moment is what prevents reuse — the same content is wanted at different moments. A stub is named content; the case says when. Trigger-less stubs fail validation as arrange in disguise. - The trace is granular. Coarse assertions let an agent silently course-correct around broken prose; a step-by-step expected trace turns that into a mismatch. The When stays coarse so the walker must still derive the path from the prose. Adds a DEVIATION: marker. Proven end to end: walked the validation lifecycle by hand in a live world, diffed against the expected fixture — the sole difference was the volatile agent-row timestamp. Co-Authored-By: Claude Opus 5 --- .claude/skills/prose-test/SKILL.md | 52 +-- design/prose-tests.md | 114 +++++-- tests/prose/README.md | 105 ++++-- tests/prose/bugfix/README.md | 8 - .../bugfix-continue-offers-investigation.md | 25 -- .../bugfix-investigation-entry-seeds.md | 22 -- .../bugfix-root-cause-validation-lifecycle.md | 45 --- ...bugfix-spec-entry-sources-investigation.md | 22 -- ...continue-bugfix-routes-to-investigation.md | 34 ++ .../continue-feature-routes-to-discussion.md | 36 ++ .../discussion-entry-seeds-from-carrier.md | 35 ++ tests/prose/feature/README.md | 7 - .../feature-continue-offers-discussion.md | 26 -- .../feature/feature-discussion-entry-seeds.md | 21 -- .../feature-implementation-first-task.md | 31 -- .../feature-planning-entry-context-gate.md | 25 -- .../feature/feature-review-entry-fresh.md | 21 -- .../feature/feature-spec-entry-handoff.md | 22 -- .../feature/feature-start-lists-fresh.md | 20 -- .../bugfix-validation-incorporated/recipe.cjs | 27 ++ .../snapshot/.claude/settings.json | 9 + .../crash-fix/root-cause-validation-001.md | 10 + .../investigation/crash-fix/state.json | 16 + .../snapshot/.workflows/.state/migrations | 52 +++ .../snapshot/.workflows/_gitignore.fixture | 2 + .../discovery/sessions/session-001.md | 40 +++ .../crash-fix/investigation/crash-fix.md | 88 +++++ .../.workflows/crash-fix/manifest.json | 16 + .../snapshot/.workflows/manifest.json | 7 + .../feature-implementation-started/recipe.cjs | 27 ++ .../snapshot/.claude/settings.json | 9 + .../.workflows/.state/environment-setup.md | 1 + .../snapshot/.workflows/.state/migrations | 52 +++ .../snapshot/.workflows/_gitignore.fixture | 2 + .../snapshot/.workflows/manifest.json | 7 + .../pay/discovery/sessions/session-001.md | 39 +++ .../snapshot/.workflows/pay/discussion/pay.md | 15 + .../snapshot/.workflows/pay/manifest.json | 70 ++++ .../.workflows/pay/planning/pay/planning.md | 8 + .../pay/planning/pay/tasks/pay-1-1.md | 10 + .../pay/planning/pay/tasks/pay-1-2.md | 10 + .../pay/specification/pay/specification.md | 11 + .../prose/implementation-picks-first-task.md | 48 +++ .../investigation-entry-seeds-from-carrier.md | 34 ++ tests/prose/lib/cases.cjs | 311 +++++++++--------- tests/prose/lib/fixtures.cjs | 47 ++- .../planning-entry-asks-for-late-context.md | 39 +++ tests/prose/review-entry-fresh.md | 31 ++ .../root-cause-validation-clean-verdict.md | 42 +++ tests/prose/run.cjs | 266 ++++++++------- tests/prose/smoke-boot-in-world.md | 28 ++ tests/prose/smoke-start-boot-structure.md | 27 ++ tests/prose/smoke/README.md | 6 - .../prose/smoke/smoke-structure-start-boot.md | 17 - tests/prose/smoke/smoke-world-boot.md | 19 -- tests/prose/spec-entry-from-discussion.md | 34 ++ tests/prose/spec-entry-from-investigation.md | 33 ++ tests/prose/start-lists-active-work.md | 31 ++ tests/prose/stubs/root-cause-validated.md | 19 ++ tests/scripts/test-prose-cases.cjs | 52 +-- 60 files changed, 1550 insertions(+), 733 deletions(-) delete mode 100644 tests/prose/bugfix/README.md delete mode 100644 tests/prose/bugfix/bugfix-continue-offers-investigation.md delete mode 100644 tests/prose/bugfix/bugfix-investigation-entry-seeds.md delete mode 100644 tests/prose/bugfix/bugfix-root-cause-validation-lifecycle.md delete mode 100644 tests/prose/bugfix/bugfix-spec-entry-sources-investigation.md create mode 100644 tests/prose/continue-bugfix-routes-to-investigation.md create mode 100644 tests/prose/continue-feature-routes-to-discussion.md create mode 100644 tests/prose/discussion-entry-seeds-from-carrier.md delete mode 100644 tests/prose/feature/README.md delete mode 100644 tests/prose/feature/feature-continue-offers-discussion.md delete mode 100644 tests/prose/feature/feature-discussion-entry-seeds.md delete mode 100644 tests/prose/feature/feature-implementation-first-task.md delete mode 100644 tests/prose/feature/feature-planning-entry-context-gate.md delete mode 100644 tests/prose/feature/feature-review-entry-fresh.md delete mode 100644 tests/prose/feature/feature-spec-entry-handoff.md delete mode 100644 tests/prose/feature/feature-start-lists-fresh.md create mode 100644 tests/prose/fixtures/bugfix-validation-incorporated/recipe.cjs create mode 100644 tests/prose/fixtures/bugfix-validation-incorporated/snapshot/.claude/settings.json create mode 100644 tests/prose/fixtures/bugfix-validation-incorporated/snapshot/.workflows/.cache/crash-fix/investigation/crash-fix/root-cause-validation-001.md create mode 100644 tests/prose/fixtures/bugfix-validation-incorporated/snapshot/.workflows/.cache/crash-fix/investigation/crash-fix/state.json create mode 100644 tests/prose/fixtures/bugfix-validation-incorporated/snapshot/.workflows/.state/migrations create mode 100644 tests/prose/fixtures/bugfix-validation-incorporated/snapshot/.workflows/_gitignore.fixture create mode 100644 tests/prose/fixtures/bugfix-validation-incorporated/snapshot/.workflows/crash-fix/discovery/sessions/session-001.md create mode 100644 tests/prose/fixtures/bugfix-validation-incorporated/snapshot/.workflows/crash-fix/investigation/crash-fix.md create mode 100644 tests/prose/fixtures/bugfix-validation-incorporated/snapshot/.workflows/crash-fix/manifest.json create mode 100644 tests/prose/fixtures/bugfix-validation-incorporated/snapshot/.workflows/manifest.json create mode 100644 tests/prose/fixtures/feature-implementation-started/recipe.cjs create mode 100644 tests/prose/fixtures/feature-implementation-started/snapshot/.claude/settings.json create mode 100644 tests/prose/fixtures/feature-implementation-started/snapshot/.workflows/.state/environment-setup.md create mode 100644 tests/prose/fixtures/feature-implementation-started/snapshot/.workflows/.state/migrations create mode 100644 tests/prose/fixtures/feature-implementation-started/snapshot/.workflows/_gitignore.fixture create mode 100644 tests/prose/fixtures/feature-implementation-started/snapshot/.workflows/manifest.json create mode 100644 tests/prose/fixtures/feature-implementation-started/snapshot/.workflows/pay/discovery/sessions/session-001.md create mode 100644 tests/prose/fixtures/feature-implementation-started/snapshot/.workflows/pay/discussion/pay.md create mode 100644 tests/prose/fixtures/feature-implementation-started/snapshot/.workflows/pay/manifest.json create mode 100644 tests/prose/fixtures/feature-implementation-started/snapshot/.workflows/pay/planning/pay/planning.md create mode 100644 tests/prose/fixtures/feature-implementation-started/snapshot/.workflows/pay/planning/pay/tasks/pay-1-1.md create mode 100644 tests/prose/fixtures/feature-implementation-started/snapshot/.workflows/pay/planning/pay/tasks/pay-1-2.md create mode 100644 tests/prose/fixtures/feature-implementation-started/snapshot/.workflows/pay/specification/pay/specification.md create mode 100644 tests/prose/implementation-picks-first-task.md create mode 100644 tests/prose/investigation-entry-seeds-from-carrier.md create mode 100644 tests/prose/planning-entry-asks-for-late-context.md create mode 100644 tests/prose/review-entry-fresh.md create mode 100644 tests/prose/root-cause-validation-clean-verdict.md create mode 100644 tests/prose/smoke-boot-in-world.md create mode 100644 tests/prose/smoke-start-boot-structure.md delete mode 100644 tests/prose/smoke/README.md delete mode 100644 tests/prose/smoke/smoke-structure-start-boot.md delete mode 100644 tests/prose/smoke/smoke-world-boot.md create mode 100644 tests/prose/spec-entry-from-discussion.md create mode 100644 tests/prose/spec-entry-from-investigation.md create mode 100644 tests/prose/start-lists-active-work.md create mode 100644 tests/prose/stubs/root-cause-validated.md diff --git a/.claude/skills/prose-test/SKILL.md b/.claude/skills/prose-test/SKILL.md index ac705b45b..2a9416657 100644 --- a/.claude/skills/prose-test/SKILL.md +++ b/.claude/skills/prose-test/SKILL.md @@ -1,13 +1,15 @@ --- name: prose-test -description: Run prose-logic test cases — walker agents execute workflow prose against materialised fixture worlds, graders check the transcripts, deterministic state assertions run in code. Scope by diff (default), case ids, or --all. +description: Run prose-logic test cases — a walker agent executes workflow prose against a materialised fixture world, an asserting agent judges the trace and the world delta. Scope by diff (default), case ids, or --all. --- # Prose Test Run prose-test cases (design: `design/prose-tests.md`; runner: -`tests/prose/run.cjs`). Walks cost tokens — this skill runs on command -only, never as part of a routine gate. +`tests/prose/run.cjs`). Cases are Given-When-Then: a world is arranged +from a committed fixture, an agent acts by walking the real prose, and a +second agent asserts the resulting trace and world. Walks cost tokens — +this skill runs on command only, never as part of a routine gate. ## Step 1: Select @@ -21,32 +23,42 @@ Parse the arguments: If the selection is empty: report that no cases intersect and stop. Otherwise show the selected case ids and proceed. -## Step 2: Walk and grade each case +## Step 2: Arrange and act Cases are independent — run up to 4 concurrently. Per case: -1. **World** (skip for `world=null` cases): `node tests/prose/run.cjs world ` — note the returned path. -2. **Walker**: `node tests/prose/run.cjs prompt [--world ]`, then dispatch a subagent (model: **sonnet**) with that prompt **verbatim and unmodified**. Never hand-assemble a walker prompt and never mention any expectation to the walker — the `prompt` command's output is the whole contract. Save the returned transcript to a scratch file. -3. **Deterministic grade**: `node tests/prose/run.cjs grade [--world ]` — state assertions pass/fail in code; the output also lists the routing claims. -4. **Grader**: dispatch a second subagent (model: **sonnet**) with: the transcript, the routing claims, and the case's scoped prose files. Instructions: verdict per claim, PASS only with a quoted line from the transcript (and prose where relevant) that satisfies it; FAIL must state what is missing or contradicting. A PASS without a quote is invalid — treat as FAIL and re-grade. +1. **World** (skip when the case has no world): `node tests/prose/run.cjs world ` — note the returned path. +2. **Walk**: `node tests/prose/run.cjs prompt [--world ]`, then dispatch a subagent (model: **sonnet**) with that prompt **verbatim and unmodified**. Never hand-assemble a walker prompt, and never mention any expectation — the `prompt` command's output is the whole contract, and it cannot contain the case's `then`. Save the returned transcript to a scratch file. -A walker that returns `UNSCRIPTED QUESTION` or `AMBIGUOUS` is a finding -in itself — carry it into the report even if claims pass. +## Step 3: Assert -## Step 3: Escalate failures +`node tests/prose/run.cjs assert [--world ]` produces the +asserting agent's prompt: the expected trace, any further claims, and the +code-computed world delta. Dispatch a second subagent (model: **sonnet**) +with that prompt plus the walk transcript. -Any failed claim (state or routing): +It returns a verdict in four parts: trace (per step, each PASS quoting the +transcript line that shows it), world (every difference in the delta +enumerated and classified volatile or material), markers, and the overall +verdict. A trace PASS without a quote is invalid — treat as FAIL. -1. Build a **fresh** world and re-run the walker on model: **opus**, then re-grade. +`UNSCRIPTED QUESTION`, `AMBIGUOUS`, or `DEVIATION` markers in a transcript +are findings in their own right — carry them into the report even when +every claim passed. + +## Step 4: Escalate failures + +Any FAIL: + +1. Build a **fresh** world and re-run the walk on model: **opus**, then re-assert. 2. Both fail → confirmed finding. 3. Sonnet-fail / Opus-pass → disagreement: report both walks with the divergent lines quoted. Never auto-resolve. -## Step 4: Report and clean up +## Step 5: Report and clean up -Report a verdict table: case id, state assertions passed/total, routing -claims passed/total, escalations, findings. Quote the evidence for every -failure. Findings are findings — never edit prose or cases mid-run; -report and stop. +Report a verdict table: case id, trace passed/total, world verdict, +markers, escalations. Quote the evidence for every failure. Findings are +findings — never edit prose, cases, or fixtures mid-run; report and stop. Destroy every world (`node tests/prose/run.cjs destroy --world `) except one you are actively citing in a failure report; name any world @@ -54,6 +66,6 @@ kept so it can be removed later. ## Rules -- The walker never sees `expect:` claims — only the `prompt` command's output reaches it. -- A failing case is a finding either way: broken prose or a stale case. The report says which is suspected, the user decides. +- The walker never sees the `then` block — only the `prompt` command's output reaches it. +- A failing case is a finding either way: broken prose, or a stale case. The report says which is suspected; the user decides. - Never regenerate fixtures or edit cases to make a run pass. diff --git a/design/prose-tests.md b/design/prose-tests.md index 1c146b02d..f32071d2b 100644 --- a/design/prose-tests.md +++ b/design/prose-tests.md @@ -29,18 +29,24 @@ testing. ## The contract -- **P1 — cases are structured natural language.** A case declares - `id`, `files:` (prose scope, drives diff-selection), `world:` (the - fixture it starts from, or none for pure prose-structure claims), - `walk:` (entry point + termination), `user:` (scripted answers for - the menus the walk hits; an unscripted question is a FAIL — the - prose asked something the case didn't predict), `expect:` (concrete - checkable claims: routing assertions graded by the agent with quoted - line evidence, end-state assertions diffed by code), `origin:` (the - finding or flow it pins). **One case per file**: - `tests/prose/{flow}/{case-id}.md`, filename equal to the id and - validated as such. A case runs as long or short as it needs to — - length never justifies bundling two into one file. +- **P1 — cases are Given-When-Then.** One case per file, flat: + `tests/prose/{case-id}.md`, filename equal to the id and validated as + such. Nothing groups cases but their `files:` scope, which is what + selection runs on. **Given**: `world_before` (a fixture) plus any + `stubs` armed, each with the moment it fires. **When**: one coarse + instruction — where to enter, what to follow, where to stop — plus + `answers`, the scripted user responses (an unscripted question is a + finding: the prose asked something the case didn't predict). + **Then**: `world_after` (a fixture, or `unchanged`), a granular + `trace` of the path the prose should have taken, and any further + `notes`. +- **P1a — coarse When, granular Then.** The When never scripts the + workflow's own steps: if the case says which commands to run, the + walker stops deriving the path from the prose and the case would pass + with the skill file deleted. Granularity belongs in the Then, where a + step-by-step expected trace catches the real threat — a walker that + silently course-corrects around broken prose instead of surfacing it. + Claims name behaviour, never coordinates. - **P2 — worlds are real, never imagined.** A fixture is authored by a **recipe** (a node script running real engine calls + perturbations against a scratch project) and committed as a **golden snapshot** @@ -52,20 +58,31 @@ testing. project; engine reads answer for real; prescribed mutations execute for real, in the sandbox. Hand-typed fixture JSON is forbidden — a hand-faked state can be a state the engine could never produce. -- **P3 — deterministic wherever possible.** The model supplies only - the walk; the facts come from code. Engine responses are real, - end-state expectations are verified by a deterministic diff, worlds - are byte-frozen snapshots. CI (`npm test`, zero tokens) holds the - deterministic perimeter: every snapshot rebuilds from its recipe - byte-identical (drift red-flags in the PR that moved the world, as - a reviewable snapshot diff), and every case's `files:` paths and - section anchors resolve. Recipes run deterministic — pinned clock - or normalised volatile fields, same constraint the simulation - already lives with. -- **P4 — walker/grader separation.** The walking agent never sees the - `expect:` block; an agent that knows the expected answer will find - it. The walker gets world + walk + user script only and returns a - transcript; grading happens after, against the transcript. +- **P3 — code states the facts, the agent judges them.** Expectations + are whole worlds, not literals: code computes the factual delta + between the acted world and the expected fixture, and the asserting + agent classifies each difference as volatile (timestamps, SHAs, + allocated ids) or material. No normalisation table to maintain, no + hand-written per-field assertions, and nothing goes unchecked + because nobody thought to assert it. Engine responses during the + walk are real. CI (`npm test`, zero tokens) holds the deterministic + perimeter: every snapshot rebuilds from its recipe byte-identical + (drift red-flags in the PR that moved the world, as a reviewable + snapshot diff), and every case's paths, anchors, worlds, stubs, and + trace resolve. +- **P4 — walker/asserter separation.** The walking agent never sees the + `then` block; an agent that knows the expected answer will find it. + The walker gets the world, the coarse instruction, the answer script + and the stubs, and returns a transcript. The asserter sees the + transcript, the expected trace, and the world delta — never the + reverse. +- **P4a — substitutions are declared and marked.** A stub is named + content; the case arming it owns the trigger, so one stub serves + many moments. The walker records `SUBSTITUTED:` for each, and the + prompt labels them as harness mechanics, so the asserter never + credits the walk for what the framework supplied. Whatever a stub + covers is not under test in that case — some other case must walk it + unstubbed. - **P5 — run on command, never in CI.** Walks cost tokens. The runner is invoked deliberately: scoped by diff-intersection (the same computation powers the PR-end suggestion — "these N cases intersect @@ -82,16 +99,17 @@ testing. ## Architecture -- `tests/prose/{flow}/` — case files per flow (research, discussion, - planning, implementation, review, discovery, …). +- `tests/prose/{case-id}.md` — the cases, flat. +- `tests/prose/stubs/{name}.md` — named substitutions: description + above a `---` fence, exact bytes below. - `tests/prose/fixtures/{name}/recipe.cjs` + `snapshot/` — the world - library. One canonical fake project (same name, same topics - throughout) so fixtures stay comparable and cases read familiarly. -- Runner (node): case parsing, diff-selection, world building, - walker-prompt assembly, end-state diffing, verdict table. Owns - everything deterministic. -- `/prose-test` skill (thin): dispatch and grading discipline — - invokes the runner, sends walkers, grades transcripts, escalates + library, holding both start and expected-end worlds. One canonical + fake project per work type so cases read familiarly. +- Runner (node): case parsing and validation, diff-selection, world + building, walker-prompt assembly, world diffing, asserter-prompt + assembly, snapshot regeneration. Owns everything deterministic. +- `/prose-test` skill (thin): dispatch and verdict discipline — + invokes the runner, sends the walker, sends the asserter, escalates failures, reports. ## Stages @@ -110,6 +128,32 @@ testing. ## Log +- 2026-07-25 — The Given-When-Then reshape (Lee, reviewing the corpus in + the TUI). Four corrections, all his: **(1)** cases are flat, one per + file, filename equal to the id — grouping by work type organised the + corpus around the build sequence rather than the prose under test, and + any single-parent taxonomy lies about cases spanning skills. + **(2)** The hand-written assertion DSL is gone. The world is a + directory that changed; the expectation is another committed world; + code diffs them and the asserting agent classifies each difference. + That deleted the grammar *and* closed the gap where anything nobody + thought to assert went unchecked — no normalisation table, no + engine-written-versus-model-written split to adjudicate. + **(3)** Stub triggers belong to the case, not the stub: binding a + substitution to a moment is what *prevents* reuse, and the same + content is wanted at different moments (first dispatch vs re-dispatch, + gaps-found vs validated). A stub is named content; the case says when. + A trigger-less stub is arrange in disguise and fails validation. + **(4)** Granularity belongs in the Then. Lee's argument, which holds: + coarse assertions let an agent silently course-correct around broken + prose — good in production, fatal in a test. The When stays coarse + (else the walker stops deriving the path and the case passes with the + skill deleted); the Then carries a step-by-step expected trace, so a + silent repair shows as a trace mismatch. Also added: a `DEVIATION:` + marker for prose that cannot be followed literally, findings in their + own right. Sections renamed given/when/then after BDD, which reads + closer to a specification than arrange/act/assert. + - 2026-07-25 — Stage 3b, the bugfix corpus. Three fixtures along `crash-fix` (created / investigating / investigated) and four cases covering only what bugfix does differently: continue-bugfix routing a diff --git a/tests/prose/README.md b/tests/prose/README.md index 8d32596dc..6d5613993 100644 --- a/tests/prose/README.md +++ b/tests/prose/README.md @@ -1,51 +1,88 @@ # Prose tests -End-to-end tests for the prose logic — natural-language cases a walker -agent executes through the real skills against a materialised fixture -world. Design and contract: [design/prose-tests.md](../../design/prose-tests.md). -Run via the `/prose-test` skill; everything deterministic lives in -`run.cjs` and `lib/`. +End-to-end tests for the prose logic. A case is Given-When-Then: a world +is **arranged** from a committed fixture, an agent **acts** by walking the +real skills as a live session would, and a second agent **asserts** the +resulting trace and world. Design and contract: +[design/prose-tests.md](../../design/prose-tests.md). Run via the +`/prose-test` skill; everything deterministic lives in `run.cjs` and +`lib/`. ## Layout -- `{flow}/{case-id}.md` — **one case per file**, filename = case id - (validated; a flow's `README.md` holds its intro and is ignored) -- `fixtures/{name}/recipe.cjs` — builds the world with real engine calls +- `{case-id}.md` — one case per file, filename = case id, flat. Nothing + groups cases but their `files:` scope, which is what selection runs on. +- `stubs/{name}.md` — a named substitution: description above a `---` + fence, exact bytes below. The **case** says when it fires. +- `fixtures/{name}/recipe.cjs` — builds a world with real engine calls under a frozen clock (`lib/fake-clock.cjs`) - `fixtures/{name}/snapshot/` — the recipe's committed golden output; regenerate with `node tests/prose/run.cjs snap {name}`, never hand-edit -- `run.cjs` — `list · select · world · prompt · grade · snap · verify · destroy` +- `run.cjs` — `list · select · world · prompt · diff · assert · snap · verify · destroy` -## Adding a case +## Writing a case -Create `{flow}/{case-id}.md` holding a single `## case:` block — the -filename must equal the id. Scope `files:` tightly — diff-selection and -the PR-end suggestion run off it. `routing:` expects are agent-graded against the walk -transcript; `state:` expects are asserted in code and require a `world:`. +**The `when` stays coarse — one instruction.** Where to enter, what to +follow, where to stop. Never a step-by-step script of the workflow: if +the case tells the walker which commands to run, the walker no longer has +to derive the path from the prose, and the case would pass with the skill +file deleted. That derivation is the thing under test. -**Stub the agents, walk the lifecycle.** Prose that dispatches a -background agent gets a `### stub` section: free text telling the walker -to play the agent itself — write the report at the path the engine's -dispatch returned, return the stated result — instead of spending a real -sub-agent analysing fixture content. What the case tests is the -lifecycle around the agent (dispatch, scan, incorporate, how the verdict -routes), never the agent's own judgment. +**The `then` is granular.** A step-by-step expected trace is what catches +a walker that silently course-corrected around broken prose — the +divergence shows up as a trace mismatch instead of relying on the walker +to confess. Plus `world_after`: a fixture name, or `unchanged` when the +walk should leave the project exactly as it found it. **Name behaviour, never coordinates.** Step numbers, arm letters, and -heading numbering in walk or expect text break on cosmetic renumbering — -failure for the wrong reason. Write "casing conventions load before the -boot pipeline", not "Step 0.1 before Step 0.2"; a behavioural claim -fails only when the behaviour changes, which is the failure the corpus -exists to catch. Anchors are substring fragments matched against heading -text (`#Boot` matches "Step 0.2: Boot") — pick the number-free part. -`npm test` validates the corpus (parse, paths, anchors, grammar) and -rebuild-compares every fixture — both token-free; only `/prose-test` -spends tokens. +heading numbering break on cosmetic renumbering — failure for the wrong +reason. Write "casing conventions load before the boot pipeline", not +"Step 0.1 before Step 0.2". Anchors are substring fragments matched +against heading text (`#Boot` matches "Step 0.2: Boot"). + +**Scope `files:` tightly** — diff-selection and the PR-end suggestion run +off it. + +## Stubs + +Prose that dispatches a background agent, or that runs an unscripted +conversation, is walked with that part substituted. A stub is named +content; the case that arms it declares the moment: + +``` +### given +world_before: bugfix-investigating +stubs: + - root-cause-validated: when the prose dispatches the validation agent +``` + +The trigger belongs to the case, not the stub, so one stub serves many +moments — first dispatch versus re-dispatch, happy path versus recovery. +A stub with no trigger is arrange in disguise, and validation rejects it. + +**Whatever a stub covers is not under test in that case.** Stubbing a +discussion's session loop means some other case must walk that loop +without stubbing it. The stub boundary is a scope decision, not a +convenience. + +## Assertion + +Deterministic where it can be, judged where it can't. Code computes the +factual delta between the acted world and the expected world; the +asserting agent classifies every difference as volatile (timestamps, git +SHAs, engine-allocated ids) or material, and checks the trace against the +transcript with quoted evidence. No normalisation and no expected-value +literals: the expectation is a whole committed world, built by a recipe, +reviewable as a diff against the world it started from. + +`npm test` validates the corpus (parse, paths, anchors, worlds, stubs, +traces) and rebuild-compares every fixture — both token-free. Only +`/prose-test` spends tokens. ## Snapshot escaping -Snapshots exclude `.git/` and `.workflows/.knowledge/` (the world -builder re-derives the store), and store `.gitignore` files as +Snapshots exclude `.git/`, `.workflows/.knowledge/` (the world builder +re-derives the store) and `.claude/skills|agents/` (copied into live +worlds, never part of a fixture), and store `.gitignore` files as `_gitignore.fixture` so the product-written `.workflows/.gitignore` -cannot ignore fixture content out of this repo; the world builder -restores them. +cannot ignore fixture content out of this repo. diff --git a/tests/prose/bugfix/README.md b/tests/prose/bugfix/README.md deleted file mode 100644 index dde030be9..000000000 --- a/tests/prose/bugfix/README.md +++ /dev/null @@ -1,8 +0,0 @@ -# Bugfix mainline — happy-path corpus - -The canonical bugfix `crash-fix` walked through what makes bugfix -distinct from the linear pipeline: investigation entry seeded from the -bug's carrier, the root-cause validation agent's full lifecycle, and a -specification sourced from an investigation rather than a discussion. -The delivery phases it shares with feature are covered by the feature -corpus and not repeated here. diff --git a/tests/prose/bugfix/bugfix-continue-offers-investigation.md b/tests/prose/bugfix/bugfix-continue-offers-investigation.md deleted file mode 100644 index 453ffce1e..000000000 --- a/tests/prose/bugfix/bugfix-continue-offers-investigation.md +++ /dev/null @@ -1,25 +0,0 @@ -## case: bugfix-continue-offers-investigation -- world: bugfix-created -- origin: bugfix mainline — continue-bugfix routes a fresh bug to investigation -- files: - - skills/workflow-continue-bugfix/SKILL.md - - skills/workflow-continue-bugfix/references/select-bugfix.md - - skills/workflow-continue-bugfix/references/bugfix-display-and-menu.md - -### walk - -Execute skills/workflow-continue-bugfix/SKILL.md with no arguments (as -routed from workflow-start with no pre-selected work unit). Follow -selection, then the bugfix state display and its menu. Record the state -display and menu verbatim, then stop — do not route into any phase -entry skill. - -### user - -1. Select the bugfix `crash-fix` (its number in the selection menu) - -### expect - -- routing: the state display shows no phase started — the pipeline is fresh -- routing: the menu's continue action routes to investigation entry (workflow-investigation-entry with bugfix and crash-fix), not to discussion or specification -- state: manifest absent crash-fix.investigation.crash-fix status diff --git a/tests/prose/bugfix/bugfix-investigation-entry-seeds.md b/tests/prose/bugfix/bugfix-investigation-entry-seeds.md deleted file mode 100644 index dacac27ba..000000000 --- a/tests/prose/bugfix/bugfix-investigation-entry-seeds.md +++ /dev/null @@ -1,22 +0,0 @@ -## case: bugfix-investigation-entry-seeds -- world: bugfix-created -- origin: bugfix mainline — investigation entry seeds from the carrier, never re-asks -- files: - - skills/workflow-investigation-entry/SKILL.md - - skills/workflow-investigation-entry/references/invoke-skill.md - -### walk - -Execute skills/workflow-investigation-entry/SKILL.md with arguments -$0=bugfix, $1=crash-fix. Follow it up to the point where the handoff to -the processing skill is constructed — record the handoff arguments -block — and stop there. Do not execute the processing skill's -instructions. - -### expect - -- routing: with no investigation item in the manifest, source is new — the phase-validation path is not taken -- routing: because a discovery session log exists, the bug context is seeded from the manifest description and that log — the gather-context questioning path is not taken, and the walk asks the user nothing -- routing: a phase note is rendered by the engine for the investigation phase, with the verb Starting -- routing: the handoff invokes workflow-investigation-process for crash-fix -- state: manifest absent crash-fix.investigation.crash-fix status diff --git a/tests/prose/bugfix/bugfix-root-cause-validation-lifecycle.md b/tests/prose/bugfix/bugfix-root-cause-validation-lifecycle.md deleted file mode 100644 index 8fda1bec2..000000000 --- a/tests/prose/bugfix/bugfix-root-cause-validation-lifecycle.md +++ /dev/null @@ -1,45 +0,0 @@ -## case: bugfix-root-cause-validation-lifecycle -- world: bugfix-investigating -- origin: bugfix mainline — the validation agent's dispatch → scan → incorporate lifecycle -- files: - - skills/workflow-investigation-process/references/root-cause-validation.md - -### walk - -The investigation is open with its root cause documented, and the -process has reached its root-cause validation step. Execute -skills/workflow-investigation-process/references/root-cause-validation.md -from the top, as the processing skill would. Stop when the reference -returns to its caller. - -### user - -1. yes — run root cause validation - -### stub - -The prose directs a validation agent to be dispatched via the Task -tool. Do not dispatch one. Instead, after recording the dispatch, write -the report yourself to the exact `file` path the dispatch response -returned, containing: - - # Root Cause Validation - - The tax context is built before the payment intent and reads the - shipping address unconditionally. Traced fresh: an address-less - order fails at that read, matching the documented root cause. - - STATUS: validated - CONFIDENCE: high - GAPS_COUNT: 0 - SUMMARY: Root cause confirmed by an independent trace. - -Then continue the prose as if the agent had returned that result. - -### expect - -- routing: the dispatch is recorded through the engine, and the report path used is the one the dispatch response returned — the prose does not invent or pre-create the file -- routing: after the report lands, the row is promoted by a scan and then closed by an incorporate — the verdict is consumed inline, never surfaced finding by finding -- routing: on a validated verdict the reference reports validation with its confidence and returns to the caller — the gaps-handling menu is not reached -- state: file exists .workflows/.cache/crash-fix/investigation/crash-fix/root-cause-validation-001.md -- state: json .workflows/.cache/crash-fix/investigation/crash-fix/state.json agents.root-cause-validation-001.status incorporated diff --git a/tests/prose/bugfix/bugfix-spec-entry-sources-investigation.md b/tests/prose/bugfix/bugfix-spec-entry-sources-investigation.md deleted file mode 100644 index 1a7fd5007..000000000 --- a/tests/prose/bugfix/bugfix-spec-entry-sources-investigation.md +++ /dev/null @@ -1,22 +0,0 @@ -## case: bugfix-spec-entry-sources-investigation -- world: bugfix-investigated -- origin: bugfix mainline — the spec's source is the investigation, not a discussion -- files: - - skills/workflow-specification-entry/SKILL.md - - skills/workflow-specification-entry/references/validate-source.md - - skills/workflow-specification-entry/references/invoke-skill.md - -### walk - -Execute skills/workflow-specification-entry/SKILL.md with arguments -$0=bugfix, $1=crash-fix. Follow it up to the point where the handoff to -the processing skill is constructed — record the handoff arguments -block — and stop there. Do not execute the processing skill's -instructions. - -### expect - -- routing: the source-material entry gate passes (empty render) — the completed investigation satisfies it for a bugfix -- routing: with no specification item in the manifest, the verb is Creating -- routing: the handoff names the investigation file .workflows/crash-fix/investigation/crash-fix.md as source material — no discussion file is named -- state: manifest absent crash-fix.specification.crash-fix status diff --git a/tests/prose/continue-bugfix-routes-to-investigation.md b/tests/prose/continue-bugfix-routes-to-investigation.md new file mode 100644 index 000000000..646d3ea2e --- /dev/null +++ b/tests/prose/continue-bugfix-routes-to-investigation.md @@ -0,0 +1,34 @@ +## case: continue-bugfix-routes-to-investigation +- origin: bugfix mainline — a fresh bug routes to investigation, not discussion +- files: + - skills/workflow-continue-bugfix/SKILL.md + - skills/workflow-continue-bugfix/references/select-bugfix.md + - skills/workflow-continue-bugfix/references/bugfix-display-and-menu.md + +### given + +world_before: bugfix-created + +### when + +Execute skills/workflow-continue-bugfix/SKILL.md with no arguments, as +it is invoked when the caller has no work unit pre-selected. Follow it +through selection and the bugfix's state display. Stop once the state +and its menu have been shown — do not invoke the route. + +answers: +1. the number listed against `crash-fix` + +### then + +world_after: unchanged + +trace: +1. initialisation is casing only — boot and migrations belong to the + entry skill +2. shows the selection display and menu rather than auto-selecting the + only bugfix +3. validates the selection, then shows the bugfix's pipeline state +4. the state shows no phase started, and the continue action routes to + investigation entry — a bugfix's first phase is investigation, never + discussion or specification diff --git a/tests/prose/continue-feature-routes-to-discussion.md b/tests/prose/continue-feature-routes-to-discussion.md new file mode 100644 index 000000000..23e5fd61f --- /dev/null +++ b/tests/prose/continue-feature-routes-to-discussion.md @@ -0,0 +1,36 @@ +## case: continue-feature-routes-to-discussion +- origin: feature mainline — navigation shows state and routes to the next phase +- files: + - skills/workflow-continue-feature/SKILL.md + - skills/workflow-continue-feature/references/select-feature.md + - skills/workflow-continue-feature/references/feature-display-and-menu.md + +### when + +Execute skills/workflow-continue-feature/SKILL.md with no arguments, as +it is invoked when the caller has no work unit pre-selected. Follow it +through selection and the feature's state display. Stop once the state +and its menu have been shown — do not invoke the route. + +answers: +1. the number listed against `pay` + +### given + +world_before: feature-created + +### then + +world_after: unchanged + +trace: +1. initialisation is casing only — no boot and no migrations, which the + entry skill has already guaranteed +2. shows the selection display and menu even though `pay` is the only + feature — never auto-selects +3. validates the selection, then shows the feature's pipeline state +4. the state shows no phase started, and the menu's continue action + routes to discussion entry for work type feature, work unit pay + +notes: +- navigation only reads: no phase is started by looking at it diff --git a/tests/prose/discussion-entry-seeds-from-carrier.md b/tests/prose/discussion-entry-seeds-from-carrier.md new file mode 100644 index 000000000..4128b458c --- /dev/null +++ b/tests/prose/discussion-entry-seeds-from-carrier.md @@ -0,0 +1,35 @@ +## case: discussion-entry-seeds-from-carrier +- origin: feature mainline — single-phase work seeds from its durable carrier +- files: + - skills/workflow-discussion-entry/SKILL.md + - skills/workflow-shared/references/ensure-discovery-item.md + +### given + +world_before: feature-created + +### when + +Execute skills/workflow-discussion-entry/SKILL.md with arguments +$0=feature, $1=pay. Follow it to the point where the handoff to the +processing skill is constructed; record the handoff block and stop. Do +not execute the processing skill's instructions. + +### then + +world_after: unchanged + +trace: +1. resolves the topic to the work unit, since only an epic is given one + explicitly +2. reads the discussion status, finds nothing, and takes the new-entry + arm — phase validation is for entries that already exist +3. ensuring a discovery item returns immediately: the map is epic-only, + and this is a feature +4. seeds the context from the manifest description and the session log's + exploration, asking the user nothing +5. hands off to the discussion processing skill for pay + +notes: +- the entry skill records nothing: the discussion item is created by the + processing skill, not by the walk to it diff --git a/tests/prose/feature/README.md b/tests/prose/feature/README.md deleted file mode 100644 index 86f290628..000000000 --- a/tests/prose/feature/README.md +++ /dev/null @@ -1,7 +0,0 @@ -# Feature mainline — happy-path corpus - -The canonical feature `pay` walked through its pipeline: navigation, -each phase entry, and the implementation pickup. Worlds are the -`feature-*` fixtures (stop points along `_shared/feature-mainline.cjs`). -Every claim here was verified against the prose at authoring time — -a failure means the behaviour moved, not the numbering. diff --git a/tests/prose/feature/feature-continue-offers-discussion.md b/tests/prose/feature/feature-continue-offers-discussion.md deleted file mode 100644 index c7e2127cf..000000000 --- a/tests/prose/feature/feature-continue-offers-discussion.md +++ /dev/null @@ -1,26 +0,0 @@ -## case: feature-continue-offers-discussion -- world: feature-created -- origin: feature mainline — continue-feature shows state and routes forward -- files: - - skills/workflow-continue-feature/SKILL.md - - skills/workflow-continue-feature/references/select-feature.md - - skills/workflow-continue-feature/references/feature-display-and-menu.md - -### walk - -Execute skills/workflow-continue-feature/SKILL.md with no arguments -(as routed from workflow-start with no pre-selected work unit). Follow -selection, then the feature state display and its menu. Record the -state display and menu verbatim, then stop — do not route into any -phase entry skill. - -### user - -1. Select the feature `pay` (its number in the selection menu) - -### expect - -- routing: the selection menu is shown even though only one feature exists — no auto-select -- routing: the feature state shows no phase started — the pipeline is fresh -- routing: the menu's forward action for the current state routes to discussion entry (workflow-discussion-entry with feature and pay) -- state: manifest absent pay.discussion.pay status diff --git a/tests/prose/feature/feature-discussion-entry-seeds.md b/tests/prose/feature/feature-discussion-entry-seeds.md deleted file mode 100644 index 57408c959..000000000 --- a/tests/prose/feature/feature-discussion-entry-seeds.md +++ /dev/null @@ -1,21 +0,0 @@ -## case: feature-discussion-entry-seeds -- world: feature-created -- origin: feature mainline — discussion entry seeds from the durable carrier -- files: - - skills/workflow-discussion-entry/SKILL.md - - skills/workflow-shared/references/ensure-discovery-item.md - -### walk - -Execute skills/workflow-discussion-entry/SKILL.md with arguments -$0=feature, $1=pay. Follow it up to the point where the handoff to the -processing skill is constructed — record the handoff arguments block — -and stop there. Do not execute the processing skill's instructions. - -### expect - -- routing: with no discussion item in the manifest, the new-entry arm is taken — no phase-status validation menu appears -- routing: ensure-discovery-item returns without creating anything — the discovery map is epic-only, and this is a feature -- routing: context is seeded from the manifest description and the session log's Exploration section — the gather-context questioning path is not taken -- routing: the handoff invokes workflow-discussion-process for pay -- state: manifest absent pay.discussion.pay status diff --git a/tests/prose/feature/feature-implementation-first-task.md b/tests/prose/feature/feature-implementation-first-task.md deleted file mode 100644 index 5bad38f40..000000000 --- a/tests/prose/feature/feature-implementation-first-task.md +++ /dev/null @@ -1,31 +0,0 @@ -## case: feature-implementation-first-task -- world: feature-planned -- origin: feature mainline — implementation entry through to the first task pickup -- files: - - skills/workflow-implementation-entry/SKILL.md - - skills/workflow-implementation-entry/references/validate-phase.md - - skills/workflow-implementation-entry/references/environment-check.md - - skills/workflow-implementation-process/SKILL.md - - skills/workflow-implementation-process/references/task-loop.md - - skills/workflow-planning-process/references/output-formats/local-markdown/reading.md - -### walk - -Execute skills/workflow-implementation-entry/SKILL.md with arguments -$0=feature, $1=pay, then continue into the processing skill it invokes. -Follow the task loop far enough to determine and start the next -available task. Stop the moment the prose directs you to begin actually -implementing that task (writing code or tests) — implement nothing. - -### user - -1. none — no special environment setup is needed - -### expect - -- routing: the plan entry gate passes (empty render) and the implementation is a new entry -- routing: with no environment-setup file recorded, the entry asks the environment question (answered none) -- routing: the processing skill's resume detection reports mode created — the fresh path, with its start-implementation commit — never the resuming-from-previous-session note -- routing: the next available task per the plan format's reading procedure is pay-1-1 — phase 1, first task, nothing completed yet -- state: manifest equals pay.implementation.pay status in-progress -- state: manifest equals pay.implementation.pay current_task pay-1-1 diff --git a/tests/prose/feature/feature-planning-entry-context-gate.md b/tests/prose/feature/feature-planning-entry-context-gate.md deleted file mode 100644 index 16976dc63..000000000 --- a/tests/prose/feature/feature-planning-entry-context-gate.md +++ /dev/null @@ -1,25 +0,0 @@ -## case: feature-planning-entry-context-gate -- world: feature-specified -- origin: feature mainline — planning entry validates the spec and asks for late context -- files: - - skills/workflow-planning-entry/SKILL.md - - skills/workflow-planning-entry/references/validate-spec.md - - skills/workflow-planning-entry/references/validate-phase.md - -### walk - -Execute skills/workflow-planning-entry/SKILL.md with arguments -$0=feature, $1=pay. Follow it up to the point where the handoff to the -processing skill is constructed — record the handoff arguments block — -and stop there. Do not execute the processing skill's instructions. - -### user - -1. continue — no additional context since the specification - -### expect - -- routing: the specification entry gate passes (empty render) — clear to plan -- routing: with no plan in the manifest, the fresh-start arm asks whether any context has changed since the specification (answered continue) -- routing: the handoff invokes workflow-planning-process for pay -- state: manifest absent pay.planning.pay status diff --git a/tests/prose/feature/feature-review-entry-fresh.md b/tests/prose/feature/feature-review-entry-fresh.md deleted file mode 100644 index 8891ef2e8..000000000 --- a/tests/prose/feature/feature-review-entry-fresh.md +++ /dev/null @@ -1,21 +0,0 @@ -## case: feature-review-entry-fresh -- world: feature-implemented -- origin: feature mainline — review entry passes both prerequisites, fresh review -- files: - - skills/workflow-review-entry/SKILL.md - - skills/workflow-review-entry/references/validate-phase.md - -### walk - -Execute skills/workflow-review-entry/SKILL.md with arguments -$0=feature, $1=pay. Follow it up to the point where the handoff to the -processing skill is constructed — record the handoff arguments block — -and stop there. Do not execute the processing skill's instructions, and -never dispatch any agent. - -### expect - -- routing: the prerequisite entry gate passes (empty render) — plan and implementation are both complete -- routing: with no review item in the manifest, the fresh path is taken — no reopen, no resume -- routing: the handoff invokes workflow-review-process for pay -- state: manifest absent pay.review.pay status diff --git a/tests/prose/feature/feature-spec-entry-handoff.md b/tests/prose/feature/feature-spec-entry-handoff.md deleted file mode 100644 index d6ded024b..000000000 --- a/tests/prose/feature/feature-spec-entry-handoff.md +++ /dev/null @@ -1,22 +0,0 @@ -## case: feature-spec-entry-handoff -- world: feature-discussed -- origin: feature mainline — specification entry validates and hands off the discussion -- files: - - skills/workflow-specification-entry/SKILL.md - - skills/workflow-specification-entry/references/validate-source.md - - skills/workflow-specification-entry/references/validate-phase.md - - skills/workflow-specification-entry/references/invoke-skill.md - -### walk - -Execute skills/workflow-specification-entry/SKILL.md with arguments -$0=feature, $1=pay. Follow it up to the point where the handoff to the -processing skill is constructed — record the handoff arguments block — -and stop there. Do not execute the processing skill's instructions. - -### expect - -- routing: the source-material entry gate passes (empty render) — the discussion is complete -- routing: with no specification item in the manifest, the verb is Creating — no resume note, no reopen -- routing: the handoff names the discussion file .workflows/pay/discussion/pay.md as source material -- state: manifest absent pay.specification.pay status diff --git a/tests/prose/feature/feature-start-lists-fresh.md b/tests/prose/feature/feature-start-lists-fresh.md deleted file mode 100644 index 6add0ffdf..000000000 --- a/tests/prose/feature/feature-start-lists-fresh.md +++ /dev/null @@ -1,20 +0,0 @@ -## case: feature-start-lists-fresh -- world: feature-created -- origin: feature mainline — workflow-start surfaces active work -- files: - - skills/workflow-start/SKILL.md - -### walk - -Execute skills/workflow-start/SKILL.md from the top. Initialisation -will find the system already migrated and the knowledge base ready, so -no migration summary or setup conversation applies. Continue until the -work dashboard and its menu are shown. Record both verbatim and stop — -select nothing. - -### expect - -- routing: initialisation runs the boot pipeline before any work is shown, and with `migrations.changed` false no confirmation gate is raised -- routing: the dashboard lists `pay` as active work -- routing: the menu offers continuing existing work (the path that routes feature work to workflow-continue-feature) -- state: manifest absent pay.discussion.pay status diff --git a/tests/prose/fixtures/bugfix-validation-incorporated/recipe.cjs b/tests/prose/fixtures/bugfix-validation-incorporated/recipe.cjs new file mode 100644 index 000000000..8e8a08ed8 --- /dev/null +++ b/tests/prose/fixtures/bugfix-validation-incorporated/recipe.cjs @@ -0,0 +1,27 @@ +'use strict'; + +// Expected end state for the root-cause validation walk: the row +// dispatched, the report landed, scan promoted it, incorporate closed it. +// The report content is read from the stub the case arms, so the bytes +// the walk writes and the bytes expected here cannot drift. + +const fs = require('fs'); +const path = require('path'); + +const m = require('../_shared/bugfix-mainline.cjs'); +const { readStub } = require('../../lib/cases.cjs'); + +module.exports = { + build(h) { + m.init(h); + m.create(h); + m.investigateToRootCause(h); + + const dispatch = JSON.parse(h.engine( + 'agent', 'dispatch', m.WU, 'investigation', m.WU, '--kind', 'root-cause-validation')); + fs.mkdirSync(path.dirname(path.join(h.dir, dispatch.file)), { recursive: true }); + h.write(dispatch.file, readStub('root-cause-validated').content); + h.engine('agent', 'scan', m.WU, 'investigation', m.WU); + h.engine('agent', 'incorporate', m.WU, 'investigation', m.WU, dispatch.id); + }, +}; diff --git a/tests/prose/fixtures/bugfix-validation-incorporated/snapshot/.claude/settings.json b/tests/prose/fixtures/bugfix-validation-incorporated/snapshot/.claude/settings.json new file mode 100644 index 000000000..ae019ede5 --- /dev/null +++ b/tests/prose/fixtures/bugfix-validation-incorporated/snapshot/.claude/settings.json @@ -0,0 +1,9 @@ +{ + "showClearContextOnPlanAccept": true, + "permissions": { + "allow": [ + "Edit(.workflows/**)", + "Bash(mv .workflows/:*)" + ] + } +} diff --git a/tests/prose/fixtures/bugfix-validation-incorporated/snapshot/.workflows/.cache/crash-fix/investigation/crash-fix/root-cause-validation-001.md b/tests/prose/fixtures/bugfix-validation-incorporated/snapshot/.workflows/.cache/crash-fix/investigation/crash-fix/root-cause-validation-001.md new file mode 100644 index 000000000..ce1c9b4c4 --- /dev/null +++ b/tests/prose/fixtures/bugfix-validation-incorporated/snapshot/.workflows/.cache/crash-fix/investigation/crash-fix/root-cause-validation-001.md @@ -0,0 +1,10 @@ +# Root Cause Validation + +The tax context is built before the payment intent and reads the shipping +address unconditionally. Traced fresh: an address-less order fails at that +read, which matches the documented root cause. + +STATUS: validated +CONFIDENCE: high +GAPS_COUNT: 0 +SUMMARY: Root cause confirmed by an independent trace. diff --git a/tests/prose/fixtures/bugfix-validation-incorporated/snapshot/.workflows/.cache/crash-fix/investigation/crash-fix/state.json b/tests/prose/fixtures/bugfix-validation-incorporated/snapshot/.workflows/.cache/crash-fix/investigation/crash-fix/state.json new file mode 100644 index 000000000..7eb5a78f1 --- /dev/null +++ b/tests/prose/fixtures/bugfix-validation-incorporated/snapshot/.workflows/.cache/crash-fix/investigation/crash-fix/state.json @@ -0,0 +1,16 @@ +{ + "agents": { + "root-cause-validation-001": { + "id": "root-cause-validation-001", + "kind": "root-cause-validation", + "phase": "investigation", + "topic": "crash-fix", + "set": "001", + "status": "incorporated", + "announced": false, + "findings": [], + "surfaced": [], + "created": "2026-01-01T00:00:00.000Z" + } + } +} diff --git a/tests/prose/fixtures/bugfix-validation-incorporated/snapshot/.workflows/.state/migrations b/tests/prose/fixtures/bugfix-validation-incorporated/snapshot/.workflows/.state/migrations new file mode 100644 index 000000000..3c7162acc --- /dev/null +++ b/tests/prose/fixtures/bugfix-validation-incorporated/snapshot/.workflows/.state/migrations @@ -0,0 +1,52 @@ +001 +002 +003 +004 +005 +006 +007 +008 +009 +010 +011 +012 +013 +014 +015 +016 +017 +018 +019 +020 +021 +022 +023 +024 +025 +026 +027 +028 +029 +030 +031 +032 +033 +034 +035 +036 +037 +038 +039 +040 +041 +042 +043 +044 +045 +046 +047 +048 +049 +050 +051 +052 diff --git a/tests/prose/fixtures/bugfix-validation-incorporated/snapshot/.workflows/_gitignore.fixture b/tests/prose/fixtures/bugfix-validation-incorporated/snapshot/.workflows/_gitignore.fixture new file mode 100644 index 000000000..779cbf28a --- /dev/null +++ b/tests/prose/fixtures/bugfix-validation-incorporated/snapshot/.workflows/_gitignore.fixture @@ -0,0 +1,2 @@ +.cache/ +.manifest.json.*.tmp diff --git a/tests/prose/fixtures/bugfix-validation-incorporated/snapshot/.workflows/crash-fix/discovery/sessions/session-001.md b/tests/prose/fixtures/bugfix-validation-incorporated/snapshot/.workflows/crash-fix/discovery/sessions/session-001.md new file mode 100644 index 000000000..6dc56d4e3 --- /dev/null +++ b/tests/prose/fixtures/bugfix-validation-incorporated/snapshot/.workflows/crash-fix/discovery/sessions/session-001.md @@ -0,0 +1,40 @@ +# Discovery Session 001 + +Date: 2026-01-01 +Work unit: crash-fix + +## Description (as of session) + +Checkout crashes when an order has no shipping address. + +## Seed + +(none) + +## Imports + +(none) + +## Map State at Start + +(n/a — single-topic work) + +## Exploration + +Reported by two users this week: the checkout page 500s at the +payment step for orders with no shipping address. Reproducible on +staging with a digital-only basket. No error surfaces to the user — +the page just fails. Confirmed as a bugfix; no design question here, +so it routes straight to investigation. + +## Edits + +(none) + +## Topics Identified + +(none) + +## Conclusion + +(none) diff --git a/tests/prose/fixtures/bugfix-validation-incorporated/snapshot/.workflows/crash-fix/investigation/crash-fix.md b/tests/prose/fixtures/bugfix-validation-incorporated/snapshot/.workflows/crash-fix/investigation/crash-fix.md new file mode 100644 index 000000000..d5fc263a9 --- /dev/null +++ b/tests/prose/fixtures/bugfix-validation-incorporated/snapshot/.workflows/crash-fix/investigation/crash-fix.md @@ -0,0 +1,88 @@ +# Investigation: Checkout Crash On Missing Shipping Address + +## Symptoms + +### Problem Description + +Checkout returns a 500 at the payment step when the order has no +shipping address. No error is surfaced to the user. + +### Manifestation + +Server error on POST to the payment step; the page fails blank. + +### Reproduction Steps + +1. Add a digital-only product to the basket. +2. Proceed to checkout without entering a shipping address. +3. Continue to the payment step — the request 500s. + +### Environment + +Reproduced on staging; reported twice in production this week. + +### Impact + +Digital-only orders cannot complete checkout. + +### References + +(none) + +## Analysis + +### Hypotheses + +- The payment step assumes a shipping address is always present. +- Tax calculation may be the first consumer of the missing address. + +### Code Trace + +The payment step builds a tax context from the order before creating +the payment intent. The tax context reads the shipping address +unconditionally; for a digital-only order the address is absent, so +the read fails and the request aborts before any handler catches it. + +### Root Cause + +The tax context treats the shipping address as mandatory. Digital-only +orders legitimately have none, so the assumption is wrong rather than +the data being wrong. + +### Contributing Factors + +Digital-only baskets were introduced after the tax context was written. + +### Why It Wasn't Caught + +No test covers a basket with no shippable line items. + +### Blast Radius + +Any flow building a tax context from an address-less order. + +## Fix Direction + +### Chosen Approach + +(pending) + +### Options Explored + +(none yet) + +### Discussion + +(none yet) + +### Testing Recommendations + +(pending) + +### Risk Assessment + +(pending) + +## Notes + +(none) diff --git a/tests/prose/fixtures/bugfix-validation-incorporated/snapshot/.workflows/crash-fix/manifest.json b/tests/prose/fixtures/bugfix-validation-incorporated/snapshot/.workflows/crash-fix/manifest.json new file mode 100644 index 000000000..88bf183ff --- /dev/null +++ b/tests/prose/fixtures/bugfix-validation-incorporated/snapshot/.workflows/crash-fix/manifest.json @@ -0,0 +1,16 @@ +{ + "name": "crash-fix", + "work_type": "bugfix", + "status": "in-progress", + "created": "2026-01-01", + "description": "Checkout crashes when an order has no shipping address", + "phases": { + "investigation": { + "items": { + "crash-fix": { + "status": "in-progress" + } + } + } + } +} diff --git a/tests/prose/fixtures/bugfix-validation-incorporated/snapshot/.workflows/manifest.json b/tests/prose/fixtures/bugfix-validation-incorporated/snapshot/.workflows/manifest.json new file mode 100644 index 000000000..1cd498ed5 --- /dev/null +++ b/tests/prose/fixtures/bugfix-validation-incorporated/snapshot/.workflows/manifest.json @@ -0,0 +1,7 @@ +{ + "work_units": { + "crash-fix": { + "work_type": "bugfix" + } + } +} diff --git a/tests/prose/fixtures/feature-implementation-started/recipe.cjs b/tests/prose/fixtures/feature-implementation-started/recipe.cjs new file mode 100644 index 000000000..6aa1cf8a2 --- /dev/null +++ b/tests/prose/fixtures/feature-implementation-started/recipe.cjs @@ -0,0 +1,27 @@ +'use strict'; + +// Expected end state for the implementation-pickup walk: the environment +// question answered "none" (which the prose records so it is never asked +// again), implementation tracking initialised by `task init`, and the +// first task started. No task work done — the walk stops at the point the +// prose directs implementation to begin. + +const m = require('../_shared/feature-mainline.cjs'); + +module.exports = { + build(h) { + m.init(h); + m.create(h); + m.discuss(h); + m.specify(h); + m.plan(h); + + // Prose order: resume detection (task init) and its commit first, then + // environment setup records the answer so it is never asked again. + h.engine('task', 'init', m.WU, m.WU); + h.engine('commit', m.WU, '-m', `impl(${m.WU}): start implementation`); + h.write('.workflows/.state/environment-setup.md', 'No special setup required.\n'); + h.engine('commit', '--workflows', '-m', `impl(${m.WU}): record environment setup`); + h.engine('task', 'start', m.WU, m.WU, `${m.WU}-1-1`); + }, +}; diff --git a/tests/prose/fixtures/feature-implementation-started/snapshot/.claude/settings.json b/tests/prose/fixtures/feature-implementation-started/snapshot/.claude/settings.json new file mode 100644 index 000000000..ae019ede5 --- /dev/null +++ b/tests/prose/fixtures/feature-implementation-started/snapshot/.claude/settings.json @@ -0,0 +1,9 @@ +{ + "showClearContextOnPlanAccept": true, + "permissions": { + "allow": [ + "Edit(.workflows/**)", + "Bash(mv .workflows/:*)" + ] + } +} diff --git a/tests/prose/fixtures/feature-implementation-started/snapshot/.workflows/.state/environment-setup.md b/tests/prose/fixtures/feature-implementation-started/snapshot/.workflows/.state/environment-setup.md new file mode 100644 index 000000000..56b13047e --- /dev/null +++ b/tests/prose/fixtures/feature-implementation-started/snapshot/.workflows/.state/environment-setup.md @@ -0,0 +1 @@ +No special setup required. diff --git a/tests/prose/fixtures/feature-implementation-started/snapshot/.workflows/.state/migrations b/tests/prose/fixtures/feature-implementation-started/snapshot/.workflows/.state/migrations new file mode 100644 index 000000000..3c7162acc --- /dev/null +++ b/tests/prose/fixtures/feature-implementation-started/snapshot/.workflows/.state/migrations @@ -0,0 +1,52 @@ +001 +002 +003 +004 +005 +006 +007 +008 +009 +010 +011 +012 +013 +014 +015 +016 +017 +018 +019 +020 +021 +022 +023 +024 +025 +026 +027 +028 +029 +030 +031 +032 +033 +034 +035 +036 +037 +038 +039 +040 +041 +042 +043 +044 +045 +046 +047 +048 +049 +050 +051 +052 diff --git a/tests/prose/fixtures/feature-implementation-started/snapshot/.workflows/_gitignore.fixture b/tests/prose/fixtures/feature-implementation-started/snapshot/.workflows/_gitignore.fixture new file mode 100644 index 000000000..779cbf28a --- /dev/null +++ b/tests/prose/fixtures/feature-implementation-started/snapshot/.workflows/_gitignore.fixture @@ -0,0 +1,2 @@ +.cache/ +.manifest.json.*.tmp diff --git a/tests/prose/fixtures/feature-implementation-started/snapshot/.workflows/manifest.json b/tests/prose/fixtures/feature-implementation-started/snapshot/.workflows/manifest.json new file mode 100644 index 000000000..987f6c236 --- /dev/null +++ b/tests/prose/fixtures/feature-implementation-started/snapshot/.workflows/manifest.json @@ -0,0 +1,7 @@ +{ + "work_units": { + "pay": { + "work_type": "feature" + } + } +} diff --git a/tests/prose/fixtures/feature-implementation-started/snapshot/.workflows/pay/discovery/sessions/session-001.md b/tests/prose/fixtures/feature-implementation-started/snapshot/.workflows/pay/discovery/sessions/session-001.md new file mode 100644 index 000000000..2aab96961 --- /dev/null +++ b/tests/prose/fixtures/feature-implementation-started/snapshot/.workflows/pay/discovery/sessions/session-001.md @@ -0,0 +1,39 @@ +# Discovery Session 001 + +Date: 2026-01-01 +Work unit: pay + +## Description (as of session) + +Accept card payments at checkout. + +## Seed + +(none) + +## Imports + +(none) + +## Map State at Start + +(n/a — single-topic work) + +## Exploration + +Shaped as a single feature: accept card payments at checkout using +the existing gateway account. Card-only for v1 came up early and was +softly agreed; wallet support was noted as a likely deferral. No +research need surfaced — routed straight to discussion. + +## Edits + +(none) + +## Topics Identified + +(none) + +## Conclusion + +(none) diff --git a/tests/prose/fixtures/feature-implementation-started/snapshot/.workflows/pay/discussion/pay.md b/tests/prose/fixtures/feature-implementation-started/snapshot/.workflows/pay/discussion/pay.md new file mode 100644 index 000000000..ec0cfc62f --- /dev/null +++ b/tests/prose/fixtures/feature-implementation-started/snapshot/.workflows/pay/discussion/pay.md @@ -0,0 +1,15 @@ +# Discussion — pay + +## Context + +Accept card payments at checkout using the existing gateway account. + +## Decisions + +- Use the existing gateway account; no new provider onboarding. +- Card-only for v1 — wallet support is deferred. +- Webhooks confirm capture; the checkout never polls. + +## Deferred + +- Wallet support (Apple/Google Pay) — revisit after v1. diff --git a/tests/prose/fixtures/feature-implementation-started/snapshot/.workflows/pay/manifest.json b/tests/prose/fixtures/feature-implementation-started/snapshot/.workflows/pay/manifest.json new file mode 100644 index 000000000..23023a119 --- /dev/null +++ b/tests/prose/fixtures/feature-implementation-started/snapshot/.workflows/pay/manifest.json @@ -0,0 +1,70 @@ +{ + "name": "pay", + "work_type": "feature", + "status": "in-progress", + "created": "2026-01-01", + "description": "Accept card payments at checkout", + "phases": { + "discussion": { + "items": { + "pay": { + "status": "completed" + } + } + }, + "specification": { + "items": { + "pay": { + "status": "completed", + "sources": { + "pay": { + "status": "incorporated" + } + } + } + } + }, + "planning": { + "items": { + "pay": { + "status": "completed", + "format": "local-markdown", + "task_list_gate_mode": "gated", + "author_gate_mode": "gated", + "finding_gate_mode": "gated", + "review_cycle": 0, + "phase": 1, + "task": null, + "task_map": { + "pay-1-1": "pay-1-1", + "pay-1-2": "pay-1-2" + }, + "storage_paths": [], + "approvals": { + "structure": "2026-01-01", + "tasks": { + "p1": "2026-01-01" + } + } + } + } + }, + "implementation": { + "items": { + "pay": { + "status": "in-progress", + "task_gate_mode": "gated", + "fix_gate_mode": "gated", + "analysis_gate_mode": "gated", + "fix_attempts": 0, + "analysis_cycle_total": 0, + "analysis_cycle_session": 0, + "linters": [], + "project_skills": [], + "current_phase": 1, + "current_task": "pay-1-1" + } + } + } + } +} diff --git a/tests/prose/fixtures/feature-implementation-started/snapshot/.workflows/pay/planning/pay/planning.md b/tests/prose/fixtures/feature-implementation-started/snapshot/.workflows/pay/planning/pay/planning.md new file mode 100644 index 000000000..9131d98c7 --- /dev/null +++ b/tests/prose/fixtures/feature-implementation-started/snapshot/.workflows/pay/planning/pay/planning.md @@ -0,0 +1,8 @@ +# Plan — pay + +## Phase 1: Payment core + +| Task | Title | +|------|-------| +| pay-1-1 | Create Payment Intent | +| pay-1-2 | Handle Capture Webhooks | diff --git a/tests/prose/fixtures/feature-implementation-started/snapshot/.workflows/pay/planning/pay/tasks/pay-1-1.md b/tests/prose/fixtures/feature-implementation-started/snapshot/.workflows/pay/planning/pay/tasks/pay-1-1.md new file mode 100644 index 000000000..f7d43a082 --- /dev/null +++ b/tests/prose/fixtures/feature-implementation-started/snapshot/.workflows/pay/planning/pay/tasks/pay-1-1.md @@ -0,0 +1,10 @@ +--- +id: pay-1-1 +phase: 1 +status: pending +created: 2026-01-01 +--- + +# Create Payment Intent + +Create a gateway payment intent when checkout begins and attach it to the order. diff --git a/tests/prose/fixtures/feature-implementation-started/snapshot/.workflows/pay/planning/pay/tasks/pay-1-2.md b/tests/prose/fixtures/feature-implementation-started/snapshot/.workflows/pay/planning/pay/tasks/pay-1-2.md new file mode 100644 index 000000000..8fb22697d --- /dev/null +++ b/tests/prose/fixtures/feature-implementation-started/snapshot/.workflows/pay/planning/pay/tasks/pay-1-2.md @@ -0,0 +1,10 @@ +--- +id: pay-1-2 +phase: 1 +status: pending +created: 2026-01-01 +--- + +# Handle Capture Webhooks + +Consume gateway capture webhooks and mark the order paid; no polling path. diff --git a/tests/prose/fixtures/feature-implementation-started/snapshot/.workflows/pay/specification/pay/specification.md b/tests/prose/fixtures/feature-implementation-started/snapshot/.workflows/pay/specification/pay/specification.md new file mode 100644 index 000000000..29a5f4322 --- /dev/null +++ b/tests/prose/fixtures/feature-implementation-started/snapshot/.workflows/pay/specification/pay/specification.md @@ -0,0 +1,11 @@ +# Specification — pay + +## Requirements + +- Checkout creates a payment intent against the existing gateway account. +- Card payments only; wallet flows are out of scope for v1. +- Capture is confirmed by gateway webhook, never by polling. + +## Out of scope + +- Wallet support (deferred by discussion). diff --git a/tests/prose/implementation-picks-first-task.md b/tests/prose/implementation-picks-first-task.md new file mode 100644 index 000000000..6771c4eb3 --- /dev/null +++ b/tests/prose/implementation-picks-first-task.md @@ -0,0 +1,48 @@ +## case: implementation-picks-first-task +- origin: feature mainline — entry through resume detection to the first task +- files: + - skills/workflow-implementation-entry/SKILL.md + - skills/workflow-implementation-entry/references/validate-phase.md + - skills/workflow-implementation-entry/references/validate-dependencies.md + - skills/workflow-implementation-entry/references/environment-check.md + - skills/workflow-implementation-process/SKILL.md + - skills/workflow-implementation-process/references/environment-setup.md + - skills/workflow-implementation-process/references/task-loop.md + - skills/workflow-planning-process/references/output-formats/local-markdown/reading.md + +### given + +world_before: feature-planned + +### when + +Execute skills/workflow-implementation-entry/SKILL.md with arguments +$0=feature, $1=pay, then continue into the processing skill it invokes. +Follow the flow until the prose directs you to begin building the first +task. Stop there — write no implementation code and no tests. + +answers: +1. none — no special setup is needed + +### then + +world_after: feature-implementation-started + +trace: +1. the plan gate renders empty, and no implementation item exists, so + this is a new entry +2. dependency validation returns immediately — external dependencies are + an epic concern +3. the environment check finds no setup document and asks the question, + gathering the answer without acting on it +4. resume detection initialises tracking and reports the created mode — + the fresh path, which commits the start of implementation, never the + resuming-from-a-previous-session note +5. environment setup records the answer as a setup document and commits + it, so the question is not asked again in a later session +6. reads the plan through the format's own reading procedure and finds + pay-1-1 next: phase one, first task, nothing completed +7. starts pay-1-1 and stops where the prose hands over to building + +notes: +- the second task is neither started nor touched diff --git a/tests/prose/investigation-entry-seeds-from-carrier.md b/tests/prose/investigation-entry-seeds-from-carrier.md new file mode 100644 index 000000000..7f6d0ad5e --- /dev/null +++ b/tests/prose/investigation-entry-seeds-from-carrier.md @@ -0,0 +1,34 @@ +## case: investigation-entry-seeds-from-carrier +- origin: bugfix mainline — a bug shaped in discovery is never re-interrogated +- files: + - skills/workflow-investigation-entry/SKILL.md + - skills/workflow-investigation-entry/references/invoke-skill.md + +### given + +world_before: bugfix-created + +### when + +Execute skills/workflow-investigation-entry/SKILL.md with arguments +$0=bugfix, $1=crash-fix. Follow it to the point where the handoff to the +processing skill is constructed; record the handoff block and stop. Do +not execute the processing skill's instructions. + +### then + +world_after: unchanged + +trace: +1. resolves the topic to the work unit, investigation being bugfix work +2. reads the investigation status, finds nothing, sets the source to new + and skips phase validation entirely +3. finds a discovery session log, so seeds the bug context from the + manifest description and that log — the context-gathering questions + are not asked +4. renders the phase note for the investigation phase with the verb + Starting, and emits it as produced +5. hands off to the investigation processing skill for crash-fix + +notes: +- the investigation file is created by the processing skill, not here diff --git a/tests/prose/lib/cases.cjs b/tests/prose/lib/cases.cjs index 8bd390985..d7af57e9e 100644 --- a/tests/prose/lib/cases.cjs +++ b/tests/prose/lib/cases.cjs @@ -2,48 +2,43 @@ // Case corpus: parsing and validation. // -// One case per file: tests/prose/{flow}/{case-id}.md, where the -// filename IS the id (validated, so a rename can never drift from the -// heading). A flow's README.md carries its prose intro and is ignored. -// Grammar, deterministic to parse: +// One case per file, flat: tests/prose/{case-id}.md, where the filename +// IS the id (validated, so a rename can never drift from the heading). +// Cases are Given-When-Then; nothing groups them but their `files:` +// scope, which is what selection actually runs on. // // ## case: {kebab-id} -// - world: {fixture-name} (optional; required by state expects) -// - origin: {free text} +// - origin: {why this case exists} // - files: // - {repo-relative path}[#heading fragment] // -// Anchors are substring fragments, matched against heading text with -// `includes` — `#Boot` matches "Step 0.2: Boot" and survives a -// renumber. Authoring rule (see README): cases name BEHAVIOUR, never -// coordinates — step numbers, arm letters, and heading numbering in -// walk or expect text rot on cosmetic edits and fail for the wrong -// reason. -// ### walk -// {free text: entry point, what to do, stop condition} -// ### user -// 1. {scripted answer, consumed in order} -// ### stub -// {free text: what to do INSTEAD of dispatching a real sub-agent} -// ### expect -// - routing: {claim graded by an agent against the walk transcript} -// - state: {deterministic assertion — see parseStateAssertion} +// ### given +// world_before: {fixture name} +// stubs: +// - {stub name}: {when it fires — the case owns the trigger} // -// `stub` is optional and free-form. Prose that dispatches background -// agents is walked with those agents stubbed: the walker plays the -// agent (writes the report the prose expects at the path the engine -// allocated, returns the stated result) instead of spending a real -// sub-agent on analysis of fixture content. The lifecycle around the -// agent — dispatch, scan, incorporate, the verdict's routing — is what -// the case tests; the agent's own judgment is not. +// ### when +// {free text: where to enter, what to follow, where to stop} +// answers: +// 1. {scripted user response, consumed in order} // -// State assertion grammar (executed by the runner, zero model judgment): -// file exists {rel} -// file absent {rel} -// manifest exists {dotpath} {field} -// manifest absent {dotpath} {field} -// manifest equals {dotpath} {field} {expected printed value} -// json {rel} {dot.path} {expected value} +// ### then +// world_after: {fixture name | unchanged} +// trace: +// 1. {behavioural step the walk should have taken} +// notes: +// - {any further claim about the walk} +// +// The `when` stays coarse on purpose — one instruction, never a +// step-by-step script. The walker must DERIVE the path from the prose; +// that derivation is the thing under test. Granularity lives in `then`, +// where a step-by-step expected trace catches a walker that silently +// course-corrected around broken prose. +// +// Anchors are substring fragments matched against heading text +// (`#Boot` matches "Step 0.2: Boot") so a renumber can't break a case. +// Claims name BEHAVIOUR, never coordinates — step numbers and arm +// letters rot on cosmetic edits and fail for the wrong reason. const fs = require('fs'); const path = require('path'); @@ -51,188 +46,186 @@ const path = require('path'); const ROOT = path.join(__dirname, '../../..'); const PROSE_DIR = path.join(ROOT, 'tests/prose'); const FIXTURES_DIR = path.join(PROSE_DIR, 'fixtures'); -const RESERVED_DIRS = new Set(['lib', 'fixtures']); +const STUBS_DIR = path.join(PROSE_DIR, 'stubs'); +const UNCHANGED = 'unchanged'; function listCaseFiles() { if (!fs.existsSync(PROSE_DIR)) return []; - const files = []; - for (const flow of fs.readdirSync(PROSE_DIR, { withFileTypes: true })) { - if (!flow.isDirectory() || RESERVED_DIRS.has(flow.name)) continue; - const flowDir = path.join(PROSE_DIR, flow.name); - for (const entry of fs.readdirSync(flowDir)) { - if (entry.endsWith('.md') && entry !== 'README.md') { - files.push({ flow: flow.name, file: path.join(flowDir, entry) }); - } - } - } - return files; -} - -function parseStateAssertion(text) { - const fileMatch = text.match(/^file (exists|absent) (\S+)$/); - if (fileMatch) { - return { kind: `file-${fileMatch[1]}`, path: fileMatch[2] }; - } - const mExists = text.match(/^manifest (exists|absent) (\S+) (\S+)$/); - if (mExists) { - return { kind: `manifest-${mExists[1]}`, dotpath: mExists[2], field: mExists[3] }; - } - const mEquals = text.match(/^manifest equals (\S+) (\S+) (.+)$/); - if (mEquals) { - return { kind: 'manifest-equals', dotpath: mEquals[1], field: mEquals[2], value: mEquals[3] }; - } - const json = text.match(/^json (\S+) (\S+) (.+)$/); - if (json) { - return { kind: 'json-equals', path: json[1], pointer: json[2], value: json[3] }; - } - return { error: `unparseable state assertion: "${text}"` }; + return fs.readdirSync(PROSE_DIR, { withFileTypes: true }) + .filter((e) => e.isFile() && e.name.endsWith('.md') && e.name !== 'README.md') + .map((e) => path.join(PROSE_DIR, e.name)); } -function parseCaseFile(file, flow) { +function parseCaseFile(file) { const lines = fs.readFileSync(file, 'utf8').split('\n'); - const cases = []; - let current = null; - let section = null; // null | 'walk' | 'user' | 'expect' - let inFiles = false; - - const push = () => { - if (current) { - current.walk = current.walk.join('\n').trim(); - current.stub = current.stub.join('\n').trim(); - cases.push(current); - } + const c = { + id: null, file: path.relative(ROOT, file), origin: null, files: [], + worldBefore: null, stubs: [], when: [], answers: [], + worldAfter: null, trace: [], notes: [], }; + let section = null; // null | given | when | then + let list = null; // files | stubs | answers | trace | notes for (const line of lines) { - const caseStart = line.match(/^## case:\s*(\S+)\s*$/); - if (caseStart) { - push(); - current = { - id: caseStart[1], flow, file: path.relative(ROOT, file), - world: null, origin: null, files: [], - walk: [], user: [], stub: [], expect: [], - }; + const head = line.match(/^## case:\s*(\S+)\s*$/); + if (head) { + c.id = head[1]; section = null; - inFiles = false; + list = null; continue; } - if (!current) continue; - - const sub = line.match(/^### (walk|user|stub|expect)\s*$/); + const sub = line.match(/^### (given|when|then)\s*$/); if (sub) { section = sub[1]; - inFiles = false; - continue; - } - if (/^#/.test(line)) { // any other heading ends the current case - push(); - current = null; + list = null; continue; } if (section === null) { - const field = line.match(/^- (world|origin):\s*(.*)$/); - if (field) { - current[field[1]] = field[2].trim(); - inFiles = false; - continue; - } - if (/^- files:\s*$/.test(line)) { - inFiles = true; - continue; - } - const fileEntry = line.match(/^\s+- (.+)$/); - if (inFiles && fileEntry) { - const [, spec] = fileEntry; + const origin = line.match(/^- origin:\s*(.*)$/); + if (origin) { c.origin = origin[1].trim(); list = null; continue; } + if (/^- files:\s*$/.test(line)) { list = 'files'; continue; } + const item = line.match(/^\s+- (.+)$/); + if (list === 'files' && item) { + const spec = item[1].trim(); const hash = spec.indexOf('#'); - current.files.push(hash === -1 - ? { path: spec.trim(), anchor: null } + c.files.push(hash === -1 + ? { path: spec, anchor: null } : { path: spec.slice(0, hash).trim(), anchor: spec.slice(hash + 1).trim() }); continue; } - if (line.trim() !== '') inFiles = false; - } else if (section === 'walk') { - current.walk.push(line); - } else if (section === 'stub') { - current.stub.push(line); - } else if (section === 'user') { + if (line.trim() !== '') list = null; + continue; + } + + if (section === 'given') { + const wb = line.match(/^world_before:\s*(\S+)\s*$/); + if (wb) { c.worldBefore = wb[1]; list = null; continue; } + if (/^stubs:\s*$/.test(line)) { list = 'stubs'; continue; } + const stub = line.match(/^\s+- ([a-z0-9][a-z0-9-]*):\s*(.+)$/); + if (list === 'stubs' && stub) { + c.stubs.push({ name: stub[1], trigger: stub[2].trim() }); + continue; + } + continue; + } + + if (section === 'when') { + if (/^answers:\s*$/.test(line)) { list = 'answers'; continue; } const answer = line.match(/^\d+\.\s+(.*)$/); - if (answer) current.user.push(answer[1].trim()); - } else if (section === 'expect') { - const claim = line.match(/^- (routing|state):\s*(.*)$/); - if (claim) current.expect.push({ kind: claim[1], text: claim[2].trim() }); + if (list === 'answers' && answer) { c.answers.push(answer[1].trim()); continue; } + if (list !== 'answers') c.when.push(line); + continue; + } + + // then + const wa = line.match(/^world_after:\s*(\S+)\s*$/); + if (wa) { c.worldAfter = wa[1]; list = null; continue; } + if (/^trace:\s*$/.test(line)) { list = 'trace'; continue; } + if (/^notes:\s*$/.test(line)) { list = 'notes'; continue; } + const step = line.match(/^\d+\.\s+(.*)$/); + if (list === 'trace' && step) { c.trace.push(step[1].trim()); continue; } + if (list === 'trace' && /^\s+\S/.test(line) && c.trace.length) { + c.trace[c.trace.length - 1] += ` ${line.trim()}`; + continue; + } + const note = line.match(/^- (.*)$/); + if (list === 'notes' && note) { c.notes.push(note[1].trim()); continue; } + if (list === 'notes' && /^\s+\S/.test(line) && c.notes.length) { + c.notes[c.notes.length - 1] += ` ${line.trim()}`; } } - push(); - return cases; + + c.when = c.when.join('\n').trim(); + return c; } function loadAllCases() { - return listCaseFiles().flatMap(({ flow, file }) => parseCaseFile(file, flow)); + return listCaseFiles().map(parseCaseFile); +} + +function listStubs() { + if (!fs.existsSync(STUBS_DIR)) return []; + return fs.readdirSync(STUBS_DIR) + .filter((f) => f.endsWith('.md') && f !== 'README.md') + .map((f) => path.basename(f, '.md')); +} + +/** A stub file is description above a `---` fence, exact bytes below. */ +function readStub(name) { + const file = path.join(STUBS_DIR, `${name}.md`); + if (!fs.existsSync(file)) throw new Error(`no stub "${name}" in tests/prose/stubs`); + const raw = fs.readFileSync(file, 'utf8'); + const fence = raw.indexOf('\n---\n'); + if (fence === -1) throw new Error(`stub "${name}" has no --- fence separating description from content`); + return { + name, + description: raw.slice(0, fence).trim(), + content: raw.slice(fence + 5).replace(/^\n+/, ''), + }; } function headingExists(absPath, anchor) { - const content = fs.readFileSync(absPath, 'utf8'); - return content.split('\n').some((line) => { + return fs.readFileSync(absPath, 'utf8').split('\n').some((line) => { const h = line.match(/^#{1,6}\s+(.*?)\s*$/); return h && h[1].includes(anchor); }); } +function fixtureExists(name) { + return fs.existsSync(path.join(FIXTURES_DIR, name, 'recipe.cjs')); +} + function validateCorpus(cases) { const errors = []; const seen = new Set(); - const perFile = new Map(); - for (const c of cases) perFile.set(c.file, (perFile.get(c.file) || 0) + 1); - - for (const [file, count] of perFile) { - if (count > 1) { - errors.push(`${file}: holds ${count} cases — one case per file, named {case-id}.md`); - } - } + const stubs = new Set(listStubs()); for (const c of cases) { - const at = `${c.file} :: ${c.id}`; + const at = c.file; + if (!c.id) { errors.push(`${at}: no "## case:" heading`); continue; } if (!/^[a-z0-9][a-z0-9-]*$/.test(c.id)) errors.push(`${at}: id is not kebab-case`); - if (seen.has(c.id)) errors.push(`${at}: duplicate case id`); + if (seen.has(c.id)) errors.push(`${at}: duplicate case id "${c.id}"`); seen.add(c.id); + if (path.basename(c.file, '.md') !== c.id) { + errors.push(`${at}: filename must equal the case id (expected ${c.id}.md)`); + } - const stem = path.basename(c.file, '.md'); - if (stem !== c.id) errors.push(`${at}: filename must equal the case id (expected ${c.id}.md)`); - + if (!c.origin) errors.push(`${at}: no origin`); if (c.files.length === 0) errors.push(`${at}: no files: scope`); for (const f of c.files) { const abs = path.join(ROOT, f.path); - if (!fs.existsSync(abs)) { - errors.push(`${at}: scoped file missing: ${f.path}`); - } else if (f.anchor && !headingExists(abs, f.anchor)) { + if (!fs.existsSync(abs)) errors.push(`${at}: scoped file missing: ${f.path}`); + else if (f.anchor && !headingExists(abs, f.anchor)) { errors.push(`${at}: anchor "#${f.anchor}" not a heading in ${f.path}`); } } - if (c.world !== null) { - if (!fs.existsSync(path.join(FIXTURES_DIR, c.world, 'recipe.cjs'))) { - errors.push(`${at}: world "${c.world}" has no fixture recipe`); + if (c.worldBefore && !fixtureExists(c.worldBefore)) { + errors.push(`${at}: world_before "${c.worldBefore}" has no fixture recipe`); + } + if (!c.worldAfter) { + errors.push(`${at}: no world_after (use "unchanged" when the walk mutates nothing)`); + } else if (c.worldAfter !== UNCHANGED) { + if (!c.worldBefore) errors.push(`${at}: world_after names a fixture but there is no world_before`); + if (!fixtureExists(c.worldAfter)) { + errors.push(`${at}: world_after "${c.worldAfter}" has no fixture recipe`); } } - if (!c.walk) errors.push(`${at}: empty walk`); - if (c.expect.length === 0) errors.push(`${at}: no expects`); - for (const e of c.expect) { - if (!e.text) errors.push(`${at}: empty ${e.kind} expect`); - if (e.kind === 'state') { - if (c.world === null) { - errors.push(`${at}: state expect requires a world`); - } - const parsed = parseStateAssertion(e.text); - if (parsed.error) errors.push(`${at}: ${parsed.error}`); - } + for (const s of c.stubs) { + if (!stubs.has(s.name)) errors.push(`${at}: no stub "${s.name}" in tests/prose/stubs`); + if (!s.trigger) errors.push(`${at}: stub "${s.name}" declares no trigger — the case owns the moment`); + if (!c.worldBefore) errors.push(`${at}: stub "${s.name}" needs a world to fire in`); } + + if (!c.when) errors.push(`${at}: empty when`); + if (c.trace.length === 0) errors.push(`${at}: no trace — then must state the expected path step by step`); } return errors; } module.exports = { - ROOT, PROSE_DIR, FIXTURES_DIR, - listCaseFiles, parseCaseFile, loadAllCases, parseStateAssertion, validateCorpus, + ROOT, PROSE_DIR, FIXTURES_DIR, STUBS_DIR, UNCHANGED, + listCaseFiles, parseCaseFile, loadAllCases, listStubs, readStub, validateCorpus, }; diff --git a/tests/prose/lib/fixtures.cjs b/tests/prose/lib/fixtures.cjs index 9c294cdc5..5892e1ac8 100644 --- a/tests/prose/lib/fixtures.cjs +++ b/tests/prose/lib/fixtures.cjs @@ -101,6 +101,11 @@ function excluded(rel) { if (parts.includes('.git')) return true; if (rel === path.join('.workflows', '.knowledge')) return true; if (rel.startsWith(path.join('.workflows', '.knowledge') + path.sep)) return true; + // Installed skills/agents: copied into live worlds by the world builder, + // never part of a snapshot. Excluding them lets an acted world be + // compared against an expected snapshot on equal terms. + if (rel.startsWith(path.join('.claude', 'skills') + path.sep)) return true; + if (rel.startsWith(path.join('.claude', 'agents') + path.sep)) return true; return false; } @@ -189,8 +194,48 @@ function materialiseSnapshot(name, destDir) { } } +/** Unified diff of two buffers, via git — no diff algorithm of our own. */ +function unifiedDiff(label, expectedBuf, actualBuf) { + const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'prose-diff-')); + try { + const a = path.join(dir, 'expected'); + const b = path.join(dir, 'actual'); + fs.writeFileSync(a, expectedBuf); + fs.writeFileSync(b, actualBuf); + const res = spawnSync('git', ['diff', '--no-index', '--unified=3', '--no-color', a, b], + { encoding: 'utf8', env: { ...process.env, GIT_CONFIG_GLOBAL: '/dev/null', GIT_CONFIG_SYSTEM: '/dev/null' } }); + const body = (res.stdout || '').split('\n').slice(4).join('\n').trimEnd(); + return `--- ${label}\n${body}`; + } finally { + fs.rmSync(dir, { recursive: true, force: true }); + } +} + +/** + * Compare a live (acted) world against a committed snapshot. Returns the + * factual delta — what a walk actually did to the world — for the + * asserting agent to classify. No normalisation: volatile fields (times, + * SHAs) surface as ordinary differences and the agent rules on them. + */ +function diffWorldAgainstSnapshot(worldDir, snapshotName) { + const expected = readSnapshot(snapshotName); + if (expected === null) throw new Error(`fixture "${snapshotName}" has no committed snapshot`); + const actual = collectTree(worldDir); + + const added = []; + const removed = []; + const changed = []; + for (const [rel, buf] of actual) { + if (!expected.has(rel)) added.push(rel); + else if (!expected.get(rel).equals(buf)) changed.push(unifiedDiff(rel, expected.get(rel), buf)); + } + for (const rel of expected.keys()) if (!actual.has(rel)) removed.push(rel); + + return { snapshot: snapshotName, added, removed, changed, identical: !added.length && !removed.length && !changed.length }; +} + module.exports = { ROOT, ENGINE, KNOWLEDGE, FIXTURES_DIR, listFixtures, runRecipe, collectTree, readSnapshot, - writeSnapshot, compareSnapshot, materialiseSnapshot, + writeSnapshot, compareSnapshot, materialiseSnapshot, diffWorldAgainstSnapshot, }; diff --git a/tests/prose/planning-entry-asks-for-late-context.md b/tests/prose/planning-entry-asks-for-late-context.md new file mode 100644 index 000000000..be116cb3b --- /dev/null +++ b/tests/prose/planning-entry-asks-for-late-context.md @@ -0,0 +1,39 @@ +## case: planning-entry-asks-for-late-context +- origin: feature mainline — planning offers a last chance to add context +- files: + - skills/workflow-planning-entry/SKILL.md + - skills/workflow-planning-entry/references/validate-spec.md + - skills/workflow-planning-entry/references/validate-phase.md + - skills/workflow-planning-entry/references/cross-cutting-context.md + +### given + +world_before: feature-specified + +### when + +Execute skills/workflow-planning-entry/SKILL.md with arguments +$0=feature, $1=pay. Follow it to the point where the handoff to the +processing skill is constructed; record the handoff block and stop. Do +not execute the processing skill's instructions. + +answers: +1. continue — nothing has changed since the specification + +### then + +world_after: unchanged + +trace: +1. the specification gate renders empty — the spec is complete and the + topic is clear to plan +2. no plan exists, so the fresh-start arm asks whether anything has + changed since the specification, and waits +3. on continue, the source is fresh and carries no additional context +4. cross-cutting context is gathered regardless of work type: it finds + no cross-cutting work units, then queries the knowledge base for + completed cross-cutting specs +5. hands off to the planning processing skill for pay + +notes: +- the knowledge base is queried, never written to, by an entry skill diff --git a/tests/prose/review-entry-fresh.md b/tests/prose/review-entry-fresh.md new file mode 100644 index 000000000..48b49b7ce --- /dev/null +++ b/tests/prose/review-entry-fresh.md @@ -0,0 +1,31 @@ +## case: review-entry-fresh +- origin: feature mainline — review refuses to start until delivery is complete +- files: + - skills/workflow-review-entry/SKILL.md + - skills/workflow-review-entry/references/validate-phase.md + +### given + +world_before: feature-implemented + +### when + +Execute skills/workflow-review-entry/SKILL.md with arguments $0=feature, +$1=pay. Follow it to the point where the handoff to the processing skill +is constructed; record the handoff block and stop. Do not execute the +processing skill's instructions, and dispatch no agents. + +### then + +world_after: unchanged + +trace: +1. resolves the topic to the work unit +2. the prerequisite gate renders empty — both the plan and the + implementation are complete, so nothing blocks entry +3. no review item exists, so the fresh path is taken — nothing is + reopened and no resume is announced +4. hands off to the review processing skill for pay + +notes: +- no verifier agents are dispatched by the entry skill diff --git a/tests/prose/root-cause-validation-clean-verdict.md b/tests/prose/root-cause-validation-clean-verdict.md new file mode 100644 index 000000000..bbc845741 --- /dev/null +++ b/tests/prose/root-cause-validation-clean-verdict.md @@ -0,0 +1,42 @@ +## case: root-cause-validation-clean-verdict +- origin: bugfix mainline — the validation agent's lifecycle on a clean verdict +- files: + - skills/workflow-investigation-process/references/root-cause-validation.md + +### given + +world_before: bugfix-investigating +stubs: + - root-cause-validated: when the prose dispatches the validation agent + +### when + +The investigation is open with its root cause documented, and the +process has reached its root-cause validation step. Walk +root-cause-validation.md from the top, as the processing skill would. +Stop when the reference returns to its caller. + +answers: +1. yes — run root cause validation + +### then + +world_after: bugfix-validation-incorporated + +trace: +1. offers the independent validation, presents the run-or-skip choice, + and waits +2. on yes, records the dispatch through the engine and creates no report + file of its own — the response carries the id and the path to use +3. announces that validation is running, then dispatches exactly one + agent, synchronously +4. once the report has landed, promotes the row with a scan and closes + it with an incorporate — the verdict is consumed whole, never + surfaced finding by finding +5. reads the report and, the verdict being validated, states the + confidence and returns to its caller — the gap-handling choice is + never reached + +notes: +- the investigation file is untouched and nothing is committed: the + validated path has no gaps to record diff --git a/tests/prose/run.cjs b/tests/prose/run.cjs index b3ab7e9fa..abecfe80d 100644 --- a/tests/prose/run.cjs +++ b/tests/prose/run.cjs @@ -2,22 +2,22 @@ 'use strict'; // Prose-test runner — everything deterministic about a prose-test run. -// The /prose-test skill drives this CLI and supplies the model layer -// (walker and grader agents). See design/prose-tests.md. +// The /prose-test skill drives this CLI and supplies the two agents: one +// walks the prose, one asserts the result. See design/prose-tests.md. // // list corpus table // select [--diff |--all|--cases a,b] cases to run, as JSON -// world materialise the case's world, print path -// prompt [--world ] walker prompt (NEVER contains expects) -// grade [--world ] run state assertions; emit routing -// claims for the grader agent +// world materialise world_before, print path +// prompt --world walker prompt (NEVER contains `then`) +// diff --world acted world vs world_after, as facts +// assert --world the asserting agent's prompt (diff + then) // snap (re)generate a fixture's golden snapshot -// verify [fixture] rebuild-compare fixture(s) against snapshot +// verify [fixture] rebuild-compare fixture(s) // destroy --world remove a world const fs = require('fs'); const path = require('path'); -const { spawnSync, execFileSync } = require('child_process'); +const { execFileSync } = require('child_process'); const cases = require('./lib/cases.cjs'); const fixtures = require('./lib/fixtures.cjs'); @@ -31,8 +31,7 @@ function die(msg) { } function loadCase(id) { - const all = cases.loadAllCases(); - const found = all.find((c) => c.id === id); + const found = cases.loadAllCases().find((c) => c.id === id); if (!found) die(`no case "${id}" in the corpus`); return found; } @@ -44,14 +43,21 @@ function flag(argv, name) { return argv[i + 1]; } +function requireWorld(argv, c) { + const dir = flag(argv, '--world'); + if (!dir) die(`case "${c.id}" needs --world (build one: run.cjs world ${c.id})`); + return dir; +} + // --- select --------------------------------------------------------------- function changedFiles(ref) { const run = (args) => execFileSync('git', args, { cwd: ROOT, encoding: 'utf8' }) .split('\n').map((l) => l.trim()).filter(Boolean); - const diff = run(['diff', '--name-only', ref]); - const status = run(['status', '--porcelain']).map((l) => l.slice(3).trim()); - return new Set([...diff, ...status]); + return new Set([ + ...run(['diff', '--name-only', ref]), + ...run(['status', '--porcelain']).map((l) => l.slice(3).trim()), + ]); } function cmdSelect(argv) { @@ -71,33 +77,30 @@ function cmdSelect(argv) { const ref = flag(argv, '--diff') || 'main'; mode = `diff:${ref}`; const changed = changedFiles(ref); - const touches = (c) => { - if (changed.has(c.file)) return true; - if (c.files.some((f) => changed.has(f.path))) return true; - if (c.world !== null) { - const prefix = path.posix.join('tests/prose/fixtures', c.world) + '/'; - for (const p of changed) if (p.startsWith(prefix)) return true; - } - return false; - }; - selected = all.filter(touches); + selected = all.filter((c) => changed.has(c.file) + || c.files.some((f) => changed.has(f.path)) + || [c.worldBefore, c.worldAfter].filter(Boolean).some((w) => + [...changed].some((p) => p.startsWith(`tests/prose/fixtures/${w}/`))) + || c.stubs.some((s) => changed.has(`tests/prose/stubs/${s.name}.md`))); } - process.stdout.write(JSON.stringify({ + process.stdout.write(`${JSON.stringify({ mode, - cases: selected.map((c) => ({ id: c.id, flow: c.flow, file: c.file, world: c.world })), - }, null, 2) + '\n'); + cases: selected.map((c) => ({ + id: c.id, file: c.file, world_before: c.worldBefore, world_after: c.worldAfter, + })), + }, null, 2)}\n`); } // --- world / destroy ------------------------------------------------------ function cmdWorld(argv) { const c = loadCase(argv[0] || die('usage: world ')); - if (c.world === null) { - process.stdout.write(JSON.stringify({ world: null, note: 'structure-only case — no world' }) + '\n'); + if (!c.worldBefore) { + process.stdout.write(`${JSON.stringify({ world: null, note: 'structure-only case — no world' })}\n`); return; } - const dir = world.buildWorld(c.world); - process.stdout.write(JSON.stringify({ world: dir, fixture: c.world }) + '\n'); + const dir = world.buildWorld(c.worldBefore); + process.stdout.write(`${JSON.stringify({ world: dir, fixture: c.worldBefore })}\n`); } function cmdDestroy(argv) { @@ -106,20 +109,15 @@ function cmdDestroy(argv) { process.stdout.write(`destroyed ${dir}\n`); } -// --- prompt --------------------------------------------------------------- +// --- prompt (the walker) -------------------------------------------------- function cmdPrompt(argv) { const c = loadCase(argv[0] || die('usage: prompt [--world ]')); - const worldDir = flag(argv, '--world'); - if (c.world !== null && !worldDir) die(`case "${c.id}" needs --world (build one: run.cjs world ${c.id})`); - - const userScript = c.user.length - ? c.user.map((a, i) => ` ${i + 1}. ${a}`).join('\n') - : ' (none — the walk should reach its stop condition without user questions)'; + const worldDir = c.worldBefore ? requireWorld(argv, c) : null; - const setting = c.world !== null + const setting = worldDir ? [ - 'You are executing this project\'s workflow prose exactly as a live session would.', + "You are executing this project's workflow prose exactly as a live session would.", '', `Project directory — your cwd for EVERY command: ${worldDir}`, 'The workflow skills are installed at .claude/skills/ inside that project.', @@ -127,39 +125,61 @@ function cmdPrompt(argv) { ] : [ 'You are walking workflow prose structurally: read the named files and trace the logic.', - 'Execute nothing — no commands, no writes. This is a read-only walk of the repository.', + 'Execute nothing — no commands, no writes. This is a read-only walk.', '', `Repository root: ${ROOT}`, ]; - const stubs = c.stub - ? ['', 'STUBS — where the prose dispatches a background agent, do NOT dispatch one.', - 'Play the agent yourself exactly as described, then continue the prose:', '', c.stub] - : []; + const answers = c.answers.length + ? c.answers.map((a, i) => ` ${i + 1}. ${a}`).join('\n') + : ' (none — the walk should reach its stop condition without user questions)'; - const prompt = [ + const substitutions = c.stubs.length ? [ + '', + 'HARNESS SUBSTITUTIONS — these are NOT part of the process you are walking.', + 'They stand in for steps this framework deliberately does not simulate.', + 'Apply each only at the moment stated, then resume the prose exactly where', + 'you left it, and record `SUBSTITUTED: ` in the transcript.', + '', + ...c.stubs.flatMap((s) => { + const stub = cases.readStub(s.name); + return [ + `### ${s.name}`, + `WHEN: ${s.trigger}`, + `WHAT IT IS: ${stub.description.replace(/^#.*\n/, '').trim()}`, + 'CONTENT (write these exact bytes where the substitution calls for a file):', + ...stub.content.split('\n').map((l) => ` ${l}`), + '', + ]; + }), + ] : []; + + process.stdout.write(`${[ ...setting, '', 'TASK', - c.walk, + c.when, '', 'SCOPE — the prose under walk:', ...c.files.map((f) => ` - ${f.path}${f.anchor ? ` (start at the heading containing "${f.anchor}")` : ''}`), - ...stubs, + ...substitutions, '', 'RULES', '- Follow the prose literally, step by step, arm by arm. Where it names an', - ' engine or knowledge call, run it (node .claude/skills/workflow-engine/', - ' scripts/engine.cjs …) from the project directory and use the real response.', + ' engine or knowledge call, run it from the project directory and use the', + ' real response to decide which arm applies.', '- You also play the user, from a fixed script. When the prose presents a', ' menu or question, consume the next scripted answer, in order:', - userScript, + answers, '- If the prose asks a question and the script has no next answer: STOP and', ' record `UNSCRIPTED QUESTION:` with the exact question text.', '- If two arms both appear to match, record `AMBIGUOUS:` naming both, then', - ' follow the one the prose\'s own ordering/guard rules select.', - '- Never repair, reinterpret, or improve the prose. Execute what is written,', - ' even where it looks wrong. You are a probe, not a reviewer.', + " follow the one the prose's own ordering/guard rules select.", + '- If the prose cannot be followed literally — a step that contradicts the', + ' state, a missing file it assumes, an instruction that cannot be executed —', + ' record `DEVIATION:` with what you could not do, then continue as best you', + ' can. NEVER silently repair, reinterpret, or improve the prose. You are a', + ' probe, not a reviewer: a broken instruction is the finding.', '- Stop at the TASK\'s stop condition, the end of the flow, an UNSCRIPTED', ' QUESTION, or a hard error — whichever comes first.', '', @@ -168,75 +188,77 @@ function cmdPrompt(argv) { ' guard line that selected it.', '2. Every command run and the first line of its output.', '3. Every menu/question encountered (verbatim) and the scripted answer used.', - '4. Every file written or edited (path only).', + '4. Every file written or edited (path only), and every SUBSTITUTED marker.', '5. Finally: `STOPPED: `.', 'Return nothing but the transcript.', - ].join('\n'); - - process.stdout.write(prompt + '\n'); + ].join('\n')}\n`); } -// --- grade ---------------------------------------------------------------- +// --- diff (the facts) ----------------------------------------------------- -function engineInWorld(worldDir, args) { - const engine = path.join(worldDir, '.claude/skills/workflow-engine/scripts/engine.cjs'); - return spawnSync('node', [engine, ...args], { cwd: worldDir, encoding: 'utf8' }); +function worldDelta(c, worldDir) { + const target = c.worldAfter === cases.UNCHANGED ? c.worldBefore : c.worldAfter; + return fixtures.diffWorldAgainstSnapshot(worldDir, target); } -function runStateAssertion(worldDir, assertion) { - const a = cases.parseStateAssertion(assertion); - if (a.error) return { assertion, pass: false, actual: a.error }; - if (a.kind === 'file-exists' || a.kind === 'file-absent') { - const present = fs.existsSync(path.join(worldDir, a.path)); - const want = a.kind === 'file-exists'; - return { assertion, pass: present === want, actual: present ? 'present' : 'absent' }; +function cmdDiff(argv) { + const c = loadCase(argv[0] || die('usage: diff --world ')); + if (!c.worldBefore) { + process.stdout.write(`${JSON.stringify({ note: 'structure-only case — no world to diff' }, null, 2)}\n`); + return; } - if (a.kind === 'json-equals') { - const full = path.join(worldDir, a.path); - if (!fs.existsSync(full)) return { assertion, pass: false, actual: 'file absent' }; - let doc; - try { - doc = JSON.parse(fs.readFileSync(full, 'utf8')); - } catch (e) { - return { assertion, pass: false, actual: `unparseable JSON: ${e.message}` }; - } - const found = a.pointer.split('.').reduce((node, key) => ( - node === undefined || node === null ? undefined : node[key] - ), doc); - const actual = found === undefined ? '' : String(found); - return { assertion, pass: actual === a.value, actual }; + process.stdout.write(`${JSON.stringify(worldDelta(c, requireWorld(argv, c)), null, 2)}\n`); +} + +// --- assert (the judging agent) ------------------------------------------- + +function cmdAssert(argv) { + const c = loadCase(argv[0] || die('usage: assert [--world ]')); + const lines = [ + 'You are asserting the result of a prose-test case. You did not perform', + 'the walk. Judge only what the evidence shows.', + '', + 'EXPECTED TRACE — the path the prose should have taken:', + ...c.trace.map((t, i) => ` ${i + 1}. ${t}`), + ]; + if (c.notes.length) { + lines.push('', 'FURTHER CLAIMS:', ...c.notes.map((n) => ` - ${n}`)); } - if (a.kind === 'manifest-exists' || a.kind === 'manifest-absent') { - const res = engineInWorld(worldDir, ['manifest', 'exists', a.dotpath, a.field]); - if (res.status !== 0) return { assertion, pass: false, actual: `engine refused: ${res.stderr.trim()}` }; - const present = res.stdout.trim() === 'true'; - const want = a.kind === 'manifest-exists'; - return { assertion, pass: present === want, actual: present ? 'present' : 'absent' }; + + if (c.worldBefore) { + const delta = worldDelta(c, requireWorld(argv, c)); + lines.push( + '', + c.worldAfter === cases.UNCHANGED + ? 'EXPECTED WORLD: unchanged — the walk should have left the project exactly as it found it.' + : `EXPECTED WORLD: the committed fixture "${c.worldAfter}".`, + '', + 'WORLD DELTA — the factual difference between the world after the walk and', + 'the expected world, computed by code. Volatile values (timestamps, git', + 'SHAs, engine-allocated ids) are NOT normalised: a difference in one of', + 'those is expected and immaterial. A difference in shape, field presence,', + 'status vocabulary, or content is material.', + '', + JSON.stringify({ added: delta.added, removed: delta.removed, identical: delta.identical }, null, 2), + ...delta.changed, + ); } - // manifest-equals - const res = engineInWorld(worldDir, ['manifest', 'get', a.dotpath, a.field]); - if (res.status !== 0) return { assertion, pass: false, actual: `engine refused: ${res.stderr.trim()}` }; - const actual = res.stdout.trim(); - return { assertion, pass: actual === a.value, actual }; -} -function cmdGrade(argv) { - const c = loadCase(argv[0] || die('usage: grade [--world ]')); - const worldDir = flag(argv, '--world'); - const stateExpects = c.expect.filter((e) => e.kind === 'state'); - if (stateExpects.length > 0 && !worldDir) die(`case "${c.id}" has state expects — --world required`); - - const state = stateExpects.map((e) => runStateAssertion(worldDir, e.text)); - const routing = c.expect.filter((e) => e.kind === 'routing').map((e) => e.text); - process.stdout.write(JSON.stringify({ - id: c.id, - state, - statePass: state.every((s) => s.pass), - routing, - note: routing.length - ? 'routing claims are graded by an agent against the walk transcript — PASS requires quoted evidence' - : 'no routing claims', - }, null, 2) + '\n'); + lines.push( + '', + 'THE TRANSCRIPT of the walk is supplied separately by the caller.', + '', + 'VERDICT — return exactly this, nothing else:', + '1. Trace: one line per expected step — PASS or FAIL, each PASS quoting the', + ' transcript line that shows it. A PASS with no quote is invalid.', + '2. World: PASS or FAIL. Enumerate EVERY difference in the delta and classify', + ' each as volatile (immaterial) or material. Any material difference fails.', + ' An empty delta passes.', + '3. Markers: list every UNSCRIPTED QUESTION, AMBIGUOUS, and DEVIATION in the', + ' transcript. Each is a finding in its own right, even if everything passed.', + '4. VERDICT: PASS or FAIL, then one sentence on what it means for the prose.', + ); + process.stdout.write(`${lines.join('\n')}\n`); } // --- snap / verify -------------------------------------------------------- @@ -258,16 +280,15 @@ function cmdVerify(argv) { for (const name of names) { const scratch = fixtures.runRecipe(name); try { - const diff = fixtures.compareSnapshot(name, scratch); - const clean = !diff.missing.length && !diff.extra.length && !diff.changed.length; - if (clean) { + const d = fixtures.compareSnapshot(name, scratch); + if (!d.missing.length && !d.extra.length && !d.changed.length) { process.stdout.write(`${name}: snapshot current\n`); } else { failed = true; process.stdout.write(`${name}: DRIFT — the recipe no longer rebuilds the snapshot\n`); - for (const f of diff.changed) process.stdout.write(` changed: ${f}\n`); - for (const f of diff.extra) process.stdout.write(` extra (rebuilt, not in snapshot): ${f}\n`); - for (const f of diff.missing) process.stdout.write(` missing (in snapshot, not rebuilt): ${f}\n`); + for (const f of d.changed) process.stdout.write(` changed: ${f}\n`); + for (const f of d.extra) process.stdout.write(` extra (rebuilt, not in snapshot): ${f}\n`); + for (const f of d.missing) process.stdout.write(` missing (in snapshot, not rebuilt): ${f}\n`); process.stdout.write(` regenerate: node tests/prose/run.cjs snap ${name}\n`); } } finally { @@ -283,10 +304,11 @@ function cmdList() { const all = cases.loadAllCases(); const errors = cases.validateCorpus(all); for (const c of all) { - const expects = `${c.expect.filter((e) => e.kind === 'routing').length}r/${c.expect.filter((e) => e.kind === 'state').length}s`; - process.stdout.write(`${c.id} [${c.flow}] world=${c.world || '-'} expects=${expects}\n`); + const stubs = c.stubs.length ? ` stubs=${c.stubs.map((s) => s.name).join(',')}` : ''; + process.stdout.write( + `${c.id}\n ${c.worldBefore || '(no world)'} → ${c.worldAfter} trace=${c.trace.length}${stubs}\n`); } - process.stdout.write(`\n${all.length} cases, ${fixtures.listFixtures().length} fixtures`); + process.stdout.write(`\n${all.length} cases, ${fixtures.listFixtures().length} fixtures, ${cases.listStubs().length} stubs`); process.stdout.write(errors.length ? `, ${errors.length} VALIDATION ERRORS:\n` : ', corpus valid\n'); for (const e of errors) process.stdout.write(` - ${e}\n`); process.exit(errors.length ? 1 : 0); @@ -297,9 +319,9 @@ function cmdList() { const [, , command, ...rest] = process.argv; const commands = { list: cmdList, select: cmdSelect, world: cmdWorld, prompt: cmdPrompt, - grade: cmdGrade, snap: cmdSnap, verify: cmdVerify, destroy: cmdDestroy, + diff: cmdDiff, assert: cmdAssert, snap: cmdSnap, verify: cmdVerify, destroy: cmdDestroy, }; if (!commands[command]) { - die('usage: run.cjs …'); + die('usage: run.cjs …'); } commands[command](rest); diff --git a/tests/prose/smoke-boot-in-world.md b/tests/prose/smoke-boot-in-world.md new file mode 100644 index 000000000..01d8ef331 --- /dev/null +++ b/tests/prose/smoke-boot-in-world.md @@ -0,0 +1,28 @@ +## case: smoke-boot-in-world +- origin: framework smoke — world build, engine-in-world, world comparison +- files: + - skills/workflow-start/SKILL.md#Boot + +### given + +world_before: base + +### when + +In the project, run the boot command exactly as the initialisation prose +of skills/workflow-start/SKILL.md prescribes — sandbox concerns do not +apply in this world, so run it directly. Capture the JSON response, then +stop; do not continue into any confirmation or later step. + +### then + +world_after: unchanged + +trace: +1. runs the boot command and reports the knowledge base ready, in + keyword-only mode +2. reports no migrations applied — this world is already migrated — so + the migrations summary and its confirmation are not raised + +notes: +- booting an already-migrated project changes nothing on disk diff --git a/tests/prose/smoke-start-boot-structure.md b/tests/prose/smoke-start-boot-structure.md new file mode 100644 index 000000000..9cc873464 --- /dev/null +++ b/tests/prose/smoke-start-boot-structure.md @@ -0,0 +1,27 @@ +## case: smoke-start-boot-structure +- origin: framework smoke — a structure-only walk, no world +- files: + - skills/workflow-start/SKILL.md#Boot + +### given + +world_after is unchanged because this case executes nothing. + +### when + +Structure-only: read the initialisation portion of +skills/workflow-start/SKILL.md — everything the skill does before any +work is shown or routed. Establish what initialisation consists of and +in what order its parts run. Stop there; do not trace the rest of the +skill. + +### then + +world_after: unchanged + +trace: +1. casing conventions are loaded before the boot pipeline runs +2. the boot pipeline is declared mandatory — it must complete before the + skill proceeds +3. the knowledge gate is reached only after boot, and branches on what + boot reported rather than on its own checks diff --git a/tests/prose/smoke/README.md b/tests/prose/smoke/README.md deleted file mode 100644 index 2296c99a9..000000000 --- a/tests/prose/smoke/README.md +++ /dev/null @@ -1,6 +0,0 @@ -# Smoke cases — the framework proving itself - -Not corpus: these two cases exercise every moving part of the machinery -(parser, anchor checks, world builder, engine-in-world, state assertions, -walker protocol) against prose stable enough to survive. If a smoke case -fails, suspect the framework before the prose. diff --git a/tests/prose/smoke/smoke-structure-start-boot.md b/tests/prose/smoke/smoke-structure-start-boot.md deleted file mode 100644 index 541813e81..000000000 --- a/tests/prose/smoke/smoke-structure-start-boot.md +++ /dev/null @@ -1,17 +0,0 @@ -## case: smoke-structure-start-boot -- origin: framework smoke — structure-only walk, no world -- files: - - skills/workflow-start/SKILL.md#Boot - -### walk - -Structure-only: read the initialisation portion of -skills/workflow-start/SKILL.md — everything the skill does before any -work is shown or routed. Establish what the initialisation consists of -and in what order its parts run. Stop there — do not trace the rest of -the skill. - -### expect - -- routing: casing conventions are loaded before the boot pipeline runs -- routing: the boot pipeline is declared mandatory — the skill must complete it before proceeding diff --git a/tests/prose/smoke/smoke-world-boot.md b/tests/prose/smoke/smoke-world-boot.md deleted file mode 100644 index fb7be3b9d..000000000 --- a/tests/prose/smoke/smoke-world-boot.md +++ /dev/null @@ -1,19 +0,0 @@ -## case: smoke-world-boot -- world: base -- origin: framework smoke — world build, engine-in-world, state assertions -- files: - - skills/workflow-start/SKILL.md#Boot - -### walk - -In the project, run the boot command exactly as the initialisation prose -of skills/workflow-start/SKILL.md prescribes (sandbox concerns do not -apply in this world — run it directly). Capture the JSON response. Stop -immediately after recording it — do not continue into any confirmation -prompt or later step. - -### expect - -- routing: the boot response reports the knowledge base ready (keyword-only mode) -- routing: the boot response reports no migrations applied on this run (the world is already migrated) -- state: file exists .workflows/.state/migrations diff --git a/tests/prose/spec-entry-from-discussion.md b/tests/prose/spec-entry-from-discussion.md new file mode 100644 index 000000000..93b05d5f6 --- /dev/null +++ b/tests/prose/spec-entry-from-discussion.md @@ -0,0 +1,34 @@ +## case: spec-entry-from-discussion +- origin: feature mainline — a feature's spec draws on its discussion +- files: + - skills/workflow-specification-entry/SKILL.md + - skills/workflow-specification-entry/references/validate-source.md + - skills/workflow-specification-entry/references/validate-phase.md + - skills/workflow-specification-entry/references/invoke-skill.md + +### given + +world_before: feature-discussed + +### when + +Execute skills/workflow-specification-entry/SKILL.md with arguments +$0=feature, $1=pay. Follow it to the point where the handoff to the +processing skill is constructed; record the handoff block and stop. Do +not execute the processing skill's instructions. + +### then + +world_after: unchanged + +trace: +1. resolves the topic to the work unit and goes straight to validating + source material — the scoped path is for an epic with no topic +2. the source-material gate renders empty: the completed discussion + satisfies the prerequisite +3. no specification item exists, so the verb is Creating — no resume + note is rendered and nothing is reopened +4. hands off naming `.workflows/pay/discussion/pay.md` as the source + +notes: +- an entry that only validates writes nothing diff --git a/tests/prose/spec-entry-from-investigation.md b/tests/prose/spec-entry-from-investigation.md new file mode 100644 index 000000000..1cdca790c --- /dev/null +++ b/tests/prose/spec-entry-from-investigation.md @@ -0,0 +1,33 @@ +## case: spec-entry-from-investigation +- origin: bugfix mainline — the same entry skill, sourced from an investigation +- files: + - skills/workflow-specification-entry/SKILL.md + - skills/workflow-specification-entry/references/validate-source.md + - skills/workflow-specification-entry/references/invoke-skill.md + +### given + +world_before: bugfix-investigated + +### when + +Execute skills/workflow-specification-entry/SKILL.md with arguments +$0=bugfix, $1=crash-fix. Follow it to the point where the handoff to the +processing skill is constructed; record the handoff block and stop. Do +not execute the processing skill's instructions. + +### then + +world_after: unchanged + +trace: +1. resolves the topic to the work unit and goes to validating source + material +2. the source-material gate renders empty: for a bugfix the completed + investigation is what satisfies it, in place of a discussion +3. no specification item exists, so the verb is Creating +4. hands off naming `.workflows/crash-fix/investigation/crash-fix.md` as + the source material — no discussion file is named + +notes: +- the same prose serves both work types; only the source arm differs diff --git a/tests/prose/start-lists-active-work.md b/tests/prose/start-lists-active-work.md new file mode 100644 index 000000000..d511ef06a --- /dev/null +++ b/tests/prose/start-lists-active-work.md @@ -0,0 +1,31 @@ +## case: start-lists-active-work +- origin: feature mainline — the entry skill surfaces work without touching it +- files: + - skills/workflow-start/SKILL.md + +### given + +world_before: feature-created + +### when + +Execute skills/workflow-start/SKILL.md from the top. Continue until the +work dashboard and its menu have been shown. Record both verbatim, then +stop — select nothing. + +### then + +world_after: unchanged + +trace: +1. loads the shared casing conventions before any state is read +2. runs the boot pipeline, and since no migrations applied and the + knowledge base is ready, raises neither the migrations confirmation + nor the knowledge gate +3. gets the workflow state from the discovery gateway script rather than + listing directories or reading files itself +4. shows `pay` as the only active work, with a menu whose continue action + leads into the per-type navigation for a feature + +notes: +- showing work is a read: nothing is created, recorded, or committed diff --git a/tests/prose/stubs/root-cause-validated.md b/tests/prose/stubs/root-cause-validated.md new file mode 100644 index 000000000..daaae4b9a --- /dev/null +++ b/tests/prose/stubs/root-cause-validated.md @@ -0,0 +1,19 @@ +# stub: root-cause-validated + +A root-cause validation agent's report: the hypothesis confirmed by an +independent trace, no gaps. The content below is written to the path the +dispatch response returned; the STATUS block is also what the agent +returns to its caller. + +--- + +# Root Cause Validation + +The tax context is built before the payment intent and reads the shipping +address unconditionally. Traced fresh: an address-less order fails at that +read, which matches the documented root cause. + +STATUS: validated +CONFIDENCE: high +GAPS_COUNT: 0 +SUMMARY: Root cause confirmed by an independent trace. diff --git a/tests/scripts/test-prose-cases.cjs b/tests/scripts/test-prose-cases.cjs index 5a0e24d10..8869ed7eb 100644 --- a/tests/scripts/test-prose-cases.cjs +++ b/tests/scripts/test-prose-cases.cjs @@ -2,8 +2,8 @@ // Prose-test corpus: the deterministic perimeter (design/prose-tests.md // P3). Every case parses, every scoped file and anchor resolves, every -// referenced world has a recipe, every state assertion is grammatical. -// The token-costing walks run on command via /prose-test — never here. +// named world and stub exists, every case states an expected trace. The +// token-costing walks run on command via /prose-test — never here. const { describe, it } = require('node:test'); const assert = require('node:assert'); @@ -12,45 +12,47 @@ const path = require('path'); const cases = require('../prose/lib/cases.cjs'); describe('prose-test corpus', () => { + const all = cases.loadAllCases(); + it('has at least one case', () => { - assert.ok(cases.loadAllCases().length > 0, 'corpus is empty'); + assert.ok(all.length > 0, 'corpus is empty'); }); it('parses and validates cleanly', () => { - const errors = cases.validateCorpus(cases.loadAllCases()); + const errors = cases.validateCorpus(all); assert.deepStrictEqual(errors, [], `corpus validation failed:\n${errors.map((e) => ` - ${e}`).join('\n')}`); }); - it('state assertion grammar rejects garbage', () => { - assert.ok(cases.parseStateAssertion('file exists .workflows/x').kind === 'file-exists'); - assert.ok(cases.parseStateAssertion('manifest equals wu.planning.t review_cycle 1').kind === 'manifest-equals'); - assert.ok(cases.parseStateAssertion('json .workflows/.cache/a/b/c/state.json agents.x-001.status incorporated').kind === 'json-equals'); - assert.ok(cases.parseStateAssertion('the manifest should look right').error); - assert.ok(cases.parseStateAssertion('file exists').error); - assert.ok(cases.parseStateAssertion('json only.json pointer').error); - }); - - it('holds exactly one case per file, filename equal to the id', () => { - const all = cases.loadAllCases(); - const seen = new Map(); + it('is flat: one case per file, filename equal to the id', () => { + const seen = new Set(); for (const c of all) { assert.ok(!seen.has(c.file), `${c.file}: more than one case in a file`); - seen.set(c.file, c.id); + seen.add(c.file); + assert.strictEqual(path.dirname(c.file), 'tests/prose', + `${c.file}: cases are flat — nothing groups them but their files: scope`); assert.strictEqual(path.basename(c.file, '.md'), c.id, `${c.file}: filename must equal the case id`); } }); - it('parses the optional stub section as free text, defaulting empty', () => { - const all = cases.loadAllCases(); - const stubbed = all.filter((c) => c.stub); - assert.ok(stubbed.length > 0, 'expected at least one stubbed case in the corpus'); - for (const c of stubbed) { - assert.ok(c.world !== null, `${c.id}: a stub without a world has nothing to stub`); + it('every stub is named, triggered by its case, and readable', () => { + for (const c of all) { + for (const s of c.stubs) { + assert.ok(s.trigger.length > 0, + `${c.id}: stub "${s.name}" has no trigger — a stub without a moment is arrange in disguise`); + const stub = cases.readStub(s.name); + assert.ok(stub.content.length > 0, `stub "${s.name}" has no content below its --- fence`); + } } - for (const c of all.filter((c) => !c.stub)) { - assert.strictEqual(c.stub, '', `${c.id}: unstubbed cases carry an empty stub`); + }); + + it('multi-line trace steps and notes join rather than truncate', () => { + for (const c of all) { + for (const t of c.trace) assert.ok(!/\s$/.test(t), `${c.id}: trace step has trailing whitespace`); + for (const n of c.notes) assert.ok(n.length > 0, `${c.id}: empty note`); } + const wrapped = all.flatMap((c) => c.notes).filter((n) => n.length > 70); + assert.ok(wrapped.length > 0, 'expected at least one note long enough to have wrapped in source'); }); }); From f7ddba7a46a3fe6c10554dcd7d67de3ce06bc25f Mon Sep 17 00:00:00 2001 From: Lee Overy Date: Sat, 25 Jul 2026 17:33:23 +0100 Subject: [PATCH 04/11] fix(prose-tests): stubs live in when, beside the answers they resemble MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Lee reading the case file: a stub declared in `given` states a trigger for a moment the reader hasn't reached yet. The format contradicted itself — scripted `answers:` already sat in `when`, and a stub is the same kind of thing: harness input consumed during the act, one playing the user, the other the dispatched agent. `given` is now the world and nothing else; `when` carries the instruction plus everything fed in while it runs. Structure-only cases drop `given` entirely. Co-Authored-By: Claude Opus 5 --- design/prose-tests.md | 10 +++++---- tests/prose/README.md | 12 ++++++++--- tests/prose/lib/cases.cjs | 21 ++++++++++--------- .../root-cause-validation-clean-verdict.md | 5 +++-- tests/prose/smoke-start-boot-structure.md | 4 ---- 5 files changed, 29 insertions(+), 23 deletions(-) diff --git a/design/prose-tests.md b/design/prose-tests.md index f32071d2b..9e3e32c9e 100644 --- a/design/prose-tests.md +++ b/design/prose-tests.md @@ -32,11 +32,13 @@ testing. - **P1 — cases are Given-When-Then.** One case per file, flat: `tests/prose/{case-id}.md`, filename equal to the id and validated as such. Nothing groups cases but their `files:` scope, which is what - selection runs on. **Given**: `world_before` (a fixture) plus any - `stubs` armed, each with the moment it fires. **When**: one coarse + selection runs on. **Given**: `world_before` (a fixture) and nothing + else — the world before anything happens. **When**: one coarse instruction — where to enter, what to follow, where to stop — plus - `answers`, the scripted user responses (an unscripted question is a - finding: the prose asked something the case didn't predict). + everything the harness feeds in *during* the act: `answers` (the + scripted user responses; an unscripted question is a finding, the + prose asked something the case didn't predict) and `stubs`, each with + the moment it fires. **Then**: `world_after` (a fixture, or `unchanged`), a granular `trace` of the path the prose should have taken, and any further `notes`. diff --git a/tests/prose/README.md b/tests/prose/README.md index 6d5613993..7da699a65 100644 --- a/tests/prose/README.md +++ b/tests/prose/README.md @@ -47,11 +47,17 @@ off it. Prose that dispatches a background agent, or that runs an unscripted conversation, is walked with that part substituted. A stub is named -content; the case that arms it declares the moment: +content; the case that arms it declares the moment. Stubs sit in `when` +beside `answers:` — both are harness input consumed *during* the act, +and `given` is the world and nothing else: ``` -### given -world_before: bugfix-investigating +### when +{the walk instruction} + +answers: +1. yes — run root cause validation + stubs: - root-cause-validated: when the prose dispatches the validation agent ``` diff --git a/tests/prose/lib/cases.cjs b/tests/prose/lib/cases.cjs index d7af57e9e..623a785c9 100644 --- a/tests/prose/lib/cases.cjs +++ b/tests/prose/lib/cases.cjs @@ -14,13 +14,17 @@ // // ### given // world_before: {fixture name} -// stubs: -// - {stub name}: {when it fires — the case owns the trigger} // // ### when // {free text: where to enter, what to follow, where to stop} // answers: // 1. {scripted user response, consumed in order} +// stubs: +// - {stub name}: {the moment it fires — the case owns the trigger} +// +// `given` is the world and nothing else. Everything the harness feeds +// in *during* the act — the user's answers, the agent substitutions — +// belongs in `when`, beside the instruction they happen during. // // ### then // world_after: {fixture name | unchanged} @@ -100,21 +104,18 @@ function parseCaseFile(file) { if (section === 'given') { const wb = line.match(/^world_before:\s*(\S+)\s*$/); - if (wb) { c.worldBefore = wb[1]; list = null; continue; } - if (/^stubs:\s*$/.test(line)) { list = 'stubs'; continue; } - const stub = line.match(/^\s+- ([a-z0-9][a-z0-9-]*):\s*(.+)$/); - if (list === 'stubs' && stub) { - c.stubs.push({ name: stub[1], trigger: stub[2].trim() }); - continue; - } + if (wb) c.worldBefore = wb[1]; continue; } if (section === 'when') { if (/^answers:\s*$/.test(line)) { list = 'answers'; continue; } + if (/^stubs:\s*$/.test(line)) { list = 'stubs'; continue; } const answer = line.match(/^\d+\.\s+(.*)$/); if (list === 'answers' && answer) { c.answers.push(answer[1].trim()); continue; } - if (list !== 'answers') c.when.push(line); + const stub = line.match(/^\s+- ([a-z0-9][a-z0-9-]*):\s*(.+)$/); + if (list === 'stubs' && stub) { c.stubs.push({ name: stub[1], trigger: stub[2].trim() }); continue; } + if (list === null) c.when.push(line); continue; } diff --git a/tests/prose/root-cause-validation-clean-verdict.md b/tests/prose/root-cause-validation-clean-verdict.md index bbc845741..45408bb51 100644 --- a/tests/prose/root-cause-validation-clean-verdict.md +++ b/tests/prose/root-cause-validation-clean-verdict.md @@ -6,8 +6,6 @@ ### given world_before: bugfix-investigating -stubs: - - root-cause-validated: when the prose dispatches the validation agent ### when @@ -19,6 +17,9 @@ Stop when the reference returns to its caller. answers: 1. yes — run root cause validation +stubs: + - root-cause-validated: when the prose dispatches the validation agent + ### then world_after: bugfix-validation-incorporated diff --git a/tests/prose/smoke-start-boot-structure.md b/tests/prose/smoke-start-boot-structure.md index 9cc873464..a7ec974cc 100644 --- a/tests/prose/smoke-start-boot-structure.md +++ b/tests/prose/smoke-start-boot-structure.md @@ -3,10 +3,6 @@ - files: - skills/workflow-start/SKILL.md#Boot -### given - -world_after is unchanged because this case executes nothing. - ### when Structure-only: read the initialisation portion of From 80c8c64444a272171938d16479733fed3de42686 Mon Sep 17 00:00:00 2001 From: Lee Overy Date: Sat, 25 Jul 2026 17:52:05 +0100 Subject: [PATCH 05/11] fix(prose-tests): given takes prose, triggers name observable events, no CI claims MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Three of Lee's points from the TUI review: - `given` carries situational prose beside world_before — what the world represents, what has already happened, where the session stands. It reaches the walker as a SITUATION block. The root-cause case had this context smuggled into its `when`, mixing the situation with the act. - A stub trigger names an observable event, not a narrative moment: "when the engine records a dispatch of kind root-cause-validation", not "when the prose dispatches the validation agent". The loose form is only unambiguous while a walk contains exactly one dispatch, and it leaves nothing in the transcript for the asserter to check the substitution against. - The docs claimed CI. This project has no .github/workflows — the gate is `npm test`, run before every commit. Reworded throughout rather than implying anything runs on push. Co-Authored-By: Claude Opus 5 --- design/prose-tests.md | 24 ++++++++++--------- tests/prose/README.md | 10 ++++++++ tests/prose/lib/cases.cjs | 7 +++++- .../root-cause-validation-clean-verdict.md | 12 ++++++---- tests/prose/run.cjs | 3 +++ 5 files changed, 39 insertions(+), 17 deletions(-) diff --git a/design/prose-tests.md b/design/prose-tests.md index 9e3e32c9e..3fb31eb2b 100644 --- a/design/prose-tests.md +++ b/design/prose-tests.md @@ -67,11 +67,12 @@ testing. allocated ids) or material. No normalisation table to maintain, no hand-written per-field assertions, and nothing goes unchecked because nobody thought to assert it. Engine responses during the - walk are real. CI (`npm test`, zero tokens) holds the deterministic - perimeter: every snapshot rebuilds from its recipe byte-identical - (drift red-flags in the PR that moved the world, as a reviewable - snapshot diff), and every case's paths, anchors, worlds, stubs, and - trace resolve. + walk are real. **`npm test`** holds the deterministic perimeter — + zero tokens, and run before every commit, this project having no + automated CI: every snapshot rebuilds from its recipe byte-identical + (so a world the engine moved goes red at the gate and lands as a + reviewable snapshot diff in the PR that moved it), and every case's + paths, anchors, worlds, stubs, and trace resolve. - **P4 — walker/asserter separation.** The walking agent never sees the `then` block; an agent that knows the expected answer will find it. The walker gets the world, the coarse instruction, the answer script @@ -85,7 +86,8 @@ testing. credits the walk for what the framework supplied. Whatever a stub covers is not under test in that case — some other case must walk it unstubbed. -- **P5 — run on command, never in CI.** Walks cost tokens. The runner +- **P5 — run on command, never in the test suite.** Walks cost tokens, + so they never ride `npm test` — only the perimeter of P3 does. The runner is invoked deliberately: scoped by diff-intersection (the same computation powers the PR-end suggestion — "these N cases intersect this PR's prose changes"), by hand-picked ids, or `--all`. An @@ -97,7 +99,7 @@ testing. - **P7 — a failing case is a finding either way.** Either the prose broke, or the world/design moved and the case is stale. The adjudication is the point; only the typo-class staleness is - pre-filtered by CI (P3). + pre-filtered by the perimeter (P3). ## Architecture @@ -217,8 +219,8 @@ testing. store re-derived at materialise), and the runner (`run.cjs`: list · select · world · prompt · grade · snap · verify · destroy). The `prompt` command is the P4 boundary — walker prompts are - machine-assembled and never contain expects. CI gains two token-free - suites (corpus validation, snapshot rebuild-compare); the `/prose-test` + machine-assembled and never contain expects. `npm test` gains two + token-free suites (corpus validation, snapshot rebuild-compare); the `/prose-test` dev skill owns the model layer (Sonnet walks, Opus confirms, quoted evidence or it didn't happen). First fixture: `base` — a post-boot keyword-only empty project, proven byte-deterministic across rebuilds. @@ -230,7 +232,7 @@ testing. prose tests own their worlds) with the engine kept at authoring time via recipes; recipe + golden snapshot over either alone (always-current *and* visible drift *and* frozen runs); on-command - invocation only — tokens are spent deliberately, CI holds the - deterministic perimeter; Sonnet→Opus escalation; happy path + invocation only — tokens are spent deliberately, `npm test` holds + the deterministic perimeter; Sonnet→Opus escalation; happy path first-class, corpus seeded from the sim's mainline enumeration before the failure harvest. diff --git a/tests/prose/README.md b/tests/prose/README.md index 7da699a65..735232ac1 100644 --- a/tests/prose/README.md +++ b/tests/prose/README.md @@ -66,6 +66,16 @@ The trigger belongs to the case, not the stub, so one stub serves many moments — first dispatch versus re-dispatch, happy path versus recovery. A stub with no trigger is arrange in disguise, and validation rejects it. +**A trigger names an observable event, not a narrative moment.** An +engine call with its arguments, a specific menu appearing, a named file +being created — something the walker can match with certainty and that +shows up in the transcript, so the asserter can confirm the substitution +fired where it should have. "When the prose dispatches an agent" is too +loose the moment a walk covers more than one dispatch; "when the engine +records a dispatch of kind root-cause-validation" is not. Add an ordinal +only where a case genuinely needs one ("on the re-dispatch after the +gaps are addressed"). + **Whatever a stub covers is not under test in that case.** Stubbing a discussion's session loop means some other case must walk that loop without stubbing it. The stub boundary is a scope decision, not a diff --git a/tests/prose/lib/cases.cjs b/tests/prose/lib/cases.cjs index 623a785c9..7219dedda 100644 --- a/tests/prose/lib/cases.cjs +++ b/tests/prose/lib/cases.cjs @@ -14,6 +14,9 @@ // // ### given // world_before: {fixture name} +// {free text: what that world represents — what has already happened, +// where the session stands. Context for the walker, never an +// instruction.} // // ### when // {free text: where to enter, what to follow, where to stop} @@ -64,7 +67,7 @@ function parseCaseFile(file) { const lines = fs.readFileSync(file, 'utf8').split('\n'); const c = { id: null, file: path.relative(ROOT, file), origin: null, files: [], - worldBefore: null, stubs: [], when: [], answers: [], + worldBefore: null, situation: [], stubs: [], when: [], answers: [], worldAfter: null, trace: [], notes: [], }; let section = null; // null | given | when | then @@ -105,6 +108,7 @@ function parseCaseFile(file) { if (section === 'given') { const wb = line.match(/^world_before:\s*(\S+)\s*$/); if (wb) c.worldBefore = wb[1]; + else c.situation.push(line); continue; } @@ -138,6 +142,7 @@ function parseCaseFile(file) { } c.when = c.when.join('\n').trim(); + c.situation = c.situation.join('\n').trim(); return c; } diff --git a/tests/prose/root-cause-validation-clean-verdict.md b/tests/prose/root-cause-validation-clean-verdict.md index 45408bb51..e7cd2b596 100644 --- a/tests/prose/root-cause-validation-clean-verdict.md +++ b/tests/prose/root-cause-validation-clean-verdict.md @@ -7,18 +7,20 @@ world_before: bugfix-investigating +The investigation is open with its symptoms gathered and its root cause +documented. The processing skill has worked down to its root-cause +validation step; nothing has been validated yet and no agent has run. + ### when -The investigation is open with its root cause documented, and the -process has reached its root-cause validation step. Walk -root-cause-validation.md from the top, as the processing skill would. -Stop when the reference returns to its caller. +Walk root-cause-validation.md from the top, as the processing skill +would. Stop when the reference returns to its caller. answers: 1. yes — run root cause validation stubs: - - root-cause-validated: when the prose dispatches the validation agent + - root-cause-validated: when the engine records a dispatch of kind root-cause-validation ### then diff --git a/tests/prose/run.cjs b/tests/prose/run.cjs index abecfe80d..918b90f1a 100644 --- a/tests/prose/run.cjs +++ b/tests/prose/run.cjs @@ -154,8 +154,11 @@ function cmdPrompt(argv) { }), ] : []; + const situation = c.situation ? ['', 'SITUATION — where the project stands as you begin:', c.situation] : []; + process.stdout.write(`${[ ...setting, + ...situation, '', 'TASK', c.when, From f3f24b3ea2dd00b937504bcc162a7103c0a71c27 Mon Sep 17 00:00:00 2001 From: Lee Overy Date: Sat, 25 Jul 2026 18:26:45 +0100 Subject: [PATCH 06/11] =?UTF-8?q?refactor(prose-tests):=20a=20case=20is=20?= =?UTF-8?q?a=20directory=20=E2=80=94=20one=20file=20per=20element?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Lee's review, three faults in the flat-markdown shape: - Code regex-parsed values out of markdown, the exact pattern the analysis-state campaign eliminated. Now case.json holds what code branches on, prose files are read whole, recipes are required as modules. Nothing is parsed and prose never shares a file with code — which also deletes ~150 lines of parser. - world_before/world_after named foreign fixtures the case merely pointed at, scattering one test across six locations. A case now owns its worlds: fixture-state.cjs and assertion-state.cjs, split because they change for different reasons, snapshots beside them. Duplication accepted; reuse moves to mainlines/ as composable stage functions. - Naming follows the stage served — fixture, act, assert — so assert.md and assertion-state.cjs sit together by design. act.md and assert.md stay separate files because that IS the walker/asserter boundary, enforced by the filesystem rather than by convention. Adds a recipe-hash skip, without which per-case worlds would grow the gate linearly. Proven both ways: 0.09s when nothing moved, and a full rebuild reporting DRIFT when a mainline's content changed. Whole pipe re-proven live — world built, lifecycle walked by hand, delta against the assertion state was the volatile timestamp alone. Co-Authored-By: Claude Opus 5 --- .claude/skills/prose-test/SKILL.md | 20 +- CLAUDE.md | 2 +- design/prose-tests.md | 68 ++-- tests/prose/README.md | 141 ++++---- .../act.md | 4 + .../assert.md | 10 + .../case.json | 11 + .../fixture-state.cjs | 12 + .../fixture/.claude/settings.json | 9 + .../fixture/.recipe-hash | 1 + .../fixture/.workflows/.state/migrations | 52 +++ .../fixture/.workflows/_gitignore.fixture | 2 + .../discovery/sessions/session-001.md | 40 +++ .../.workflows/crash-fix/manifest.json | 8 + .../fixture/.workflows/manifest.json | 7 + .../act.md | 4 + .../assert.md | 13 + .../case.json | 11 + .../fixture-state.cjs | 12 + .../fixture/.claude/settings.json | 9 + .../fixture/.recipe-hash | 1 + .../fixture/.workflows/.state/migrations | 52 +++ .../fixture/.workflows/_gitignore.fixture | 2 + .../fixture/.workflows/manifest.json | 7 + .../pay/discovery/sessions/session-001.md | 39 +++ .../fixture/.workflows/pay/manifest.json | 8 + .../act.md | 4 + .../assert.md} | 25 +- .../case.json | 7 + .../fixture-state.cjs | 12 + .../fixture/.claude/settings.json | 9 + .../fixture/.recipe-hash | 1 + .../fixture/.workflows/.state/migrations | 52 +++ .../fixture/.workflows/_gitignore.fixture | 2 + .../fixture/.workflows/manifest.json | 7 + .../pay/discovery/sessions/session-001.md | 39 +++ .../fixture/.workflows/pay/manifest.json | 8 + .../implementation-picks-first-task/act.md | 4 + .../implementation-picks-first-task/assert.md | 20 ++ .../assertion-state.cjs | 22 ++ .../assertion/.claude/settings.json | 9 + .../assertion/.recipe-hash | 1 + .../.workflows/.state/environment-setup.md | 1 + .../assertion/.workflows/.state/migrations | 52 +++ .../assertion/.workflows/_gitignore.fixture | 2 + .../assertion/.workflows/manifest.json | 7 + .../pay/discovery/sessions/session-001.md | 39 +++ .../.workflows/pay/discussion/pay.md | 15 + .../assertion/.workflows/pay/manifest.json | 70 ++++ .../.workflows/pay/planning/pay/planning.md | 8 + .../pay/planning/pay/tasks/pay-1-1.md | 10 + .../pay/planning/pay/tasks/pay-1-2.md | 10 + .../pay/specification/pay/specification.md | 11 + .../implementation-picks-first-task/case.json | 16 + .../fixture-state.cjs | 15 + .../fixture/.claude/settings.json | 9 + .../fixture/.recipe-hash | 1 + .../fixture/.workflows/.state/migrations | 52 +++ .../fixture/.workflows/_gitignore.fixture | 2 + .../fixture/.workflows/manifest.json | 7 + .../pay/discovery/sessions/session-001.md | 39 +++ .../fixture/.workflows/pay/discussion/pay.md | 15 + .../fixture/.workflows/pay/manifest.json | 53 +++ .../.workflows/pay/planning/pay/planning.md | 8 + .../pay/planning/pay/tasks/pay-1-1.md | 10 + .../pay/planning/pay/tasks/pay-1-2.md | 10 + .../pay/specification/pay/specification.md | 11 + .../act.md | 4 + .../assert.md | 15 + .../case.json | 7 + .../fixture-state.cjs | 12 + .../fixture/.claude/settings.json | 9 + .../fixture/.recipe-hash | 1 + .../fixture/.workflows/.state/migrations | 52 +++ .../fixture/.workflows/_gitignore.fixture | 2 + .../discovery/sessions/session-001.md | 40 +++ .../.workflows/crash-fix/manifest.json | 8 + .../fixture/.workflows/manifest.json | 7 + .../act.md | 4 + .../assert.md | 15 + .../case.json | 12 + .../fixture-state.cjs | 14 + .../fixture/.claude/settings.json | 9 + .../fixture/.recipe-hash | 1 + .../fixture/.workflows/.state/migrations | 52 +++ .../fixture/.workflows/_gitignore.fixture | 2 + .../fixture/.workflows/manifest.json | 7 + .../pay/discovery/sessions/session-001.md | 39 +++ .../fixture/.workflows/pay/discussion/pay.md | 15 + .../fixture/.workflows/pay/manifest.json | 28 ++ .../pay/specification/pay/specification.md | 11 + tests/prose/cases/review-entry-fresh/act.md | 4 + .../prose/cases/review-entry-fresh/assert.md | 12 + .../prose/cases/review-entry-fresh/case.json | 7 + .../review-entry-fresh/fixture-state.cjs | 16 + .../fixture/.claude/settings.json | 9 + .../review-entry-fresh/fixture/.recipe-hash | 1 + .../fixture/.workflows/.state/migrations | 52 +++ .../fixture/.workflows/_gitignore.fixture | 2 + .../fixture/.workflows/manifest.json | 7 + .../pay/discovery/sessions/session-001.md | 39 +++ .../fixture/.workflows/pay/discussion/pay.md | 15 + .../fixture/.workflows/pay/manifest.json | 77 +++++ .../.workflows/pay/planning/pay/planning.md | 8 + .../pay/planning/pay/tasks/pay-1-1.md | 10 + .../pay/planning/pay/tasks/pay-1-2.md | 10 + .../pay/specification/pay/specification.md | 11 + .../act.md | 2 + .../assert.md | 19 + .../assertion-state.cjs | 27 ++ .../assertion/.claude/settings.json | 9 + .../assertion/.recipe-hash | 1 + .../crash-fix/root-cause-validation-001.md | 10 + .../investigation/crash-fix/state.json | 16 + .../assertion/.workflows/.state/migrations | 52 +++ .../assertion/.workflows/_gitignore.fixture | 2 + .../discovery/sessions/session-001.md | 40 +++ .../crash-fix/investigation/crash-fix.md | 88 +++++ .../.workflows/crash-fix/manifest.json | 16 + .../assertion/.workflows/manifest.json | 7 + .../case.json | 12 + .../fixture-state.cjs | 13 + .../fixture.md | 3 + .../fixture/.claude/settings.json | 9 + .../fixture/.recipe-hash | 1 + .../fixture/.workflows/.state/migrations | 52 +++ .../fixture/.workflows/_gitignore.fixture | 2 + .../discovery/sessions/session-001.md | 40 +++ .../crash-fix/investigation/crash-fix.md | 88 +++++ .../.workflows/crash-fix/manifest.json | 16 + .../fixture/.workflows/manifest.json | 7 + tests/prose/cases/smoke-boot-in-world/act.md | 4 + .../prose/cases/smoke-boot-in-world/assert.md | 10 + .../prose/cases/smoke-boot-in-world/case.json | 6 + .../smoke-boot-in-world/fixture-state.cjs | 11 + .../fixture/.claude/settings.json | 9 + .../smoke-boot-in-world/fixture/.recipe-hash | 1 + .../fixture/.workflows/.state/migrations | 52 +++ .../fixture/.workflows/_gitignore.fixture | 2 + .../cases/smoke-start-boot-structure/act.md | 5 + .../smoke-start-boot-structure/assert.md | 7 + .../smoke-start-boot-structure/case.json | 6 + .../cases/spec-entry-from-discussion/act.md | 4 + .../spec-entry-from-discussion/assert.md | 13 + .../spec-entry-from-discussion/case.json | 9 + .../fixture-state.cjs | 13 + .../fixture/.claude/settings.json | 9 + .../fixture/.recipe-hash | 1 + .../fixture/.workflows/.state/migrations | 52 +++ .../fixture/.workflows/_gitignore.fixture | 2 + .../fixture/.workflows/manifest.json | 7 + .../pay/discovery/sessions/session-001.md | 39 +++ .../fixture/.workflows/pay/discussion/pay.md | 15 + .../fixture/.workflows/pay/manifest.json | 16 + .../spec-entry-from-investigation/act.md | 4 + .../spec-entry-from-investigation/assert.md | 13 + .../spec-entry-from-investigation/case.json | 8 + .../fixture-state.cjs | 14 + .../fixture/.claude/settings.json | 9 + .../fixture/.recipe-hash | 1 + .../fixture/.workflows/.state/migrations | 52 +++ .../fixture/.workflows/_gitignore.fixture | 2 + .../discovery/sessions/session-001.md | 40 +++ .../crash-fix/investigation/crash-fix.md | 90 +++++ .../.workflows/crash-fix/manifest.json | 16 + .../fixture/.workflows/manifest.json | 7 + .../cases/start-lists-active-work/act.md | 3 + .../start-lists-active-work/assert.md} | 23 +- .../cases/start-lists-active-work/case.json | 6 + .../start-lists-active-work/fixture-state.cjs | 12 + .../fixture/.claude/settings.json | 9 + .../fixture/.recipe-hash | 1 + .../fixture/.workflows/.state/migrations | 52 +++ .../fixture/.workflows/_gitignore.fixture | 2 + .../fixture/.workflows/manifest.json | 7 + .../pay/discovery/sessions/session-001.md | 39 +++ .../fixture/.workflows/pay/manifest.json | 8 + ...continue-bugfix-routes-to-investigation.md | 34 -- .../continue-feature-routes-to-discussion.md | 36 -- .../prose/implementation-picks-first-task.md | 48 --- .../investigation-entry-seeds-from-carrier.md | 34 -- tests/prose/lib/cases.cjs | 278 +++++++-------- tests/prose/lib/fixtures.cjs | 241 ------------- tests/prose/lib/world.cjs | 75 ---- tests/prose/lib/worlds.cjs | 326 ++++++++++++++++++ .../bugfix.cjs} | 0 .../feature.cjs} | 0 .../planning-entry-asks-for-late-context.md | 39 --- tests/prose/review-entry-fresh.md | 31 -- .../root-cause-validation-clean-verdict.md | 45 --- tests/prose/run.cjs | 146 ++++---- tests/prose/smoke-boot-in-world.md | 28 -- tests/prose/smoke-start-boot-structure.md | 23 -- tests/prose/spec-entry-from-discussion.md | 34 -- tests/prose/spec-entry-from-investigation.md | 33 -- tests/scripts/test-prose-cases.cjs | 51 +-- tests/scripts/test-prose-fixtures.cjs | 59 ++-- 197 files changed, 3571 insertions(+), 1119 deletions(-) create mode 100644 tests/prose/cases/continue-bugfix-routes-to-investigation/act.md create mode 100644 tests/prose/cases/continue-bugfix-routes-to-investigation/assert.md create mode 100644 tests/prose/cases/continue-bugfix-routes-to-investigation/case.json create mode 100644 tests/prose/cases/continue-bugfix-routes-to-investigation/fixture-state.cjs create mode 100644 tests/prose/cases/continue-bugfix-routes-to-investigation/fixture/.claude/settings.json create mode 100644 tests/prose/cases/continue-bugfix-routes-to-investigation/fixture/.recipe-hash create mode 100644 tests/prose/cases/continue-bugfix-routes-to-investigation/fixture/.workflows/.state/migrations create mode 100644 tests/prose/cases/continue-bugfix-routes-to-investigation/fixture/.workflows/_gitignore.fixture create mode 100644 tests/prose/cases/continue-bugfix-routes-to-investigation/fixture/.workflows/crash-fix/discovery/sessions/session-001.md create mode 100644 tests/prose/cases/continue-bugfix-routes-to-investigation/fixture/.workflows/crash-fix/manifest.json create mode 100644 tests/prose/cases/continue-bugfix-routes-to-investigation/fixture/.workflows/manifest.json create mode 100644 tests/prose/cases/continue-feature-routes-to-discussion/act.md create mode 100644 tests/prose/cases/continue-feature-routes-to-discussion/assert.md create mode 100644 tests/prose/cases/continue-feature-routes-to-discussion/case.json create mode 100644 tests/prose/cases/continue-feature-routes-to-discussion/fixture-state.cjs create mode 100644 tests/prose/cases/continue-feature-routes-to-discussion/fixture/.claude/settings.json create mode 100644 tests/prose/cases/continue-feature-routes-to-discussion/fixture/.recipe-hash create mode 100644 tests/prose/cases/continue-feature-routes-to-discussion/fixture/.workflows/.state/migrations create mode 100644 tests/prose/cases/continue-feature-routes-to-discussion/fixture/.workflows/_gitignore.fixture create mode 100644 tests/prose/cases/continue-feature-routes-to-discussion/fixture/.workflows/manifest.json create mode 100644 tests/prose/cases/continue-feature-routes-to-discussion/fixture/.workflows/pay/discovery/sessions/session-001.md create mode 100644 tests/prose/cases/continue-feature-routes-to-discussion/fixture/.workflows/pay/manifest.json create mode 100644 tests/prose/cases/discussion-entry-seeds-from-carrier/act.md rename tests/prose/{discussion-entry-seeds-from-carrier.md => cases/discussion-entry-seeds-from-carrier/assert.md} (50%) create mode 100644 tests/prose/cases/discussion-entry-seeds-from-carrier/case.json create mode 100644 tests/prose/cases/discussion-entry-seeds-from-carrier/fixture-state.cjs create mode 100644 tests/prose/cases/discussion-entry-seeds-from-carrier/fixture/.claude/settings.json create mode 100644 tests/prose/cases/discussion-entry-seeds-from-carrier/fixture/.recipe-hash create mode 100644 tests/prose/cases/discussion-entry-seeds-from-carrier/fixture/.workflows/.state/migrations create mode 100644 tests/prose/cases/discussion-entry-seeds-from-carrier/fixture/.workflows/_gitignore.fixture create mode 100644 tests/prose/cases/discussion-entry-seeds-from-carrier/fixture/.workflows/manifest.json create mode 100644 tests/prose/cases/discussion-entry-seeds-from-carrier/fixture/.workflows/pay/discovery/sessions/session-001.md create mode 100644 tests/prose/cases/discussion-entry-seeds-from-carrier/fixture/.workflows/pay/manifest.json create mode 100644 tests/prose/cases/implementation-picks-first-task/act.md create mode 100644 tests/prose/cases/implementation-picks-first-task/assert.md create mode 100644 tests/prose/cases/implementation-picks-first-task/assertion-state.cjs create mode 100644 tests/prose/cases/implementation-picks-first-task/assertion/.claude/settings.json create mode 100644 tests/prose/cases/implementation-picks-first-task/assertion/.recipe-hash create mode 100644 tests/prose/cases/implementation-picks-first-task/assertion/.workflows/.state/environment-setup.md create mode 100644 tests/prose/cases/implementation-picks-first-task/assertion/.workflows/.state/migrations create mode 100644 tests/prose/cases/implementation-picks-first-task/assertion/.workflows/_gitignore.fixture create mode 100644 tests/prose/cases/implementation-picks-first-task/assertion/.workflows/manifest.json create mode 100644 tests/prose/cases/implementation-picks-first-task/assertion/.workflows/pay/discovery/sessions/session-001.md create mode 100644 tests/prose/cases/implementation-picks-first-task/assertion/.workflows/pay/discussion/pay.md create mode 100644 tests/prose/cases/implementation-picks-first-task/assertion/.workflows/pay/manifest.json create mode 100644 tests/prose/cases/implementation-picks-first-task/assertion/.workflows/pay/planning/pay/planning.md create mode 100644 tests/prose/cases/implementation-picks-first-task/assertion/.workflows/pay/planning/pay/tasks/pay-1-1.md create mode 100644 tests/prose/cases/implementation-picks-first-task/assertion/.workflows/pay/planning/pay/tasks/pay-1-2.md create mode 100644 tests/prose/cases/implementation-picks-first-task/assertion/.workflows/pay/specification/pay/specification.md create mode 100644 tests/prose/cases/implementation-picks-first-task/case.json create mode 100644 tests/prose/cases/implementation-picks-first-task/fixture-state.cjs create mode 100644 tests/prose/cases/implementation-picks-first-task/fixture/.claude/settings.json create mode 100644 tests/prose/cases/implementation-picks-first-task/fixture/.recipe-hash create mode 100644 tests/prose/cases/implementation-picks-first-task/fixture/.workflows/.state/migrations create mode 100644 tests/prose/cases/implementation-picks-first-task/fixture/.workflows/_gitignore.fixture create mode 100644 tests/prose/cases/implementation-picks-first-task/fixture/.workflows/manifest.json create mode 100644 tests/prose/cases/implementation-picks-first-task/fixture/.workflows/pay/discovery/sessions/session-001.md create mode 100644 tests/prose/cases/implementation-picks-first-task/fixture/.workflows/pay/discussion/pay.md create mode 100644 tests/prose/cases/implementation-picks-first-task/fixture/.workflows/pay/manifest.json create mode 100644 tests/prose/cases/implementation-picks-first-task/fixture/.workflows/pay/planning/pay/planning.md create mode 100644 tests/prose/cases/implementation-picks-first-task/fixture/.workflows/pay/planning/pay/tasks/pay-1-1.md create mode 100644 tests/prose/cases/implementation-picks-first-task/fixture/.workflows/pay/planning/pay/tasks/pay-1-2.md create mode 100644 tests/prose/cases/implementation-picks-first-task/fixture/.workflows/pay/specification/pay/specification.md create mode 100644 tests/prose/cases/investigation-entry-seeds-from-carrier/act.md create mode 100644 tests/prose/cases/investigation-entry-seeds-from-carrier/assert.md create mode 100644 tests/prose/cases/investigation-entry-seeds-from-carrier/case.json create mode 100644 tests/prose/cases/investigation-entry-seeds-from-carrier/fixture-state.cjs create mode 100644 tests/prose/cases/investigation-entry-seeds-from-carrier/fixture/.claude/settings.json create mode 100644 tests/prose/cases/investigation-entry-seeds-from-carrier/fixture/.recipe-hash create mode 100644 tests/prose/cases/investigation-entry-seeds-from-carrier/fixture/.workflows/.state/migrations create mode 100644 tests/prose/cases/investigation-entry-seeds-from-carrier/fixture/.workflows/_gitignore.fixture create mode 100644 tests/prose/cases/investigation-entry-seeds-from-carrier/fixture/.workflows/crash-fix/discovery/sessions/session-001.md create mode 100644 tests/prose/cases/investigation-entry-seeds-from-carrier/fixture/.workflows/crash-fix/manifest.json create mode 100644 tests/prose/cases/investigation-entry-seeds-from-carrier/fixture/.workflows/manifest.json create mode 100644 tests/prose/cases/planning-entry-asks-for-late-context/act.md create mode 100644 tests/prose/cases/planning-entry-asks-for-late-context/assert.md create mode 100644 tests/prose/cases/planning-entry-asks-for-late-context/case.json create mode 100644 tests/prose/cases/planning-entry-asks-for-late-context/fixture-state.cjs create mode 100644 tests/prose/cases/planning-entry-asks-for-late-context/fixture/.claude/settings.json create mode 100644 tests/prose/cases/planning-entry-asks-for-late-context/fixture/.recipe-hash create mode 100644 tests/prose/cases/planning-entry-asks-for-late-context/fixture/.workflows/.state/migrations create mode 100644 tests/prose/cases/planning-entry-asks-for-late-context/fixture/.workflows/_gitignore.fixture create mode 100644 tests/prose/cases/planning-entry-asks-for-late-context/fixture/.workflows/manifest.json create mode 100644 tests/prose/cases/planning-entry-asks-for-late-context/fixture/.workflows/pay/discovery/sessions/session-001.md create mode 100644 tests/prose/cases/planning-entry-asks-for-late-context/fixture/.workflows/pay/discussion/pay.md create mode 100644 tests/prose/cases/planning-entry-asks-for-late-context/fixture/.workflows/pay/manifest.json create mode 100644 tests/prose/cases/planning-entry-asks-for-late-context/fixture/.workflows/pay/specification/pay/specification.md create mode 100644 tests/prose/cases/review-entry-fresh/act.md create mode 100644 tests/prose/cases/review-entry-fresh/assert.md create mode 100644 tests/prose/cases/review-entry-fresh/case.json create mode 100644 tests/prose/cases/review-entry-fresh/fixture-state.cjs create mode 100644 tests/prose/cases/review-entry-fresh/fixture/.claude/settings.json create mode 100644 tests/prose/cases/review-entry-fresh/fixture/.recipe-hash create mode 100644 tests/prose/cases/review-entry-fresh/fixture/.workflows/.state/migrations create mode 100644 tests/prose/cases/review-entry-fresh/fixture/.workflows/_gitignore.fixture create mode 100644 tests/prose/cases/review-entry-fresh/fixture/.workflows/manifest.json create mode 100644 tests/prose/cases/review-entry-fresh/fixture/.workflows/pay/discovery/sessions/session-001.md create mode 100644 tests/prose/cases/review-entry-fresh/fixture/.workflows/pay/discussion/pay.md create mode 100644 tests/prose/cases/review-entry-fresh/fixture/.workflows/pay/manifest.json create mode 100644 tests/prose/cases/review-entry-fresh/fixture/.workflows/pay/planning/pay/planning.md create mode 100644 tests/prose/cases/review-entry-fresh/fixture/.workflows/pay/planning/pay/tasks/pay-1-1.md create mode 100644 tests/prose/cases/review-entry-fresh/fixture/.workflows/pay/planning/pay/tasks/pay-1-2.md create mode 100644 tests/prose/cases/review-entry-fresh/fixture/.workflows/pay/specification/pay/specification.md create mode 100644 tests/prose/cases/root-cause-validation-clean-verdict/act.md create mode 100644 tests/prose/cases/root-cause-validation-clean-verdict/assert.md create mode 100644 tests/prose/cases/root-cause-validation-clean-verdict/assertion-state.cjs create mode 100644 tests/prose/cases/root-cause-validation-clean-verdict/assertion/.claude/settings.json create mode 100644 tests/prose/cases/root-cause-validation-clean-verdict/assertion/.recipe-hash create mode 100644 tests/prose/cases/root-cause-validation-clean-verdict/assertion/.workflows/.cache/crash-fix/investigation/crash-fix/root-cause-validation-001.md create mode 100644 tests/prose/cases/root-cause-validation-clean-verdict/assertion/.workflows/.cache/crash-fix/investigation/crash-fix/state.json create mode 100644 tests/prose/cases/root-cause-validation-clean-verdict/assertion/.workflows/.state/migrations create mode 100644 tests/prose/cases/root-cause-validation-clean-verdict/assertion/.workflows/_gitignore.fixture create mode 100644 tests/prose/cases/root-cause-validation-clean-verdict/assertion/.workflows/crash-fix/discovery/sessions/session-001.md create mode 100644 tests/prose/cases/root-cause-validation-clean-verdict/assertion/.workflows/crash-fix/investigation/crash-fix.md create mode 100644 tests/prose/cases/root-cause-validation-clean-verdict/assertion/.workflows/crash-fix/manifest.json create mode 100644 tests/prose/cases/root-cause-validation-clean-verdict/assertion/.workflows/manifest.json create mode 100644 tests/prose/cases/root-cause-validation-clean-verdict/case.json create mode 100644 tests/prose/cases/root-cause-validation-clean-verdict/fixture-state.cjs create mode 100644 tests/prose/cases/root-cause-validation-clean-verdict/fixture.md create mode 100644 tests/prose/cases/root-cause-validation-clean-verdict/fixture/.claude/settings.json create mode 100644 tests/prose/cases/root-cause-validation-clean-verdict/fixture/.recipe-hash create mode 100644 tests/prose/cases/root-cause-validation-clean-verdict/fixture/.workflows/.state/migrations create mode 100644 tests/prose/cases/root-cause-validation-clean-verdict/fixture/.workflows/_gitignore.fixture create mode 100644 tests/prose/cases/root-cause-validation-clean-verdict/fixture/.workflows/crash-fix/discovery/sessions/session-001.md create mode 100644 tests/prose/cases/root-cause-validation-clean-verdict/fixture/.workflows/crash-fix/investigation/crash-fix.md create mode 100644 tests/prose/cases/root-cause-validation-clean-verdict/fixture/.workflows/crash-fix/manifest.json create mode 100644 tests/prose/cases/root-cause-validation-clean-verdict/fixture/.workflows/manifest.json create mode 100644 tests/prose/cases/smoke-boot-in-world/act.md create mode 100644 tests/prose/cases/smoke-boot-in-world/assert.md create mode 100644 tests/prose/cases/smoke-boot-in-world/case.json create mode 100644 tests/prose/cases/smoke-boot-in-world/fixture-state.cjs create mode 100644 tests/prose/cases/smoke-boot-in-world/fixture/.claude/settings.json create mode 100644 tests/prose/cases/smoke-boot-in-world/fixture/.recipe-hash create mode 100644 tests/prose/cases/smoke-boot-in-world/fixture/.workflows/.state/migrations create mode 100644 tests/prose/cases/smoke-boot-in-world/fixture/.workflows/_gitignore.fixture create mode 100644 tests/prose/cases/smoke-start-boot-structure/act.md create mode 100644 tests/prose/cases/smoke-start-boot-structure/assert.md create mode 100644 tests/prose/cases/smoke-start-boot-structure/case.json create mode 100644 tests/prose/cases/spec-entry-from-discussion/act.md create mode 100644 tests/prose/cases/spec-entry-from-discussion/assert.md create mode 100644 tests/prose/cases/spec-entry-from-discussion/case.json create mode 100644 tests/prose/cases/spec-entry-from-discussion/fixture-state.cjs create mode 100644 tests/prose/cases/spec-entry-from-discussion/fixture/.claude/settings.json create mode 100644 tests/prose/cases/spec-entry-from-discussion/fixture/.recipe-hash create mode 100644 tests/prose/cases/spec-entry-from-discussion/fixture/.workflows/.state/migrations create mode 100644 tests/prose/cases/spec-entry-from-discussion/fixture/.workflows/_gitignore.fixture create mode 100644 tests/prose/cases/spec-entry-from-discussion/fixture/.workflows/manifest.json create mode 100644 tests/prose/cases/spec-entry-from-discussion/fixture/.workflows/pay/discovery/sessions/session-001.md create mode 100644 tests/prose/cases/spec-entry-from-discussion/fixture/.workflows/pay/discussion/pay.md create mode 100644 tests/prose/cases/spec-entry-from-discussion/fixture/.workflows/pay/manifest.json create mode 100644 tests/prose/cases/spec-entry-from-investigation/act.md create mode 100644 tests/prose/cases/spec-entry-from-investigation/assert.md create mode 100644 tests/prose/cases/spec-entry-from-investigation/case.json create mode 100644 tests/prose/cases/spec-entry-from-investigation/fixture-state.cjs create mode 100644 tests/prose/cases/spec-entry-from-investigation/fixture/.claude/settings.json create mode 100644 tests/prose/cases/spec-entry-from-investigation/fixture/.recipe-hash create mode 100644 tests/prose/cases/spec-entry-from-investigation/fixture/.workflows/.state/migrations create mode 100644 tests/prose/cases/spec-entry-from-investigation/fixture/.workflows/_gitignore.fixture create mode 100644 tests/prose/cases/spec-entry-from-investigation/fixture/.workflows/crash-fix/discovery/sessions/session-001.md create mode 100644 tests/prose/cases/spec-entry-from-investigation/fixture/.workflows/crash-fix/investigation/crash-fix.md create mode 100644 tests/prose/cases/spec-entry-from-investigation/fixture/.workflows/crash-fix/manifest.json create mode 100644 tests/prose/cases/spec-entry-from-investigation/fixture/.workflows/manifest.json create mode 100644 tests/prose/cases/start-lists-active-work/act.md rename tests/prose/{start-lists-active-work.md => cases/start-lists-active-work/assert.md} (56%) create mode 100644 tests/prose/cases/start-lists-active-work/case.json create mode 100644 tests/prose/cases/start-lists-active-work/fixture-state.cjs create mode 100644 tests/prose/cases/start-lists-active-work/fixture/.claude/settings.json create mode 100644 tests/prose/cases/start-lists-active-work/fixture/.recipe-hash create mode 100644 tests/prose/cases/start-lists-active-work/fixture/.workflows/.state/migrations create mode 100644 tests/prose/cases/start-lists-active-work/fixture/.workflows/_gitignore.fixture create mode 100644 tests/prose/cases/start-lists-active-work/fixture/.workflows/manifest.json create mode 100644 tests/prose/cases/start-lists-active-work/fixture/.workflows/pay/discovery/sessions/session-001.md create mode 100644 tests/prose/cases/start-lists-active-work/fixture/.workflows/pay/manifest.json delete mode 100644 tests/prose/continue-bugfix-routes-to-investigation.md delete mode 100644 tests/prose/continue-feature-routes-to-discussion.md delete mode 100644 tests/prose/implementation-picks-first-task.md delete mode 100644 tests/prose/investigation-entry-seeds-from-carrier.md delete mode 100644 tests/prose/lib/fixtures.cjs delete mode 100644 tests/prose/lib/world.cjs create mode 100644 tests/prose/lib/worlds.cjs rename tests/prose/{fixtures/_shared/bugfix-mainline.cjs => mainlines/bugfix.cjs} (100%) rename tests/prose/{fixtures/_shared/feature-mainline.cjs => mainlines/feature.cjs} (100%) delete mode 100644 tests/prose/planning-entry-asks-for-late-context.md delete mode 100644 tests/prose/review-entry-fresh.md delete mode 100644 tests/prose/root-cause-validation-clean-verdict.md delete mode 100644 tests/prose/smoke-boot-in-world.md delete mode 100644 tests/prose/smoke-start-boot-structure.md delete mode 100644 tests/prose/spec-entry-from-discussion.md delete mode 100644 tests/prose/spec-entry-from-investigation.md diff --git a/.claude/skills/prose-test/SKILL.md b/.claude/skills/prose-test/SKILL.md index 2a9416657..280eae39b 100644 --- a/.claude/skills/prose-test/SKILL.md +++ b/.claude/skills/prose-test/SKILL.md @@ -1,6 +1,6 @@ --- name: prose-test -description: Run prose-logic test cases — a walker agent executes workflow prose against a materialised fixture world, an asserting agent judges the trace and the world delta. Scope by diff (default), case ids, or --all. +description: Run prose-logic test cases — a walker agent executes workflow prose against a materialised fixture world, an asserting agent judges the path taken and the world delta. Scope by diff (default), case ids, or --all. --- # Prose Test @@ -8,7 +8,7 @@ description: Run prose-logic test cases — a walker agent executes workflow pro Run prose-test cases (design: `design/prose-tests.md`; runner: `tests/prose/run.cjs`). Cases are Given-When-Then: a world is arranged from a committed fixture, an agent acts by walking the real prose, and a -second agent asserts the resulting trace and world. Walks cost tokens — +second agent asserts the resulting path and world. Walks cost tokens — this skill runs on command only, never as part of a routine gate. ## Step 1: Select @@ -28,19 +28,19 @@ Otherwise show the selected case ids and proceed. Cases are independent — run up to 4 concurrently. Per case: 1. **World** (skip when the case has no world): `node tests/prose/run.cjs world ` — note the returned path. -2. **Walk**: `node tests/prose/run.cjs prompt [--world ]`, then dispatch a subagent (model: **sonnet**) with that prompt **verbatim and unmodified**. Never hand-assemble a walker prompt, and never mention any expectation — the `prompt` command's output is the whole contract, and it cannot contain the case's `then`. Save the returned transcript to a scratch file. +2. **Walk**: `node tests/prose/run.cjs prompt [--world ]`, then dispatch a subagent (model: **sonnet**) with that prompt **verbatim and unmodified**. Never hand-assemble a walker prompt, and never mention any expectation — the `prompt` command's output is the whole contract, and it cannot contain assert.md. Save the returned transcript to a scratch file. ## Step 3: Assert `node tests/prose/run.cjs assert [--world ]` produces the -asserting agent's prompt: the expected trace, any further claims, and the +asserting agent's prompt: the expected path, any further claims, and the code-computed world delta. Dispatch a second subagent (model: **sonnet**) with that prompt plus the walk transcript. -It returns a verdict in four parts: trace (per step, each PASS quoting the +It returns a verdict in four parts: path (per step, each PASS quoting the transcript line that shows it), world (every difference in the delta enumerated and classified volatile or material), markers, and the overall -verdict. A trace PASS without a quote is invalid — treat as FAIL. +verdict. A path PASS without a quote is invalid — treat as FAIL. `UNSCRIPTED QUESTION`, `AMBIGUOUS`, or `DEVIATION` markers in a transcript are findings in their own right — carry them into the report even when @@ -56,9 +56,9 @@ Any FAIL: ## Step 5: Report and clean up -Report a verdict table: case id, trace passed/total, world verdict, +Report a verdict table: case id, path passed/total, world verdict, markers, escalations. Quote the evidence for every failure. Findings are -findings — never edit prose, cases, or fixtures mid-run; report and stop. +findings — never edit prose, cases, or snapshots mid-run; report and stop. Destroy every world (`node tests/prose/run.cjs destroy --world `) except one you are actively citing in a failure report; name any world @@ -66,6 +66,6 @@ kept so it can be removed later. ## Rules -- The walker never sees the `then` block — only the `prompt` command's output reaches it. +- The walker never sees `assert.md` — only the `prompt` command's output reaches it. - A failing case is a finding either way: broken prose, or a stale case. The report says which is suspected; the user decides. -- Never regenerate fixtures or edit cases to make a run pass. +- Never regenerate snapshots or edit cases to make a run pass. diff --git a/CLAUDE.md b/CLAUDE.md index d161a8968..c76990ced 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -210,7 +210,7 @@ Add or update a test alongside any change to engine scripts, adapters, migration ## Prose Tests -`tests/prose/` — end-to-end tests for the prose logic (design: `design/prose-tests.md`). Natural-language cases a walker agent executes through the real skills against a materialised fixture world; run on command via the `/prose-test` dev skill — walks cost tokens, so they are never part of a routine gate. The deterministic perimeter runs under `npm test`: corpus validation (parse, scoped paths, anchors, state-assertion grammar) and fixture golden checks — every `tests/prose/fixtures/{name}/snapshot/` must rebuild byte-identical from its `recipe.cjs` under the frozen clock. When an engine change moves a world, regenerate with `node tests/prose/run.cjs snap {name}` and land the snapshot diff in the same PR; never hand-edit a snapshot. Every review finding lands twice: the fix, and the case that would have caught it. At the end of a PR that touches skill prose, run `node tests/prose/run.cjs select --diff main` and suggest running the intersecting cases. +`tests/prose/` — end-to-end tests for the prose logic (design: `design/prose-tests.md`). A case is a directory under `tests/prose/cases/{case-id}/`: `case.json` (values code branches on), `fixture.md` + `fixture-state.cjs` (the starting world), `act.md` (the coarse instruction a walker agent follows through the real skills), `assert.md` (the expected path, seen only by the asserting agent), optional `assertion-state.cjs` (the world the walk should produce; absent = unchanged), and the two generated snapshots. Nothing is parsed and prose never shares a file with code. Shared pipeline stages live in `tests/prose/mainlines/`. Walks cost tokens — run on command via the `/prose-test` dev skill, never as part of a routine gate. The deterministic perimeter runs under `npm test`: corpus validation plus snapshot goldens — every snapshot must rebuild byte-identical from its recipe under the frozen clock, skipped when nothing feeding it changed. When an engine change moves a world, regenerate with `node tests/prose/run.cjs snap {case-id}` and land the snapshot diff in the same PR; never hand-edit a snapshot. Every review finding lands twice: the fix, and the case that would have caught it. At the end of a PR that touches skill prose, run `node tests/prose/run.cjs select --diff main` and suggest running the intersecting cases. ## Knowledge Base Subsystem diff --git a/design/prose-tests.md b/design/prose-tests.md index 3fb31eb2b..4ccfe21cb 100644 --- a/design/prose-tests.md +++ b/design/prose-tests.md @@ -29,24 +29,32 @@ testing. ## The contract -- **P1 — cases are Given-When-Then.** One case per file, flat: - `tests/prose/{case-id}.md`, filename equal to the id and validated as - such. Nothing groups cases but their `files:` scope, which is what - selection runs on. **Given**: `world_before` (a fixture) and nothing - else — the world before anything happens. **When**: one coarse - instruction — where to enter, what to follow, where to stop — plus - everything the harness feeds in *during* the act: `answers` (the - scripted user responses; an unscripted question is a finding, the - prose asked something the case didn't predict) and `stubs`, each with - the moment it fires. +- **P1 — a case is a directory, one file per element.** + `tests/prose/cases/{case-id}/`: `case.json` (the values code branches + on — origin, scoped files, scripted answers, stub bindings), + `fixture.md` + `fixture-state.cjs` (the starting world in prose and in + engine calls), `act.md` (the coarse instruction), `assert.md` (the + expected path), `assertion-state.cjs` (the world the walk should + produce; absent means "unchanged"), and the two generated snapshots. + Nothing is parsed: JSON is JSON, prose files are read whole, recipes + are required as modules. Prose and code never share a file. +- **P1a — the file boundaries are load-bearing.** `act.md` and + `assert.md` are separate because that is the P4 boundary — the walker + must never see the expected path, and a file boundary enforces it + structurally rather than by convention. `fixture-state.cjs` and + `assertion-state.cjs` are separate because they change for different + reasons: the fixture when the precondition changes, the assertion when + the prose's behaviour changes. Worlds are per-case; duplication is + accepted, because a case readable in one directory beats a + deduplicated fixture library chased across the tree. **Then**: `world_after` (a fixture, or `unchanged`), a granular `trace` of the path the prose should have taken, and any further `notes`. -- **P1a — coarse When, granular Then.** The When never scripts the +- **P1b — coarse act, granular assertion.** `act.md` never scripts the workflow's own steps: if the case says which commands to run, the walker stops deriving the path from the prose and the case would pass - with the skill file deleted. Granularity belongs in the Then, where a - step-by-step expected trace catches the real threat — a walker that + with the skill file deleted. Granularity belongs in `assert.md`, where + a step-by-step expected path catches the real threat — a walker that silently course-corrects around broken prose instead of surfacing it. Claims name behaviour, never coordinates. - **P2 — worlds are real, never imagined.** A fixture is authored by a @@ -103,15 +111,17 @@ testing. ## Architecture -- `tests/prose/{case-id}.md` — the cases, flat. +- `tests/prose/cases/{case-id}/` — the cases, one directory each. +- `tests/prose/mainlines/{work-type}.cjs` — shared pipeline stages, so + a case's state recipes are a few lines of composition. This is where + reuse lives: as functions, not as deduplicated snapshots. - `tests/prose/stubs/{name}.md` — named substitutions: description above a `---` fence, exact bytes below. -- `tests/prose/fixtures/{name}/recipe.cjs` + `snapshot/` — the world - library, holding both start and expected-end worlds. One canonical - fake project per work type so cases read familiarly. -- Runner (node): case parsing and validation, diff-selection, world - building, walker-prompt assembly, world diffing, asserter-prompt - assembly, snapshot regeneration. Owns everything deterministic. +- `tests/prose/lib/` — `cases.cjs` (load and validate a case + directory), `worlds.cjs` (run recipes, snapshot, hash-skip, diff, + materialise), `fake-clock.cjs`. +- `run.cjs` — the deterministic CLI: `list · select · world · prompt · + diff · assert · snap · verify · destroy`. - `/prose-test` skill (thin): dispatch and verdict discipline — invokes the runner, sends the walker, sends the asserter, escalates failures, reports. @@ -132,6 +142,24 @@ testing. ## Log +- 2026-07-25 — A case becomes a directory (Lee, still reviewing in the + TUI). Three faults in the flat-markdown shape, all his: **(1)** code + regex-parsed values out of markdown — the exact pattern this project + spent the analysis-state campaign eliminating. Now `case.json` holds + what code branches on, prose files are read whole, recipes are + modules; nothing is parsed and prose never shares a file with code. + **(2)** `world_before`/`world_after` named foreign fixtures the case + merely pointed at, and the scatter was real — six locations for one + test. A case now owns its worlds: `fixture-state.cjs` and + `assertion-state.cjs`, split because they change for different + reasons, with their snapshots beside them. Duplication accepted; + reuse moved to `mainlines/` as functions. **(3)** The naming follows + the stage it serves — fixture, act, assert — so `assert.md` and + `assertion-state.cjs` sit together by design. Added with it: a + recipe-hash skip, without which per-case worlds would make the gate + grow linearly (proven: 0.09s when nothing moved, full rebuild and + DRIFT when a mainline's content changed). + - 2026-07-25 — The Given-When-Then reshape (Lee, reviewing the corpus in the TUI). Four corrections, all his: **(1)** cases are flat, one per file, filename equal to the id — grouping by work type organised the diff --git a/tests/prose/README.md b/tests/prose/README.md index 735232ac1..0bea54d01 100644 --- a/tests/prose/README.md +++ b/tests/prose/README.md @@ -1,104 +1,115 @@ # Prose tests -End-to-end tests for the prose logic. A case is Given-When-Then: a world -is **arranged** from a committed fixture, an agent **acts** by walking the -real skills as a live session would, and a second agent **asserts** the -resulting trace and world. Design and contract: -[design/prose-tests.md](../../design/prose-tests.md). Run via the -`/prose-test` skill; everything deterministic lives in `run.cjs` and -`lib/`. - -## Layout - -- `{case-id}.md` — one case per file, filename = case id, flat. Nothing - groups cases but their `files:` scope, which is what selection runs on. -- `stubs/{name}.md` — a named substitution: description above a `---` - fence, exact bytes below. The **case** says when it fires. -- `fixtures/{name}/recipe.cjs` — builds a world with real engine calls - under a frozen clock (`lib/fake-clock.cjs`) -- `fixtures/{name}/snapshot/` — the recipe's committed golden output; - regenerate with `node tests/prose/run.cjs snap {name}`, never hand-edit -- `run.cjs` — `list · select · world · prompt · diff · assert · snap · verify · destroy` +End-to-end tests for the prose logic. A case arranges a world from a +committed fixture, an agent acts by walking the real skills as a live +session would, and a second agent asserts the resulting path and world. +Design and contract: [design/prose-tests.md](../../design/prose-tests.md). +Run via the `/prose-test` skill. + +## A case is a directory + +``` +cases/{case-id}/ + case.json the values code branches on + fixture.md optional — prose describing the starting world + fixture-state.cjs builds that world, in engine calls + act.md the coarse instruction given to the walker + assert.md the expected path, given only to the asserter + assertion-state.cjs optional — builds the world the walk should produce + fixture/ generated snapshot + assertion/ generated snapshot +``` + +**Nothing is parsed.** JSON is JSON, prose files are read whole and +relayed into prompts, recipes are `require`d as modules. There is no +format to mis-read: a value code branches on lives in `case.json`, and a +blob code merely relays lives in markdown. + +`act.md` and `assert.md` are separate files because that is the +walker/asserter boundary — the walker must never see the expected path, +and a file boundary enforces that structurally rather than by convention. + +`fixture-state.cjs` and `assertion-state.cjs` are separate because they +change for different reasons: the fixture when the precondition changes, +the assertion when the prose's *behaviour* changes. The assertion state +composes the fixture state plus whatever the walk should have done. A +case with no `assertion-state.cjs` expects its world back untouched — +which is most entry-skill cases, and a strong claim in itself. + +Shared pipeline stages live in `mainlines/{work-type}.cjs`, so a +fixture-state is usually three lines of composition. Worlds are per-case +and duplication is accepted: a case you can read in one directory beats +a deduplicated fixture library you have to chase across the tree. ## Writing a case -**The `when` stays coarse — one instruction.** Where to enter, what to +**The act stays coarse — one instruction.** Where to enter, what to follow, where to stop. Never a step-by-step script of the workflow: if the case tells the walker which commands to run, the walker no longer has to derive the path from the prose, and the case would pass with the skill file deleted. That derivation is the thing under test. -**The `then` is granular.** A step-by-step expected trace is what catches -a walker that silently course-corrected around broken prose — the -divergence shows up as a trace mismatch instead of relying on the walker -to confess. Plus `world_after`: a fixture name, or `unchanged` when the -walk should leave the project exactly as it found it. +**The assertion is granular.** A step-by-step expected path is what +catches a walker that silently course-corrected around broken prose — +the divergence shows up as a mismatch instead of relying on the walker to +confess it. -**Name behaviour, never coordinates.** Step numbers, arm letters, and +**Name behaviour, never coordinates.** Step numbers, arm letters and heading numbering break on cosmetic renumbering — failure for the wrong reason. Write "casing conventions load before the boot pipeline", not -"Step 0.1 before Step 0.2". Anchors are substring fragments matched -against heading text (`#Boot` matches "Step 0.2: Boot"). +"Step 0.1 before Step 0.2". Anchors in `case.json` are substring +fragments matched against heading text (`#Boot` matches "Step 0.2: Boot"). -**Scope `files:` tightly** — diff-selection and the PR-end suggestion run -off it. +**Scope `files` tightly** — selection and the PR-end suggestion run off it. ## Stubs -Prose that dispatches a background agent, or that runs an unscripted +Prose that dispatches a background agent, or runs an unscripted conversation, is walked with that part substituted. A stub is named -content; the case that arms it declares the moment. Stubs sit in `when` -beside `answers:` — both are harness input consumed *during* the act, -and `given` is the world and nothing else: - -``` -### when -{the walk instruction} - -answers: -1. yes — run root cause validation +content in `stubs/{name}.md` — description above a `---` fence, exact +bytes below — and the case arming it owns the moment: -stubs: - - root-cause-validated: when the prose dispatches the validation agent +```json +"stubs": { + "root-cause-validated": "when the engine records a dispatch of kind root-cause-validation" +} ``` The trigger belongs to the case, not the stub, so one stub serves many moments — first dispatch versus re-dispatch, happy path versus recovery. -A stub with no trigger is arrange in disguise, and validation rejects it. +A stub with no trigger is a fixture in disguise, and validation rejects it. **A trigger names an observable event, not a narrative moment.** An engine call with its arguments, a specific menu appearing, a named file being created — something the walker can match with certainty and that shows up in the transcript, so the asserter can confirm the substitution -fired where it should have. "When the prose dispatches an agent" is too -loose the moment a walk covers more than one dispatch; "when the engine -records a dispatch of kind root-cause-validation" is not. Add an ordinal -only where a case genuinely needs one ("on the re-dispatch after the -gaps are addressed"). +fired where it should have. **Whatever a stub covers is not under test in that case.** Stubbing a discussion's session loop means some other case must walk that loop -without stubbing it. The stub boundary is a scope decision, not a -convenience. +unstubbed. The stub boundary is a scope decision, not a convenience. ## Assertion -Deterministic where it can be, judged where it can't. Code computes the -factual delta between the acted world and the expected world; the -asserting agent classifies every difference as volatile (timestamps, git -SHAs, engine-allocated ids) or material, and checks the trace against the -transcript with quoted evidence. No normalisation and no expected-value -literals: the expectation is a whole committed world, built by a recipe, -reviewable as a diff against the world it started from. +Code computes the factual delta between the acted world and the expected +world; the asserting agent classifies every difference as volatile +(timestamps, git SHAs, engine-allocated ids) or material, and checks the +expected path against the transcript with quoted evidence. No +normalisation table, no expected-value literals: the expectation is a +whole committed world, built by a recipe, reviewable as a diff against +the world it started from. -`npm test` validates the corpus (parse, paths, anchors, worlds, stubs, -traces) and rebuild-compares every fixture — both token-free. Only -`/prose-test` spends tokens. +## The gate -## Snapshot escaping +`npm test` validates the corpus and rebuilds every snapshot to +byte-compare it — both token-free, and this repo's only gate since there +is no automated CI. Rebuilds are skipped when nothing feeding a world has +changed; the hash covers each case's recipes, the shared mainlines, and +the engine and knowledge sources, so an engine change invalidates every +hash and forces a full rebuild. Only `/prose-test` spends tokens. Snapshots exclude `.git/`, `.workflows/.knowledge/` (the world builder re-derives the store) and `.claude/skills|agents/` (copied into live -worlds, never part of a fixture), and store `.gitignore` files as -`_gitignore.fixture` so the product-written `.workflows/.gitignore` -cannot ignore fixture content out of this repo. +worlds, never part of a world's own state), and store `.gitignore` files +as `_gitignore.fixture` so the product-written `.workflows/.gitignore` +cannot ignore snapshot content out of this repo. diff --git a/tests/prose/cases/continue-bugfix-routes-to-investigation/act.md b/tests/prose/cases/continue-bugfix-routes-to-investigation/act.md new file mode 100644 index 000000000..0f11a5af9 --- /dev/null +++ b/tests/prose/cases/continue-bugfix-routes-to-investigation/act.md @@ -0,0 +1,4 @@ +Execute skills/workflow-continue-bugfix/SKILL.md with no arguments, as +it is invoked when the caller has no work unit pre-selected. Follow it +through selection and the bugfix's state display. Stop once the state +and its menu have been shown — do not invoke the route. diff --git a/tests/prose/cases/continue-bugfix-routes-to-investigation/assert.md b/tests/prose/cases/continue-bugfix-routes-to-investigation/assert.md new file mode 100644 index 000000000..f975a31a2 --- /dev/null +++ b/tests/prose/cases/continue-bugfix-routes-to-investigation/assert.md @@ -0,0 +1,10 @@ +The prose should have taken this path: + +1. initialisation is casing only — boot and migrations belong to the + entry skill +2. shows the selection display and menu rather than auto-selecting the + only bugfix +3. validates the selection, then shows the bugfix's pipeline state +4. the state shows no phase started, and the continue action routes to + investigation entry — a bugfix's first phase is investigation, never + discussion or specification diff --git a/tests/prose/cases/continue-bugfix-routes-to-investigation/case.json b/tests/prose/cases/continue-bugfix-routes-to-investigation/case.json new file mode 100644 index 000000000..7a221fd7f --- /dev/null +++ b/tests/prose/cases/continue-bugfix-routes-to-investigation/case.json @@ -0,0 +1,11 @@ +{ + "origin": "bugfix mainline — a fresh bug routes to investigation, not discussion", + "files": [ + "skills/workflow-continue-bugfix/SKILL.md", + "skills/workflow-continue-bugfix/references/select-bugfix.md", + "skills/workflow-continue-bugfix/references/bugfix-display-and-menu.md" + ], + "answers": [ + "the number listed against `crash-fix`" + ] +} diff --git a/tests/prose/cases/continue-bugfix-routes-to-investigation/fixture-state.cjs b/tests/prose/cases/continue-bugfix-routes-to-investigation/fixture-state.cjs new file mode 100644 index 000000000..6cd47642e --- /dev/null +++ b/tests/prose/cases/continue-bugfix-routes-to-investigation/fixture-state.cjs @@ -0,0 +1,12 @@ +'use strict'; + +// The bug is captured and nothing has been started. + +const m = require('../../mainlines/bugfix.cjs'); + +module.exports = { + build(h) { + m.init(h); + m.create(h); + }, +}; diff --git a/tests/prose/cases/continue-bugfix-routes-to-investigation/fixture/.claude/settings.json b/tests/prose/cases/continue-bugfix-routes-to-investigation/fixture/.claude/settings.json new file mode 100644 index 000000000..ae019ede5 --- /dev/null +++ b/tests/prose/cases/continue-bugfix-routes-to-investigation/fixture/.claude/settings.json @@ -0,0 +1,9 @@ +{ + "showClearContextOnPlanAccept": true, + "permissions": { + "allow": [ + "Edit(.workflows/**)", + "Bash(mv .workflows/:*)" + ] + } +} diff --git a/tests/prose/cases/continue-bugfix-routes-to-investigation/fixture/.recipe-hash b/tests/prose/cases/continue-bugfix-routes-to-investigation/fixture/.recipe-hash new file mode 100644 index 000000000..259e11ddc --- /dev/null +++ b/tests/prose/cases/continue-bugfix-routes-to-investigation/fixture/.recipe-hash @@ -0,0 +1 @@ +f2d22a09d2680d82822f6042b87baf77dacbf41d4d8cb2b6d26a33d5c4129ff9 diff --git a/tests/prose/cases/continue-bugfix-routes-to-investigation/fixture/.workflows/.state/migrations b/tests/prose/cases/continue-bugfix-routes-to-investigation/fixture/.workflows/.state/migrations new file mode 100644 index 000000000..3c7162acc --- /dev/null +++ b/tests/prose/cases/continue-bugfix-routes-to-investigation/fixture/.workflows/.state/migrations @@ -0,0 +1,52 @@ +001 +002 +003 +004 +005 +006 +007 +008 +009 +010 +011 +012 +013 +014 +015 +016 +017 +018 +019 +020 +021 +022 +023 +024 +025 +026 +027 +028 +029 +030 +031 +032 +033 +034 +035 +036 +037 +038 +039 +040 +041 +042 +043 +044 +045 +046 +047 +048 +049 +050 +051 +052 diff --git a/tests/prose/cases/continue-bugfix-routes-to-investigation/fixture/.workflows/_gitignore.fixture b/tests/prose/cases/continue-bugfix-routes-to-investigation/fixture/.workflows/_gitignore.fixture new file mode 100644 index 000000000..779cbf28a --- /dev/null +++ b/tests/prose/cases/continue-bugfix-routes-to-investigation/fixture/.workflows/_gitignore.fixture @@ -0,0 +1,2 @@ +.cache/ +.manifest.json.*.tmp diff --git a/tests/prose/cases/continue-bugfix-routes-to-investigation/fixture/.workflows/crash-fix/discovery/sessions/session-001.md b/tests/prose/cases/continue-bugfix-routes-to-investigation/fixture/.workflows/crash-fix/discovery/sessions/session-001.md new file mode 100644 index 000000000..6dc56d4e3 --- /dev/null +++ b/tests/prose/cases/continue-bugfix-routes-to-investigation/fixture/.workflows/crash-fix/discovery/sessions/session-001.md @@ -0,0 +1,40 @@ +# Discovery Session 001 + +Date: 2026-01-01 +Work unit: crash-fix + +## Description (as of session) + +Checkout crashes when an order has no shipping address. + +## Seed + +(none) + +## Imports + +(none) + +## Map State at Start + +(n/a — single-topic work) + +## Exploration + +Reported by two users this week: the checkout page 500s at the +payment step for orders with no shipping address. Reproducible on +staging with a digital-only basket. No error surfaces to the user — +the page just fails. Confirmed as a bugfix; no design question here, +so it routes straight to investigation. + +## Edits + +(none) + +## Topics Identified + +(none) + +## Conclusion + +(none) diff --git a/tests/prose/cases/continue-bugfix-routes-to-investigation/fixture/.workflows/crash-fix/manifest.json b/tests/prose/cases/continue-bugfix-routes-to-investigation/fixture/.workflows/crash-fix/manifest.json new file mode 100644 index 000000000..eaaa56199 --- /dev/null +++ b/tests/prose/cases/continue-bugfix-routes-to-investigation/fixture/.workflows/crash-fix/manifest.json @@ -0,0 +1,8 @@ +{ + "name": "crash-fix", + "work_type": "bugfix", + "status": "in-progress", + "created": "2026-01-01", + "description": "Checkout crashes when an order has no shipping address", + "phases": {} +} diff --git a/tests/prose/cases/continue-bugfix-routes-to-investigation/fixture/.workflows/manifest.json b/tests/prose/cases/continue-bugfix-routes-to-investigation/fixture/.workflows/manifest.json new file mode 100644 index 000000000..1cd498ed5 --- /dev/null +++ b/tests/prose/cases/continue-bugfix-routes-to-investigation/fixture/.workflows/manifest.json @@ -0,0 +1,7 @@ +{ + "work_units": { + "crash-fix": { + "work_type": "bugfix" + } + } +} diff --git a/tests/prose/cases/continue-feature-routes-to-discussion/act.md b/tests/prose/cases/continue-feature-routes-to-discussion/act.md new file mode 100644 index 000000000..93e469eef --- /dev/null +++ b/tests/prose/cases/continue-feature-routes-to-discussion/act.md @@ -0,0 +1,4 @@ +Execute skills/workflow-continue-feature/SKILL.md with no arguments, as +it is invoked when the caller has no work unit pre-selected. Follow it +through selection and the feature's state display. Stop once the state +and its menu have been shown — do not invoke the route. diff --git a/tests/prose/cases/continue-feature-routes-to-discussion/assert.md b/tests/prose/cases/continue-feature-routes-to-discussion/assert.md new file mode 100644 index 000000000..b789b6b49 --- /dev/null +++ b/tests/prose/cases/continue-feature-routes-to-discussion/assert.md @@ -0,0 +1,13 @@ +The prose should have taken this path: + +1. initialisation is casing only — no boot and no migrations, which the + entry skill has already guaranteed +2. shows the selection display and menu even though `pay` is the only + feature — never auto-selects +3. validates the selection, then shows the feature's pipeline state +4. the state shows no phase started, and the menu's continue action + routes to discussion entry for work type feature, work unit pay + +Further claims: + +- navigation only reads: no phase is started by looking at it diff --git a/tests/prose/cases/continue-feature-routes-to-discussion/case.json b/tests/prose/cases/continue-feature-routes-to-discussion/case.json new file mode 100644 index 000000000..33c362b36 --- /dev/null +++ b/tests/prose/cases/continue-feature-routes-to-discussion/case.json @@ -0,0 +1,11 @@ +{ + "origin": "feature mainline — navigation shows state and routes to the next phase", + "files": [ + "skills/workflow-continue-feature/SKILL.md", + "skills/workflow-continue-feature/references/select-feature.md", + "skills/workflow-continue-feature/references/feature-display-and-menu.md" + ], + "answers": [ + "the number listed against `pay`" + ] +} diff --git a/tests/prose/cases/continue-feature-routes-to-discussion/fixture-state.cjs b/tests/prose/cases/continue-feature-routes-to-discussion/fixture-state.cjs new file mode 100644 index 000000000..e4b78cd71 --- /dev/null +++ b/tests/prose/cases/continue-feature-routes-to-discussion/fixture-state.cjs @@ -0,0 +1,12 @@ +'use strict'; + +// The feature exists and nothing has been started. + +const m = require('../../mainlines/feature.cjs'); + +module.exports = { + build(h) { + m.init(h); + m.create(h); + }, +}; diff --git a/tests/prose/cases/continue-feature-routes-to-discussion/fixture/.claude/settings.json b/tests/prose/cases/continue-feature-routes-to-discussion/fixture/.claude/settings.json new file mode 100644 index 000000000..ae019ede5 --- /dev/null +++ b/tests/prose/cases/continue-feature-routes-to-discussion/fixture/.claude/settings.json @@ -0,0 +1,9 @@ +{ + "showClearContextOnPlanAccept": true, + "permissions": { + "allow": [ + "Edit(.workflows/**)", + "Bash(mv .workflows/:*)" + ] + } +} diff --git a/tests/prose/cases/continue-feature-routes-to-discussion/fixture/.recipe-hash b/tests/prose/cases/continue-feature-routes-to-discussion/fixture/.recipe-hash new file mode 100644 index 000000000..92519878f --- /dev/null +++ b/tests/prose/cases/continue-feature-routes-to-discussion/fixture/.recipe-hash @@ -0,0 +1 @@ +5b0a7d802b3a436a4656bc000122451bb015f2250f8fef47b1d05f3ea51ba96b diff --git a/tests/prose/cases/continue-feature-routes-to-discussion/fixture/.workflows/.state/migrations b/tests/prose/cases/continue-feature-routes-to-discussion/fixture/.workflows/.state/migrations new file mode 100644 index 000000000..3c7162acc --- /dev/null +++ b/tests/prose/cases/continue-feature-routes-to-discussion/fixture/.workflows/.state/migrations @@ -0,0 +1,52 @@ +001 +002 +003 +004 +005 +006 +007 +008 +009 +010 +011 +012 +013 +014 +015 +016 +017 +018 +019 +020 +021 +022 +023 +024 +025 +026 +027 +028 +029 +030 +031 +032 +033 +034 +035 +036 +037 +038 +039 +040 +041 +042 +043 +044 +045 +046 +047 +048 +049 +050 +051 +052 diff --git a/tests/prose/cases/continue-feature-routes-to-discussion/fixture/.workflows/_gitignore.fixture b/tests/prose/cases/continue-feature-routes-to-discussion/fixture/.workflows/_gitignore.fixture new file mode 100644 index 000000000..779cbf28a --- /dev/null +++ b/tests/prose/cases/continue-feature-routes-to-discussion/fixture/.workflows/_gitignore.fixture @@ -0,0 +1,2 @@ +.cache/ +.manifest.json.*.tmp diff --git a/tests/prose/cases/continue-feature-routes-to-discussion/fixture/.workflows/manifest.json b/tests/prose/cases/continue-feature-routes-to-discussion/fixture/.workflows/manifest.json new file mode 100644 index 000000000..987f6c236 --- /dev/null +++ b/tests/prose/cases/continue-feature-routes-to-discussion/fixture/.workflows/manifest.json @@ -0,0 +1,7 @@ +{ + "work_units": { + "pay": { + "work_type": "feature" + } + } +} diff --git a/tests/prose/cases/continue-feature-routes-to-discussion/fixture/.workflows/pay/discovery/sessions/session-001.md b/tests/prose/cases/continue-feature-routes-to-discussion/fixture/.workflows/pay/discovery/sessions/session-001.md new file mode 100644 index 000000000..2aab96961 --- /dev/null +++ b/tests/prose/cases/continue-feature-routes-to-discussion/fixture/.workflows/pay/discovery/sessions/session-001.md @@ -0,0 +1,39 @@ +# Discovery Session 001 + +Date: 2026-01-01 +Work unit: pay + +## Description (as of session) + +Accept card payments at checkout. + +## Seed + +(none) + +## Imports + +(none) + +## Map State at Start + +(n/a — single-topic work) + +## Exploration + +Shaped as a single feature: accept card payments at checkout using +the existing gateway account. Card-only for v1 came up early and was +softly agreed; wallet support was noted as a likely deferral. No +research need surfaced — routed straight to discussion. + +## Edits + +(none) + +## Topics Identified + +(none) + +## Conclusion + +(none) diff --git a/tests/prose/cases/continue-feature-routes-to-discussion/fixture/.workflows/pay/manifest.json b/tests/prose/cases/continue-feature-routes-to-discussion/fixture/.workflows/pay/manifest.json new file mode 100644 index 000000000..1cf8afb0e --- /dev/null +++ b/tests/prose/cases/continue-feature-routes-to-discussion/fixture/.workflows/pay/manifest.json @@ -0,0 +1,8 @@ +{ + "name": "pay", + "work_type": "feature", + "status": "in-progress", + "created": "2026-01-01", + "description": "Accept card payments at checkout", + "phases": {} +} diff --git a/tests/prose/cases/discussion-entry-seeds-from-carrier/act.md b/tests/prose/cases/discussion-entry-seeds-from-carrier/act.md new file mode 100644 index 000000000..6e82ff2ab --- /dev/null +++ b/tests/prose/cases/discussion-entry-seeds-from-carrier/act.md @@ -0,0 +1,4 @@ +Execute skills/workflow-discussion-entry/SKILL.md with arguments +$0=feature, $1=pay. Follow it to the point where the handoff to the +processing skill is constructed; record the handoff block and stop. Do +not execute the processing skill's instructions. diff --git a/tests/prose/discussion-entry-seeds-from-carrier.md b/tests/prose/cases/discussion-entry-seeds-from-carrier/assert.md similarity index 50% rename from tests/prose/discussion-entry-seeds-from-carrier.md rename to tests/prose/cases/discussion-entry-seeds-from-carrier/assert.md index 4128b458c..e0a0c97ba 100644 --- a/tests/prose/discussion-entry-seeds-from-carrier.md +++ b/tests/prose/cases/discussion-entry-seeds-from-carrier/assert.md @@ -1,25 +1,5 @@ -## case: discussion-entry-seeds-from-carrier -- origin: feature mainline — single-phase work seeds from its durable carrier -- files: - - skills/workflow-discussion-entry/SKILL.md - - skills/workflow-shared/references/ensure-discovery-item.md +The prose should have taken this path: -### given - -world_before: feature-created - -### when - -Execute skills/workflow-discussion-entry/SKILL.md with arguments -$0=feature, $1=pay. Follow it to the point where the handoff to the -processing skill is constructed; record the handoff block and stop. Do -not execute the processing skill's instructions. - -### then - -world_after: unchanged - -trace: 1. resolves the topic to the work unit, since only an epic is given one explicitly 2. reads the discussion status, finds nothing, and takes the new-entry @@ -30,6 +10,7 @@ trace: exploration, asking the user nothing 5. hands off to the discussion processing skill for pay -notes: +Further claims: + - the entry skill records nothing: the discussion item is created by the processing skill, not by the walk to it diff --git a/tests/prose/cases/discussion-entry-seeds-from-carrier/case.json b/tests/prose/cases/discussion-entry-seeds-from-carrier/case.json new file mode 100644 index 000000000..f5bc80e4b --- /dev/null +++ b/tests/prose/cases/discussion-entry-seeds-from-carrier/case.json @@ -0,0 +1,7 @@ +{ + "origin": "feature mainline — single-phase work seeds from its durable carrier", + "files": [ + "skills/workflow-discussion-entry/SKILL.md", + "skills/workflow-shared/references/ensure-discovery-item.md" + ] +} diff --git a/tests/prose/cases/discussion-entry-seeds-from-carrier/fixture-state.cjs b/tests/prose/cases/discussion-entry-seeds-from-carrier/fixture-state.cjs new file mode 100644 index 000000000..e4b78cd71 --- /dev/null +++ b/tests/prose/cases/discussion-entry-seeds-from-carrier/fixture-state.cjs @@ -0,0 +1,12 @@ +'use strict'; + +// The feature exists and nothing has been started. + +const m = require('../../mainlines/feature.cjs'); + +module.exports = { + build(h) { + m.init(h); + m.create(h); + }, +}; diff --git a/tests/prose/cases/discussion-entry-seeds-from-carrier/fixture/.claude/settings.json b/tests/prose/cases/discussion-entry-seeds-from-carrier/fixture/.claude/settings.json new file mode 100644 index 000000000..ae019ede5 --- /dev/null +++ b/tests/prose/cases/discussion-entry-seeds-from-carrier/fixture/.claude/settings.json @@ -0,0 +1,9 @@ +{ + "showClearContextOnPlanAccept": true, + "permissions": { + "allow": [ + "Edit(.workflows/**)", + "Bash(mv .workflows/:*)" + ] + } +} diff --git a/tests/prose/cases/discussion-entry-seeds-from-carrier/fixture/.recipe-hash b/tests/prose/cases/discussion-entry-seeds-from-carrier/fixture/.recipe-hash new file mode 100644 index 000000000..92519878f --- /dev/null +++ b/tests/prose/cases/discussion-entry-seeds-from-carrier/fixture/.recipe-hash @@ -0,0 +1 @@ +5b0a7d802b3a436a4656bc000122451bb015f2250f8fef47b1d05f3ea51ba96b diff --git a/tests/prose/cases/discussion-entry-seeds-from-carrier/fixture/.workflows/.state/migrations b/tests/prose/cases/discussion-entry-seeds-from-carrier/fixture/.workflows/.state/migrations new file mode 100644 index 000000000..3c7162acc --- /dev/null +++ b/tests/prose/cases/discussion-entry-seeds-from-carrier/fixture/.workflows/.state/migrations @@ -0,0 +1,52 @@ +001 +002 +003 +004 +005 +006 +007 +008 +009 +010 +011 +012 +013 +014 +015 +016 +017 +018 +019 +020 +021 +022 +023 +024 +025 +026 +027 +028 +029 +030 +031 +032 +033 +034 +035 +036 +037 +038 +039 +040 +041 +042 +043 +044 +045 +046 +047 +048 +049 +050 +051 +052 diff --git a/tests/prose/cases/discussion-entry-seeds-from-carrier/fixture/.workflows/_gitignore.fixture b/tests/prose/cases/discussion-entry-seeds-from-carrier/fixture/.workflows/_gitignore.fixture new file mode 100644 index 000000000..779cbf28a --- /dev/null +++ b/tests/prose/cases/discussion-entry-seeds-from-carrier/fixture/.workflows/_gitignore.fixture @@ -0,0 +1,2 @@ +.cache/ +.manifest.json.*.tmp diff --git a/tests/prose/cases/discussion-entry-seeds-from-carrier/fixture/.workflows/manifest.json b/tests/prose/cases/discussion-entry-seeds-from-carrier/fixture/.workflows/manifest.json new file mode 100644 index 000000000..987f6c236 --- /dev/null +++ b/tests/prose/cases/discussion-entry-seeds-from-carrier/fixture/.workflows/manifest.json @@ -0,0 +1,7 @@ +{ + "work_units": { + "pay": { + "work_type": "feature" + } + } +} diff --git a/tests/prose/cases/discussion-entry-seeds-from-carrier/fixture/.workflows/pay/discovery/sessions/session-001.md b/tests/prose/cases/discussion-entry-seeds-from-carrier/fixture/.workflows/pay/discovery/sessions/session-001.md new file mode 100644 index 000000000..2aab96961 --- /dev/null +++ b/tests/prose/cases/discussion-entry-seeds-from-carrier/fixture/.workflows/pay/discovery/sessions/session-001.md @@ -0,0 +1,39 @@ +# Discovery Session 001 + +Date: 2026-01-01 +Work unit: pay + +## Description (as of session) + +Accept card payments at checkout. + +## Seed + +(none) + +## Imports + +(none) + +## Map State at Start + +(n/a — single-topic work) + +## Exploration + +Shaped as a single feature: accept card payments at checkout using +the existing gateway account. Card-only for v1 came up early and was +softly agreed; wallet support was noted as a likely deferral. No +research need surfaced — routed straight to discussion. + +## Edits + +(none) + +## Topics Identified + +(none) + +## Conclusion + +(none) diff --git a/tests/prose/cases/discussion-entry-seeds-from-carrier/fixture/.workflows/pay/manifest.json b/tests/prose/cases/discussion-entry-seeds-from-carrier/fixture/.workflows/pay/manifest.json new file mode 100644 index 000000000..1cf8afb0e --- /dev/null +++ b/tests/prose/cases/discussion-entry-seeds-from-carrier/fixture/.workflows/pay/manifest.json @@ -0,0 +1,8 @@ +{ + "name": "pay", + "work_type": "feature", + "status": "in-progress", + "created": "2026-01-01", + "description": "Accept card payments at checkout", + "phases": {} +} diff --git a/tests/prose/cases/implementation-picks-first-task/act.md b/tests/prose/cases/implementation-picks-first-task/act.md new file mode 100644 index 000000000..cdf6faee0 --- /dev/null +++ b/tests/prose/cases/implementation-picks-first-task/act.md @@ -0,0 +1,4 @@ +Execute skills/workflow-implementation-entry/SKILL.md with arguments +$0=feature, $1=pay, then continue into the processing skill it invokes. +Follow the flow until the prose directs you to begin building the first +task. Stop there — write no implementation code and no tests. diff --git a/tests/prose/cases/implementation-picks-first-task/assert.md b/tests/prose/cases/implementation-picks-first-task/assert.md new file mode 100644 index 000000000..e09c80080 --- /dev/null +++ b/tests/prose/cases/implementation-picks-first-task/assert.md @@ -0,0 +1,20 @@ +The prose should have taken this path: + +1. the plan gate renders empty, and no implementation item exists, so + this is a new entry +2. dependency validation returns immediately — external dependencies are + an epic concern +3. the environment check finds no setup document and asks the question, + gathering the answer without acting on it +4. resume detection initialises tracking and reports the created mode — + the fresh path, which commits the start of implementation, never the + resuming-from-a-previous-session note +5. environment setup records the answer as a setup document and commits + it, so the question is not asked again in a later session +6. reads the plan through the format's own reading procedure and finds + pay-1-1 next: phase one, first task, nothing completed +7. starts pay-1-1 and stops where the prose hands over to building + +Further claims: + +- the second task is neither started nor touched diff --git a/tests/prose/cases/implementation-picks-first-task/assertion-state.cjs b/tests/prose/cases/implementation-picks-first-task/assertion-state.cjs new file mode 100644 index 000000000..c065db590 --- /dev/null +++ b/tests/prose/cases/implementation-picks-first-task/assertion-state.cjs @@ -0,0 +1,22 @@ +'use strict'; + +// The fixture state plus what the walk should have done: resume +// detection initialised tracking and committed the start of +// implementation, the environment answer was recorded so it is never +// asked again, and the first task was started. No task work — the walk +// stops where the prose hands over to building. + +const fixture = require('./fixture-state.cjs'); +const m = require('../../mainlines/feature.cjs'); + +module.exports = { + build(h) { + fixture.build(h); + + h.engine('task', 'init', m.WU, m.WU); + h.engine('commit', m.WU, '-m', `impl(${m.WU}): start implementation`); + h.write('.workflows/.state/environment-setup.md', 'No special setup required.\n'); + h.engine('commit', '--workflows', '-m', `impl(${m.WU}): record environment setup`); + h.engine('task', 'start', m.WU, m.WU, `${m.WU}-1-1`); + }, +}; diff --git a/tests/prose/cases/implementation-picks-first-task/assertion/.claude/settings.json b/tests/prose/cases/implementation-picks-first-task/assertion/.claude/settings.json new file mode 100644 index 000000000..ae019ede5 --- /dev/null +++ b/tests/prose/cases/implementation-picks-first-task/assertion/.claude/settings.json @@ -0,0 +1,9 @@ +{ + "showClearContextOnPlanAccept": true, + "permissions": { + "allow": [ + "Edit(.workflows/**)", + "Bash(mv .workflows/:*)" + ] + } +} diff --git a/tests/prose/cases/implementation-picks-first-task/assertion/.recipe-hash b/tests/prose/cases/implementation-picks-first-task/assertion/.recipe-hash new file mode 100644 index 000000000..a29d91438 --- /dev/null +++ b/tests/prose/cases/implementation-picks-first-task/assertion/.recipe-hash @@ -0,0 +1 @@ +e963357b5150c81c98156db6f527db5bf50bce292bb39b107e14ca98c6f5366a diff --git a/tests/prose/cases/implementation-picks-first-task/assertion/.workflows/.state/environment-setup.md b/tests/prose/cases/implementation-picks-first-task/assertion/.workflows/.state/environment-setup.md new file mode 100644 index 000000000..56b13047e --- /dev/null +++ b/tests/prose/cases/implementation-picks-first-task/assertion/.workflows/.state/environment-setup.md @@ -0,0 +1 @@ +No special setup required. diff --git a/tests/prose/cases/implementation-picks-first-task/assertion/.workflows/.state/migrations b/tests/prose/cases/implementation-picks-first-task/assertion/.workflows/.state/migrations new file mode 100644 index 000000000..3c7162acc --- /dev/null +++ b/tests/prose/cases/implementation-picks-first-task/assertion/.workflows/.state/migrations @@ -0,0 +1,52 @@ +001 +002 +003 +004 +005 +006 +007 +008 +009 +010 +011 +012 +013 +014 +015 +016 +017 +018 +019 +020 +021 +022 +023 +024 +025 +026 +027 +028 +029 +030 +031 +032 +033 +034 +035 +036 +037 +038 +039 +040 +041 +042 +043 +044 +045 +046 +047 +048 +049 +050 +051 +052 diff --git a/tests/prose/cases/implementation-picks-first-task/assertion/.workflows/_gitignore.fixture b/tests/prose/cases/implementation-picks-first-task/assertion/.workflows/_gitignore.fixture new file mode 100644 index 000000000..779cbf28a --- /dev/null +++ b/tests/prose/cases/implementation-picks-first-task/assertion/.workflows/_gitignore.fixture @@ -0,0 +1,2 @@ +.cache/ +.manifest.json.*.tmp diff --git a/tests/prose/cases/implementation-picks-first-task/assertion/.workflows/manifest.json b/tests/prose/cases/implementation-picks-first-task/assertion/.workflows/manifest.json new file mode 100644 index 000000000..987f6c236 --- /dev/null +++ b/tests/prose/cases/implementation-picks-first-task/assertion/.workflows/manifest.json @@ -0,0 +1,7 @@ +{ + "work_units": { + "pay": { + "work_type": "feature" + } + } +} diff --git a/tests/prose/cases/implementation-picks-first-task/assertion/.workflows/pay/discovery/sessions/session-001.md b/tests/prose/cases/implementation-picks-first-task/assertion/.workflows/pay/discovery/sessions/session-001.md new file mode 100644 index 000000000..2aab96961 --- /dev/null +++ b/tests/prose/cases/implementation-picks-first-task/assertion/.workflows/pay/discovery/sessions/session-001.md @@ -0,0 +1,39 @@ +# Discovery Session 001 + +Date: 2026-01-01 +Work unit: pay + +## Description (as of session) + +Accept card payments at checkout. + +## Seed + +(none) + +## Imports + +(none) + +## Map State at Start + +(n/a — single-topic work) + +## Exploration + +Shaped as a single feature: accept card payments at checkout using +the existing gateway account. Card-only for v1 came up early and was +softly agreed; wallet support was noted as a likely deferral. No +research need surfaced — routed straight to discussion. + +## Edits + +(none) + +## Topics Identified + +(none) + +## Conclusion + +(none) diff --git a/tests/prose/cases/implementation-picks-first-task/assertion/.workflows/pay/discussion/pay.md b/tests/prose/cases/implementation-picks-first-task/assertion/.workflows/pay/discussion/pay.md new file mode 100644 index 000000000..ec0cfc62f --- /dev/null +++ b/tests/prose/cases/implementation-picks-first-task/assertion/.workflows/pay/discussion/pay.md @@ -0,0 +1,15 @@ +# Discussion — pay + +## Context + +Accept card payments at checkout using the existing gateway account. + +## Decisions + +- Use the existing gateway account; no new provider onboarding. +- Card-only for v1 — wallet support is deferred. +- Webhooks confirm capture; the checkout never polls. + +## Deferred + +- Wallet support (Apple/Google Pay) — revisit after v1. diff --git a/tests/prose/cases/implementation-picks-first-task/assertion/.workflows/pay/manifest.json b/tests/prose/cases/implementation-picks-first-task/assertion/.workflows/pay/manifest.json new file mode 100644 index 000000000..23023a119 --- /dev/null +++ b/tests/prose/cases/implementation-picks-first-task/assertion/.workflows/pay/manifest.json @@ -0,0 +1,70 @@ +{ + "name": "pay", + "work_type": "feature", + "status": "in-progress", + "created": "2026-01-01", + "description": "Accept card payments at checkout", + "phases": { + "discussion": { + "items": { + "pay": { + "status": "completed" + } + } + }, + "specification": { + "items": { + "pay": { + "status": "completed", + "sources": { + "pay": { + "status": "incorporated" + } + } + } + } + }, + "planning": { + "items": { + "pay": { + "status": "completed", + "format": "local-markdown", + "task_list_gate_mode": "gated", + "author_gate_mode": "gated", + "finding_gate_mode": "gated", + "review_cycle": 0, + "phase": 1, + "task": null, + "task_map": { + "pay-1-1": "pay-1-1", + "pay-1-2": "pay-1-2" + }, + "storage_paths": [], + "approvals": { + "structure": "2026-01-01", + "tasks": { + "p1": "2026-01-01" + } + } + } + } + }, + "implementation": { + "items": { + "pay": { + "status": "in-progress", + "task_gate_mode": "gated", + "fix_gate_mode": "gated", + "analysis_gate_mode": "gated", + "fix_attempts": 0, + "analysis_cycle_total": 0, + "analysis_cycle_session": 0, + "linters": [], + "project_skills": [], + "current_phase": 1, + "current_task": "pay-1-1" + } + } + } + } +} diff --git a/tests/prose/cases/implementation-picks-first-task/assertion/.workflows/pay/planning/pay/planning.md b/tests/prose/cases/implementation-picks-first-task/assertion/.workflows/pay/planning/pay/planning.md new file mode 100644 index 000000000..9131d98c7 --- /dev/null +++ b/tests/prose/cases/implementation-picks-first-task/assertion/.workflows/pay/planning/pay/planning.md @@ -0,0 +1,8 @@ +# Plan — pay + +## Phase 1: Payment core + +| Task | Title | +|------|-------| +| pay-1-1 | Create Payment Intent | +| pay-1-2 | Handle Capture Webhooks | diff --git a/tests/prose/cases/implementation-picks-first-task/assertion/.workflows/pay/planning/pay/tasks/pay-1-1.md b/tests/prose/cases/implementation-picks-first-task/assertion/.workflows/pay/planning/pay/tasks/pay-1-1.md new file mode 100644 index 000000000..f7d43a082 --- /dev/null +++ b/tests/prose/cases/implementation-picks-first-task/assertion/.workflows/pay/planning/pay/tasks/pay-1-1.md @@ -0,0 +1,10 @@ +--- +id: pay-1-1 +phase: 1 +status: pending +created: 2026-01-01 +--- + +# Create Payment Intent + +Create a gateway payment intent when checkout begins and attach it to the order. diff --git a/tests/prose/cases/implementation-picks-first-task/assertion/.workflows/pay/planning/pay/tasks/pay-1-2.md b/tests/prose/cases/implementation-picks-first-task/assertion/.workflows/pay/planning/pay/tasks/pay-1-2.md new file mode 100644 index 000000000..8fb22697d --- /dev/null +++ b/tests/prose/cases/implementation-picks-first-task/assertion/.workflows/pay/planning/pay/tasks/pay-1-2.md @@ -0,0 +1,10 @@ +--- +id: pay-1-2 +phase: 1 +status: pending +created: 2026-01-01 +--- + +# Handle Capture Webhooks + +Consume gateway capture webhooks and mark the order paid; no polling path. diff --git a/tests/prose/cases/implementation-picks-first-task/assertion/.workflows/pay/specification/pay/specification.md b/tests/prose/cases/implementation-picks-first-task/assertion/.workflows/pay/specification/pay/specification.md new file mode 100644 index 000000000..29a5f4322 --- /dev/null +++ b/tests/prose/cases/implementation-picks-first-task/assertion/.workflows/pay/specification/pay/specification.md @@ -0,0 +1,11 @@ +# Specification — pay + +## Requirements + +- Checkout creates a payment intent against the existing gateway account. +- Card payments only; wallet flows are out of scope for v1. +- Capture is confirmed by gateway webhook, never by polling. + +## Out of scope + +- Wallet support (deferred by discussion). diff --git a/tests/prose/cases/implementation-picks-first-task/case.json b/tests/prose/cases/implementation-picks-first-task/case.json new file mode 100644 index 000000000..2a3a9a5d0 --- /dev/null +++ b/tests/prose/cases/implementation-picks-first-task/case.json @@ -0,0 +1,16 @@ +{ + "origin": "feature mainline — entry through resume detection to the first task", + "files": [ + "skills/workflow-implementation-entry/SKILL.md", + "skills/workflow-implementation-entry/references/validate-phase.md", + "skills/workflow-implementation-entry/references/validate-dependencies.md", + "skills/workflow-implementation-entry/references/environment-check.md", + "skills/workflow-implementation-process/SKILL.md", + "skills/workflow-implementation-process/references/environment-setup.md", + "skills/workflow-implementation-process/references/task-loop.md", + "skills/workflow-planning-process/references/output-formats/local-markdown/reading.md" + ], + "answers": [ + "none — no special setup is needed" + ] +} diff --git a/tests/prose/cases/implementation-picks-first-task/fixture-state.cjs b/tests/prose/cases/implementation-picks-first-task/fixture-state.cjs new file mode 100644 index 000000000..88f1d7dfe --- /dev/null +++ b/tests/prose/cases/implementation-picks-first-task/fixture-state.cjs @@ -0,0 +1,15 @@ +'use strict'; + +// The plan is authored and complete; implementation has not begun. + +const m = require('../../mainlines/feature.cjs'); + +module.exports = { + build(h) { + m.init(h); + m.create(h); + m.discuss(h); + m.specify(h); + m.plan(h); + }, +}; diff --git a/tests/prose/cases/implementation-picks-first-task/fixture/.claude/settings.json b/tests/prose/cases/implementation-picks-first-task/fixture/.claude/settings.json new file mode 100644 index 000000000..ae019ede5 --- /dev/null +++ b/tests/prose/cases/implementation-picks-first-task/fixture/.claude/settings.json @@ -0,0 +1,9 @@ +{ + "showClearContextOnPlanAccept": true, + "permissions": { + "allow": [ + "Edit(.workflows/**)", + "Bash(mv .workflows/:*)" + ] + } +} diff --git a/tests/prose/cases/implementation-picks-first-task/fixture/.recipe-hash b/tests/prose/cases/implementation-picks-first-task/fixture/.recipe-hash new file mode 100644 index 000000000..a29d91438 --- /dev/null +++ b/tests/prose/cases/implementation-picks-first-task/fixture/.recipe-hash @@ -0,0 +1 @@ +e963357b5150c81c98156db6f527db5bf50bce292bb39b107e14ca98c6f5366a diff --git a/tests/prose/cases/implementation-picks-first-task/fixture/.workflows/.state/migrations b/tests/prose/cases/implementation-picks-first-task/fixture/.workflows/.state/migrations new file mode 100644 index 000000000..3c7162acc --- /dev/null +++ b/tests/prose/cases/implementation-picks-first-task/fixture/.workflows/.state/migrations @@ -0,0 +1,52 @@ +001 +002 +003 +004 +005 +006 +007 +008 +009 +010 +011 +012 +013 +014 +015 +016 +017 +018 +019 +020 +021 +022 +023 +024 +025 +026 +027 +028 +029 +030 +031 +032 +033 +034 +035 +036 +037 +038 +039 +040 +041 +042 +043 +044 +045 +046 +047 +048 +049 +050 +051 +052 diff --git a/tests/prose/cases/implementation-picks-first-task/fixture/.workflows/_gitignore.fixture b/tests/prose/cases/implementation-picks-first-task/fixture/.workflows/_gitignore.fixture new file mode 100644 index 000000000..779cbf28a --- /dev/null +++ b/tests/prose/cases/implementation-picks-first-task/fixture/.workflows/_gitignore.fixture @@ -0,0 +1,2 @@ +.cache/ +.manifest.json.*.tmp diff --git a/tests/prose/cases/implementation-picks-first-task/fixture/.workflows/manifest.json b/tests/prose/cases/implementation-picks-first-task/fixture/.workflows/manifest.json new file mode 100644 index 000000000..987f6c236 --- /dev/null +++ b/tests/prose/cases/implementation-picks-first-task/fixture/.workflows/manifest.json @@ -0,0 +1,7 @@ +{ + "work_units": { + "pay": { + "work_type": "feature" + } + } +} diff --git a/tests/prose/cases/implementation-picks-first-task/fixture/.workflows/pay/discovery/sessions/session-001.md b/tests/prose/cases/implementation-picks-first-task/fixture/.workflows/pay/discovery/sessions/session-001.md new file mode 100644 index 000000000..2aab96961 --- /dev/null +++ b/tests/prose/cases/implementation-picks-first-task/fixture/.workflows/pay/discovery/sessions/session-001.md @@ -0,0 +1,39 @@ +# Discovery Session 001 + +Date: 2026-01-01 +Work unit: pay + +## Description (as of session) + +Accept card payments at checkout. + +## Seed + +(none) + +## Imports + +(none) + +## Map State at Start + +(n/a — single-topic work) + +## Exploration + +Shaped as a single feature: accept card payments at checkout using +the existing gateway account. Card-only for v1 came up early and was +softly agreed; wallet support was noted as a likely deferral. No +research need surfaced — routed straight to discussion. + +## Edits + +(none) + +## Topics Identified + +(none) + +## Conclusion + +(none) diff --git a/tests/prose/cases/implementation-picks-first-task/fixture/.workflows/pay/discussion/pay.md b/tests/prose/cases/implementation-picks-first-task/fixture/.workflows/pay/discussion/pay.md new file mode 100644 index 000000000..ec0cfc62f --- /dev/null +++ b/tests/prose/cases/implementation-picks-first-task/fixture/.workflows/pay/discussion/pay.md @@ -0,0 +1,15 @@ +# Discussion — pay + +## Context + +Accept card payments at checkout using the existing gateway account. + +## Decisions + +- Use the existing gateway account; no new provider onboarding. +- Card-only for v1 — wallet support is deferred. +- Webhooks confirm capture; the checkout never polls. + +## Deferred + +- Wallet support (Apple/Google Pay) — revisit after v1. diff --git a/tests/prose/cases/implementation-picks-first-task/fixture/.workflows/pay/manifest.json b/tests/prose/cases/implementation-picks-first-task/fixture/.workflows/pay/manifest.json new file mode 100644 index 000000000..91f99aef4 --- /dev/null +++ b/tests/prose/cases/implementation-picks-first-task/fixture/.workflows/pay/manifest.json @@ -0,0 +1,53 @@ +{ + "name": "pay", + "work_type": "feature", + "status": "in-progress", + "created": "2026-01-01", + "description": "Accept card payments at checkout", + "phases": { + "discussion": { + "items": { + "pay": { + "status": "completed" + } + } + }, + "specification": { + "items": { + "pay": { + "status": "completed", + "sources": { + "pay": { + "status": "incorporated" + } + } + } + } + }, + "planning": { + "items": { + "pay": { + "status": "completed", + "format": "local-markdown", + "task_list_gate_mode": "gated", + "author_gate_mode": "gated", + "finding_gate_mode": "gated", + "review_cycle": 0, + "phase": 1, + "task": null, + "task_map": { + "pay-1-1": "pay-1-1", + "pay-1-2": "pay-1-2" + }, + "storage_paths": [], + "approvals": { + "structure": "2026-01-01", + "tasks": { + "p1": "2026-01-01" + } + } + } + } + } + } +} diff --git a/tests/prose/cases/implementation-picks-first-task/fixture/.workflows/pay/planning/pay/planning.md b/tests/prose/cases/implementation-picks-first-task/fixture/.workflows/pay/planning/pay/planning.md new file mode 100644 index 000000000..9131d98c7 --- /dev/null +++ b/tests/prose/cases/implementation-picks-first-task/fixture/.workflows/pay/planning/pay/planning.md @@ -0,0 +1,8 @@ +# Plan — pay + +## Phase 1: Payment core + +| Task | Title | +|------|-------| +| pay-1-1 | Create Payment Intent | +| pay-1-2 | Handle Capture Webhooks | diff --git a/tests/prose/cases/implementation-picks-first-task/fixture/.workflows/pay/planning/pay/tasks/pay-1-1.md b/tests/prose/cases/implementation-picks-first-task/fixture/.workflows/pay/planning/pay/tasks/pay-1-1.md new file mode 100644 index 000000000..f7d43a082 --- /dev/null +++ b/tests/prose/cases/implementation-picks-first-task/fixture/.workflows/pay/planning/pay/tasks/pay-1-1.md @@ -0,0 +1,10 @@ +--- +id: pay-1-1 +phase: 1 +status: pending +created: 2026-01-01 +--- + +# Create Payment Intent + +Create a gateway payment intent when checkout begins and attach it to the order. diff --git a/tests/prose/cases/implementation-picks-first-task/fixture/.workflows/pay/planning/pay/tasks/pay-1-2.md b/tests/prose/cases/implementation-picks-first-task/fixture/.workflows/pay/planning/pay/tasks/pay-1-2.md new file mode 100644 index 000000000..8fb22697d --- /dev/null +++ b/tests/prose/cases/implementation-picks-first-task/fixture/.workflows/pay/planning/pay/tasks/pay-1-2.md @@ -0,0 +1,10 @@ +--- +id: pay-1-2 +phase: 1 +status: pending +created: 2026-01-01 +--- + +# Handle Capture Webhooks + +Consume gateway capture webhooks and mark the order paid; no polling path. diff --git a/tests/prose/cases/implementation-picks-first-task/fixture/.workflows/pay/specification/pay/specification.md b/tests/prose/cases/implementation-picks-first-task/fixture/.workflows/pay/specification/pay/specification.md new file mode 100644 index 000000000..29a5f4322 --- /dev/null +++ b/tests/prose/cases/implementation-picks-first-task/fixture/.workflows/pay/specification/pay/specification.md @@ -0,0 +1,11 @@ +# Specification — pay + +## Requirements + +- Checkout creates a payment intent against the existing gateway account. +- Card payments only; wallet flows are out of scope for v1. +- Capture is confirmed by gateway webhook, never by polling. + +## Out of scope + +- Wallet support (deferred by discussion). diff --git a/tests/prose/cases/investigation-entry-seeds-from-carrier/act.md b/tests/prose/cases/investigation-entry-seeds-from-carrier/act.md new file mode 100644 index 000000000..7165e44fd --- /dev/null +++ b/tests/prose/cases/investigation-entry-seeds-from-carrier/act.md @@ -0,0 +1,4 @@ +Execute skills/workflow-investigation-entry/SKILL.md with arguments +$0=bugfix, $1=crash-fix. Follow it to the point where the handoff to the +processing skill is constructed; record the handoff block and stop. Do +not execute the processing skill's instructions. diff --git a/tests/prose/cases/investigation-entry-seeds-from-carrier/assert.md b/tests/prose/cases/investigation-entry-seeds-from-carrier/assert.md new file mode 100644 index 000000000..7117e3603 --- /dev/null +++ b/tests/prose/cases/investigation-entry-seeds-from-carrier/assert.md @@ -0,0 +1,15 @@ +The prose should have taken this path: + +1. resolves the topic to the work unit, investigation being bugfix work +2. reads the investigation status, finds nothing, sets the source to new + and skips phase validation entirely +3. finds a discovery session log, so seeds the bug context from the + manifest description and that log — the context-gathering questions + are not asked +4. renders the phase note for the investigation phase with the verb + Starting, and emits it as produced +5. hands off to the investigation processing skill for crash-fix + +Further claims: + +- the investigation file is created by the processing skill, not here diff --git a/tests/prose/cases/investigation-entry-seeds-from-carrier/case.json b/tests/prose/cases/investigation-entry-seeds-from-carrier/case.json new file mode 100644 index 000000000..3eb224b9c --- /dev/null +++ b/tests/prose/cases/investigation-entry-seeds-from-carrier/case.json @@ -0,0 +1,7 @@ +{ + "origin": "bugfix mainline — a bug shaped in discovery is never re-interrogated", + "files": [ + "skills/workflow-investigation-entry/SKILL.md", + "skills/workflow-investigation-entry/references/invoke-skill.md" + ] +} diff --git a/tests/prose/cases/investigation-entry-seeds-from-carrier/fixture-state.cjs b/tests/prose/cases/investigation-entry-seeds-from-carrier/fixture-state.cjs new file mode 100644 index 000000000..6cd47642e --- /dev/null +++ b/tests/prose/cases/investigation-entry-seeds-from-carrier/fixture-state.cjs @@ -0,0 +1,12 @@ +'use strict'; + +// The bug is captured and nothing has been started. + +const m = require('../../mainlines/bugfix.cjs'); + +module.exports = { + build(h) { + m.init(h); + m.create(h); + }, +}; diff --git a/tests/prose/cases/investigation-entry-seeds-from-carrier/fixture/.claude/settings.json b/tests/prose/cases/investigation-entry-seeds-from-carrier/fixture/.claude/settings.json new file mode 100644 index 000000000..ae019ede5 --- /dev/null +++ b/tests/prose/cases/investigation-entry-seeds-from-carrier/fixture/.claude/settings.json @@ -0,0 +1,9 @@ +{ + "showClearContextOnPlanAccept": true, + "permissions": { + "allow": [ + "Edit(.workflows/**)", + "Bash(mv .workflows/:*)" + ] + } +} diff --git a/tests/prose/cases/investigation-entry-seeds-from-carrier/fixture/.recipe-hash b/tests/prose/cases/investigation-entry-seeds-from-carrier/fixture/.recipe-hash new file mode 100644 index 000000000..259e11ddc --- /dev/null +++ b/tests/prose/cases/investigation-entry-seeds-from-carrier/fixture/.recipe-hash @@ -0,0 +1 @@ +f2d22a09d2680d82822f6042b87baf77dacbf41d4d8cb2b6d26a33d5c4129ff9 diff --git a/tests/prose/cases/investigation-entry-seeds-from-carrier/fixture/.workflows/.state/migrations b/tests/prose/cases/investigation-entry-seeds-from-carrier/fixture/.workflows/.state/migrations new file mode 100644 index 000000000..3c7162acc --- /dev/null +++ b/tests/prose/cases/investigation-entry-seeds-from-carrier/fixture/.workflows/.state/migrations @@ -0,0 +1,52 @@ +001 +002 +003 +004 +005 +006 +007 +008 +009 +010 +011 +012 +013 +014 +015 +016 +017 +018 +019 +020 +021 +022 +023 +024 +025 +026 +027 +028 +029 +030 +031 +032 +033 +034 +035 +036 +037 +038 +039 +040 +041 +042 +043 +044 +045 +046 +047 +048 +049 +050 +051 +052 diff --git a/tests/prose/cases/investigation-entry-seeds-from-carrier/fixture/.workflows/_gitignore.fixture b/tests/prose/cases/investigation-entry-seeds-from-carrier/fixture/.workflows/_gitignore.fixture new file mode 100644 index 000000000..779cbf28a --- /dev/null +++ b/tests/prose/cases/investigation-entry-seeds-from-carrier/fixture/.workflows/_gitignore.fixture @@ -0,0 +1,2 @@ +.cache/ +.manifest.json.*.tmp diff --git a/tests/prose/cases/investigation-entry-seeds-from-carrier/fixture/.workflows/crash-fix/discovery/sessions/session-001.md b/tests/prose/cases/investigation-entry-seeds-from-carrier/fixture/.workflows/crash-fix/discovery/sessions/session-001.md new file mode 100644 index 000000000..6dc56d4e3 --- /dev/null +++ b/tests/prose/cases/investigation-entry-seeds-from-carrier/fixture/.workflows/crash-fix/discovery/sessions/session-001.md @@ -0,0 +1,40 @@ +# Discovery Session 001 + +Date: 2026-01-01 +Work unit: crash-fix + +## Description (as of session) + +Checkout crashes when an order has no shipping address. + +## Seed + +(none) + +## Imports + +(none) + +## Map State at Start + +(n/a — single-topic work) + +## Exploration + +Reported by two users this week: the checkout page 500s at the +payment step for orders with no shipping address. Reproducible on +staging with a digital-only basket. No error surfaces to the user — +the page just fails. Confirmed as a bugfix; no design question here, +so it routes straight to investigation. + +## Edits + +(none) + +## Topics Identified + +(none) + +## Conclusion + +(none) diff --git a/tests/prose/cases/investigation-entry-seeds-from-carrier/fixture/.workflows/crash-fix/manifest.json b/tests/prose/cases/investigation-entry-seeds-from-carrier/fixture/.workflows/crash-fix/manifest.json new file mode 100644 index 000000000..eaaa56199 --- /dev/null +++ b/tests/prose/cases/investigation-entry-seeds-from-carrier/fixture/.workflows/crash-fix/manifest.json @@ -0,0 +1,8 @@ +{ + "name": "crash-fix", + "work_type": "bugfix", + "status": "in-progress", + "created": "2026-01-01", + "description": "Checkout crashes when an order has no shipping address", + "phases": {} +} diff --git a/tests/prose/cases/investigation-entry-seeds-from-carrier/fixture/.workflows/manifest.json b/tests/prose/cases/investigation-entry-seeds-from-carrier/fixture/.workflows/manifest.json new file mode 100644 index 000000000..1cd498ed5 --- /dev/null +++ b/tests/prose/cases/investigation-entry-seeds-from-carrier/fixture/.workflows/manifest.json @@ -0,0 +1,7 @@ +{ + "work_units": { + "crash-fix": { + "work_type": "bugfix" + } + } +} diff --git a/tests/prose/cases/planning-entry-asks-for-late-context/act.md b/tests/prose/cases/planning-entry-asks-for-late-context/act.md new file mode 100644 index 000000000..0ff96c62f --- /dev/null +++ b/tests/prose/cases/planning-entry-asks-for-late-context/act.md @@ -0,0 +1,4 @@ +Execute skills/workflow-planning-entry/SKILL.md with arguments +$0=feature, $1=pay. Follow it to the point where the handoff to the +processing skill is constructed; record the handoff block and stop. Do +not execute the processing skill's instructions. diff --git a/tests/prose/cases/planning-entry-asks-for-late-context/assert.md b/tests/prose/cases/planning-entry-asks-for-late-context/assert.md new file mode 100644 index 000000000..f1bf44650 --- /dev/null +++ b/tests/prose/cases/planning-entry-asks-for-late-context/assert.md @@ -0,0 +1,15 @@ +The prose should have taken this path: + +1. the specification gate renders empty — the spec is complete and the + topic is clear to plan +2. no plan exists, so the fresh-start arm asks whether anything has + changed since the specification, and waits +3. on continue, the source is fresh and carries no additional context +4. cross-cutting context is gathered regardless of work type: it finds + no cross-cutting work units, then queries the knowledge base for + completed cross-cutting specs +5. hands off to the planning processing skill for pay + +Further claims: + +- the knowledge base is queried, never written to, by an entry skill diff --git a/tests/prose/cases/planning-entry-asks-for-late-context/case.json b/tests/prose/cases/planning-entry-asks-for-late-context/case.json new file mode 100644 index 000000000..e5e2d1d46 --- /dev/null +++ b/tests/prose/cases/planning-entry-asks-for-late-context/case.json @@ -0,0 +1,12 @@ +{ + "origin": "feature mainline — planning offers a last chance to add context", + "files": [ + "skills/workflow-planning-entry/SKILL.md", + "skills/workflow-planning-entry/references/validate-spec.md", + "skills/workflow-planning-entry/references/validate-phase.md", + "skills/workflow-planning-entry/references/cross-cutting-context.md" + ], + "answers": [ + "continue — nothing has changed since the specification" + ] +} diff --git a/tests/prose/cases/planning-entry-asks-for-late-context/fixture-state.cjs b/tests/prose/cases/planning-entry-asks-for-late-context/fixture-state.cjs new file mode 100644 index 000000000..4113f7768 --- /dev/null +++ b/tests/prose/cases/planning-entry-asks-for-late-context/fixture-state.cjs @@ -0,0 +1,14 @@ +'use strict'; + +// The specification is complete; planning has not begun. + +const m = require('../../mainlines/feature.cjs'); + +module.exports = { + build(h) { + m.init(h); + m.create(h); + m.discuss(h); + m.specify(h); + }, +}; diff --git a/tests/prose/cases/planning-entry-asks-for-late-context/fixture/.claude/settings.json b/tests/prose/cases/planning-entry-asks-for-late-context/fixture/.claude/settings.json new file mode 100644 index 000000000..ae019ede5 --- /dev/null +++ b/tests/prose/cases/planning-entry-asks-for-late-context/fixture/.claude/settings.json @@ -0,0 +1,9 @@ +{ + "showClearContextOnPlanAccept": true, + "permissions": { + "allow": [ + "Edit(.workflows/**)", + "Bash(mv .workflows/:*)" + ] + } +} diff --git a/tests/prose/cases/planning-entry-asks-for-late-context/fixture/.recipe-hash b/tests/prose/cases/planning-entry-asks-for-late-context/fixture/.recipe-hash new file mode 100644 index 000000000..deb107e54 --- /dev/null +++ b/tests/prose/cases/planning-entry-asks-for-late-context/fixture/.recipe-hash @@ -0,0 +1 @@ +7c074ebb1f3729ec5ba32e89896c1d55de4c288de4cd3ddc8b93191813015936 diff --git a/tests/prose/cases/planning-entry-asks-for-late-context/fixture/.workflows/.state/migrations b/tests/prose/cases/planning-entry-asks-for-late-context/fixture/.workflows/.state/migrations new file mode 100644 index 000000000..3c7162acc --- /dev/null +++ b/tests/prose/cases/planning-entry-asks-for-late-context/fixture/.workflows/.state/migrations @@ -0,0 +1,52 @@ +001 +002 +003 +004 +005 +006 +007 +008 +009 +010 +011 +012 +013 +014 +015 +016 +017 +018 +019 +020 +021 +022 +023 +024 +025 +026 +027 +028 +029 +030 +031 +032 +033 +034 +035 +036 +037 +038 +039 +040 +041 +042 +043 +044 +045 +046 +047 +048 +049 +050 +051 +052 diff --git a/tests/prose/cases/planning-entry-asks-for-late-context/fixture/.workflows/_gitignore.fixture b/tests/prose/cases/planning-entry-asks-for-late-context/fixture/.workflows/_gitignore.fixture new file mode 100644 index 000000000..779cbf28a --- /dev/null +++ b/tests/prose/cases/planning-entry-asks-for-late-context/fixture/.workflows/_gitignore.fixture @@ -0,0 +1,2 @@ +.cache/ +.manifest.json.*.tmp diff --git a/tests/prose/cases/planning-entry-asks-for-late-context/fixture/.workflows/manifest.json b/tests/prose/cases/planning-entry-asks-for-late-context/fixture/.workflows/manifest.json new file mode 100644 index 000000000..987f6c236 --- /dev/null +++ b/tests/prose/cases/planning-entry-asks-for-late-context/fixture/.workflows/manifest.json @@ -0,0 +1,7 @@ +{ + "work_units": { + "pay": { + "work_type": "feature" + } + } +} diff --git a/tests/prose/cases/planning-entry-asks-for-late-context/fixture/.workflows/pay/discovery/sessions/session-001.md b/tests/prose/cases/planning-entry-asks-for-late-context/fixture/.workflows/pay/discovery/sessions/session-001.md new file mode 100644 index 000000000..2aab96961 --- /dev/null +++ b/tests/prose/cases/planning-entry-asks-for-late-context/fixture/.workflows/pay/discovery/sessions/session-001.md @@ -0,0 +1,39 @@ +# Discovery Session 001 + +Date: 2026-01-01 +Work unit: pay + +## Description (as of session) + +Accept card payments at checkout. + +## Seed + +(none) + +## Imports + +(none) + +## Map State at Start + +(n/a — single-topic work) + +## Exploration + +Shaped as a single feature: accept card payments at checkout using +the existing gateway account. Card-only for v1 came up early and was +softly agreed; wallet support was noted as a likely deferral. No +research need surfaced — routed straight to discussion. + +## Edits + +(none) + +## Topics Identified + +(none) + +## Conclusion + +(none) diff --git a/tests/prose/cases/planning-entry-asks-for-late-context/fixture/.workflows/pay/discussion/pay.md b/tests/prose/cases/planning-entry-asks-for-late-context/fixture/.workflows/pay/discussion/pay.md new file mode 100644 index 000000000..ec0cfc62f --- /dev/null +++ b/tests/prose/cases/planning-entry-asks-for-late-context/fixture/.workflows/pay/discussion/pay.md @@ -0,0 +1,15 @@ +# Discussion — pay + +## Context + +Accept card payments at checkout using the existing gateway account. + +## Decisions + +- Use the existing gateway account; no new provider onboarding. +- Card-only for v1 — wallet support is deferred. +- Webhooks confirm capture; the checkout never polls. + +## Deferred + +- Wallet support (Apple/Google Pay) — revisit after v1. diff --git a/tests/prose/cases/planning-entry-asks-for-late-context/fixture/.workflows/pay/manifest.json b/tests/prose/cases/planning-entry-asks-for-late-context/fixture/.workflows/pay/manifest.json new file mode 100644 index 000000000..029755ae0 --- /dev/null +++ b/tests/prose/cases/planning-entry-asks-for-late-context/fixture/.workflows/pay/manifest.json @@ -0,0 +1,28 @@ +{ + "name": "pay", + "work_type": "feature", + "status": "in-progress", + "created": "2026-01-01", + "description": "Accept card payments at checkout", + "phases": { + "discussion": { + "items": { + "pay": { + "status": "completed" + } + } + }, + "specification": { + "items": { + "pay": { + "status": "completed", + "sources": { + "pay": { + "status": "incorporated" + } + } + } + } + } + } +} diff --git a/tests/prose/cases/planning-entry-asks-for-late-context/fixture/.workflows/pay/specification/pay/specification.md b/tests/prose/cases/planning-entry-asks-for-late-context/fixture/.workflows/pay/specification/pay/specification.md new file mode 100644 index 000000000..29a5f4322 --- /dev/null +++ b/tests/prose/cases/planning-entry-asks-for-late-context/fixture/.workflows/pay/specification/pay/specification.md @@ -0,0 +1,11 @@ +# Specification — pay + +## Requirements + +- Checkout creates a payment intent against the existing gateway account. +- Card payments only; wallet flows are out of scope for v1. +- Capture is confirmed by gateway webhook, never by polling. + +## Out of scope + +- Wallet support (deferred by discussion). diff --git a/tests/prose/cases/review-entry-fresh/act.md b/tests/prose/cases/review-entry-fresh/act.md new file mode 100644 index 000000000..df21d57cb --- /dev/null +++ b/tests/prose/cases/review-entry-fresh/act.md @@ -0,0 +1,4 @@ +Execute skills/workflow-review-entry/SKILL.md with arguments $0=feature, +$1=pay. Follow it to the point where the handoff to the processing skill +is constructed; record the handoff block and stop. Do not execute the +processing skill's instructions, and dispatch no agents. diff --git a/tests/prose/cases/review-entry-fresh/assert.md b/tests/prose/cases/review-entry-fresh/assert.md new file mode 100644 index 000000000..c1305bb9b --- /dev/null +++ b/tests/prose/cases/review-entry-fresh/assert.md @@ -0,0 +1,12 @@ +The prose should have taken this path: + +1. resolves the topic to the work unit +2. the prerequisite gate renders empty — both the plan and the + implementation are complete, so nothing blocks entry +3. no review item exists, so the fresh path is taken — nothing is + reopened and no resume is announced +4. hands off to the review processing skill for pay + +Further claims: + +- no verifier agents are dispatched by the entry skill diff --git a/tests/prose/cases/review-entry-fresh/case.json b/tests/prose/cases/review-entry-fresh/case.json new file mode 100644 index 000000000..c67cada98 --- /dev/null +++ b/tests/prose/cases/review-entry-fresh/case.json @@ -0,0 +1,7 @@ +{ + "origin": "feature mainline — review refuses to start until delivery is complete", + "files": [ + "skills/workflow-review-entry/SKILL.md", + "skills/workflow-review-entry/references/validate-phase.md" + ] +} diff --git a/tests/prose/cases/review-entry-fresh/fixture-state.cjs b/tests/prose/cases/review-entry-fresh/fixture-state.cjs new file mode 100644 index 000000000..3502e883d --- /dev/null +++ b/tests/prose/cases/review-entry-fresh/fixture-state.cjs @@ -0,0 +1,16 @@ +'use strict'; + +// Every task is built and implementation is complete; review has not begun. + +const m = require('../../mainlines/feature.cjs'); + +module.exports = { + build(h) { + m.init(h); + m.create(h); + m.discuss(h); + m.specify(h); + m.plan(h); + m.implement(h); + }, +}; diff --git a/tests/prose/cases/review-entry-fresh/fixture/.claude/settings.json b/tests/prose/cases/review-entry-fresh/fixture/.claude/settings.json new file mode 100644 index 000000000..ae019ede5 --- /dev/null +++ b/tests/prose/cases/review-entry-fresh/fixture/.claude/settings.json @@ -0,0 +1,9 @@ +{ + "showClearContextOnPlanAccept": true, + "permissions": { + "allow": [ + "Edit(.workflows/**)", + "Bash(mv .workflows/:*)" + ] + } +} diff --git a/tests/prose/cases/review-entry-fresh/fixture/.recipe-hash b/tests/prose/cases/review-entry-fresh/fixture/.recipe-hash new file mode 100644 index 000000000..e04b82514 --- /dev/null +++ b/tests/prose/cases/review-entry-fresh/fixture/.recipe-hash @@ -0,0 +1 @@ +7289bcc8afeefb182d37e5ca576f35e76a28fec2c91f3738062049801773d0cb diff --git a/tests/prose/cases/review-entry-fresh/fixture/.workflows/.state/migrations b/tests/prose/cases/review-entry-fresh/fixture/.workflows/.state/migrations new file mode 100644 index 000000000..3c7162acc --- /dev/null +++ b/tests/prose/cases/review-entry-fresh/fixture/.workflows/.state/migrations @@ -0,0 +1,52 @@ +001 +002 +003 +004 +005 +006 +007 +008 +009 +010 +011 +012 +013 +014 +015 +016 +017 +018 +019 +020 +021 +022 +023 +024 +025 +026 +027 +028 +029 +030 +031 +032 +033 +034 +035 +036 +037 +038 +039 +040 +041 +042 +043 +044 +045 +046 +047 +048 +049 +050 +051 +052 diff --git a/tests/prose/cases/review-entry-fresh/fixture/.workflows/_gitignore.fixture b/tests/prose/cases/review-entry-fresh/fixture/.workflows/_gitignore.fixture new file mode 100644 index 000000000..779cbf28a --- /dev/null +++ b/tests/prose/cases/review-entry-fresh/fixture/.workflows/_gitignore.fixture @@ -0,0 +1,2 @@ +.cache/ +.manifest.json.*.tmp diff --git a/tests/prose/cases/review-entry-fresh/fixture/.workflows/manifest.json b/tests/prose/cases/review-entry-fresh/fixture/.workflows/manifest.json new file mode 100644 index 000000000..987f6c236 --- /dev/null +++ b/tests/prose/cases/review-entry-fresh/fixture/.workflows/manifest.json @@ -0,0 +1,7 @@ +{ + "work_units": { + "pay": { + "work_type": "feature" + } + } +} diff --git a/tests/prose/cases/review-entry-fresh/fixture/.workflows/pay/discovery/sessions/session-001.md b/tests/prose/cases/review-entry-fresh/fixture/.workflows/pay/discovery/sessions/session-001.md new file mode 100644 index 000000000..2aab96961 --- /dev/null +++ b/tests/prose/cases/review-entry-fresh/fixture/.workflows/pay/discovery/sessions/session-001.md @@ -0,0 +1,39 @@ +# Discovery Session 001 + +Date: 2026-01-01 +Work unit: pay + +## Description (as of session) + +Accept card payments at checkout. + +## Seed + +(none) + +## Imports + +(none) + +## Map State at Start + +(n/a — single-topic work) + +## Exploration + +Shaped as a single feature: accept card payments at checkout using +the existing gateway account. Card-only for v1 came up early and was +softly agreed; wallet support was noted as a likely deferral. No +research need surfaced — routed straight to discussion. + +## Edits + +(none) + +## Topics Identified + +(none) + +## Conclusion + +(none) diff --git a/tests/prose/cases/review-entry-fresh/fixture/.workflows/pay/discussion/pay.md b/tests/prose/cases/review-entry-fresh/fixture/.workflows/pay/discussion/pay.md new file mode 100644 index 000000000..ec0cfc62f --- /dev/null +++ b/tests/prose/cases/review-entry-fresh/fixture/.workflows/pay/discussion/pay.md @@ -0,0 +1,15 @@ +# Discussion — pay + +## Context + +Accept card payments at checkout using the existing gateway account. + +## Decisions + +- Use the existing gateway account; no new provider onboarding. +- Card-only for v1 — wallet support is deferred. +- Webhooks confirm capture; the checkout never polls. + +## Deferred + +- Wallet support (Apple/Google Pay) — revisit after v1. diff --git a/tests/prose/cases/review-entry-fresh/fixture/.workflows/pay/manifest.json b/tests/prose/cases/review-entry-fresh/fixture/.workflows/pay/manifest.json new file mode 100644 index 000000000..269cab533 --- /dev/null +++ b/tests/prose/cases/review-entry-fresh/fixture/.workflows/pay/manifest.json @@ -0,0 +1,77 @@ +{ + "name": "pay", + "work_type": "feature", + "status": "in-progress", + "created": "2026-01-01", + "description": "Accept card payments at checkout", + "phases": { + "discussion": { + "items": { + "pay": { + "status": "completed" + } + } + }, + "specification": { + "items": { + "pay": { + "status": "completed", + "sources": { + "pay": { + "status": "incorporated" + } + } + } + } + }, + "planning": { + "items": { + "pay": { + "status": "completed", + "format": "local-markdown", + "task_list_gate_mode": "gated", + "author_gate_mode": "gated", + "finding_gate_mode": "gated", + "review_cycle": 0, + "phase": 1, + "task": null, + "task_map": { + "pay-1-1": "pay-1-1", + "pay-1-2": "pay-1-2" + }, + "storage_paths": [], + "approvals": { + "structure": "2026-01-01", + "tasks": { + "p1": "2026-01-01" + } + } + } + } + }, + "implementation": { + "items": { + "pay": { + "status": "completed", + "task_gate_mode": "gated", + "fix_gate_mode": "gated", + "analysis_gate_mode": "gated", + "fix_attempts": 0, + "analysis_cycle_total": 0, + "analysis_cycle_session": 0, + "linters": [], + "project_skills": [], + "current_phase": 1, + "current_task": null, + "completed_tasks": [ + "pay-1-1", + "pay-1-2" + ], + "completed_phases": [ + 1 + ] + } + } + } + } +} diff --git a/tests/prose/cases/review-entry-fresh/fixture/.workflows/pay/planning/pay/planning.md b/tests/prose/cases/review-entry-fresh/fixture/.workflows/pay/planning/pay/planning.md new file mode 100644 index 000000000..9131d98c7 --- /dev/null +++ b/tests/prose/cases/review-entry-fresh/fixture/.workflows/pay/planning/pay/planning.md @@ -0,0 +1,8 @@ +# Plan — pay + +## Phase 1: Payment core + +| Task | Title | +|------|-------| +| pay-1-1 | Create Payment Intent | +| pay-1-2 | Handle Capture Webhooks | diff --git a/tests/prose/cases/review-entry-fresh/fixture/.workflows/pay/planning/pay/tasks/pay-1-1.md b/tests/prose/cases/review-entry-fresh/fixture/.workflows/pay/planning/pay/tasks/pay-1-1.md new file mode 100644 index 000000000..8ceaac9af --- /dev/null +++ b/tests/prose/cases/review-entry-fresh/fixture/.workflows/pay/planning/pay/tasks/pay-1-1.md @@ -0,0 +1,10 @@ +--- +id: pay-1-1 +phase: 1 +status: completed +created: 2026-01-01 +--- + +# Create Payment Intent + +Create a gateway payment intent when checkout begins and attach it to the order. diff --git a/tests/prose/cases/review-entry-fresh/fixture/.workflows/pay/planning/pay/tasks/pay-1-2.md b/tests/prose/cases/review-entry-fresh/fixture/.workflows/pay/planning/pay/tasks/pay-1-2.md new file mode 100644 index 000000000..23c692d2c --- /dev/null +++ b/tests/prose/cases/review-entry-fresh/fixture/.workflows/pay/planning/pay/tasks/pay-1-2.md @@ -0,0 +1,10 @@ +--- +id: pay-1-2 +phase: 1 +status: completed +created: 2026-01-01 +--- + +# Handle Capture Webhooks + +Consume gateway capture webhooks and mark the order paid; no polling path. diff --git a/tests/prose/cases/review-entry-fresh/fixture/.workflows/pay/specification/pay/specification.md b/tests/prose/cases/review-entry-fresh/fixture/.workflows/pay/specification/pay/specification.md new file mode 100644 index 000000000..29a5f4322 --- /dev/null +++ b/tests/prose/cases/review-entry-fresh/fixture/.workflows/pay/specification/pay/specification.md @@ -0,0 +1,11 @@ +# Specification — pay + +## Requirements + +- Checkout creates a payment intent against the existing gateway account. +- Card payments only; wallet flows are out of scope for v1. +- Capture is confirmed by gateway webhook, never by polling. + +## Out of scope + +- Wallet support (deferred by discussion). diff --git a/tests/prose/cases/root-cause-validation-clean-verdict/act.md b/tests/prose/cases/root-cause-validation-clean-verdict/act.md new file mode 100644 index 000000000..c3225285c --- /dev/null +++ b/tests/prose/cases/root-cause-validation-clean-verdict/act.md @@ -0,0 +1,2 @@ +Walk root-cause-validation.md from the top, as the processing skill +would. Stop when the reference returns to its caller. diff --git a/tests/prose/cases/root-cause-validation-clean-verdict/assert.md b/tests/prose/cases/root-cause-validation-clean-verdict/assert.md new file mode 100644 index 000000000..f00e364c7 --- /dev/null +++ b/tests/prose/cases/root-cause-validation-clean-verdict/assert.md @@ -0,0 +1,19 @@ +The prose should have taken this path: + +1. offers the independent validation, presents the run-or-skip choice, + and waits +2. on yes, records the dispatch through the engine and creates no report + file of its own — the response carries the id and the path to use +3. announces that validation is running, then dispatches exactly one + agent, synchronously +4. once the report has landed, promotes the row with a scan and closes + it with an incorporate — the verdict is consumed whole, never + surfaced finding by finding +5. reads the report and, the verdict being validated, states the + confidence and returns to its caller — the gap-handling choice is + never reached + +Further claims: + +- the investigation file is untouched and nothing is committed: the + validated path has no gaps to record diff --git a/tests/prose/cases/root-cause-validation-clean-verdict/assertion-state.cjs b/tests/prose/cases/root-cause-validation-clean-verdict/assertion-state.cjs new file mode 100644 index 000000000..12c381e52 --- /dev/null +++ b/tests/prose/cases/root-cause-validation-clean-verdict/assertion-state.cjs @@ -0,0 +1,27 @@ +'use strict'; + +// The fixture state plus what the walk should have done: the dispatch +// recorded, the agent's report landed at the path the dispatch returned, +// scan promoting the row and incorporate closing it. The report content +// comes from the same stub the case arms, so the bytes the walk writes +// and the bytes expected here cannot drift apart. + +const fs = require('fs'); +const path = require('path'); + +const fixture = require('./fixture-state.cjs'); +const m = require('../../mainlines/bugfix.cjs'); +const { readStub } = require('../../lib/cases.cjs'); + +module.exports = { + build(h) { + fixture.build(h); + + const dispatch = JSON.parse(h.engine( + 'agent', 'dispatch', m.WU, 'investigation', m.WU, '--kind', 'root-cause-validation')); + fs.mkdirSync(path.dirname(path.join(h.dir, dispatch.file)), { recursive: true }); + h.write(dispatch.file, readStub('root-cause-validated').content); + h.engine('agent', 'scan', m.WU, 'investigation', m.WU); + h.engine('agent', 'incorporate', m.WU, 'investigation', m.WU, dispatch.id); + }, +}; diff --git a/tests/prose/cases/root-cause-validation-clean-verdict/assertion/.claude/settings.json b/tests/prose/cases/root-cause-validation-clean-verdict/assertion/.claude/settings.json new file mode 100644 index 000000000..ae019ede5 --- /dev/null +++ b/tests/prose/cases/root-cause-validation-clean-verdict/assertion/.claude/settings.json @@ -0,0 +1,9 @@ +{ + "showClearContextOnPlanAccept": true, + "permissions": { + "allow": [ + "Edit(.workflows/**)", + "Bash(mv .workflows/:*)" + ] + } +} diff --git a/tests/prose/cases/root-cause-validation-clean-verdict/assertion/.recipe-hash b/tests/prose/cases/root-cause-validation-clean-verdict/assertion/.recipe-hash new file mode 100644 index 000000000..d38ba55a6 --- /dev/null +++ b/tests/prose/cases/root-cause-validation-clean-verdict/assertion/.recipe-hash @@ -0,0 +1 @@ +b65c43b1fced1a2d5742f96a2d6d1a8278f67d44c143552dbb42bee06badded6 diff --git a/tests/prose/cases/root-cause-validation-clean-verdict/assertion/.workflows/.cache/crash-fix/investigation/crash-fix/root-cause-validation-001.md b/tests/prose/cases/root-cause-validation-clean-verdict/assertion/.workflows/.cache/crash-fix/investigation/crash-fix/root-cause-validation-001.md new file mode 100644 index 000000000..ce1c9b4c4 --- /dev/null +++ b/tests/prose/cases/root-cause-validation-clean-verdict/assertion/.workflows/.cache/crash-fix/investigation/crash-fix/root-cause-validation-001.md @@ -0,0 +1,10 @@ +# Root Cause Validation + +The tax context is built before the payment intent and reads the shipping +address unconditionally. Traced fresh: an address-less order fails at that +read, which matches the documented root cause. + +STATUS: validated +CONFIDENCE: high +GAPS_COUNT: 0 +SUMMARY: Root cause confirmed by an independent trace. diff --git a/tests/prose/cases/root-cause-validation-clean-verdict/assertion/.workflows/.cache/crash-fix/investigation/crash-fix/state.json b/tests/prose/cases/root-cause-validation-clean-verdict/assertion/.workflows/.cache/crash-fix/investigation/crash-fix/state.json new file mode 100644 index 000000000..7eb5a78f1 --- /dev/null +++ b/tests/prose/cases/root-cause-validation-clean-verdict/assertion/.workflows/.cache/crash-fix/investigation/crash-fix/state.json @@ -0,0 +1,16 @@ +{ + "agents": { + "root-cause-validation-001": { + "id": "root-cause-validation-001", + "kind": "root-cause-validation", + "phase": "investigation", + "topic": "crash-fix", + "set": "001", + "status": "incorporated", + "announced": false, + "findings": [], + "surfaced": [], + "created": "2026-01-01T00:00:00.000Z" + } + } +} diff --git a/tests/prose/cases/root-cause-validation-clean-verdict/assertion/.workflows/.state/migrations b/tests/prose/cases/root-cause-validation-clean-verdict/assertion/.workflows/.state/migrations new file mode 100644 index 000000000..3c7162acc --- /dev/null +++ b/tests/prose/cases/root-cause-validation-clean-verdict/assertion/.workflows/.state/migrations @@ -0,0 +1,52 @@ +001 +002 +003 +004 +005 +006 +007 +008 +009 +010 +011 +012 +013 +014 +015 +016 +017 +018 +019 +020 +021 +022 +023 +024 +025 +026 +027 +028 +029 +030 +031 +032 +033 +034 +035 +036 +037 +038 +039 +040 +041 +042 +043 +044 +045 +046 +047 +048 +049 +050 +051 +052 diff --git a/tests/prose/cases/root-cause-validation-clean-verdict/assertion/.workflows/_gitignore.fixture b/tests/prose/cases/root-cause-validation-clean-verdict/assertion/.workflows/_gitignore.fixture new file mode 100644 index 000000000..779cbf28a --- /dev/null +++ b/tests/prose/cases/root-cause-validation-clean-verdict/assertion/.workflows/_gitignore.fixture @@ -0,0 +1,2 @@ +.cache/ +.manifest.json.*.tmp diff --git a/tests/prose/cases/root-cause-validation-clean-verdict/assertion/.workflows/crash-fix/discovery/sessions/session-001.md b/tests/prose/cases/root-cause-validation-clean-verdict/assertion/.workflows/crash-fix/discovery/sessions/session-001.md new file mode 100644 index 000000000..6dc56d4e3 --- /dev/null +++ b/tests/prose/cases/root-cause-validation-clean-verdict/assertion/.workflows/crash-fix/discovery/sessions/session-001.md @@ -0,0 +1,40 @@ +# Discovery Session 001 + +Date: 2026-01-01 +Work unit: crash-fix + +## Description (as of session) + +Checkout crashes when an order has no shipping address. + +## Seed + +(none) + +## Imports + +(none) + +## Map State at Start + +(n/a — single-topic work) + +## Exploration + +Reported by two users this week: the checkout page 500s at the +payment step for orders with no shipping address. Reproducible on +staging with a digital-only basket. No error surfaces to the user — +the page just fails. Confirmed as a bugfix; no design question here, +so it routes straight to investigation. + +## Edits + +(none) + +## Topics Identified + +(none) + +## Conclusion + +(none) diff --git a/tests/prose/cases/root-cause-validation-clean-verdict/assertion/.workflows/crash-fix/investigation/crash-fix.md b/tests/prose/cases/root-cause-validation-clean-verdict/assertion/.workflows/crash-fix/investigation/crash-fix.md new file mode 100644 index 000000000..d5fc263a9 --- /dev/null +++ b/tests/prose/cases/root-cause-validation-clean-verdict/assertion/.workflows/crash-fix/investigation/crash-fix.md @@ -0,0 +1,88 @@ +# Investigation: Checkout Crash On Missing Shipping Address + +## Symptoms + +### Problem Description + +Checkout returns a 500 at the payment step when the order has no +shipping address. No error is surfaced to the user. + +### Manifestation + +Server error on POST to the payment step; the page fails blank. + +### Reproduction Steps + +1. Add a digital-only product to the basket. +2. Proceed to checkout without entering a shipping address. +3. Continue to the payment step — the request 500s. + +### Environment + +Reproduced on staging; reported twice in production this week. + +### Impact + +Digital-only orders cannot complete checkout. + +### References + +(none) + +## Analysis + +### Hypotheses + +- The payment step assumes a shipping address is always present. +- Tax calculation may be the first consumer of the missing address. + +### Code Trace + +The payment step builds a tax context from the order before creating +the payment intent. The tax context reads the shipping address +unconditionally; for a digital-only order the address is absent, so +the read fails and the request aborts before any handler catches it. + +### Root Cause + +The tax context treats the shipping address as mandatory. Digital-only +orders legitimately have none, so the assumption is wrong rather than +the data being wrong. + +### Contributing Factors + +Digital-only baskets were introduced after the tax context was written. + +### Why It Wasn't Caught + +No test covers a basket with no shippable line items. + +### Blast Radius + +Any flow building a tax context from an address-less order. + +## Fix Direction + +### Chosen Approach + +(pending) + +### Options Explored + +(none yet) + +### Discussion + +(none yet) + +### Testing Recommendations + +(pending) + +### Risk Assessment + +(pending) + +## Notes + +(none) diff --git a/tests/prose/cases/root-cause-validation-clean-verdict/assertion/.workflows/crash-fix/manifest.json b/tests/prose/cases/root-cause-validation-clean-verdict/assertion/.workflows/crash-fix/manifest.json new file mode 100644 index 000000000..88bf183ff --- /dev/null +++ b/tests/prose/cases/root-cause-validation-clean-verdict/assertion/.workflows/crash-fix/manifest.json @@ -0,0 +1,16 @@ +{ + "name": "crash-fix", + "work_type": "bugfix", + "status": "in-progress", + "created": "2026-01-01", + "description": "Checkout crashes when an order has no shipping address", + "phases": { + "investigation": { + "items": { + "crash-fix": { + "status": "in-progress" + } + } + } + } +} diff --git a/tests/prose/cases/root-cause-validation-clean-verdict/assertion/.workflows/manifest.json b/tests/prose/cases/root-cause-validation-clean-verdict/assertion/.workflows/manifest.json new file mode 100644 index 000000000..1cd498ed5 --- /dev/null +++ b/tests/prose/cases/root-cause-validation-clean-verdict/assertion/.workflows/manifest.json @@ -0,0 +1,7 @@ +{ + "work_units": { + "crash-fix": { + "work_type": "bugfix" + } + } +} diff --git a/tests/prose/cases/root-cause-validation-clean-verdict/case.json b/tests/prose/cases/root-cause-validation-clean-verdict/case.json new file mode 100644 index 000000000..aab675685 --- /dev/null +++ b/tests/prose/cases/root-cause-validation-clean-verdict/case.json @@ -0,0 +1,12 @@ +{ + "origin": "bugfix mainline — the validation agent's lifecycle on a clean verdict", + "files": [ + "skills/workflow-investigation-process/references/root-cause-validation.md" + ], + "answers": [ + "yes — run root cause validation" + ], + "stubs": { + "root-cause-validated": "when the engine records a dispatch of kind root-cause-validation" + } +} diff --git a/tests/prose/cases/root-cause-validation-clean-verdict/fixture-state.cjs b/tests/prose/cases/root-cause-validation-clean-verdict/fixture-state.cjs new file mode 100644 index 000000000..67d91fdbf --- /dev/null +++ b/tests/prose/cases/root-cause-validation-clean-verdict/fixture-state.cjs @@ -0,0 +1,13 @@ +'use strict'; + +// The investigation is open with its root cause documented, and no agent has run. + +const m = require('../../mainlines/bugfix.cjs'); + +module.exports = { + build(h) { + m.init(h); + m.create(h); + m.investigateToRootCause(h); + }, +}; diff --git a/tests/prose/cases/root-cause-validation-clean-verdict/fixture.md b/tests/prose/cases/root-cause-validation-clean-verdict/fixture.md new file mode 100644 index 000000000..95d92d942 --- /dev/null +++ b/tests/prose/cases/root-cause-validation-clean-verdict/fixture.md @@ -0,0 +1,3 @@ +The investigation is open with its symptoms gathered and its root cause +documented. The processing skill has worked down to its root-cause +validation step; nothing has been validated yet and no agent has run. diff --git a/tests/prose/cases/root-cause-validation-clean-verdict/fixture/.claude/settings.json b/tests/prose/cases/root-cause-validation-clean-verdict/fixture/.claude/settings.json new file mode 100644 index 000000000..ae019ede5 --- /dev/null +++ b/tests/prose/cases/root-cause-validation-clean-verdict/fixture/.claude/settings.json @@ -0,0 +1,9 @@ +{ + "showClearContextOnPlanAccept": true, + "permissions": { + "allow": [ + "Edit(.workflows/**)", + "Bash(mv .workflows/:*)" + ] + } +} diff --git a/tests/prose/cases/root-cause-validation-clean-verdict/fixture/.recipe-hash b/tests/prose/cases/root-cause-validation-clean-verdict/fixture/.recipe-hash new file mode 100644 index 000000000..d38ba55a6 --- /dev/null +++ b/tests/prose/cases/root-cause-validation-clean-verdict/fixture/.recipe-hash @@ -0,0 +1 @@ +b65c43b1fced1a2d5742f96a2d6d1a8278f67d44c143552dbb42bee06badded6 diff --git a/tests/prose/cases/root-cause-validation-clean-verdict/fixture/.workflows/.state/migrations b/tests/prose/cases/root-cause-validation-clean-verdict/fixture/.workflows/.state/migrations new file mode 100644 index 000000000..3c7162acc --- /dev/null +++ b/tests/prose/cases/root-cause-validation-clean-verdict/fixture/.workflows/.state/migrations @@ -0,0 +1,52 @@ +001 +002 +003 +004 +005 +006 +007 +008 +009 +010 +011 +012 +013 +014 +015 +016 +017 +018 +019 +020 +021 +022 +023 +024 +025 +026 +027 +028 +029 +030 +031 +032 +033 +034 +035 +036 +037 +038 +039 +040 +041 +042 +043 +044 +045 +046 +047 +048 +049 +050 +051 +052 diff --git a/tests/prose/cases/root-cause-validation-clean-verdict/fixture/.workflows/_gitignore.fixture b/tests/prose/cases/root-cause-validation-clean-verdict/fixture/.workflows/_gitignore.fixture new file mode 100644 index 000000000..779cbf28a --- /dev/null +++ b/tests/prose/cases/root-cause-validation-clean-verdict/fixture/.workflows/_gitignore.fixture @@ -0,0 +1,2 @@ +.cache/ +.manifest.json.*.tmp diff --git a/tests/prose/cases/root-cause-validation-clean-verdict/fixture/.workflows/crash-fix/discovery/sessions/session-001.md b/tests/prose/cases/root-cause-validation-clean-verdict/fixture/.workflows/crash-fix/discovery/sessions/session-001.md new file mode 100644 index 000000000..6dc56d4e3 --- /dev/null +++ b/tests/prose/cases/root-cause-validation-clean-verdict/fixture/.workflows/crash-fix/discovery/sessions/session-001.md @@ -0,0 +1,40 @@ +# Discovery Session 001 + +Date: 2026-01-01 +Work unit: crash-fix + +## Description (as of session) + +Checkout crashes when an order has no shipping address. + +## Seed + +(none) + +## Imports + +(none) + +## Map State at Start + +(n/a — single-topic work) + +## Exploration + +Reported by two users this week: the checkout page 500s at the +payment step for orders with no shipping address. Reproducible on +staging with a digital-only basket. No error surfaces to the user — +the page just fails. Confirmed as a bugfix; no design question here, +so it routes straight to investigation. + +## Edits + +(none) + +## Topics Identified + +(none) + +## Conclusion + +(none) diff --git a/tests/prose/cases/root-cause-validation-clean-verdict/fixture/.workflows/crash-fix/investigation/crash-fix.md b/tests/prose/cases/root-cause-validation-clean-verdict/fixture/.workflows/crash-fix/investigation/crash-fix.md new file mode 100644 index 000000000..d5fc263a9 --- /dev/null +++ b/tests/prose/cases/root-cause-validation-clean-verdict/fixture/.workflows/crash-fix/investigation/crash-fix.md @@ -0,0 +1,88 @@ +# Investigation: Checkout Crash On Missing Shipping Address + +## Symptoms + +### Problem Description + +Checkout returns a 500 at the payment step when the order has no +shipping address. No error is surfaced to the user. + +### Manifestation + +Server error on POST to the payment step; the page fails blank. + +### Reproduction Steps + +1. Add a digital-only product to the basket. +2. Proceed to checkout without entering a shipping address. +3. Continue to the payment step — the request 500s. + +### Environment + +Reproduced on staging; reported twice in production this week. + +### Impact + +Digital-only orders cannot complete checkout. + +### References + +(none) + +## Analysis + +### Hypotheses + +- The payment step assumes a shipping address is always present. +- Tax calculation may be the first consumer of the missing address. + +### Code Trace + +The payment step builds a tax context from the order before creating +the payment intent. The tax context reads the shipping address +unconditionally; for a digital-only order the address is absent, so +the read fails and the request aborts before any handler catches it. + +### Root Cause + +The tax context treats the shipping address as mandatory. Digital-only +orders legitimately have none, so the assumption is wrong rather than +the data being wrong. + +### Contributing Factors + +Digital-only baskets were introduced after the tax context was written. + +### Why It Wasn't Caught + +No test covers a basket with no shippable line items. + +### Blast Radius + +Any flow building a tax context from an address-less order. + +## Fix Direction + +### Chosen Approach + +(pending) + +### Options Explored + +(none yet) + +### Discussion + +(none yet) + +### Testing Recommendations + +(pending) + +### Risk Assessment + +(pending) + +## Notes + +(none) diff --git a/tests/prose/cases/root-cause-validation-clean-verdict/fixture/.workflows/crash-fix/manifest.json b/tests/prose/cases/root-cause-validation-clean-verdict/fixture/.workflows/crash-fix/manifest.json new file mode 100644 index 000000000..88bf183ff --- /dev/null +++ b/tests/prose/cases/root-cause-validation-clean-verdict/fixture/.workflows/crash-fix/manifest.json @@ -0,0 +1,16 @@ +{ + "name": "crash-fix", + "work_type": "bugfix", + "status": "in-progress", + "created": "2026-01-01", + "description": "Checkout crashes when an order has no shipping address", + "phases": { + "investigation": { + "items": { + "crash-fix": { + "status": "in-progress" + } + } + } + } +} diff --git a/tests/prose/cases/root-cause-validation-clean-verdict/fixture/.workflows/manifest.json b/tests/prose/cases/root-cause-validation-clean-verdict/fixture/.workflows/manifest.json new file mode 100644 index 000000000..1cd498ed5 --- /dev/null +++ b/tests/prose/cases/root-cause-validation-clean-verdict/fixture/.workflows/manifest.json @@ -0,0 +1,7 @@ +{ + "work_units": { + "crash-fix": { + "work_type": "bugfix" + } + } +} diff --git a/tests/prose/cases/smoke-boot-in-world/act.md b/tests/prose/cases/smoke-boot-in-world/act.md new file mode 100644 index 000000000..bb4d2a56b --- /dev/null +++ b/tests/prose/cases/smoke-boot-in-world/act.md @@ -0,0 +1,4 @@ +In the project, run the boot command exactly as the initialisation prose +of skills/workflow-start/SKILL.md prescribes — sandbox concerns do not +apply in this world, so run it directly. Capture the JSON response, then +stop; do not continue into any confirmation or later step. diff --git a/tests/prose/cases/smoke-boot-in-world/assert.md b/tests/prose/cases/smoke-boot-in-world/assert.md new file mode 100644 index 000000000..da99ef3ce --- /dev/null +++ b/tests/prose/cases/smoke-boot-in-world/assert.md @@ -0,0 +1,10 @@ +The prose should have taken this path: + +1. runs the boot command and reports the knowledge base ready, in + keyword-only mode +2. reports no migrations applied — this world is already migrated — so + the migrations summary and its confirmation are not raised + +Further claims: + +- booting an already-migrated project changes nothing on disk diff --git a/tests/prose/cases/smoke-boot-in-world/case.json b/tests/prose/cases/smoke-boot-in-world/case.json new file mode 100644 index 000000000..047c10686 --- /dev/null +++ b/tests/prose/cases/smoke-boot-in-world/case.json @@ -0,0 +1,6 @@ +{ + "origin": "framework smoke — world build, engine-in-world, world comparison", + "files": [ + "skills/workflow-start/SKILL.md#Boot" + ] +} diff --git a/tests/prose/cases/smoke-boot-in-world/fixture-state.cjs b/tests/prose/cases/smoke-boot-in-world/fixture-state.cjs new file mode 100644 index 000000000..8e39fface --- /dev/null +++ b/tests/prose/cases/smoke-boot-in-world/fixture-state.cjs @@ -0,0 +1,11 @@ +'use strict'; + +// A bare installed project: migrations applied, knowledge base set up +// keyword-only, no work units at all. + +module.exports = { + build(h) { + h.knowledge('setup', '--keyword-only'); + h.engine('boot'); + }, +}; diff --git a/tests/prose/cases/smoke-boot-in-world/fixture/.claude/settings.json b/tests/prose/cases/smoke-boot-in-world/fixture/.claude/settings.json new file mode 100644 index 000000000..ae019ede5 --- /dev/null +++ b/tests/prose/cases/smoke-boot-in-world/fixture/.claude/settings.json @@ -0,0 +1,9 @@ +{ + "showClearContextOnPlanAccept": true, + "permissions": { + "allow": [ + "Edit(.workflows/**)", + "Bash(mv .workflows/:*)" + ] + } +} diff --git a/tests/prose/cases/smoke-boot-in-world/fixture/.recipe-hash b/tests/prose/cases/smoke-boot-in-world/fixture/.recipe-hash new file mode 100644 index 000000000..27528f6e9 --- /dev/null +++ b/tests/prose/cases/smoke-boot-in-world/fixture/.recipe-hash @@ -0,0 +1 @@ +e75a843d66eafa8a47fd143e1e3b2eb41e0a6b7b2b2d225753bec1cdee9892c5 diff --git a/tests/prose/cases/smoke-boot-in-world/fixture/.workflows/.state/migrations b/tests/prose/cases/smoke-boot-in-world/fixture/.workflows/.state/migrations new file mode 100644 index 000000000..3c7162acc --- /dev/null +++ b/tests/prose/cases/smoke-boot-in-world/fixture/.workflows/.state/migrations @@ -0,0 +1,52 @@ +001 +002 +003 +004 +005 +006 +007 +008 +009 +010 +011 +012 +013 +014 +015 +016 +017 +018 +019 +020 +021 +022 +023 +024 +025 +026 +027 +028 +029 +030 +031 +032 +033 +034 +035 +036 +037 +038 +039 +040 +041 +042 +043 +044 +045 +046 +047 +048 +049 +050 +051 +052 diff --git a/tests/prose/cases/smoke-boot-in-world/fixture/.workflows/_gitignore.fixture b/tests/prose/cases/smoke-boot-in-world/fixture/.workflows/_gitignore.fixture new file mode 100644 index 000000000..779cbf28a --- /dev/null +++ b/tests/prose/cases/smoke-boot-in-world/fixture/.workflows/_gitignore.fixture @@ -0,0 +1,2 @@ +.cache/ +.manifest.json.*.tmp diff --git a/tests/prose/cases/smoke-start-boot-structure/act.md b/tests/prose/cases/smoke-start-boot-structure/act.md new file mode 100644 index 000000000..cc3f6a902 --- /dev/null +++ b/tests/prose/cases/smoke-start-boot-structure/act.md @@ -0,0 +1,5 @@ +Structure-only: read the initialisation portion of +skills/workflow-start/SKILL.md — everything the skill does before any +work is shown or routed. Establish what initialisation consists of and +in what order its parts run. Stop there; do not trace the rest of the +skill. diff --git a/tests/prose/cases/smoke-start-boot-structure/assert.md b/tests/prose/cases/smoke-start-boot-structure/assert.md new file mode 100644 index 000000000..058974b50 --- /dev/null +++ b/tests/prose/cases/smoke-start-boot-structure/assert.md @@ -0,0 +1,7 @@ +The prose should have taken this path: + +1. casing conventions are loaded before the boot pipeline runs +2. the boot pipeline is declared mandatory — it must complete before the + skill proceeds +3. the knowledge gate is reached only after boot, and branches on what + boot reported rather than on its own checks diff --git a/tests/prose/cases/smoke-start-boot-structure/case.json b/tests/prose/cases/smoke-start-boot-structure/case.json new file mode 100644 index 000000000..07b9e0424 --- /dev/null +++ b/tests/prose/cases/smoke-start-boot-structure/case.json @@ -0,0 +1,6 @@ +{ + "origin": "framework smoke — a structure-only walk, no world", + "files": [ + "skills/workflow-start/SKILL.md#Boot" + ] +} diff --git a/tests/prose/cases/spec-entry-from-discussion/act.md b/tests/prose/cases/spec-entry-from-discussion/act.md new file mode 100644 index 000000000..a76fbb90c --- /dev/null +++ b/tests/prose/cases/spec-entry-from-discussion/act.md @@ -0,0 +1,4 @@ +Execute skills/workflow-specification-entry/SKILL.md with arguments +$0=feature, $1=pay. Follow it to the point where the handoff to the +processing skill is constructed; record the handoff block and stop. Do +not execute the processing skill's instructions. diff --git a/tests/prose/cases/spec-entry-from-discussion/assert.md b/tests/prose/cases/spec-entry-from-discussion/assert.md new file mode 100644 index 000000000..59ebce857 --- /dev/null +++ b/tests/prose/cases/spec-entry-from-discussion/assert.md @@ -0,0 +1,13 @@ +The prose should have taken this path: + +1. resolves the topic to the work unit and goes straight to validating + source material — the scoped path is for an epic with no topic +2. the source-material gate renders empty: the completed discussion + satisfies the prerequisite +3. no specification item exists, so the verb is Creating — no resume + note is rendered and nothing is reopened +4. hands off naming `.workflows/pay/discussion/pay.md` as the source + +Further claims: + +- an entry that only validates writes nothing diff --git a/tests/prose/cases/spec-entry-from-discussion/case.json b/tests/prose/cases/spec-entry-from-discussion/case.json new file mode 100644 index 000000000..f380e57ce --- /dev/null +++ b/tests/prose/cases/spec-entry-from-discussion/case.json @@ -0,0 +1,9 @@ +{ + "origin": "feature mainline — a feature's spec draws on its discussion", + "files": [ + "skills/workflow-specification-entry/SKILL.md", + "skills/workflow-specification-entry/references/validate-source.md", + "skills/workflow-specification-entry/references/validate-phase.md", + "skills/workflow-specification-entry/references/invoke-skill.md" + ] +} diff --git a/tests/prose/cases/spec-entry-from-discussion/fixture-state.cjs b/tests/prose/cases/spec-entry-from-discussion/fixture-state.cjs new file mode 100644 index 000000000..a1a1b7964 --- /dev/null +++ b/tests/prose/cases/spec-entry-from-discussion/fixture-state.cjs @@ -0,0 +1,13 @@ +'use strict'; + +// The discussion is complete; the specification has not begun. + +const m = require('../../mainlines/feature.cjs'); + +module.exports = { + build(h) { + m.init(h); + m.create(h); + m.discuss(h); + }, +}; diff --git a/tests/prose/cases/spec-entry-from-discussion/fixture/.claude/settings.json b/tests/prose/cases/spec-entry-from-discussion/fixture/.claude/settings.json new file mode 100644 index 000000000..ae019ede5 --- /dev/null +++ b/tests/prose/cases/spec-entry-from-discussion/fixture/.claude/settings.json @@ -0,0 +1,9 @@ +{ + "showClearContextOnPlanAccept": true, + "permissions": { + "allow": [ + "Edit(.workflows/**)", + "Bash(mv .workflows/:*)" + ] + } +} diff --git a/tests/prose/cases/spec-entry-from-discussion/fixture/.recipe-hash b/tests/prose/cases/spec-entry-from-discussion/fixture/.recipe-hash new file mode 100644 index 000000000..336903a57 --- /dev/null +++ b/tests/prose/cases/spec-entry-from-discussion/fixture/.recipe-hash @@ -0,0 +1 @@ +ed6d01566a29220645f5ff7b62dd7d748b55eec7036c00af9bd68089a6091c61 diff --git a/tests/prose/cases/spec-entry-from-discussion/fixture/.workflows/.state/migrations b/tests/prose/cases/spec-entry-from-discussion/fixture/.workflows/.state/migrations new file mode 100644 index 000000000..3c7162acc --- /dev/null +++ b/tests/prose/cases/spec-entry-from-discussion/fixture/.workflows/.state/migrations @@ -0,0 +1,52 @@ +001 +002 +003 +004 +005 +006 +007 +008 +009 +010 +011 +012 +013 +014 +015 +016 +017 +018 +019 +020 +021 +022 +023 +024 +025 +026 +027 +028 +029 +030 +031 +032 +033 +034 +035 +036 +037 +038 +039 +040 +041 +042 +043 +044 +045 +046 +047 +048 +049 +050 +051 +052 diff --git a/tests/prose/cases/spec-entry-from-discussion/fixture/.workflows/_gitignore.fixture b/tests/prose/cases/spec-entry-from-discussion/fixture/.workflows/_gitignore.fixture new file mode 100644 index 000000000..779cbf28a --- /dev/null +++ b/tests/prose/cases/spec-entry-from-discussion/fixture/.workflows/_gitignore.fixture @@ -0,0 +1,2 @@ +.cache/ +.manifest.json.*.tmp diff --git a/tests/prose/cases/spec-entry-from-discussion/fixture/.workflows/manifest.json b/tests/prose/cases/spec-entry-from-discussion/fixture/.workflows/manifest.json new file mode 100644 index 000000000..987f6c236 --- /dev/null +++ b/tests/prose/cases/spec-entry-from-discussion/fixture/.workflows/manifest.json @@ -0,0 +1,7 @@ +{ + "work_units": { + "pay": { + "work_type": "feature" + } + } +} diff --git a/tests/prose/cases/spec-entry-from-discussion/fixture/.workflows/pay/discovery/sessions/session-001.md b/tests/prose/cases/spec-entry-from-discussion/fixture/.workflows/pay/discovery/sessions/session-001.md new file mode 100644 index 000000000..2aab96961 --- /dev/null +++ b/tests/prose/cases/spec-entry-from-discussion/fixture/.workflows/pay/discovery/sessions/session-001.md @@ -0,0 +1,39 @@ +# Discovery Session 001 + +Date: 2026-01-01 +Work unit: pay + +## Description (as of session) + +Accept card payments at checkout. + +## Seed + +(none) + +## Imports + +(none) + +## Map State at Start + +(n/a — single-topic work) + +## Exploration + +Shaped as a single feature: accept card payments at checkout using +the existing gateway account. Card-only for v1 came up early and was +softly agreed; wallet support was noted as a likely deferral. No +research need surfaced — routed straight to discussion. + +## Edits + +(none) + +## Topics Identified + +(none) + +## Conclusion + +(none) diff --git a/tests/prose/cases/spec-entry-from-discussion/fixture/.workflows/pay/discussion/pay.md b/tests/prose/cases/spec-entry-from-discussion/fixture/.workflows/pay/discussion/pay.md new file mode 100644 index 000000000..ec0cfc62f --- /dev/null +++ b/tests/prose/cases/spec-entry-from-discussion/fixture/.workflows/pay/discussion/pay.md @@ -0,0 +1,15 @@ +# Discussion — pay + +## Context + +Accept card payments at checkout using the existing gateway account. + +## Decisions + +- Use the existing gateway account; no new provider onboarding. +- Card-only for v1 — wallet support is deferred. +- Webhooks confirm capture; the checkout never polls. + +## Deferred + +- Wallet support (Apple/Google Pay) — revisit after v1. diff --git a/tests/prose/cases/spec-entry-from-discussion/fixture/.workflows/pay/manifest.json b/tests/prose/cases/spec-entry-from-discussion/fixture/.workflows/pay/manifest.json new file mode 100644 index 000000000..d4a3fca6c --- /dev/null +++ b/tests/prose/cases/spec-entry-from-discussion/fixture/.workflows/pay/manifest.json @@ -0,0 +1,16 @@ +{ + "name": "pay", + "work_type": "feature", + "status": "in-progress", + "created": "2026-01-01", + "description": "Accept card payments at checkout", + "phases": { + "discussion": { + "items": { + "pay": { + "status": "completed" + } + } + } + } +} diff --git a/tests/prose/cases/spec-entry-from-investigation/act.md b/tests/prose/cases/spec-entry-from-investigation/act.md new file mode 100644 index 000000000..a2bd56697 --- /dev/null +++ b/tests/prose/cases/spec-entry-from-investigation/act.md @@ -0,0 +1,4 @@ +Execute skills/workflow-specification-entry/SKILL.md with arguments +$0=bugfix, $1=crash-fix. Follow it to the point where the handoff to the +processing skill is constructed; record the handoff block and stop. Do +not execute the processing skill's instructions. diff --git a/tests/prose/cases/spec-entry-from-investigation/assert.md b/tests/prose/cases/spec-entry-from-investigation/assert.md new file mode 100644 index 000000000..85c1245e4 --- /dev/null +++ b/tests/prose/cases/spec-entry-from-investigation/assert.md @@ -0,0 +1,13 @@ +The prose should have taken this path: + +1. resolves the topic to the work unit and goes to validating source + material +2. the source-material gate renders empty: for a bugfix the completed + investigation is what satisfies it, in place of a discussion +3. no specification item exists, so the verb is Creating +4. hands off naming `.workflows/crash-fix/investigation/crash-fix.md` as + the source material — no discussion file is named + +Further claims: + +- the same prose serves both work types; only the source arm differs diff --git a/tests/prose/cases/spec-entry-from-investigation/case.json b/tests/prose/cases/spec-entry-from-investigation/case.json new file mode 100644 index 000000000..3eed6746c --- /dev/null +++ b/tests/prose/cases/spec-entry-from-investigation/case.json @@ -0,0 +1,8 @@ +{ + "origin": "bugfix mainline — the same entry skill, sourced from an investigation", + "files": [ + "skills/workflow-specification-entry/SKILL.md", + "skills/workflow-specification-entry/references/validate-source.md", + "skills/workflow-specification-entry/references/invoke-skill.md" + ] +} diff --git a/tests/prose/cases/spec-entry-from-investigation/fixture-state.cjs b/tests/prose/cases/spec-entry-from-investigation/fixture-state.cjs new file mode 100644 index 000000000..a97e20061 --- /dev/null +++ b/tests/prose/cases/spec-entry-from-investigation/fixture-state.cjs @@ -0,0 +1,14 @@ +'use strict'; + +// The investigation is concluded; the specification has not begun. + +const m = require('../../mainlines/bugfix.cjs'); + +module.exports = { + build(h) { + m.init(h); + m.create(h); + m.investigateToRootCause(h); + m.concludeInvestigation(h); + }, +}; diff --git a/tests/prose/cases/spec-entry-from-investigation/fixture/.claude/settings.json b/tests/prose/cases/spec-entry-from-investigation/fixture/.claude/settings.json new file mode 100644 index 000000000..ae019ede5 --- /dev/null +++ b/tests/prose/cases/spec-entry-from-investigation/fixture/.claude/settings.json @@ -0,0 +1,9 @@ +{ + "showClearContextOnPlanAccept": true, + "permissions": { + "allow": [ + "Edit(.workflows/**)", + "Bash(mv .workflows/:*)" + ] + } +} diff --git a/tests/prose/cases/spec-entry-from-investigation/fixture/.recipe-hash b/tests/prose/cases/spec-entry-from-investigation/fixture/.recipe-hash new file mode 100644 index 000000000..1d4381905 --- /dev/null +++ b/tests/prose/cases/spec-entry-from-investigation/fixture/.recipe-hash @@ -0,0 +1 @@ +b34ddf2eff3b10056fb8d6c0e69ede001d06a863d3990e8752b0a5d4fdcbd1a3 diff --git a/tests/prose/cases/spec-entry-from-investigation/fixture/.workflows/.state/migrations b/tests/prose/cases/spec-entry-from-investigation/fixture/.workflows/.state/migrations new file mode 100644 index 000000000..3c7162acc --- /dev/null +++ b/tests/prose/cases/spec-entry-from-investigation/fixture/.workflows/.state/migrations @@ -0,0 +1,52 @@ +001 +002 +003 +004 +005 +006 +007 +008 +009 +010 +011 +012 +013 +014 +015 +016 +017 +018 +019 +020 +021 +022 +023 +024 +025 +026 +027 +028 +029 +030 +031 +032 +033 +034 +035 +036 +037 +038 +039 +040 +041 +042 +043 +044 +045 +046 +047 +048 +049 +050 +051 +052 diff --git a/tests/prose/cases/spec-entry-from-investigation/fixture/.workflows/_gitignore.fixture b/tests/prose/cases/spec-entry-from-investigation/fixture/.workflows/_gitignore.fixture new file mode 100644 index 000000000..779cbf28a --- /dev/null +++ b/tests/prose/cases/spec-entry-from-investigation/fixture/.workflows/_gitignore.fixture @@ -0,0 +1,2 @@ +.cache/ +.manifest.json.*.tmp diff --git a/tests/prose/cases/spec-entry-from-investigation/fixture/.workflows/crash-fix/discovery/sessions/session-001.md b/tests/prose/cases/spec-entry-from-investigation/fixture/.workflows/crash-fix/discovery/sessions/session-001.md new file mode 100644 index 000000000..6dc56d4e3 --- /dev/null +++ b/tests/prose/cases/spec-entry-from-investigation/fixture/.workflows/crash-fix/discovery/sessions/session-001.md @@ -0,0 +1,40 @@ +# Discovery Session 001 + +Date: 2026-01-01 +Work unit: crash-fix + +## Description (as of session) + +Checkout crashes when an order has no shipping address. + +## Seed + +(none) + +## Imports + +(none) + +## Map State at Start + +(n/a — single-topic work) + +## Exploration + +Reported by two users this week: the checkout page 500s at the +payment step for orders with no shipping address. Reproducible on +staging with a digital-only basket. No error surfaces to the user — +the page just fails. Confirmed as a bugfix; no design question here, +so it routes straight to investigation. + +## Edits + +(none) + +## Topics Identified + +(none) + +## Conclusion + +(none) diff --git a/tests/prose/cases/spec-entry-from-investigation/fixture/.workflows/crash-fix/investigation/crash-fix.md b/tests/prose/cases/spec-entry-from-investigation/fixture/.workflows/crash-fix/investigation/crash-fix.md new file mode 100644 index 000000000..d09b4daba --- /dev/null +++ b/tests/prose/cases/spec-entry-from-investigation/fixture/.workflows/crash-fix/investigation/crash-fix.md @@ -0,0 +1,90 @@ +# Investigation: Checkout Crash On Missing Shipping Address + +## Symptoms + +### Problem Description + +Checkout returns a 500 at the payment step when the order has no +shipping address. No error is surfaced to the user. + +### Manifestation + +Server error on POST to the payment step; the page fails blank. + +### Reproduction Steps + +1. Add a digital-only product to the basket. +2. Proceed to checkout without entering a shipping address. +3. Continue to the payment step — the request 500s. + +### Environment + +Reproduced on staging; reported twice in production this week. + +### Impact + +Digital-only orders cannot complete checkout. + +### References + +(none) + +## Analysis + +### Hypotheses + +- The payment step assumes a shipping address is always present. +- Tax calculation may be the first consumer of the missing address. + +### Code Trace + +The payment step builds a tax context from the order before creating +the payment intent. The tax context reads the shipping address +unconditionally; for a digital-only order the address is absent, so +the read fails and the request aborts before any handler catches it. + +### Root Cause + +The tax context treats the shipping address as mandatory. Digital-only +orders legitimately have none, so the assumption is wrong rather than +the data being wrong. + +### Contributing Factors + +Digital-only baskets were introduced after the tax context was written. + +### Why It Wasn't Caught + +No test covers a basket with no shippable line items. + +### Blast Radius + +Any flow building a tax context from an address-less order. + +## Fix Direction + +### Chosen Approach + +Make the tax context treat the shipping address as optional: when +an order has no shippable items, build the context from the billing +address instead. + +### Options Explored + +(none yet) + +### Discussion + +(none yet) + +### Testing Recommendations + +Cover a digital-only basket end to end through the payment step. + +### Risk Assessment + +Low — the change is confined to tax-context construction. + +## Notes + +(none) diff --git a/tests/prose/cases/spec-entry-from-investigation/fixture/.workflows/crash-fix/manifest.json b/tests/prose/cases/spec-entry-from-investigation/fixture/.workflows/crash-fix/manifest.json new file mode 100644 index 000000000..4c5ece2e7 --- /dev/null +++ b/tests/prose/cases/spec-entry-from-investigation/fixture/.workflows/crash-fix/manifest.json @@ -0,0 +1,16 @@ +{ + "name": "crash-fix", + "work_type": "bugfix", + "status": "in-progress", + "created": "2026-01-01", + "description": "Checkout crashes when an order has no shipping address", + "phases": { + "investigation": { + "items": { + "crash-fix": { + "status": "completed" + } + } + } + } +} diff --git a/tests/prose/cases/spec-entry-from-investigation/fixture/.workflows/manifest.json b/tests/prose/cases/spec-entry-from-investigation/fixture/.workflows/manifest.json new file mode 100644 index 000000000..1cd498ed5 --- /dev/null +++ b/tests/prose/cases/spec-entry-from-investigation/fixture/.workflows/manifest.json @@ -0,0 +1,7 @@ +{ + "work_units": { + "crash-fix": { + "work_type": "bugfix" + } + } +} diff --git a/tests/prose/cases/start-lists-active-work/act.md b/tests/prose/cases/start-lists-active-work/act.md new file mode 100644 index 000000000..b4fa96524 --- /dev/null +++ b/tests/prose/cases/start-lists-active-work/act.md @@ -0,0 +1,3 @@ +Execute skills/workflow-start/SKILL.md from the top. Continue until the +work dashboard and its menu have been shown. Record both verbatim, then +stop — select nothing. diff --git a/tests/prose/start-lists-active-work.md b/tests/prose/cases/start-lists-active-work/assert.md similarity index 56% rename from tests/prose/start-lists-active-work.md rename to tests/prose/cases/start-lists-active-work/assert.md index d511ef06a..e0842d0b6 100644 --- a/tests/prose/start-lists-active-work.md +++ b/tests/prose/cases/start-lists-active-work/assert.md @@ -1,23 +1,5 @@ -## case: start-lists-active-work -- origin: feature mainline — the entry skill surfaces work without touching it -- files: - - skills/workflow-start/SKILL.md +The prose should have taken this path: -### given - -world_before: feature-created - -### when - -Execute skills/workflow-start/SKILL.md from the top. Continue until the -work dashboard and its menu have been shown. Record both verbatim, then -stop — select nothing. - -### then - -world_after: unchanged - -trace: 1. loads the shared casing conventions before any state is read 2. runs the boot pipeline, and since no migrations applied and the knowledge base is ready, raises neither the migrations confirmation @@ -27,5 +9,6 @@ trace: 4. shows `pay` as the only active work, with a menu whose continue action leads into the per-type navigation for a feature -notes: +Further claims: + - showing work is a read: nothing is created, recorded, or committed diff --git a/tests/prose/cases/start-lists-active-work/case.json b/tests/prose/cases/start-lists-active-work/case.json new file mode 100644 index 000000000..59711da9f --- /dev/null +++ b/tests/prose/cases/start-lists-active-work/case.json @@ -0,0 +1,6 @@ +{ + "origin": "feature mainline — the entry skill surfaces work without touching it", + "files": [ + "skills/workflow-start/SKILL.md" + ] +} diff --git a/tests/prose/cases/start-lists-active-work/fixture-state.cjs b/tests/prose/cases/start-lists-active-work/fixture-state.cjs new file mode 100644 index 000000000..e4b78cd71 --- /dev/null +++ b/tests/prose/cases/start-lists-active-work/fixture-state.cjs @@ -0,0 +1,12 @@ +'use strict'; + +// The feature exists and nothing has been started. + +const m = require('../../mainlines/feature.cjs'); + +module.exports = { + build(h) { + m.init(h); + m.create(h); + }, +}; diff --git a/tests/prose/cases/start-lists-active-work/fixture/.claude/settings.json b/tests/prose/cases/start-lists-active-work/fixture/.claude/settings.json new file mode 100644 index 000000000..ae019ede5 --- /dev/null +++ b/tests/prose/cases/start-lists-active-work/fixture/.claude/settings.json @@ -0,0 +1,9 @@ +{ + "showClearContextOnPlanAccept": true, + "permissions": { + "allow": [ + "Edit(.workflows/**)", + "Bash(mv .workflows/:*)" + ] + } +} diff --git a/tests/prose/cases/start-lists-active-work/fixture/.recipe-hash b/tests/prose/cases/start-lists-active-work/fixture/.recipe-hash new file mode 100644 index 000000000..92519878f --- /dev/null +++ b/tests/prose/cases/start-lists-active-work/fixture/.recipe-hash @@ -0,0 +1 @@ +5b0a7d802b3a436a4656bc000122451bb015f2250f8fef47b1d05f3ea51ba96b diff --git a/tests/prose/cases/start-lists-active-work/fixture/.workflows/.state/migrations b/tests/prose/cases/start-lists-active-work/fixture/.workflows/.state/migrations new file mode 100644 index 000000000..3c7162acc --- /dev/null +++ b/tests/prose/cases/start-lists-active-work/fixture/.workflows/.state/migrations @@ -0,0 +1,52 @@ +001 +002 +003 +004 +005 +006 +007 +008 +009 +010 +011 +012 +013 +014 +015 +016 +017 +018 +019 +020 +021 +022 +023 +024 +025 +026 +027 +028 +029 +030 +031 +032 +033 +034 +035 +036 +037 +038 +039 +040 +041 +042 +043 +044 +045 +046 +047 +048 +049 +050 +051 +052 diff --git a/tests/prose/cases/start-lists-active-work/fixture/.workflows/_gitignore.fixture b/tests/prose/cases/start-lists-active-work/fixture/.workflows/_gitignore.fixture new file mode 100644 index 000000000..779cbf28a --- /dev/null +++ b/tests/prose/cases/start-lists-active-work/fixture/.workflows/_gitignore.fixture @@ -0,0 +1,2 @@ +.cache/ +.manifest.json.*.tmp diff --git a/tests/prose/cases/start-lists-active-work/fixture/.workflows/manifest.json b/tests/prose/cases/start-lists-active-work/fixture/.workflows/manifest.json new file mode 100644 index 000000000..987f6c236 --- /dev/null +++ b/tests/prose/cases/start-lists-active-work/fixture/.workflows/manifest.json @@ -0,0 +1,7 @@ +{ + "work_units": { + "pay": { + "work_type": "feature" + } + } +} diff --git a/tests/prose/cases/start-lists-active-work/fixture/.workflows/pay/discovery/sessions/session-001.md b/tests/prose/cases/start-lists-active-work/fixture/.workflows/pay/discovery/sessions/session-001.md new file mode 100644 index 000000000..2aab96961 --- /dev/null +++ b/tests/prose/cases/start-lists-active-work/fixture/.workflows/pay/discovery/sessions/session-001.md @@ -0,0 +1,39 @@ +# Discovery Session 001 + +Date: 2026-01-01 +Work unit: pay + +## Description (as of session) + +Accept card payments at checkout. + +## Seed + +(none) + +## Imports + +(none) + +## Map State at Start + +(n/a — single-topic work) + +## Exploration + +Shaped as a single feature: accept card payments at checkout using +the existing gateway account. Card-only for v1 came up early and was +softly agreed; wallet support was noted as a likely deferral. No +research need surfaced — routed straight to discussion. + +## Edits + +(none) + +## Topics Identified + +(none) + +## Conclusion + +(none) diff --git a/tests/prose/cases/start-lists-active-work/fixture/.workflows/pay/manifest.json b/tests/prose/cases/start-lists-active-work/fixture/.workflows/pay/manifest.json new file mode 100644 index 000000000..1cf8afb0e --- /dev/null +++ b/tests/prose/cases/start-lists-active-work/fixture/.workflows/pay/manifest.json @@ -0,0 +1,8 @@ +{ + "name": "pay", + "work_type": "feature", + "status": "in-progress", + "created": "2026-01-01", + "description": "Accept card payments at checkout", + "phases": {} +} diff --git a/tests/prose/continue-bugfix-routes-to-investigation.md b/tests/prose/continue-bugfix-routes-to-investigation.md deleted file mode 100644 index 646d3ea2e..000000000 --- a/tests/prose/continue-bugfix-routes-to-investigation.md +++ /dev/null @@ -1,34 +0,0 @@ -## case: continue-bugfix-routes-to-investigation -- origin: bugfix mainline — a fresh bug routes to investigation, not discussion -- files: - - skills/workflow-continue-bugfix/SKILL.md - - skills/workflow-continue-bugfix/references/select-bugfix.md - - skills/workflow-continue-bugfix/references/bugfix-display-and-menu.md - -### given - -world_before: bugfix-created - -### when - -Execute skills/workflow-continue-bugfix/SKILL.md with no arguments, as -it is invoked when the caller has no work unit pre-selected. Follow it -through selection and the bugfix's state display. Stop once the state -and its menu have been shown — do not invoke the route. - -answers: -1. the number listed against `crash-fix` - -### then - -world_after: unchanged - -trace: -1. initialisation is casing only — boot and migrations belong to the - entry skill -2. shows the selection display and menu rather than auto-selecting the - only bugfix -3. validates the selection, then shows the bugfix's pipeline state -4. the state shows no phase started, and the continue action routes to - investigation entry — a bugfix's first phase is investigation, never - discussion or specification diff --git a/tests/prose/continue-feature-routes-to-discussion.md b/tests/prose/continue-feature-routes-to-discussion.md deleted file mode 100644 index 23e5fd61f..000000000 --- a/tests/prose/continue-feature-routes-to-discussion.md +++ /dev/null @@ -1,36 +0,0 @@ -## case: continue-feature-routes-to-discussion -- origin: feature mainline — navigation shows state and routes to the next phase -- files: - - skills/workflow-continue-feature/SKILL.md - - skills/workflow-continue-feature/references/select-feature.md - - skills/workflow-continue-feature/references/feature-display-and-menu.md - -### when - -Execute skills/workflow-continue-feature/SKILL.md with no arguments, as -it is invoked when the caller has no work unit pre-selected. Follow it -through selection and the feature's state display. Stop once the state -and its menu have been shown — do not invoke the route. - -answers: -1. the number listed against `pay` - -### given - -world_before: feature-created - -### then - -world_after: unchanged - -trace: -1. initialisation is casing only — no boot and no migrations, which the - entry skill has already guaranteed -2. shows the selection display and menu even though `pay` is the only - feature — never auto-selects -3. validates the selection, then shows the feature's pipeline state -4. the state shows no phase started, and the menu's continue action - routes to discussion entry for work type feature, work unit pay - -notes: -- navigation only reads: no phase is started by looking at it diff --git a/tests/prose/implementation-picks-first-task.md b/tests/prose/implementation-picks-first-task.md deleted file mode 100644 index 6771c4eb3..000000000 --- a/tests/prose/implementation-picks-first-task.md +++ /dev/null @@ -1,48 +0,0 @@ -## case: implementation-picks-first-task -- origin: feature mainline — entry through resume detection to the first task -- files: - - skills/workflow-implementation-entry/SKILL.md - - skills/workflow-implementation-entry/references/validate-phase.md - - skills/workflow-implementation-entry/references/validate-dependencies.md - - skills/workflow-implementation-entry/references/environment-check.md - - skills/workflow-implementation-process/SKILL.md - - skills/workflow-implementation-process/references/environment-setup.md - - skills/workflow-implementation-process/references/task-loop.md - - skills/workflow-planning-process/references/output-formats/local-markdown/reading.md - -### given - -world_before: feature-planned - -### when - -Execute skills/workflow-implementation-entry/SKILL.md with arguments -$0=feature, $1=pay, then continue into the processing skill it invokes. -Follow the flow until the prose directs you to begin building the first -task. Stop there — write no implementation code and no tests. - -answers: -1. none — no special setup is needed - -### then - -world_after: feature-implementation-started - -trace: -1. the plan gate renders empty, and no implementation item exists, so - this is a new entry -2. dependency validation returns immediately — external dependencies are - an epic concern -3. the environment check finds no setup document and asks the question, - gathering the answer without acting on it -4. resume detection initialises tracking and reports the created mode — - the fresh path, which commits the start of implementation, never the - resuming-from-a-previous-session note -5. environment setup records the answer as a setup document and commits - it, so the question is not asked again in a later session -6. reads the plan through the format's own reading procedure and finds - pay-1-1 next: phase one, first task, nothing completed -7. starts pay-1-1 and stops where the prose hands over to building - -notes: -- the second task is neither started nor touched diff --git a/tests/prose/investigation-entry-seeds-from-carrier.md b/tests/prose/investigation-entry-seeds-from-carrier.md deleted file mode 100644 index 7f6d0ad5e..000000000 --- a/tests/prose/investigation-entry-seeds-from-carrier.md +++ /dev/null @@ -1,34 +0,0 @@ -## case: investigation-entry-seeds-from-carrier -- origin: bugfix mainline — a bug shaped in discovery is never re-interrogated -- files: - - skills/workflow-investigation-entry/SKILL.md - - skills/workflow-investigation-entry/references/invoke-skill.md - -### given - -world_before: bugfix-created - -### when - -Execute skills/workflow-investigation-entry/SKILL.md with arguments -$0=bugfix, $1=crash-fix. Follow it to the point where the handoff to the -processing skill is constructed; record the handoff block and stop. Do -not execute the processing skill's instructions. - -### then - -world_after: unchanged - -trace: -1. resolves the topic to the work unit, investigation being bugfix work -2. reads the investigation status, finds nothing, sets the source to new - and skips phase validation entirely -3. finds a discovery session log, so seeds the bug context from the - manifest description and that log — the context-gathering questions - are not asked -4. renders the phase note for the investigation phase with the verb - Starting, and emits it as produced -5. hands off to the investigation processing skill for crash-fix - -notes: -- the investigation file is created by the processing skill, not here diff --git a/tests/prose/lib/cases.cjs b/tests/prose/lib/cases.cjs index 7219dedda..006534a5d 100644 --- a/tests/prose/lib/cases.cjs +++ b/tests/prose/lib/cases.cjs @@ -1,153 +1,121 @@ 'use strict'; -// Case corpus: parsing and validation. +// Case corpus: loading and validation. // -// One case per file, flat: tests/prose/{case-id}.md, where the filename -// IS the id (validated, so a rename can never drift from the heading). -// Cases are Given-When-Then; nothing groups them but their `files:` -// scope, which is what selection actually runs on. +// A case is a directory, tests/prose/cases/{case-id}/, holding one file +// per element of the test. Nothing is parsed: JSON is JSON, prose files +// are read whole and relayed into prompts, recipes are required as +// modules. // -// ## case: {kebab-id} -// - origin: {why this case exists} -// - files: -// - {repo-relative path}[#heading fragment] +// case.json the values code branches on: +// { origin, files[], answers[], stubs{name: trigger} } +// fixture.md optional. Prose describing the starting world — +// what has already happened, where the session +// stands. Given to the walker. +// fixture-state.cjs exports build(h): the starting world, in engine +// calls. Composed from tests/prose/mainlines/. +// act.md the coarse instruction: where to enter, what to +// follow, where to stop. Given to the walker. +// assert.md the expected trace, step by step, plus any +// further claims. Given ONLY to the asserter. +// assertion-state.cjs optional. exports build(h): the world the walk +// should produce. Absent means "unchanged" — the +// walk should leave the fixture state untouched. +// fixture/ generated snapshot of the starting world +// assertion/ generated snapshot of the expected world // -// ### given -// world_before: {fixture name} -// {free text: what that world represents — what has already happened, -// where the session stands. Context for the walker, never an -// instruction.} +// act.md and assert.md are separate files because that is the P4 +// boundary: the walker must never see the expected trace, and a file +// boundary enforces it structurally rather than by convention. // -// ### when -// {free text: where to enter, what to follow, where to stop} -// answers: -// 1. {scripted user response, consumed in order} -// stubs: -// - {stub name}: {the moment it fires — the case owns the trigger} -// -// `given` is the world and nothing else. Everything the harness feeds -// in *during* the act — the user's answers, the agent substitutions — -// belongs in `when`, beside the instruction they happen during. -// -// ### then -// world_after: {fixture name | unchanged} -// trace: -// 1. {behavioural step the walk should have taken} -// notes: -// - {any further claim about the walk} -// -// The `when` stays coarse on purpose — one instruction, never a -// step-by-step script. The walker must DERIVE the path from the prose; -// that derivation is the thing under test. Granularity lives in `then`, -// where a step-by-step expected trace catches a walker that silently -// course-corrected around broken prose. -// -// Anchors are substring fragments matched against heading text -// (`#Boot` matches "Step 0.2: Boot") so a renumber can't break a case. -// Claims name BEHAVIOUR, never coordinates — step numbers and arm -// letters rot on cosmetic edits and fail for the wrong reason. +// The act stays coarse on purpose — the walker must DERIVE the path from +// the prose, and that derivation is the thing under test. Granularity +// lives in assert.md, where a step-by-step expected trace catches a +// walker that silently course-corrected around broken prose. const fs = require('fs'); const path = require('path'); const ROOT = path.join(__dirname, '../../..'); const PROSE_DIR = path.join(ROOT, 'tests/prose'); -const FIXTURES_DIR = path.join(PROSE_DIR, 'fixtures'); +const CASES_DIR = path.join(PROSE_DIR, 'cases'); const STUBS_DIR = path.join(PROSE_DIR, 'stubs'); -const UNCHANGED = 'unchanged'; -function listCaseFiles() { - if (!fs.existsSync(PROSE_DIR)) return []; - return fs.readdirSync(PROSE_DIR, { withFileTypes: true }) - .filter((e) => e.isFile() && e.name.endsWith('.md') && e.name !== 'README.md') - .map((e) => path.join(PROSE_DIR, e.name)); +const FILES = { + meta: 'case.json', + situation: 'fixture.md', + fixtureState: 'fixture-state.cjs', + act: 'act.md', + assert: 'assert.md', + assertionState: 'assertion-state.cjs', +}; +const SNAPSHOTS = { fixture: 'fixture', assertion: 'assertion' }; + +function listCaseIds() { + if (!fs.existsSync(CASES_DIR)) return []; + return fs.readdirSync(CASES_DIR, { withFileTypes: true }) + .filter((e) => e.isDirectory() && !e.name.startsWith('_')) + .map((e) => e.name) + .sort(); } -function parseCaseFile(file) { - const lines = fs.readFileSync(file, 'utf8').split('\n'); - const c = { - id: null, file: path.relative(ROOT, file), origin: null, files: [], - worldBefore: null, situation: [], stubs: [], when: [], answers: [], - worldAfter: null, trace: [], notes: [], - }; - let section = null; // null | given | when | then - let list = null; // files | stubs | answers | trace | notes - - for (const line of lines) { - const head = line.match(/^## case:\s*(\S+)\s*$/); - if (head) { - c.id = head[1]; - section = null; - list = null; - continue; - } - const sub = line.match(/^### (given|when|then)\s*$/); - if (sub) { - section = sub[1]; - list = null; - continue; - } - - if (section === null) { - const origin = line.match(/^- origin:\s*(.*)$/); - if (origin) { c.origin = origin[1].trim(); list = null; continue; } - if (/^- files:\s*$/.test(line)) { list = 'files'; continue; } - const item = line.match(/^\s+- (.+)$/); - if (list === 'files' && item) { - const spec = item[1].trim(); - const hash = spec.indexOf('#'); - c.files.push(hash === -1 - ? { path: spec, anchor: null } - : { path: spec.slice(0, hash).trim(), anchor: spec.slice(hash + 1).trim() }); - continue; - } - if (line.trim() !== '') list = null; - continue; - } - - if (section === 'given') { - const wb = line.match(/^world_before:\s*(\S+)\s*$/); - if (wb) c.worldBefore = wb[1]; - else c.situation.push(line); - continue; - } - - if (section === 'when') { - if (/^answers:\s*$/.test(line)) { list = 'answers'; continue; } - if (/^stubs:\s*$/.test(line)) { list = 'stubs'; continue; } - const answer = line.match(/^\d+\.\s+(.*)$/); - if (list === 'answers' && answer) { c.answers.push(answer[1].trim()); continue; } - const stub = line.match(/^\s+- ([a-z0-9][a-z0-9-]*):\s*(.+)$/); - if (list === 'stubs' && stub) { c.stubs.push({ name: stub[1], trigger: stub[2].trim() }); continue; } - if (list === null) c.when.push(line); - continue; - } +function readIf(dir, name) { + const file = path.join(dir, name); + return fs.existsSync(file) ? fs.readFileSync(file, 'utf8').trim() : null; +} - // then - const wa = line.match(/^world_after:\s*(\S+)\s*$/); - if (wa) { c.worldAfter = wa[1]; list = null; continue; } - if (/^trace:\s*$/.test(line)) { list = 'trace'; continue; } - if (/^notes:\s*$/.test(line)) { list = 'notes'; continue; } - const step = line.match(/^\d+\.\s+(.*)$/); - if (list === 'trace' && step) { c.trace.push(step[1].trim()); continue; } - if (list === 'trace' && /^\s+\S/.test(line) && c.trace.length) { - c.trace[c.trace.length - 1] += ` ${line.trim()}`; - continue; - } - const note = line.match(/^- (.*)$/); - if (list === 'notes' && note) { c.notes.push(note[1].trim()); continue; } - if (list === 'notes' && /^\s+\S/.test(line) && c.notes.length) { - c.notes[c.notes.length - 1] += ` ${line.trim()}`; +function loadCase(id) { + const dir = path.join(CASES_DIR, id); + if (!fs.existsSync(dir)) throw new Error(`no case "${id}" in tests/prose/cases`); + + const metaPath = path.join(dir, FILES.meta); + let meta = {}; + let metaError = null; + if (!fs.existsSync(metaPath)) metaError = `missing ${FILES.meta}`; + else { + try { + meta = JSON.parse(fs.readFileSync(metaPath, 'utf8')); + } catch (e) { + metaError = `${FILES.meta} does not parse: ${e.message}`; } } - c.when = c.when.join('\n').trim(); - c.situation = c.situation.join('\n').trim(); - return c; + return { + id, + dir, + rel: path.relative(ROOT, dir), + metaError, + origin: meta.origin || null, + files: (meta.files || []).map((spec) => { + const hash = spec.indexOf('#'); + return hash === -1 + ? { path: spec, anchor: null } + : { path: spec.slice(0, hash).trim(), anchor: spec.slice(hash + 1).trim() }; + }), + answers: meta.answers || [], + stubs: Object.entries(meta.stubs || {}).map(([name, trigger]) => ({ name, trigger })), + situation: readIf(dir, FILES.situation), + act: readIf(dir, FILES.act), + assert: readIf(dir, FILES.assert), + hasFixtureState: fs.existsSync(path.join(dir, FILES.fixtureState)), + hasAssertionState: fs.existsSync(path.join(dir, FILES.assertionState)), + }; } function loadAllCases() { - return listCaseFiles().map(parseCaseFile); + return listCaseIds().map(loadCase); +} + +/** A world builder module: exports build(h). */ +function requireState(caseId, which) { + const file = path.join(CASES_DIR, caseId, FILES[which]); + if (!fs.existsSync(file)) return null; + delete require.cache[require.resolve(file)]; + const mod = require(file); + if (typeof mod.build !== 'function') { + throw new Error(`${caseId}/${FILES[which]} must export build(h)`); + } + return mod; } function listStubs() { @@ -166,7 +134,7 @@ function readStub(name) { if (fence === -1) throw new Error(`stub "${name}" has no --- fence separating description from content`); return { name, - description: raw.slice(0, fence).trim(), + description: raw.slice(0, fence).replace(/^#.*\n/, '').trim(), content: raw.slice(fence + 5).replace(/^\n+/, ''), }; } @@ -178,27 +146,17 @@ function headingExists(absPath, anchor) { }); } -function fixtureExists(name) { - return fs.existsSync(path.join(FIXTURES_DIR, name, 'recipe.cjs')); -} - function validateCorpus(cases) { const errors = []; - const seen = new Set(); const stubs = new Set(listStubs()); for (const c of cases) { - const at = c.file; - if (!c.id) { errors.push(`${at}: no "## case:" heading`); continue; } - if (!/^[a-z0-9][a-z0-9-]*$/.test(c.id)) errors.push(`${at}: id is not kebab-case`); - if (seen.has(c.id)) errors.push(`${at}: duplicate case id "${c.id}"`); - seen.add(c.id); - if (path.basename(c.file, '.md') !== c.id) { - errors.push(`${at}: filename must equal the case id (expected ${c.id}.md)`); - } + const at = c.rel; + if (c.metaError) { errors.push(`${at}: ${c.metaError}`); continue; } + if (!/^[a-z0-9][a-z0-9-]*$/.test(c.id)) errors.push(`${at}: case directory is not kebab-case`); + if (!c.origin) errors.push(`${at}: ${FILES.meta} has no origin`); - if (!c.origin) errors.push(`${at}: no origin`); - if (c.files.length === 0) errors.push(`${at}: no files: scope`); + if (c.files.length === 0) errors.push(`${at}: ${FILES.meta} scopes no files`); for (const f of c.files) { const abs = path.join(ROOT, f.path); if (!fs.existsSync(abs)) errors.push(`${at}: scoped file missing: ${f.path}`); @@ -207,31 +165,35 @@ function validateCorpus(cases) { } } - if (c.worldBefore && !fixtureExists(c.worldBefore)) { - errors.push(`${at}: world_before "${c.worldBefore}" has no fixture recipe`); + if (!c.act) errors.push(`${at}: no ${FILES.act}`); + if (!c.assert) errors.push(`${at}: no ${FILES.assert} — the expected trace is what catches a silent repair`); + + if (c.hasAssertionState && !c.hasFixtureState) { + errors.push(`${at}: ${FILES.assertionState} without ${FILES.fixtureState} — nothing to act on`); } - if (!c.worldAfter) { - errors.push(`${at}: no world_after (use "unchanged" when the walk mutates nothing)`); - } else if (c.worldAfter !== UNCHANGED) { - if (!c.worldBefore) errors.push(`${at}: world_after names a fixture but there is no world_before`); - if (!fixtureExists(c.worldAfter)) { - errors.push(`${at}: world_after "${c.worldAfter}" has no fixture recipe`); + if (c.situation && !c.hasFixtureState) { + errors.push(`${at}: ${FILES.situation} describes a world this case does not build`); + } + for (const which of ['fixtureState', 'assertionState']) { + try { + requireState(c.id, which); + } catch (e) { + errors.push(`${at}: ${e.message}`); } } for (const s of c.stubs) { if (!stubs.has(s.name)) errors.push(`${at}: no stub "${s.name}" in tests/prose/stubs`); - if (!s.trigger) errors.push(`${at}: stub "${s.name}" declares no trigger — the case owns the moment`); - if (!c.worldBefore) errors.push(`${at}: stub "${s.name}" needs a world to fire in`); + if (!s.trigger || !s.trigger.trim()) { + errors.push(`${at}: stub "${s.name}" declares no trigger — the case owns the moment`); + } + if (!c.hasFixtureState) errors.push(`${at}: stub "${s.name}" needs a world to fire in`); } - - if (!c.when) errors.push(`${at}: empty when`); - if (c.trace.length === 0) errors.push(`${at}: no trace — then must state the expected path step by step`); } return errors; } module.exports = { - ROOT, PROSE_DIR, FIXTURES_DIR, STUBS_DIR, UNCHANGED, - listCaseFiles, parseCaseFile, loadAllCases, listStubs, readStub, validateCorpus, + ROOT, PROSE_DIR, CASES_DIR, STUBS_DIR, FILES, SNAPSHOTS, + listCaseIds, loadCase, loadAllCases, requireState, listStubs, readStub, validateCorpus, }; diff --git a/tests/prose/lib/fixtures.cjs b/tests/prose/lib/fixtures.cjs deleted file mode 100644 index 5892e1ac8..000000000 --- a/tests/prose/lib/fixtures.cjs +++ /dev/null @@ -1,241 +0,0 @@ -'use strict'; - -// Fixture recipes and golden snapshots. -// -// A fixture is tests/prose/fixtures/{name}/: a recipe.cjs (a module whose -// build(harness) drives real engine/knowledge calls against a scratch -// project) and a committed snapshot/ (the recipe's byte-exact output). -// The recipe is the source of truth; the snapshot is what makes drift -// visible — CI rebuilds every recipe and byte-compares (P2/P3 in -// design/prose-tests.md). Hand-editing a snapshot is forbidden: -// regenerate with `node tests/prose/run.cjs snap {name}`. -// -// Determinism: recipe subprocesses run under the frozen-clock preload and -// pinned git identity/dates. Snapshots exclude `.git/` (SHAs live in -// manifest values where recipes record them, not as a tree) and -// `.workflows/.knowledge/` (binary store; the world builder re-derives it -// at materialise time). `.gitignore` files inside a snapshot are stored -// escaped so the product-written `.workflows/.gitignore` cannot ignore -// fixture content out of THIS repo's git. - -const fs = require('fs'); -const os = require('os'); -const path = require('path'); -const { execFileSync, spawnSync } = require('child_process'); - -const ROOT = path.join(__dirname, '../../..'); -const ENGINE = path.join(ROOT, 'skills/workflow-engine/scripts/engine.cjs'); -const KNOWLEDGE = path.join(ROOT, 'skills/workflow-knowledge/scripts/knowledge.cjs'); -const FIXTURES_DIR = path.join(ROOT, 'tests/prose/fixtures'); -const CLOCK = path.join(__dirname, 'fake-clock.cjs'); - -const GITIGNORE = '.gitignore'; -const GITIGNORE_ESCAPED = '_gitignore.fixture'; - -function recipeEnv() { - if (/\s/.test(CLOCK)) { - throw new Error(`fake-clock path contains whitespace — NODE_OPTIONS cannot carry it: ${CLOCK}`); - } - return { - ...process.env, - NODE_OPTIONS: [process.env.NODE_OPTIONS, `--require ${CLOCK}`].filter(Boolean).join(' '), - GIT_CONFIG_GLOBAL: '/dev/null', - GIT_CONFIG_SYSTEM: '/dev/null', - GIT_AUTHOR_DATE: '2026-01-01T00:00:00Z', - GIT_COMMITTER_DATE: '2026-01-01T00:00:00Z', - TZ: 'UTC', - }; -} - -function makeHarness(dir) { - const env = recipeEnv(); - const node = (script, args) => { - const res = spawnSync('node', [script, ...args], { cwd: dir, encoding: 'utf8', env }); - if (res.status !== 0) { - throw new Error(`recipe call failed: ${path.basename(script)} ${args.join(' ')}\n` + - `stdout: ${res.stdout}\nstderr: ${res.stderr}`); - } - return res.stdout; - }; - return { - dir, - engine: (...args) => node(ENGINE, args), - knowledge: (...args) => node(KNOWLEDGE, args), - git: (...args) => execFileSync('git', args, { cwd: dir, encoding: 'utf8', env }), - write(rel, content) { - const full = path.join(dir, rel); - fs.mkdirSync(path.dirname(full), { recursive: true }); - fs.writeFileSync(full, content); - }, - remove(rel) { - fs.rmSync(path.join(dir, rel), { recursive: true, force: true }); - }, - }; -} - -function listFixtures() { - if (!fs.existsSync(FIXTURES_DIR)) return []; - return fs.readdirSync(FIXTURES_DIR, { withFileTypes: true }) - .filter((e) => e.isDirectory() && fs.existsSync(path.join(FIXTURES_DIR, e.name, 'recipe.cjs'))) - .map((e) => e.name); -} - -/** Build a fixture's world into a fresh scratch dir; caller removes it. */ -function runRecipe(name) { - const recipePath = path.join(FIXTURES_DIR, name, 'recipe.cjs'); - if (!fs.existsSync(recipePath)) throw new Error(`no recipe for fixture "${name}"`); - const dir = fs.mkdtempSync(path.join(os.tmpdir(), `prose-recipe-${name}-`)); - const h = makeHarness(dir); - h.git('init', '-q', '-b', 'main'); - h.git('config', 'user.email', 'prose@example.com'); - h.git('config', 'user.name', 'Prose Fixture'); - h.git('config', 'commit.gpgsign', 'false'); - fs.mkdirSync(path.join(dir, '.workflows'), { recursive: true }); - delete require.cache[require.resolve(recipePath)]; - require(recipePath).build(h); - return dir; -} - -function excluded(rel) { - const parts = rel.split(path.sep); - if (parts.includes('.git')) return true; - if (rel === path.join('.workflows', '.knowledge')) return true; - if (rel.startsWith(path.join('.workflows', '.knowledge') + path.sep)) return true; - // Installed skills/agents: copied into live worlds by the world builder, - // never part of a snapshot. Excluding them lets an acted world be - // compared against an expected snapshot on equal terms. - if (rel.startsWith(path.join('.claude', 'skills') + path.sep)) return true; - if (rel.startsWith(path.join('.claude', 'agents') + path.sep)) return true; - return false; -} - -/** Scratch tree → Map(snapshot-relative path → Buffer), escaped + filtered. */ -function collectTree(root) { - const files = new Map(); - const walk = (rel) => { - const abs = path.join(root, rel); - for (const entry of fs.readdirSync(abs, { withFileTypes: true })) { - const childRel = rel === '' ? entry.name : path.join(rel, entry.name); - if (excluded(childRel)) continue; - if (entry.isDirectory()) { - walk(childRel); - } else if (entry.isFile()) { - const stored = entry.name === GITIGNORE - ? path.join(path.dirname(childRel), GITIGNORE_ESCAPED) - : childRel; - files.set(stored, fs.readFileSync(path.join(root, childRel))); - } - } - }; - walk(''); - return files; -} - -/** Committed snapshot dir → Map(snapshot-relative path → Buffer), verbatim. */ -function readSnapshot(name) { - const snapDir = path.join(FIXTURES_DIR, name, 'snapshot'); - if (!fs.existsSync(snapDir)) return null; - const files = new Map(); - const walk = (rel) => { - for (const entry of fs.readdirSync(path.join(snapDir, rel), { withFileTypes: true })) { - const childRel = rel === '' ? entry.name : path.join(rel, entry.name); - if (entry.isDirectory()) walk(childRel); - else if (entry.isFile()) files.set(childRel, fs.readFileSync(path.join(snapDir, childRel))); - } - }; - walk(''); - return files; -} - -/** Regenerate the committed snapshot from a built scratch tree. */ -function writeSnapshot(name, scratchDir) { - const snapDir = path.join(FIXTURES_DIR, name, 'snapshot'); - fs.rmSync(snapDir, { recursive: true, force: true }); - const files = collectTree(scratchDir); - for (const [rel, buf] of files) { - const dest = path.join(snapDir, rel); - fs.mkdirSync(path.dirname(dest), { recursive: true }); - fs.writeFileSync(dest, buf); - } - return files.size; -} - -/** Rebuild-compare: {missing, extra, changed} — all empty means current. */ -function compareSnapshot(name, scratchDir) { - const snap = readSnapshot(name); - if (snap === null) return { missing: [''], extra: [], changed: [] }; - const built = collectTree(scratchDir); - const missing = []; - const extra = []; - const changed = []; - for (const [rel, buf] of built) { - if (!snap.has(rel)) extra.push(rel); - else if (!snap.get(rel).equals(buf)) changed.push(rel); - } - for (const rel of snap.keys()) { - if (!built.has(rel)) missing.push(rel); - } - return { missing, extra, changed }; -} - -/** Snapshot → real project tree at dest, unescaping .gitignore files. */ -function materialiseSnapshot(name, destDir) { - const snap = readSnapshot(name); - if (snap === null) { - throw new Error(`fixture "${name}" has no committed snapshot — run: node tests/prose/run.cjs snap ${name}`); - } - for (const [rel, buf] of snap) { - const real = path.basename(rel) === GITIGNORE_ESCAPED - ? path.join(path.dirname(rel), GITIGNORE) - : rel; - const dest = path.join(destDir, real); - fs.mkdirSync(path.dirname(dest), { recursive: true }); - fs.writeFileSync(dest, buf); - } -} - -/** Unified diff of two buffers, via git — no diff algorithm of our own. */ -function unifiedDiff(label, expectedBuf, actualBuf) { - const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'prose-diff-')); - try { - const a = path.join(dir, 'expected'); - const b = path.join(dir, 'actual'); - fs.writeFileSync(a, expectedBuf); - fs.writeFileSync(b, actualBuf); - const res = spawnSync('git', ['diff', '--no-index', '--unified=3', '--no-color', a, b], - { encoding: 'utf8', env: { ...process.env, GIT_CONFIG_GLOBAL: '/dev/null', GIT_CONFIG_SYSTEM: '/dev/null' } }); - const body = (res.stdout || '').split('\n').slice(4).join('\n').trimEnd(); - return `--- ${label}\n${body}`; - } finally { - fs.rmSync(dir, { recursive: true, force: true }); - } -} - -/** - * Compare a live (acted) world against a committed snapshot. Returns the - * factual delta — what a walk actually did to the world — for the - * asserting agent to classify. No normalisation: volatile fields (times, - * SHAs) surface as ordinary differences and the agent rules on them. - */ -function diffWorldAgainstSnapshot(worldDir, snapshotName) { - const expected = readSnapshot(snapshotName); - if (expected === null) throw new Error(`fixture "${snapshotName}" has no committed snapshot`); - const actual = collectTree(worldDir); - - const added = []; - const removed = []; - const changed = []; - for (const [rel, buf] of actual) { - if (!expected.has(rel)) added.push(rel); - else if (!expected.get(rel).equals(buf)) changed.push(unifiedDiff(rel, expected.get(rel), buf)); - } - for (const rel of expected.keys()) if (!actual.has(rel)) removed.push(rel); - - return { snapshot: snapshotName, added, removed, changed, identical: !added.length && !removed.length && !changed.length }; -} - -module.exports = { - ROOT, ENGINE, KNOWLEDGE, FIXTURES_DIR, - listFixtures, runRecipe, collectTree, readSnapshot, - writeSnapshot, compareSnapshot, materialiseSnapshot, diffWorldAgainstSnapshot, -}; diff --git a/tests/prose/lib/world.cjs b/tests/prose/lib/world.cjs deleted file mode 100644 index f3dc2e7b6..000000000 --- a/tests/prose/lib/world.cjs +++ /dev/null @@ -1,75 +0,0 @@ -'use strict'; - -// World builder — materialise a fixture into a live, parallel-safe fake -// project a walker agent can work in as if it were a real install: -// -// 1. copy the fixture snapshot in (unescaping .gitignore files), -// 2. copy the repo's current skills/ and agents/ to the installed -// layout (.claude/skills, .claude/agents) so prose paths resolve, -// 3. git init + commit (hermetic identity — no user config leaks), -// 4. `knowledge setup --keyword-only` — re-derives the keyword index -// from the snapshot's artifacts (the binary store is never -// snapshotted), leaving every entry skill's knowledge gate green. -// -// Worlds run on the REAL clock: walker runs are live sessions, never -// byte-compared. Mutations during a walk land here and nowhere else. - -const fs = require('fs'); -const os = require('os'); -const path = require('path'); -const { execFileSync, spawnSync } = require('child_process'); - -const fixtures = require('./fixtures.cjs'); - -const WORLD_PREFIX = 'prose-world-'; - -function worldEnv() { - return { - ...process.env, - GIT_CONFIG_GLOBAL: '/dev/null', - GIT_CONFIG_SYSTEM: '/dev/null', - }; -} - -function buildWorld(fixtureName) { - const dir = fs.mkdtempSync(path.join(os.tmpdir(), WORLD_PREFIX)); - const env = worldEnv(); - const git = (...args) => execFileSync('git', args, { cwd: dir, encoding: 'utf8', env }); - - fixtures.materialiseSnapshot(fixtureName, dir); - - for (const layer of ['skills', 'agents']) { - const src = path.join(fixtures.ROOT, layer); - if (fs.existsSync(src)) { - fs.cpSync(src, path.join(dir, '.claude', layer), { recursive: true }); - } - } - - git('init', '-q', '-b', 'main'); - git('config', 'user.email', 'prose@example.com'); - git('config', 'user.name', 'Prose World'); - git('config', 'commit.gpgsign', 'false'); - git('add', '-A'); - git('commit', '-q', '-m', `world: ${fixtureName}`); - - const knowledge = path.join(dir, '.claude/skills/workflow-knowledge/scripts/knowledge.cjs'); - const setup = spawnSync('node', [knowledge, 'setup', '--keyword-only'], - { cwd: dir, encoding: 'utf8', env }); - if (setup.status !== 0) { - fs.rmSync(dir, { recursive: true, force: true }); - throw new Error(`knowledge setup failed in world:\nstdout: ${setup.stdout}\nstderr: ${setup.stderr}`); - } - git('add', '-A'); - git('commit', '-q', '-m', 'chore(knowledge): initialise store'); - - return dir; -} - -function destroyWorld(dir) { - if (!path.basename(dir).startsWith(WORLD_PREFIX)) { - throw new Error(`refusing to remove non-world directory: ${dir}`); - } - fs.rmSync(dir, { recursive: true, force: true, maxRetries: 5, retryDelay: 100 }); -} - -module.exports = { buildWorld, destroyWorld, WORLD_PREFIX }; diff --git a/tests/prose/lib/worlds.cjs b/tests/prose/lib/worlds.cjs new file mode 100644 index 000000000..2c4697889 --- /dev/null +++ b/tests/prose/lib/worlds.cjs @@ -0,0 +1,326 @@ +'use strict'; + +// World building: run a case's state recipe, snapshot it, compare it. +// +// A recipe (fixture-state.cjs / assertion-state.cjs) drives real engine +// calls against a scratch project under a frozen clock; its committed +// output lives beside it as fixture/ or assertion/. The recipe is the +// source of truth, the snapshot is the golden that makes drift visible: +// `npm test` rebuilds and byte-compares, so an engine change that moves +// a world goes red at the gate and lands as a reviewable diff. +// +// Rebuilds are skipped when nothing that feeds them has changed — the +// hash of the case's recipes, the shared mainlines, and the engine and +// knowledge sources is stored inside each snapshot. An engine change +// invalidates every hash, so drift can never hide behind the skip. +// +// Snapshots exclude `.git/` (SHAs), `.workflows/.knowledge/` (binary +// store, re-derived at materialise) and `.claude/skills|agents/` (copied +// into live worlds, never part of a world's own state), and store +// `.gitignore` files escaped so the product-written `.workflows/.gitignore` +// cannot ignore snapshot content out of this repo. + +const crypto = require('crypto'); +const fs = require('fs'); +const os = require('os'); +const path = require('path'); +const { execFileSync, spawnSync } = require('child_process'); + +const cases = require('./cases.cjs'); + +const ROOT = cases.ROOT; +const ENGINE = path.join(ROOT, 'skills/workflow-engine/scripts/engine.cjs'); +const KNOWLEDGE = path.join(ROOT, 'skills/workflow-knowledge/scripts/knowledge.cjs'); +const MAINLINES_DIR = path.join(cases.PROSE_DIR, 'mainlines'); +const CLOCK = path.join(__dirname, 'fake-clock.cjs'); + +const GITIGNORE = '.gitignore'; +const GITIGNORE_ESCAPED = '_gitignore.fixture'; +const HASH_FILE = '.recipe-hash'; + +// --- the recipe harness --------------------------------------------------- + +function recipeEnv() { + if (/\s/.test(CLOCK)) { + throw new Error(`fake-clock path contains whitespace — NODE_OPTIONS cannot carry it: ${CLOCK}`); + } + return { + ...process.env, + NODE_OPTIONS: [process.env.NODE_OPTIONS, `--require ${CLOCK}`].filter(Boolean).join(' '), + GIT_CONFIG_GLOBAL: '/dev/null', + GIT_CONFIG_SYSTEM: '/dev/null', + GIT_AUTHOR_DATE: '2026-01-01T00:00:00Z', + GIT_COMMITTER_DATE: '2026-01-01T00:00:00Z', + TZ: 'UTC', + }; +} + +function makeHarness(dir) { + const env = recipeEnv(); + const node = (script, args) => { + const res = spawnSync('node', [script, ...args], { cwd: dir, encoding: 'utf8', env }); + if (res.status !== 0) { + throw new Error(`recipe call failed: ${path.basename(script)} ${args.join(' ')}\n` + + `stdout: ${res.stdout}\nstderr: ${res.stderr}`); + } + return res.stdout; + }; + return { + dir, + engine: (...args) => node(ENGINE, args), + knowledge: (...args) => node(KNOWLEDGE, args), + git: (...args) => execFileSync('git', args, { cwd: dir, encoding: 'utf8', env }), + write(rel, content) { + const full = path.join(dir, rel); + fs.mkdirSync(path.dirname(full), { recursive: true }); + fs.writeFileSync(full, content); + }, + remove(rel) { + fs.rmSync(path.join(dir, rel), { recursive: true, force: true }); + }, + }; +} + +/** Build a case's world into a fresh scratch dir; the caller removes it. */ +function runRecipe(caseId, which) { + const state = cases.requireState(caseId, which); + if (!state) throw new Error(`case "${caseId}" has no ${cases.FILES[which]}`); + const dir = fs.mkdtempSync(path.join(os.tmpdir(), `prose-recipe-${caseId}-`)); + const h = makeHarness(dir); + h.git('init', '-q', '-b', 'main'); + h.git('config', 'user.email', 'prose@example.com'); + h.git('config', 'user.name', 'Prose Fixture'); + h.git('config', 'commit.gpgsign', 'false'); + fs.mkdirSync(path.join(dir, '.workflows'), { recursive: true }); + state.build(h); + return dir; +} + +// --- trees ---------------------------------------------------------------- + +function excluded(rel) { + const parts = rel.split(path.sep); + if (parts.includes('.git')) return true; + if (rel === HASH_FILE) return true; + if (rel === path.join('.workflows', '.knowledge')) return true; + if (rel.startsWith(path.join('.workflows', '.knowledge') + path.sep)) return true; + if (rel.startsWith(path.join('.claude', 'skills') + path.sep)) return true; + if (rel.startsWith(path.join('.claude', 'agents') + path.sep)) return true; + return false; +} + +/** Directory → Map(snapshot-relative path → Buffer), escaped + filtered. */ +function collectTree(root) { + const files = new Map(); + const walk = (rel) => { + for (const entry of fs.readdirSync(path.join(root, rel), { withFileTypes: true })) { + const childRel = rel === '' ? entry.name : path.join(rel, entry.name); + if (excluded(childRel)) continue; + if (entry.isDirectory()) walk(childRel); + else if (entry.isFile()) { + const stored = entry.name === GITIGNORE + ? path.join(path.dirname(childRel), GITIGNORE_ESCAPED) + : childRel; + files.set(stored, fs.readFileSync(path.join(root, childRel))); + } + } + }; + walk(''); + return files; +} + +function snapshotDir(caseId, which) { + return path.join(cases.CASES_DIR, caseId, cases.SNAPSHOTS[which]); +} + +function readSnapshot(caseId, which) { + const dir = snapshotDir(caseId, which); + return fs.existsSync(dir) ? collectTree(dir) : null; +} + +// --- recipe hashing ------------------------------------------------------- + +function hashDir(hash, dir) { + if (!fs.existsSync(dir)) return; + for (const entry of fs.readdirSync(dir, { withFileTypes: true }).sort((a, b) => a.name.localeCompare(b.name))) { + const full = path.join(dir, entry.name); + if (entry.isDirectory()) hashDir(hash, full); + else if (entry.isFile()) hash.update(entry.name).update(fs.readFileSync(full)); + } +} + +/** Everything a world's content can depend on: its recipes, the shared + * mainlines, and the engine and knowledge sources that execute them. */ +function recipeHash(caseId) { + const hash = crypto.createHash('sha256'); + for (const which of ['fixtureState', 'assertionState']) { + const file = path.join(cases.CASES_DIR, caseId, cases.FILES[which]); + if (fs.existsSync(file)) hash.update(fs.readFileSync(file)); + } + hashDir(hash, MAINLINES_DIR); + hashDir(hash, path.join(ROOT, 'skills/workflow-engine/scripts')); + hash.update(fs.readFileSync(KNOWLEDGE)); + return hash.digest('hex'); +} + +function storedHash(caseId, which) { + const file = path.join(snapshotDir(caseId, which), HASH_FILE); + return fs.existsSync(file) ? fs.readFileSync(file, 'utf8').trim() : null; +} + +// --- snapshot write / verify ---------------------------------------------- + +// Snapshot name ('fixture'|'assertion') → the module that builds it. +const STATE_OF = { fixture: 'fixtureState', assertion: 'assertionState' }; + +function writeSnapshot(caseId, which) { + const scratch = runRecipe(caseId, STATE_OF[which]); + try { + const dir = snapshotDir(caseId, which); + fs.rmSync(dir, { recursive: true, force: true }); + const files = collectTree(scratch); + for (const [rel, buf] of files) { + const dest = path.join(dir, rel); + fs.mkdirSync(path.dirname(dest), { recursive: true }); + fs.writeFileSync(dest, buf); + } + fs.writeFileSync(path.join(dir, HASH_FILE), `${recipeHash(caseId)}\n`); + return files.size; + } finally { + fs.rmSync(scratch, { recursive: true, force: true }); + } +} + +/** + * Rebuild and byte-compare, unless nothing feeding this world has moved. + * Returns {skipped} or {missing, extra, changed}. + */ +function verifySnapshot(caseId, which) { + const snap = readSnapshot(caseId, which); + if (snap === null) return { missing: [''], extra: [], changed: [] }; + if (storedHash(caseId, which) === recipeHash(caseId)) return { skipped: true }; + + const scratch = runRecipe(caseId, STATE_OF[which]); + try { + const built = collectTree(scratch); + const missing = []; + const extra = []; + const changed = []; + for (const [rel, buf] of built) { + if (!snap.has(rel)) extra.push(rel); + else if (!snap.get(rel).equals(buf)) changed.push(rel); + } + for (const rel of snap.keys()) if (!built.has(rel)) missing.push(rel); + return { missing, extra, changed }; + } finally { + fs.rmSync(scratch, { recursive: true, force: true }); + } +} + +// --- diffing a live world ------------------------------------------------- + +/** Unified diff of two buffers, via git — no diff algorithm of our own. */ +function unifiedDiff(label, expectedBuf, actualBuf) { + const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'prose-diff-')); + try { + const a = path.join(dir, 'expected'); + const b = path.join(dir, 'actual'); + fs.writeFileSync(a, expectedBuf); + fs.writeFileSync(b, actualBuf); + const res = spawnSync('git', ['diff', '--no-index', '--unified=3', '--no-color', a, b], { + encoding: 'utf8', + env: { ...process.env, GIT_CONFIG_GLOBAL: '/dev/null', GIT_CONFIG_SYSTEM: '/dev/null' }, + }); + return `--- ${label}\n${(res.stdout || '').split('\n').slice(4).join('\n').trimEnd()}`; + } finally { + fs.rmSync(dir, { recursive: true, force: true }); + } +} + +/** + * The factual delta between the world after a walk and the world the case + * expects — for the asserting agent to classify. Nothing is normalised: + * volatile values surface as ordinary differences and the agent rules on + * them. A case with no assertion-state expects its fixture back unchanged. + */ +function diffWorld(caseId, worldDir) { + const which = fs.existsSync(snapshotDir(caseId, 'assertion')) ? 'assertion' : 'fixture'; + const expected = readSnapshot(caseId, which); + if (expected === null) throw new Error(`case "${caseId}" has no committed ${which} snapshot`); + const actual = collectTree(worldDir); + + const added = []; + const removed = []; + const changed = []; + for (const [rel, buf] of actual) { + if (!expected.has(rel)) added.push(rel); + else if (!expected.get(rel).equals(buf)) changed.push(unifiedDiff(rel, expected.get(rel), buf)); + } + for (const rel of expected.keys()) if (!actual.has(rel)) removed.push(rel); + + return { + expecting: which === 'assertion' ? 'the assertion state' : 'the fixture state, unchanged', + added, + removed, + changed, + identical: !added.length && !removed.length && !changed.length, + }; +} + +// --- materialising a live world ------------------------------------------- + +const WORLD_PREFIX = 'prose-world-'; + +function buildWorld(caseId) { + const snap = readSnapshot(caseId, 'fixture'); + if (snap === null) { + throw new Error(`case "${caseId}" has no committed fixture — run: node tests/prose/run.cjs snap ${caseId}`); + } + const dir = fs.mkdtempSync(path.join(os.tmpdir(), WORLD_PREFIX)); + const env = { ...process.env, GIT_CONFIG_GLOBAL: '/dev/null', GIT_CONFIG_SYSTEM: '/dev/null' }; + const git = (...args) => execFileSync('git', args, { cwd: dir, encoding: 'utf8', env }); + + for (const [rel, buf] of snap) { + const real = path.basename(rel) === GITIGNORE_ESCAPED + ? path.join(path.dirname(rel), GITIGNORE) + : rel; + const dest = path.join(dir, real); + fs.mkdirSync(path.dirname(dest), { recursive: true }); + fs.writeFileSync(dest, buf); + } + for (const layer of ['skills', 'agents']) { + const src = path.join(ROOT, layer); + if (fs.existsSync(src)) fs.cpSync(src, path.join(dir, '.claude', layer), { recursive: true }); + } + + git('init', '-q', '-b', 'main'); + git('config', 'user.email', 'prose@example.com'); + git('config', 'user.name', 'Prose World'); + git('config', 'commit.gpgsign', 'false'); + git('add', '-A'); + git('commit', '-q', '-m', `world: ${caseId}`); + + const knowledge = path.join(dir, '.claude/skills/workflow-knowledge/scripts/knowledge.cjs'); + const setup = spawnSync('node', [knowledge, 'setup', '--keyword-only'], { cwd: dir, encoding: 'utf8', env }); + if (setup.status !== 0) { + fs.rmSync(dir, { recursive: true, force: true }); + throw new Error(`knowledge setup failed in world:\nstdout: ${setup.stdout}\nstderr: ${setup.stderr}`); + } + git('add', '-A'); + git('commit', '-q', '-m', 'chore(knowledge): initialise store'); + + return dir; +} + +function destroyWorld(dir) { + if (!path.basename(dir).startsWith(WORLD_PREFIX)) { + throw new Error(`refusing to remove non-world directory: ${dir}`); + } + fs.rmSync(dir, { recursive: true, force: true, maxRetries: 5, retryDelay: 100 }); +} + +module.exports = { + ROOT, ENGINE, KNOWLEDGE, MAINLINES_DIR, WORLD_PREFIX, + runRecipe, collectTree, readSnapshot, snapshotDir, recipeHash, storedHash, + writeSnapshot, verifySnapshot, diffWorld, buildWorld, destroyWorld, +}; diff --git a/tests/prose/fixtures/_shared/bugfix-mainline.cjs b/tests/prose/mainlines/bugfix.cjs similarity index 100% rename from tests/prose/fixtures/_shared/bugfix-mainline.cjs rename to tests/prose/mainlines/bugfix.cjs diff --git a/tests/prose/fixtures/_shared/feature-mainline.cjs b/tests/prose/mainlines/feature.cjs similarity index 100% rename from tests/prose/fixtures/_shared/feature-mainline.cjs rename to tests/prose/mainlines/feature.cjs diff --git a/tests/prose/planning-entry-asks-for-late-context.md b/tests/prose/planning-entry-asks-for-late-context.md deleted file mode 100644 index be116cb3b..000000000 --- a/tests/prose/planning-entry-asks-for-late-context.md +++ /dev/null @@ -1,39 +0,0 @@ -## case: planning-entry-asks-for-late-context -- origin: feature mainline — planning offers a last chance to add context -- files: - - skills/workflow-planning-entry/SKILL.md - - skills/workflow-planning-entry/references/validate-spec.md - - skills/workflow-planning-entry/references/validate-phase.md - - skills/workflow-planning-entry/references/cross-cutting-context.md - -### given - -world_before: feature-specified - -### when - -Execute skills/workflow-planning-entry/SKILL.md with arguments -$0=feature, $1=pay. Follow it to the point where the handoff to the -processing skill is constructed; record the handoff block and stop. Do -not execute the processing skill's instructions. - -answers: -1. continue — nothing has changed since the specification - -### then - -world_after: unchanged - -trace: -1. the specification gate renders empty — the spec is complete and the - topic is clear to plan -2. no plan exists, so the fresh-start arm asks whether anything has - changed since the specification, and waits -3. on continue, the source is fresh and carries no additional context -4. cross-cutting context is gathered regardless of work type: it finds - no cross-cutting work units, then queries the knowledge base for - completed cross-cutting specs -5. hands off to the planning processing skill for pay - -notes: -- the knowledge base is queried, never written to, by an entry skill diff --git a/tests/prose/review-entry-fresh.md b/tests/prose/review-entry-fresh.md deleted file mode 100644 index 48b49b7ce..000000000 --- a/tests/prose/review-entry-fresh.md +++ /dev/null @@ -1,31 +0,0 @@ -## case: review-entry-fresh -- origin: feature mainline — review refuses to start until delivery is complete -- files: - - skills/workflow-review-entry/SKILL.md - - skills/workflow-review-entry/references/validate-phase.md - -### given - -world_before: feature-implemented - -### when - -Execute skills/workflow-review-entry/SKILL.md with arguments $0=feature, -$1=pay. Follow it to the point where the handoff to the processing skill -is constructed; record the handoff block and stop. Do not execute the -processing skill's instructions, and dispatch no agents. - -### then - -world_after: unchanged - -trace: -1. resolves the topic to the work unit -2. the prerequisite gate renders empty — both the plan and the - implementation are complete, so nothing blocks entry -3. no review item exists, so the fresh path is taken — nothing is - reopened and no resume is announced -4. hands off to the review processing skill for pay - -notes: -- no verifier agents are dispatched by the entry skill diff --git a/tests/prose/root-cause-validation-clean-verdict.md b/tests/prose/root-cause-validation-clean-verdict.md deleted file mode 100644 index e7cd2b596..000000000 --- a/tests/prose/root-cause-validation-clean-verdict.md +++ /dev/null @@ -1,45 +0,0 @@ -## case: root-cause-validation-clean-verdict -- origin: bugfix mainline — the validation agent's lifecycle on a clean verdict -- files: - - skills/workflow-investigation-process/references/root-cause-validation.md - -### given - -world_before: bugfix-investigating - -The investigation is open with its symptoms gathered and its root cause -documented. The processing skill has worked down to its root-cause -validation step; nothing has been validated yet and no agent has run. - -### when - -Walk root-cause-validation.md from the top, as the processing skill -would. Stop when the reference returns to its caller. - -answers: -1. yes — run root cause validation - -stubs: - - root-cause-validated: when the engine records a dispatch of kind root-cause-validation - -### then - -world_after: bugfix-validation-incorporated - -trace: -1. offers the independent validation, presents the run-or-skip choice, - and waits -2. on yes, records the dispatch through the engine and creates no report - file of its own — the response carries the id and the path to use -3. announces that validation is running, then dispatches exactly one - agent, synchronously -4. once the report has landed, promotes the row with a scan and closes - it with an incorporate — the verdict is consumed whole, never - surfaced finding by finding -5. reads the report and, the verdict being validated, states the - confidence and returns to its caller — the gap-handling choice is - never reached - -notes: -- the investigation file is untouched and nothing is committed: the - validated path has no gaps to record diff --git a/tests/prose/run.cjs b/tests/prose/run.cjs index 918b90f1a..26872856f 100644 --- a/tests/prose/run.cjs +++ b/tests/prose/run.cjs @@ -7,12 +7,12 @@ // // list corpus table // select [--diff |--all|--cases a,b] cases to run, as JSON -// world materialise world_before, print path -// prompt --world walker prompt (NEVER contains `then`) -// diff --world acted world vs world_after, as facts -// assert --world the asserting agent's prompt (diff + then) -// snap (re)generate a fixture's golden snapshot -// verify [fixture] rebuild-compare fixture(s) +// world materialise the fixture state, print path +// prompt --world walker prompt (NEVER contains assert.md) +// diff --world acted world vs expected world, as facts +// assert --world the asserting agent's prompt +// snap (re)generate a case's snapshots +// verify [case-id] rebuild-compare snapshot(s) // destroy --world remove a world const fs = require('fs'); @@ -20,8 +20,7 @@ const path = require('path'); const { execFileSync } = require('child_process'); const cases = require('./lib/cases.cjs'); -const fixtures = require('./lib/fixtures.cjs'); -const world = require('./lib/world.cjs'); +const worlds = require('./lib/worlds.cjs'); const ROOT = cases.ROOT; @@ -30,10 +29,10 @@ function die(msg) { process.exit(1); } -function loadCase(id) { - const found = cases.loadAllCases().find((c) => c.id === id); - if (!found) die(`no case "${id}" in the corpus`); - return found; +function getCase(id) { + if (!id) die('a case id is required'); + if (!cases.listCaseIds().includes(id)) die(`no case "${id}" in tests/prose/cases`); + return cases.loadCase(id); } function flag(argv, name) { @@ -70,23 +69,23 @@ function cmdSelect(argv) { } else if (flag(argv, '--cases')) { mode = 'ids'; const ids = flag(argv, '--cases').split(',').map((s) => s.trim()); - const byId = new Map(all.map((c) => [c.id, c])); - for (const id of ids) if (!byId.has(id)) die(`no case "${id}" in the corpus`); - selected = ids.map((id) => byId.get(id)); + for (const id of ids) getCase(id); + selected = all.filter((c) => ids.includes(c.id)); } else { const ref = flag(argv, '--diff') || 'main'; mode = `diff:${ref}`; const changed = changedFiles(ref); - selected = all.filter((c) => changed.has(c.file) - || c.files.some((f) => changed.has(f.path)) - || [c.worldBefore, c.worldAfter].filter(Boolean).some((w) => - [...changed].some((p) => p.startsWith(`tests/prose/fixtures/${w}/`))) - || c.stubs.some((s) => changed.has(`tests/prose/stubs/${s.name}.md`))); + selected = all.filter((c) => c.files.some((f) => changed.has(f.path)) + || [...changed].some((p) => p.startsWith(`${c.rel}/`)) + || c.stubs.some((s) => changed.has(`tests/prose/stubs/${s.name}.md`)) + || [...changed].some((p) => p.startsWith('tests/prose/mainlines/'))); } process.stdout.write(`${JSON.stringify({ mode, cases: selected.map((c) => ({ - id: c.id, file: c.file, world_before: c.worldBefore, world_after: c.worldAfter, + id: c.id, + dir: c.rel, + expects: c.hasAssertionState ? 'assertion state' : 'fixture state unchanged', })), }, null, 2)}\n`); } @@ -94,26 +93,25 @@ function cmdSelect(argv) { // --- world / destroy ------------------------------------------------------ function cmdWorld(argv) { - const c = loadCase(argv[0] || die('usage: world ')); - if (!c.worldBefore) { + const c = getCase(argv[0]); + if (!c.hasFixtureState) { process.stdout.write(`${JSON.stringify({ world: null, note: 'structure-only case — no world' })}\n`); return; } - const dir = world.buildWorld(c.worldBefore); - process.stdout.write(`${JSON.stringify({ world: dir, fixture: c.worldBefore })}\n`); + process.stdout.write(`${JSON.stringify({ world: worlds.buildWorld(c.id), case: c.id })}\n`); } function cmdDestroy(argv) { const dir = flag(argv, '--world') || die('usage: destroy --world '); - world.destroyWorld(dir); + worlds.destroyWorld(dir); process.stdout.write(`destroyed ${dir}\n`); } // --- prompt (the walker) -------------------------------------------------- function cmdPrompt(argv) { - const c = loadCase(argv[0] || die('usage: prompt [--world ]')); - const worldDir = c.worldBefore ? requireWorld(argv, c) : null; + const c = getCase(argv[0]); + const worldDir = c.hasFixtureState ? requireWorld(argv, c) : null; const setting = worldDir ? [ @@ -146,7 +144,7 @@ function cmdPrompt(argv) { return [ `### ${s.name}`, `WHEN: ${s.trigger}`, - `WHAT IT IS: ${stub.description.replace(/^#.*\n/, '').trim()}`, + `WHAT IT IS: ${stub.description}`, 'CONTENT (write these exact bytes where the substitution calls for a file):', ...stub.content.split('\n').map((l) => ` ${l}`), '', @@ -154,14 +152,12 @@ function cmdPrompt(argv) { }), ] : []; - const situation = c.situation ? ['', 'SITUATION — where the project stands as you begin:', c.situation] : []; - process.stdout.write(`${[ ...setting, - ...situation, + ...(c.situation ? ['', 'SITUATION — where the project stands as you begin:', c.situation] : []), '', 'TASK', - c.when, + c.act, '', 'SCOPE — the prose under walk:', ...c.files.map((f) => ` - ${f.path}${f.anchor ? ` (start at the heading containing "${f.anchor}")` : ''}`), @@ -183,7 +179,7 @@ function cmdPrompt(argv) { ' record `DEVIATION:` with what you could not do, then continue as best you', ' can. NEVER silently repair, reinterpret, or improve the prose. You are a', ' probe, not a reviewer: a broken instruction is the finding.', - '- Stop at the TASK\'s stop condition, the end of the flow, an UNSCRIPTED', + "- Stop at the TASK's stop condition, the end of the flow, an UNSCRIPTED", ' QUESTION, or a hard error — whichever comes first.', '', 'TRANSCRIPT — your entire final output, in order of events:', @@ -199,42 +195,33 @@ function cmdPrompt(argv) { // --- diff (the facts) ----------------------------------------------------- -function worldDelta(c, worldDir) { - const target = c.worldAfter === cases.UNCHANGED ? c.worldBefore : c.worldAfter; - return fixtures.diffWorldAgainstSnapshot(worldDir, target); -} - function cmdDiff(argv) { - const c = loadCase(argv[0] || die('usage: diff --world ')); - if (!c.worldBefore) { + const c = getCase(argv[0]); + if (!c.hasFixtureState) { process.stdout.write(`${JSON.stringify({ note: 'structure-only case — no world to diff' }, null, 2)}\n`); return; } - process.stdout.write(`${JSON.stringify(worldDelta(c, requireWorld(argv, c)), null, 2)}\n`); + process.stdout.write(`${JSON.stringify(worlds.diffWorld(c.id, requireWorld(argv, c)), null, 2)}\n`); } // --- assert (the judging agent) ------------------------------------------- function cmdAssert(argv) { - const c = loadCase(argv[0] || die('usage: assert [--world ]')); + const c = getCase(argv[0]); const lines = [ 'You are asserting the result of a prose-test case. You did not perform', 'the walk. Judge only what the evidence shows.', '', - 'EXPECTED TRACE — the path the prose should have taken:', - ...c.trace.map((t, i) => ` ${i + 1}. ${t}`), + 'WHAT THE PROSE SHOULD HAVE DONE:', + '', + c.assert, ]; - if (c.notes.length) { - lines.push('', 'FURTHER CLAIMS:', ...c.notes.map((n) => ` - ${n}`)); - } - if (c.worldBefore) { - const delta = worldDelta(c, requireWorld(argv, c)); + if (c.hasFixtureState) { + const delta = worlds.diffWorld(c.id, requireWorld(argv, c)); lines.push( '', - c.worldAfter === cases.UNCHANGED - ? 'EXPECTED WORLD: unchanged — the walk should have left the project exactly as it found it.' - : `EXPECTED WORLD: the committed fixture "${c.worldAfter}".`, + `EXPECTED WORLD: ${delta.expecting}.`, '', 'WORLD DELTA — the factual difference between the world after the walk and', 'the expected world, computed by code. Volatile values (timestamps, git', @@ -252,7 +239,7 @@ function cmdAssert(argv) { 'THE TRANSCRIPT of the walk is supplied separately by the caller.', '', 'VERDICT — return exactly this, nothing else:', - '1. Trace: one line per expected step — PASS or FAIL, each PASS quoting the', + '1. Path: one line per expected step — PASS or FAIL, each PASS quoting the', ' transcript line that shows it. A PASS with no quote is invalid.', '2. World: PASS or FAIL. Enumerate EVERY difference in the delta and classify', ' each as volatile (immaterial) or material. Any material difference fails.', @@ -266,36 +253,43 @@ function cmdAssert(argv) { // --- snap / verify -------------------------------------------------------- +function statesOf(c) { + return [ + c.hasFixtureState ? 'fixture' : null, + c.hasAssertionState ? 'assertion' : null, + ].filter(Boolean); +} + function cmdSnap(argv) { - const name = argv[0] || die('usage: snap '); - const scratch = fixtures.runRecipe(name); - try { - const count = fixtures.writeSnapshot(name, scratch); - process.stdout.write(`snapshot ${name}: ${count} files written — review the diff before committing\n`); - } finally { - fs.rmSync(scratch, { recursive: true, force: true }); + const ids = argv[0] ? [getCase(argv[0]).id] : cases.listCaseIds(); + for (const id of ids) { + const c = cases.loadCase(id); + for (const which of statesOf(c)) { + const count = worlds.writeSnapshot(id, which); + process.stdout.write(`${id}/${which}: ${count} files — review the diff before committing\n`); + } } } function cmdVerify(argv) { - const names = argv[0] ? [argv[0]] : fixtures.listFixtures(); + const ids = argv[0] ? [getCase(argv[0]).id] : cases.listCaseIds(); let failed = false; - for (const name of names) { - const scratch = fixtures.runRecipe(name); - try { - const d = fixtures.compareSnapshot(name, scratch); - if (!d.missing.length && !d.extra.length && !d.changed.length) { - process.stdout.write(`${name}: snapshot current\n`); + for (const id of ids) { + const c = cases.loadCase(id); + for (const which of statesOf(c)) { + const d = worlds.verifySnapshot(id, which); + if (d.skipped) { + process.stdout.write(`${id}/${which}: unchanged since last build\n`); + } else if (!d.missing.length && !d.extra.length && !d.changed.length) { + process.stdout.write(`${id}/${which}: snapshot current\n`); } else { failed = true; - process.stdout.write(`${name}: DRIFT — the recipe no longer rebuilds the snapshot\n`); + process.stdout.write(`${id}/${which}: DRIFT — the recipe no longer rebuilds the snapshot\n`); for (const f of d.changed) process.stdout.write(` changed: ${f}\n`); for (const f of d.extra) process.stdout.write(` extra (rebuilt, not in snapshot): ${f}\n`); for (const f of d.missing) process.stdout.write(` missing (in snapshot, not rebuilt): ${f}\n`); - process.stdout.write(` regenerate: node tests/prose/run.cjs snap ${name}\n`); + process.stdout.write(` regenerate: node tests/prose/run.cjs snap ${id}\n`); } - } finally { - fs.rmSync(scratch, { recursive: true, force: true }); } } process.exit(failed ? 1 : 0); @@ -307,11 +301,11 @@ function cmdList() { const all = cases.loadAllCases(); const errors = cases.validateCorpus(all); for (const c of all) { - const stubs = c.stubs.length ? ` stubs=${c.stubs.map((s) => s.name).join(',')}` : ''; - process.stdout.write( - `${c.id}\n ${c.worldBefore || '(no world)'} → ${c.worldAfter} trace=${c.trace.length}${stubs}\n`); + const stubs = c.stubs.length ? ` stubs=${c.stubs.map((s) => s.name).join(',')}` : ''; + const expects = c.hasAssertionState ? 'assertion state' : 'no change'; + process.stdout.write(`${c.id}\n ${c.hasFixtureState ? 'fixture' : 'no world'} → ${expects}${stubs}\n`); } - process.stdout.write(`\n${all.length} cases, ${fixtures.listFixtures().length} fixtures, ${cases.listStubs().length} stubs`); + process.stdout.write(`\n${all.length} cases, ${cases.listStubs().length} stubs`); process.stdout.write(errors.length ? `, ${errors.length} VALIDATION ERRORS:\n` : ', corpus valid\n'); for (const e of errors) process.stdout.write(` - ${e}\n`); process.exit(errors.length ? 1 : 0); diff --git a/tests/prose/smoke-boot-in-world.md b/tests/prose/smoke-boot-in-world.md deleted file mode 100644 index 01d8ef331..000000000 --- a/tests/prose/smoke-boot-in-world.md +++ /dev/null @@ -1,28 +0,0 @@ -## case: smoke-boot-in-world -- origin: framework smoke — world build, engine-in-world, world comparison -- files: - - skills/workflow-start/SKILL.md#Boot - -### given - -world_before: base - -### when - -In the project, run the boot command exactly as the initialisation prose -of skills/workflow-start/SKILL.md prescribes — sandbox concerns do not -apply in this world, so run it directly. Capture the JSON response, then -stop; do not continue into any confirmation or later step. - -### then - -world_after: unchanged - -trace: -1. runs the boot command and reports the knowledge base ready, in - keyword-only mode -2. reports no migrations applied — this world is already migrated — so - the migrations summary and its confirmation are not raised - -notes: -- booting an already-migrated project changes nothing on disk diff --git a/tests/prose/smoke-start-boot-structure.md b/tests/prose/smoke-start-boot-structure.md deleted file mode 100644 index a7ec974cc..000000000 --- a/tests/prose/smoke-start-boot-structure.md +++ /dev/null @@ -1,23 +0,0 @@ -## case: smoke-start-boot-structure -- origin: framework smoke — a structure-only walk, no world -- files: - - skills/workflow-start/SKILL.md#Boot - -### when - -Structure-only: read the initialisation portion of -skills/workflow-start/SKILL.md — everything the skill does before any -work is shown or routed. Establish what initialisation consists of and -in what order its parts run. Stop there; do not trace the rest of the -skill. - -### then - -world_after: unchanged - -trace: -1. casing conventions are loaded before the boot pipeline runs -2. the boot pipeline is declared mandatory — it must complete before the - skill proceeds -3. the knowledge gate is reached only after boot, and branches on what - boot reported rather than on its own checks diff --git a/tests/prose/spec-entry-from-discussion.md b/tests/prose/spec-entry-from-discussion.md deleted file mode 100644 index 93b05d5f6..000000000 --- a/tests/prose/spec-entry-from-discussion.md +++ /dev/null @@ -1,34 +0,0 @@ -## case: spec-entry-from-discussion -- origin: feature mainline — a feature's spec draws on its discussion -- files: - - skills/workflow-specification-entry/SKILL.md - - skills/workflow-specification-entry/references/validate-source.md - - skills/workflow-specification-entry/references/validate-phase.md - - skills/workflow-specification-entry/references/invoke-skill.md - -### given - -world_before: feature-discussed - -### when - -Execute skills/workflow-specification-entry/SKILL.md with arguments -$0=feature, $1=pay. Follow it to the point where the handoff to the -processing skill is constructed; record the handoff block and stop. Do -not execute the processing skill's instructions. - -### then - -world_after: unchanged - -trace: -1. resolves the topic to the work unit and goes straight to validating - source material — the scoped path is for an epic with no topic -2. the source-material gate renders empty: the completed discussion - satisfies the prerequisite -3. no specification item exists, so the verb is Creating — no resume - note is rendered and nothing is reopened -4. hands off naming `.workflows/pay/discussion/pay.md` as the source - -notes: -- an entry that only validates writes nothing diff --git a/tests/prose/spec-entry-from-investigation.md b/tests/prose/spec-entry-from-investigation.md deleted file mode 100644 index 1cdca790c..000000000 --- a/tests/prose/spec-entry-from-investigation.md +++ /dev/null @@ -1,33 +0,0 @@ -## case: spec-entry-from-investigation -- origin: bugfix mainline — the same entry skill, sourced from an investigation -- files: - - skills/workflow-specification-entry/SKILL.md - - skills/workflow-specification-entry/references/validate-source.md - - skills/workflow-specification-entry/references/invoke-skill.md - -### given - -world_before: bugfix-investigated - -### when - -Execute skills/workflow-specification-entry/SKILL.md with arguments -$0=bugfix, $1=crash-fix. Follow it to the point where the handoff to the -processing skill is constructed; record the handoff block and stop. Do -not execute the processing skill's instructions. - -### then - -world_after: unchanged - -trace: -1. resolves the topic to the work unit and goes to validating source - material -2. the source-material gate renders empty: for a bugfix the completed - investigation is what satisfies it, in place of a discussion -3. no specification item exists, so the verb is Creating -4. hands off naming `.workflows/crash-fix/investigation/crash-fix.md` as - the source material — no discussion file is named - -notes: -- the same prose serves both work types; only the source arm differs diff --git a/tests/scripts/test-prose-cases.cjs b/tests/scripts/test-prose-cases.cjs index 8869ed7eb..847624496 100644 --- a/tests/scripts/test-prose-cases.cjs +++ b/tests/scripts/test-prose-cases.cjs @@ -1,12 +1,13 @@ 'use strict'; // Prose-test corpus: the deterministic perimeter (design/prose-tests.md -// P3). Every case parses, every scoped file and anchor resolves, every -// named world and stub exists, every case states an expected trace. The +// P3). Every case directory is well-formed, every scoped file and anchor +// resolves, every stub is named and triggered, every recipe loads. The // token-costing walks run on command via /prose-test — never here. const { describe, it } = require('node:test'); const assert = require('node:assert'); +const fs = require('fs'); const path = require('path'); const cases = require('../prose/lib/cases.cjs'); @@ -18,41 +19,51 @@ describe('prose-test corpus', () => { assert.ok(all.length > 0, 'corpus is empty'); }); - it('parses and validates cleanly', () => { + it('validates cleanly', () => { const errors = cases.validateCorpus(all); assert.deepStrictEqual(errors, [], `corpus validation failed:\n${errors.map((e) => ` - ${e}`).join('\n')}`); }); - it('is flat: one case per file, filename equal to the id', () => { - const seen = new Set(); + it('is one directory per case, holding only known files', () => { + const known = new Set([...Object.values(cases.FILES), ...Object.values(cases.SNAPSHOTS)]); for (const c of all) { - assert.ok(!seen.has(c.file), `${c.file}: more than one case in a file`); - seen.add(c.file); - assert.strictEqual(path.dirname(c.file), 'tests/prose', - `${c.file}: cases are flat — nothing groups them but their files: scope`); - assert.strictEqual(path.basename(c.file, '.md'), c.id, - `${c.file}: filename must equal the case id`); + for (const entry of fs.readdirSync(c.dir)) { + assert.ok(known.has(entry), + `${c.rel}/${entry}: unexpected file — a case holds only ${[...known].join(', ')}`); + } + } + }); + + it('keeps the walker and asserter prose in separate files', () => { + // The P4 boundary is enforced by the filesystem: one file goes to the + // walker, the other only to the asserter. Never merge them. + for (const c of all) { + assert.ok(c.act, `${c.rel}: no ${cases.FILES.act}`); + assert.ok(c.assert, `${c.rel}: no ${cases.FILES.assert}`); + assert.notStrictEqual(cases.FILES.act, cases.FILES.assert); } }); it('every stub is named, triggered by its case, and readable', () => { for (const c of all) { for (const s of c.stubs) { - assert.ok(s.trigger.length > 0, - `${c.id}: stub "${s.name}" has no trigger — a stub without a moment is arrange in disguise`); - const stub = cases.readStub(s.name); - assert.ok(stub.content.length > 0, `stub "${s.name}" has no content below its --- fence`); + assert.ok(s.trigger.trim().length > 0, + `${c.rel}: stub "${s.name}" has no trigger — a stub without a moment is a fixture in disguise`); + assert.ok(cases.readStub(s.name).content.length > 0, + `stub "${s.name}" has no content below its --- fence`); } } }); - it('multi-line trace steps and notes join rather than truncate', () => { + it('every state recipe loads and exports build()', () => { for (const c of all) { - for (const t of c.trace) assert.ok(!/\s$/.test(t), `${c.id}: trace step has trailing whitespace`); - for (const n of c.notes) assert.ok(n.length > 0, `${c.id}: empty note`); + for (const which of ['fixtureState', 'assertionState']) { + const file = path.join(c.dir, cases.FILES[which]); + if (!fs.existsSync(file)) continue; + assert.strictEqual(typeof cases.requireState(c.id, which).build, 'function', + `${c.rel}/${cases.FILES[which]} must export build(h)`); + } } - const wrapped = all.flatMap((c) => c.notes).filter((n) => n.length > 70); - assert.ok(wrapped.length > 0, 'expected at least one note long enough to have wrapped in source'); }); }); diff --git a/tests/scripts/test-prose-fixtures.cjs b/tests/scripts/test-prose-fixtures.cjs index c1c652e18..fc3303aa2 100644 --- a/tests/scripts/test-prose-fixtures.cjs +++ b/tests/scripts/test-prose-fixtures.cjs @@ -1,43 +1,50 @@ 'use strict'; -// Fixture golden check (design/prose-tests.md P2/P3): every committed +// Snapshot goldens (design/prose-tests.md P2/P3): every committed // snapshot rebuilds byte-identical from its recipe under the frozen -// clock. A red run here means the engine moved the world — regenerate -// with `node tests/prose/run.cjs snap ` and review the snapshot -// diff in the same PR as the change that moved it. Never hand-edit a +// clock. A red run means the engine moved the world — regenerate with +// `node tests/prose/run.cjs snap ` and land the snapshot diff in +// the same commit as the change that moved it. Never hand-edit a // snapshot to green this. +// +// Rebuilds are skipped for worlds whose recipes, shared mainlines, and +// engine sources are all unchanged since the snapshot was written — so a +// normal run costs nothing, and any engine change invalidates every hash +// and rebuilds the lot. const { describe, it } = require('node:test'); const assert = require('node:assert'); -const fs = require('fs'); -const fixtures = require('../prose/lib/fixtures.cjs'); +const cases = require('../prose/lib/cases.cjs'); +const worlds = require('../prose/lib/worlds.cjs'); -describe('prose-test fixtures', () => { - const names = fixtures.listFixtures(); +describe('prose-test snapshots', () => { + const all = cases.loadAllCases(); - it('has at least one fixture', () => { - assert.ok(names.length > 0, 'no fixtures found'); + it('has at least one built world', () => { + assert.ok(all.some((c) => c.hasFixtureState), 'no case builds a world'); }); - for (const name of names) { - it(`${name}: recipe rebuilds the committed snapshot byte-identical`, () => { - assert.ok(fixtures.readSnapshot(name) !== null, - `fixture "${name}" has no committed snapshot — run: node tests/prose/run.cjs snap ${name}`); - const scratch = fixtures.runRecipe(name); - try { - const diff = fixtures.compareSnapshot(name, scratch); + for (const c of all) { + const states = [ + c.hasFixtureState ? 'fixture' : null, + c.hasAssertionState ? 'assertion' : null, + ].filter(Boolean); + + for (const which of states) { + it(`${c.id}/${which}: rebuilds byte-identical`, () => { + assert.ok(worlds.readSnapshot(c.id, which) !== null, + `${c.id}/${which} has no committed snapshot — run: node tests/prose/run.cjs snap ${c.id}`); + const d = worlds.verifySnapshot(c.id, which); + if (d.skipped) return; const report = [ - ...diff.changed.map((f) => `changed: ${f}`), - ...diff.extra.map((f) => `extra (rebuilt, not in snapshot): ${f}`), - ...diff.missing.map((f) => `missing (in snapshot, not rebuilt): ${f}`), + ...d.changed.map((f) => `changed: ${f}`), + ...d.extra.map((f) => `extra (rebuilt, not in snapshot): ${f}`), + ...d.missing.map((f) => `missing (in snapshot, not rebuilt): ${f}`), ]; assert.deepStrictEqual(report, [], - `world moved — regenerate: node tests/prose/run.cjs snap ${name}\n` + - report.map((l) => ` ${l}`).join('\n')); - } finally { - fs.rmSync(scratch, { recursive: true, force: true }); - } - }); + `world moved — regenerate: node tests/prose/run.cjs snap ${c.id}\n${report.map((l) => ` ${l}`).join('\n')}`); + }); + } } }); From d427c08ac6f83209ba65984e2cfd422263378432 Mon Sep 17 00:00:00 2001 From: Lee Overy Date: Sat, 25 Jul 2026 18:46:47 +0100 Subject: [PATCH 07/11] chore(prose-tests): delete the dead fixture library The case-directory restructure was meant to remove tests/prose/fixtures but the rm ran from the wrong working directory and silently deleted nothing, leaving 98 committed files that no code references. Spotted by Lee reading the PR. Co-Authored-By: Claude Opus 5 --- tests/prose/fixtures/base/recipe.cjs | 13 --- .../base/snapshot/.claude/settings.json | 9 -- .../snapshot/.workflows/.state/migrations | 52 ----------- .../snapshot/.workflows/_gitignore.fixture | 2 - .../prose/fixtures/bugfix-created/recipe.cjs | 11 --- .../snapshot/.claude/settings.json | 9 -- .../snapshot/.workflows/.state/migrations | 52 ----------- .../snapshot/.workflows/_gitignore.fixture | 2 - .../discovery/sessions/session-001.md | 40 --------- .../.workflows/crash-fix/manifest.json | 8 -- .../snapshot/.workflows/manifest.json | 7 -- .../fixtures/bugfix-investigated/recipe.cjs | 11 --- .../snapshot/.claude/settings.json | 9 -- .../snapshot/.workflows/.state/migrations | 52 ----------- .../snapshot/.workflows/_gitignore.fixture | 2 - .../discovery/sessions/session-001.md | 40 --------- .../crash-fix/investigation/crash-fix.md | 90 ------------------- .../.workflows/crash-fix/manifest.json | 16 ---- .../snapshot/.workflows/manifest.json | 7 -- .../fixtures/bugfix-investigating/recipe.cjs | 11 --- .../snapshot/.claude/settings.json | 9 -- .../snapshot/.workflows/.state/migrations | 52 ----------- .../snapshot/.workflows/_gitignore.fixture | 2 - .../discovery/sessions/session-001.md | 40 --------- .../crash-fix/investigation/crash-fix.md | 88 ------------------ .../.workflows/crash-fix/manifest.json | 16 ---- .../snapshot/.workflows/manifest.json | 7 -- .../bugfix-validation-incorporated/recipe.cjs | 27 ------ .../snapshot/.claude/settings.json | 9 -- .../crash-fix/root-cause-validation-001.md | 10 --- .../investigation/crash-fix/state.json | 16 ---- .../snapshot/.workflows/.state/migrations | 52 ----------- .../snapshot/.workflows/_gitignore.fixture | 2 - .../discovery/sessions/session-001.md | 40 --------- .../crash-fix/investigation/crash-fix.md | 88 ------------------ .../.workflows/crash-fix/manifest.json | 16 ---- .../snapshot/.workflows/manifest.json | 7 -- .../prose/fixtures/feature-created/recipe.cjs | 11 --- .../snapshot/.claude/settings.json | 9 -- .../snapshot/.workflows/.state/migrations | 52 ----------- .../snapshot/.workflows/_gitignore.fixture | 2 - .../snapshot/.workflows/manifest.json | 7 -- .../pay/discovery/sessions/session-001.md | 39 -------- .../snapshot/.workflows/pay/manifest.json | 8 -- .../fixtures/feature-discussed/recipe.cjs | 11 --- .../snapshot/.claude/settings.json | 9 -- .../snapshot/.workflows/.state/migrations | 52 ----------- .../snapshot/.workflows/_gitignore.fixture | 2 - .../snapshot/.workflows/manifest.json | 7 -- .../pay/discovery/sessions/session-001.md | 39 -------- .../snapshot/.workflows/pay/discussion/pay.md | 15 ---- .../snapshot/.workflows/pay/manifest.json | 16 ---- .../feature-implementation-started/recipe.cjs | 27 ------ .../snapshot/.claude/settings.json | 9 -- .../.workflows/.state/environment-setup.md | 1 - .../snapshot/.workflows/.state/migrations | 52 ----------- .../snapshot/.workflows/_gitignore.fixture | 2 - .../snapshot/.workflows/manifest.json | 7 -- .../pay/discovery/sessions/session-001.md | 39 -------- .../snapshot/.workflows/pay/discussion/pay.md | 15 ---- .../snapshot/.workflows/pay/manifest.json | 70 --------------- .../.workflows/pay/planning/pay/planning.md | 8 -- .../pay/planning/pay/tasks/pay-1-1.md | 10 --- .../pay/planning/pay/tasks/pay-1-2.md | 10 --- .../pay/specification/pay/specification.md | 11 --- .../fixtures/feature-implemented/recipe.cjs | 11 --- .../snapshot/.claude/settings.json | 9 -- .../snapshot/.workflows/.state/migrations | 52 ----------- .../snapshot/.workflows/_gitignore.fixture | 2 - .../snapshot/.workflows/manifest.json | 7 -- .../pay/discovery/sessions/session-001.md | 39 -------- .../snapshot/.workflows/pay/discussion/pay.md | 15 ---- .../snapshot/.workflows/pay/manifest.json | 77 ---------------- .../.workflows/pay/planning/pay/planning.md | 8 -- .../pay/planning/pay/tasks/pay-1-1.md | 10 --- .../pay/planning/pay/tasks/pay-1-2.md | 10 --- .../pay/specification/pay/specification.md | 11 --- .../prose/fixtures/feature-planned/recipe.cjs | 11 --- .../snapshot/.claude/settings.json | 9 -- .../snapshot/.workflows/.state/migrations | 52 ----------- .../snapshot/.workflows/_gitignore.fixture | 2 - .../snapshot/.workflows/manifest.json | 7 -- .../pay/discovery/sessions/session-001.md | 39 -------- .../snapshot/.workflows/pay/discussion/pay.md | 15 ---- .../snapshot/.workflows/pay/manifest.json | 53 ----------- .../.workflows/pay/planning/pay/planning.md | 8 -- .../pay/planning/pay/tasks/pay-1-1.md | 10 --- .../pay/planning/pay/tasks/pay-1-2.md | 10 --- .../pay/specification/pay/specification.md | 11 --- .../fixtures/feature-specified/recipe.cjs | 11 --- .../snapshot/.claude/settings.json | 9 -- .../snapshot/.workflows/.state/migrations | 52 ----------- .../snapshot/.workflows/_gitignore.fixture | 2 - .../snapshot/.workflows/manifest.json | 7 -- .../pay/discovery/sessions/session-001.md | 39 -------- .../snapshot/.workflows/pay/discussion/pay.md | 15 ---- .../snapshot/.workflows/pay/manifest.json | 28 ------ .../pay/specification/pay/specification.md | 11 --- 98 files changed, 2116 deletions(-) delete mode 100644 tests/prose/fixtures/base/recipe.cjs delete mode 100644 tests/prose/fixtures/base/snapshot/.claude/settings.json delete mode 100644 tests/prose/fixtures/base/snapshot/.workflows/.state/migrations delete mode 100644 tests/prose/fixtures/base/snapshot/.workflows/_gitignore.fixture delete mode 100644 tests/prose/fixtures/bugfix-created/recipe.cjs delete mode 100644 tests/prose/fixtures/bugfix-created/snapshot/.claude/settings.json delete mode 100644 tests/prose/fixtures/bugfix-created/snapshot/.workflows/.state/migrations delete mode 100644 tests/prose/fixtures/bugfix-created/snapshot/.workflows/_gitignore.fixture delete mode 100644 tests/prose/fixtures/bugfix-created/snapshot/.workflows/crash-fix/discovery/sessions/session-001.md delete mode 100644 tests/prose/fixtures/bugfix-created/snapshot/.workflows/crash-fix/manifest.json delete mode 100644 tests/prose/fixtures/bugfix-created/snapshot/.workflows/manifest.json delete mode 100644 tests/prose/fixtures/bugfix-investigated/recipe.cjs delete mode 100644 tests/prose/fixtures/bugfix-investigated/snapshot/.claude/settings.json delete mode 100644 tests/prose/fixtures/bugfix-investigated/snapshot/.workflows/.state/migrations delete mode 100644 tests/prose/fixtures/bugfix-investigated/snapshot/.workflows/_gitignore.fixture delete mode 100644 tests/prose/fixtures/bugfix-investigated/snapshot/.workflows/crash-fix/discovery/sessions/session-001.md delete mode 100644 tests/prose/fixtures/bugfix-investigated/snapshot/.workflows/crash-fix/investigation/crash-fix.md delete mode 100644 tests/prose/fixtures/bugfix-investigated/snapshot/.workflows/crash-fix/manifest.json delete mode 100644 tests/prose/fixtures/bugfix-investigated/snapshot/.workflows/manifest.json delete mode 100644 tests/prose/fixtures/bugfix-investigating/recipe.cjs delete mode 100644 tests/prose/fixtures/bugfix-investigating/snapshot/.claude/settings.json delete mode 100644 tests/prose/fixtures/bugfix-investigating/snapshot/.workflows/.state/migrations delete mode 100644 tests/prose/fixtures/bugfix-investigating/snapshot/.workflows/_gitignore.fixture delete mode 100644 tests/prose/fixtures/bugfix-investigating/snapshot/.workflows/crash-fix/discovery/sessions/session-001.md delete mode 100644 tests/prose/fixtures/bugfix-investigating/snapshot/.workflows/crash-fix/investigation/crash-fix.md delete mode 100644 tests/prose/fixtures/bugfix-investigating/snapshot/.workflows/crash-fix/manifest.json delete mode 100644 tests/prose/fixtures/bugfix-investigating/snapshot/.workflows/manifest.json delete mode 100644 tests/prose/fixtures/bugfix-validation-incorporated/recipe.cjs delete mode 100644 tests/prose/fixtures/bugfix-validation-incorporated/snapshot/.claude/settings.json delete mode 100644 tests/prose/fixtures/bugfix-validation-incorporated/snapshot/.workflows/.cache/crash-fix/investigation/crash-fix/root-cause-validation-001.md delete mode 100644 tests/prose/fixtures/bugfix-validation-incorporated/snapshot/.workflows/.cache/crash-fix/investigation/crash-fix/state.json delete mode 100644 tests/prose/fixtures/bugfix-validation-incorporated/snapshot/.workflows/.state/migrations delete mode 100644 tests/prose/fixtures/bugfix-validation-incorporated/snapshot/.workflows/_gitignore.fixture delete mode 100644 tests/prose/fixtures/bugfix-validation-incorporated/snapshot/.workflows/crash-fix/discovery/sessions/session-001.md delete mode 100644 tests/prose/fixtures/bugfix-validation-incorporated/snapshot/.workflows/crash-fix/investigation/crash-fix.md delete mode 100644 tests/prose/fixtures/bugfix-validation-incorporated/snapshot/.workflows/crash-fix/manifest.json delete mode 100644 tests/prose/fixtures/bugfix-validation-incorporated/snapshot/.workflows/manifest.json delete mode 100644 tests/prose/fixtures/feature-created/recipe.cjs delete mode 100644 tests/prose/fixtures/feature-created/snapshot/.claude/settings.json delete mode 100644 tests/prose/fixtures/feature-created/snapshot/.workflows/.state/migrations delete mode 100644 tests/prose/fixtures/feature-created/snapshot/.workflows/_gitignore.fixture delete mode 100644 tests/prose/fixtures/feature-created/snapshot/.workflows/manifest.json delete mode 100644 tests/prose/fixtures/feature-created/snapshot/.workflows/pay/discovery/sessions/session-001.md delete mode 100644 tests/prose/fixtures/feature-created/snapshot/.workflows/pay/manifest.json delete mode 100644 tests/prose/fixtures/feature-discussed/recipe.cjs delete mode 100644 tests/prose/fixtures/feature-discussed/snapshot/.claude/settings.json delete mode 100644 tests/prose/fixtures/feature-discussed/snapshot/.workflows/.state/migrations delete mode 100644 tests/prose/fixtures/feature-discussed/snapshot/.workflows/_gitignore.fixture delete mode 100644 tests/prose/fixtures/feature-discussed/snapshot/.workflows/manifest.json delete mode 100644 tests/prose/fixtures/feature-discussed/snapshot/.workflows/pay/discovery/sessions/session-001.md delete mode 100644 tests/prose/fixtures/feature-discussed/snapshot/.workflows/pay/discussion/pay.md delete mode 100644 tests/prose/fixtures/feature-discussed/snapshot/.workflows/pay/manifest.json delete mode 100644 tests/prose/fixtures/feature-implementation-started/recipe.cjs delete mode 100644 tests/prose/fixtures/feature-implementation-started/snapshot/.claude/settings.json delete mode 100644 tests/prose/fixtures/feature-implementation-started/snapshot/.workflows/.state/environment-setup.md delete mode 100644 tests/prose/fixtures/feature-implementation-started/snapshot/.workflows/.state/migrations delete mode 100644 tests/prose/fixtures/feature-implementation-started/snapshot/.workflows/_gitignore.fixture delete mode 100644 tests/prose/fixtures/feature-implementation-started/snapshot/.workflows/manifest.json delete mode 100644 tests/prose/fixtures/feature-implementation-started/snapshot/.workflows/pay/discovery/sessions/session-001.md delete mode 100644 tests/prose/fixtures/feature-implementation-started/snapshot/.workflows/pay/discussion/pay.md delete mode 100644 tests/prose/fixtures/feature-implementation-started/snapshot/.workflows/pay/manifest.json delete mode 100644 tests/prose/fixtures/feature-implementation-started/snapshot/.workflows/pay/planning/pay/planning.md delete mode 100644 tests/prose/fixtures/feature-implementation-started/snapshot/.workflows/pay/planning/pay/tasks/pay-1-1.md delete mode 100644 tests/prose/fixtures/feature-implementation-started/snapshot/.workflows/pay/planning/pay/tasks/pay-1-2.md delete mode 100644 tests/prose/fixtures/feature-implementation-started/snapshot/.workflows/pay/specification/pay/specification.md delete mode 100644 tests/prose/fixtures/feature-implemented/recipe.cjs delete mode 100644 tests/prose/fixtures/feature-implemented/snapshot/.claude/settings.json delete mode 100644 tests/prose/fixtures/feature-implemented/snapshot/.workflows/.state/migrations delete mode 100644 tests/prose/fixtures/feature-implemented/snapshot/.workflows/_gitignore.fixture delete mode 100644 tests/prose/fixtures/feature-implemented/snapshot/.workflows/manifest.json delete mode 100644 tests/prose/fixtures/feature-implemented/snapshot/.workflows/pay/discovery/sessions/session-001.md delete mode 100644 tests/prose/fixtures/feature-implemented/snapshot/.workflows/pay/discussion/pay.md delete mode 100644 tests/prose/fixtures/feature-implemented/snapshot/.workflows/pay/manifest.json delete mode 100644 tests/prose/fixtures/feature-implemented/snapshot/.workflows/pay/planning/pay/planning.md delete mode 100644 tests/prose/fixtures/feature-implemented/snapshot/.workflows/pay/planning/pay/tasks/pay-1-1.md delete mode 100644 tests/prose/fixtures/feature-implemented/snapshot/.workflows/pay/planning/pay/tasks/pay-1-2.md delete mode 100644 tests/prose/fixtures/feature-implemented/snapshot/.workflows/pay/specification/pay/specification.md delete mode 100644 tests/prose/fixtures/feature-planned/recipe.cjs delete mode 100644 tests/prose/fixtures/feature-planned/snapshot/.claude/settings.json delete mode 100644 tests/prose/fixtures/feature-planned/snapshot/.workflows/.state/migrations delete mode 100644 tests/prose/fixtures/feature-planned/snapshot/.workflows/_gitignore.fixture delete mode 100644 tests/prose/fixtures/feature-planned/snapshot/.workflows/manifest.json delete mode 100644 tests/prose/fixtures/feature-planned/snapshot/.workflows/pay/discovery/sessions/session-001.md delete mode 100644 tests/prose/fixtures/feature-planned/snapshot/.workflows/pay/discussion/pay.md delete mode 100644 tests/prose/fixtures/feature-planned/snapshot/.workflows/pay/manifest.json delete mode 100644 tests/prose/fixtures/feature-planned/snapshot/.workflows/pay/planning/pay/planning.md delete mode 100644 tests/prose/fixtures/feature-planned/snapshot/.workflows/pay/planning/pay/tasks/pay-1-1.md delete mode 100644 tests/prose/fixtures/feature-planned/snapshot/.workflows/pay/planning/pay/tasks/pay-1-2.md delete mode 100644 tests/prose/fixtures/feature-planned/snapshot/.workflows/pay/specification/pay/specification.md delete mode 100644 tests/prose/fixtures/feature-specified/recipe.cjs delete mode 100644 tests/prose/fixtures/feature-specified/snapshot/.claude/settings.json delete mode 100644 tests/prose/fixtures/feature-specified/snapshot/.workflows/.state/migrations delete mode 100644 tests/prose/fixtures/feature-specified/snapshot/.workflows/_gitignore.fixture delete mode 100644 tests/prose/fixtures/feature-specified/snapshot/.workflows/manifest.json delete mode 100644 tests/prose/fixtures/feature-specified/snapshot/.workflows/pay/discovery/sessions/session-001.md delete mode 100644 tests/prose/fixtures/feature-specified/snapshot/.workflows/pay/discussion/pay.md delete mode 100644 tests/prose/fixtures/feature-specified/snapshot/.workflows/pay/manifest.json delete mode 100644 tests/prose/fixtures/feature-specified/snapshot/.workflows/pay/specification/pay/specification.md diff --git a/tests/prose/fixtures/base/recipe.cjs b/tests/prose/fixtures/base/recipe.cjs deleted file mode 100644 index 06ac01e21..000000000 --- a/tests/prose/fixtures/base/recipe.cjs +++ /dev/null @@ -1,13 +0,0 @@ -'use strict'; - -// The canonical empty project — a real install straight after -// /workflow-start Step 0: migrations applied (the log is the durable -// trace), knowledge set up keyword-only, boot run. No work units yet. -// Every richer fixture starts from this world's shape. - -module.exports = { - build(h) { - h.knowledge('setup', '--keyword-only'); - h.engine('boot'); - }, -}; diff --git a/tests/prose/fixtures/base/snapshot/.claude/settings.json b/tests/prose/fixtures/base/snapshot/.claude/settings.json deleted file mode 100644 index ae019ede5..000000000 --- a/tests/prose/fixtures/base/snapshot/.claude/settings.json +++ /dev/null @@ -1,9 +0,0 @@ -{ - "showClearContextOnPlanAccept": true, - "permissions": { - "allow": [ - "Edit(.workflows/**)", - "Bash(mv .workflows/:*)" - ] - } -} diff --git a/tests/prose/fixtures/base/snapshot/.workflows/.state/migrations b/tests/prose/fixtures/base/snapshot/.workflows/.state/migrations deleted file mode 100644 index 3c7162acc..000000000 --- a/tests/prose/fixtures/base/snapshot/.workflows/.state/migrations +++ /dev/null @@ -1,52 +0,0 @@ -001 -002 -003 -004 -005 -006 -007 -008 -009 -010 -011 -012 -013 -014 -015 -016 -017 -018 -019 -020 -021 -022 -023 -024 -025 -026 -027 -028 -029 -030 -031 -032 -033 -034 -035 -036 -037 -038 -039 -040 -041 -042 -043 -044 -045 -046 -047 -048 -049 -050 -051 -052 diff --git a/tests/prose/fixtures/base/snapshot/.workflows/_gitignore.fixture b/tests/prose/fixtures/base/snapshot/.workflows/_gitignore.fixture deleted file mode 100644 index 779cbf28a..000000000 --- a/tests/prose/fixtures/base/snapshot/.workflows/_gitignore.fixture +++ /dev/null @@ -1,2 +0,0 @@ -.cache/ -.manifest.json.*.tmp diff --git a/tests/prose/fixtures/bugfix-created/recipe.cjs b/tests/prose/fixtures/bugfix-created/recipe.cjs deleted file mode 100644 index 012ec0df2..000000000 --- a/tests/prose/fixtures/bugfix-created/recipe.cjs +++ /dev/null @@ -1,11 +0,0 @@ -'use strict'; - -// Bugfix mainline stop point — see _shared/bugfix-mainline.cjs. - -const m = require('../_shared/bugfix-mainline.cjs'); - -module.exports = { - build(h) { - m.init(h); m.create(h); - }, -}; diff --git a/tests/prose/fixtures/bugfix-created/snapshot/.claude/settings.json b/tests/prose/fixtures/bugfix-created/snapshot/.claude/settings.json deleted file mode 100644 index ae019ede5..000000000 --- a/tests/prose/fixtures/bugfix-created/snapshot/.claude/settings.json +++ /dev/null @@ -1,9 +0,0 @@ -{ - "showClearContextOnPlanAccept": true, - "permissions": { - "allow": [ - "Edit(.workflows/**)", - "Bash(mv .workflows/:*)" - ] - } -} diff --git a/tests/prose/fixtures/bugfix-created/snapshot/.workflows/.state/migrations b/tests/prose/fixtures/bugfix-created/snapshot/.workflows/.state/migrations deleted file mode 100644 index 3c7162acc..000000000 --- a/tests/prose/fixtures/bugfix-created/snapshot/.workflows/.state/migrations +++ /dev/null @@ -1,52 +0,0 @@ -001 -002 -003 -004 -005 -006 -007 -008 -009 -010 -011 -012 -013 -014 -015 -016 -017 -018 -019 -020 -021 -022 -023 -024 -025 -026 -027 -028 -029 -030 -031 -032 -033 -034 -035 -036 -037 -038 -039 -040 -041 -042 -043 -044 -045 -046 -047 -048 -049 -050 -051 -052 diff --git a/tests/prose/fixtures/bugfix-created/snapshot/.workflows/_gitignore.fixture b/tests/prose/fixtures/bugfix-created/snapshot/.workflows/_gitignore.fixture deleted file mode 100644 index 779cbf28a..000000000 --- a/tests/prose/fixtures/bugfix-created/snapshot/.workflows/_gitignore.fixture +++ /dev/null @@ -1,2 +0,0 @@ -.cache/ -.manifest.json.*.tmp diff --git a/tests/prose/fixtures/bugfix-created/snapshot/.workflows/crash-fix/discovery/sessions/session-001.md b/tests/prose/fixtures/bugfix-created/snapshot/.workflows/crash-fix/discovery/sessions/session-001.md deleted file mode 100644 index 6dc56d4e3..000000000 --- a/tests/prose/fixtures/bugfix-created/snapshot/.workflows/crash-fix/discovery/sessions/session-001.md +++ /dev/null @@ -1,40 +0,0 @@ -# Discovery Session 001 - -Date: 2026-01-01 -Work unit: crash-fix - -## Description (as of session) - -Checkout crashes when an order has no shipping address. - -## Seed - -(none) - -## Imports - -(none) - -## Map State at Start - -(n/a — single-topic work) - -## Exploration - -Reported by two users this week: the checkout page 500s at the -payment step for orders with no shipping address. Reproducible on -staging with a digital-only basket. No error surfaces to the user — -the page just fails. Confirmed as a bugfix; no design question here, -so it routes straight to investigation. - -## Edits - -(none) - -## Topics Identified - -(none) - -## Conclusion - -(none) diff --git a/tests/prose/fixtures/bugfix-created/snapshot/.workflows/crash-fix/manifest.json b/tests/prose/fixtures/bugfix-created/snapshot/.workflows/crash-fix/manifest.json deleted file mode 100644 index eaaa56199..000000000 --- a/tests/prose/fixtures/bugfix-created/snapshot/.workflows/crash-fix/manifest.json +++ /dev/null @@ -1,8 +0,0 @@ -{ - "name": "crash-fix", - "work_type": "bugfix", - "status": "in-progress", - "created": "2026-01-01", - "description": "Checkout crashes when an order has no shipping address", - "phases": {} -} diff --git a/tests/prose/fixtures/bugfix-created/snapshot/.workflows/manifest.json b/tests/prose/fixtures/bugfix-created/snapshot/.workflows/manifest.json deleted file mode 100644 index 1cd498ed5..000000000 --- a/tests/prose/fixtures/bugfix-created/snapshot/.workflows/manifest.json +++ /dev/null @@ -1,7 +0,0 @@ -{ - "work_units": { - "crash-fix": { - "work_type": "bugfix" - } - } -} diff --git a/tests/prose/fixtures/bugfix-investigated/recipe.cjs b/tests/prose/fixtures/bugfix-investigated/recipe.cjs deleted file mode 100644 index 85cf787f4..000000000 --- a/tests/prose/fixtures/bugfix-investigated/recipe.cjs +++ /dev/null @@ -1,11 +0,0 @@ -'use strict'; - -// Bugfix mainline stop point — see _shared/bugfix-mainline.cjs. - -const m = require('../_shared/bugfix-mainline.cjs'); - -module.exports = { - build(h) { - m.init(h); m.create(h); m.investigateToRootCause(h); m.concludeInvestigation(h); - }, -}; diff --git a/tests/prose/fixtures/bugfix-investigated/snapshot/.claude/settings.json b/tests/prose/fixtures/bugfix-investigated/snapshot/.claude/settings.json deleted file mode 100644 index ae019ede5..000000000 --- a/tests/prose/fixtures/bugfix-investigated/snapshot/.claude/settings.json +++ /dev/null @@ -1,9 +0,0 @@ -{ - "showClearContextOnPlanAccept": true, - "permissions": { - "allow": [ - "Edit(.workflows/**)", - "Bash(mv .workflows/:*)" - ] - } -} diff --git a/tests/prose/fixtures/bugfix-investigated/snapshot/.workflows/.state/migrations b/tests/prose/fixtures/bugfix-investigated/snapshot/.workflows/.state/migrations deleted file mode 100644 index 3c7162acc..000000000 --- a/tests/prose/fixtures/bugfix-investigated/snapshot/.workflows/.state/migrations +++ /dev/null @@ -1,52 +0,0 @@ -001 -002 -003 -004 -005 -006 -007 -008 -009 -010 -011 -012 -013 -014 -015 -016 -017 -018 -019 -020 -021 -022 -023 -024 -025 -026 -027 -028 -029 -030 -031 -032 -033 -034 -035 -036 -037 -038 -039 -040 -041 -042 -043 -044 -045 -046 -047 -048 -049 -050 -051 -052 diff --git a/tests/prose/fixtures/bugfix-investigated/snapshot/.workflows/_gitignore.fixture b/tests/prose/fixtures/bugfix-investigated/snapshot/.workflows/_gitignore.fixture deleted file mode 100644 index 779cbf28a..000000000 --- a/tests/prose/fixtures/bugfix-investigated/snapshot/.workflows/_gitignore.fixture +++ /dev/null @@ -1,2 +0,0 @@ -.cache/ -.manifest.json.*.tmp diff --git a/tests/prose/fixtures/bugfix-investigated/snapshot/.workflows/crash-fix/discovery/sessions/session-001.md b/tests/prose/fixtures/bugfix-investigated/snapshot/.workflows/crash-fix/discovery/sessions/session-001.md deleted file mode 100644 index 6dc56d4e3..000000000 --- a/tests/prose/fixtures/bugfix-investigated/snapshot/.workflows/crash-fix/discovery/sessions/session-001.md +++ /dev/null @@ -1,40 +0,0 @@ -# Discovery Session 001 - -Date: 2026-01-01 -Work unit: crash-fix - -## Description (as of session) - -Checkout crashes when an order has no shipping address. - -## Seed - -(none) - -## Imports - -(none) - -## Map State at Start - -(n/a — single-topic work) - -## Exploration - -Reported by two users this week: the checkout page 500s at the -payment step for orders with no shipping address. Reproducible on -staging with a digital-only basket. No error surfaces to the user — -the page just fails. Confirmed as a bugfix; no design question here, -so it routes straight to investigation. - -## Edits - -(none) - -## Topics Identified - -(none) - -## Conclusion - -(none) diff --git a/tests/prose/fixtures/bugfix-investigated/snapshot/.workflows/crash-fix/investigation/crash-fix.md b/tests/prose/fixtures/bugfix-investigated/snapshot/.workflows/crash-fix/investigation/crash-fix.md deleted file mode 100644 index d09b4daba..000000000 --- a/tests/prose/fixtures/bugfix-investigated/snapshot/.workflows/crash-fix/investigation/crash-fix.md +++ /dev/null @@ -1,90 +0,0 @@ -# Investigation: Checkout Crash On Missing Shipping Address - -## Symptoms - -### Problem Description - -Checkout returns a 500 at the payment step when the order has no -shipping address. No error is surfaced to the user. - -### Manifestation - -Server error on POST to the payment step; the page fails blank. - -### Reproduction Steps - -1. Add a digital-only product to the basket. -2. Proceed to checkout without entering a shipping address. -3. Continue to the payment step — the request 500s. - -### Environment - -Reproduced on staging; reported twice in production this week. - -### Impact - -Digital-only orders cannot complete checkout. - -### References - -(none) - -## Analysis - -### Hypotheses - -- The payment step assumes a shipping address is always present. -- Tax calculation may be the first consumer of the missing address. - -### Code Trace - -The payment step builds a tax context from the order before creating -the payment intent. The tax context reads the shipping address -unconditionally; for a digital-only order the address is absent, so -the read fails and the request aborts before any handler catches it. - -### Root Cause - -The tax context treats the shipping address as mandatory. Digital-only -orders legitimately have none, so the assumption is wrong rather than -the data being wrong. - -### Contributing Factors - -Digital-only baskets were introduced after the tax context was written. - -### Why It Wasn't Caught - -No test covers a basket with no shippable line items. - -### Blast Radius - -Any flow building a tax context from an address-less order. - -## Fix Direction - -### Chosen Approach - -Make the tax context treat the shipping address as optional: when -an order has no shippable items, build the context from the billing -address instead. - -### Options Explored - -(none yet) - -### Discussion - -(none yet) - -### Testing Recommendations - -Cover a digital-only basket end to end through the payment step. - -### Risk Assessment - -Low — the change is confined to tax-context construction. - -## Notes - -(none) diff --git a/tests/prose/fixtures/bugfix-investigated/snapshot/.workflows/crash-fix/manifest.json b/tests/prose/fixtures/bugfix-investigated/snapshot/.workflows/crash-fix/manifest.json deleted file mode 100644 index 4c5ece2e7..000000000 --- a/tests/prose/fixtures/bugfix-investigated/snapshot/.workflows/crash-fix/manifest.json +++ /dev/null @@ -1,16 +0,0 @@ -{ - "name": "crash-fix", - "work_type": "bugfix", - "status": "in-progress", - "created": "2026-01-01", - "description": "Checkout crashes when an order has no shipping address", - "phases": { - "investigation": { - "items": { - "crash-fix": { - "status": "completed" - } - } - } - } -} diff --git a/tests/prose/fixtures/bugfix-investigated/snapshot/.workflows/manifest.json b/tests/prose/fixtures/bugfix-investigated/snapshot/.workflows/manifest.json deleted file mode 100644 index 1cd498ed5..000000000 --- a/tests/prose/fixtures/bugfix-investigated/snapshot/.workflows/manifest.json +++ /dev/null @@ -1,7 +0,0 @@ -{ - "work_units": { - "crash-fix": { - "work_type": "bugfix" - } - } -} diff --git a/tests/prose/fixtures/bugfix-investigating/recipe.cjs b/tests/prose/fixtures/bugfix-investigating/recipe.cjs deleted file mode 100644 index a0e1bea2c..000000000 --- a/tests/prose/fixtures/bugfix-investigating/recipe.cjs +++ /dev/null @@ -1,11 +0,0 @@ -'use strict'; - -// Bugfix mainline stop point — see _shared/bugfix-mainline.cjs. - -const m = require('../_shared/bugfix-mainline.cjs'); - -module.exports = { - build(h) { - m.init(h); m.create(h); m.investigateToRootCause(h); - }, -}; diff --git a/tests/prose/fixtures/bugfix-investigating/snapshot/.claude/settings.json b/tests/prose/fixtures/bugfix-investigating/snapshot/.claude/settings.json deleted file mode 100644 index ae019ede5..000000000 --- a/tests/prose/fixtures/bugfix-investigating/snapshot/.claude/settings.json +++ /dev/null @@ -1,9 +0,0 @@ -{ - "showClearContextOnPlanAccept": true, - "permissions": { - "allow": [ - "Edit(.workflows/**)", - "Bash(mv .workflows/:*)" - ] - } -} diff --git a/tests/prose/fixtures/bugfix-investigating/snapshot/.workflows/.state/migrations b/tests/prose/fixtures/bugfix-investigating/snapshot/.workflows/.state/migrations deleted file mode 100644 index 3c7162acc..000000000 --- a/tests/prose/fixtures/bugfix-investigating/snapshot/.workflows/.state/migrations +++ /dev/null @@ -1,52 +0,0 @@ -001 -002 -003 -004 -005 -006 -007 -008 -009 -010 -011 -012 -013 -014 -015 -016 -017 -018 -019 -020 -021 -022 -023 -024 -025 -026 -027 -028 -029 -030 -031 -032 -033 -034 -035 -036 -037 -038 -039 -040 -041 -042 -043 -044 -045 -046 -047 -048 -049 -050 -051 -052 diff --git a/tests/prose/fixtures/bugfix-investigating/snapshot/.workflows/_gitignore.fixture b/tests/prose/fixtures/bugfix-investigating/snapshot/.workflows/_gitignore.fixture deleted file mode 100644 index 779cbf28a..000000000 --- a/tests/prose/fixtures/bugfix-investigating/snapshot/.workflows/_gitignore.fixture +++ /dev/null @@ -1,2 +0,0 @@ -.cache/ -.manifest.json.*.tmp diff --git a/tests/prose/fixtures/bugfix-investigating/snapshot/.workflows/crash-fix/discovery/sessions/session-001.md b/tests/prose/fixtures/bugfix-investigating/snapshot/.workflows/crash-fix/discovery/sessions/session-001.md deleted file mode 100644 index 6dc56d4e3..000000000 --- a/tests/prose/fixtures/bugfix-investigating/snapshot/.workflows/crash-fix/discovery/sessions/session-001.md +++ /dev/null @@ -1,40 +0,0 @@ -# Discovery Session 001 - -Date: 2026-01-01 -Work unit: crash-fix - -## Description (as of session) - -Checkout crashes when an order has no shipping address. - -## Seed - -(none) - -## Imports - -(none) - -## Map State at Start - -(n/a — single-topic work) - -## Exploration - -Reported by two users this week: the checkout page 500s at the -payment step for orders with no shipping address. Reproducible on -staging with a digital-only basket. No error surfaces to the user — -the page just fails. Confirmed as a bugfix; no design question here, -so it routes straight to investigation. - -## Edits - -(none) - -## Topics Identified - -(none) - -## Conclusion - -(none) diff --git a/tests/prose/fixtures/bugfix-investigating/snapshot/.workflows/crash-fix/investigation/crash-fix.md b/tests/prose/fixtures/bugfix-investigating/snapshot/.workflows/crash-fix/investigation/crash-fix.md deleted file mode 100644 index d5fc263a9..000000000 --- a/tests/prose/fixtures/bugfix-investigating/snapshot/.workflows/crash-fix/investigation/crash-fix.md +++ /dev/null @@ -1,88 +0,0 @@ -# Investigation: Checkout Crash On Missing Shipping Address - -## Symptoms - -### Problem Description - -Checkout returns a 500 at the payment step when the order has no -shipping address. No error is surfaced to the user. - -### Manifestation - -Server error on POST to the payment step; the page fails blank. - -### Reproduction Steps - -1. Add a digital-only product to the basket. -2. Proceed to checkout without entering a shipping address. -3. Continue to the payment step — the request 500s. - -### Environment - -Reproduced on staging; reported twice in production this week. - -### Impact - -Digital-only orders cannot complete checkout. - -### References - -(none) - -## Analysis - -### Hypotheses - -- The payment step assumes a shipping address is always present. -- Tax calculation may be the first consumer of the missing address. - -### Code Trace - -The payment step builds a tax context from the order before creating -the payment intent. The tax context reads the shipping address -unconditionally; for a digital-only order the address is absent, so -the read fails and the request aborts before any handler catches it. - -### Root Cause - -The tax context treats the shipping address as mandatory. Digital-only -orders legitimately have none, so the assumption is wrong rather than -the data being wrong. - -### Contributing Factors - -Digital-only baskets were introduced after the tax context was written. - -### Why It Wasn't Caught - -No test covers a basket with no shippable line items. - -### Blast Radius - -Any flow building a tax context from an address-less order. - -## Fix Direction - -### Chosen Approach - -(pending) - -### Options Explored - -(none yet) - -### Discussion - -(none yet) - -### Testing Recommendations - -(pending) - -### Risk Assessment - -(pending) - -## Notes - -(none) diff --git a/tests/prose/fixtures/bugfix-investigating/snapshot/.workflows/crash-fix/manifest.json b/tests/prose/fixtures/bugfix-investigating/snapshot/.workflows/crash-fix/manifest.json deleted file mode 100644 index 88bf183ff..000000000 --- a/tests/prose/fixtures/bugfix-investigating/snapshot/.workflows/crash-fix/manifest.json +++ /dev/null @@ -1,16 +0,0 @@ -{ - "name": "crash-fix", - "work_type": "bugfix", - "status": "in-progress", - "created": "2026-01-01", - "description": "Checkout crashes when an order has no shipping address", - "phases": { - "investigation": { - "items": { - "crash-fix": { - "status": "in-progress" - } - } - } - } -} diff --git a/tests/prose/fixtures/bugfix-investigating/snapshot/.workflows/manifest.json b/tests/prose/fixtures/bugfix-investigating/snapshot/.workflows/manifest.json deleted file mode 100644 index 1cd498ed5..000000000 --- a/tests/prose/fixtures/bugfix-investigating/snapshot/.workflows/manifest.json +++ /dev/null @@ -1,7 +0,0 @@ -{ - "work_units": { - "crash-fix": { - "work_type": "bugfix" - } - } -} diff --git a/tests/prose/fixtures/bugfix-validation-incorporated/recipe.cjs b/tests/prose/fixtures/bugfix-validation-incorporated/recipe.cjs deleted file mode 100644 index 8e8a08ed8..000000000 --- a/tests/prose/fixtures/bugfix-validation-incorporated/recipe.cjs +++ /dev/null @@ -1,27 +0,0 @@ -'use strict'; - -// Expected end state for the root-cause validation walk: the row -// dispatched, the report landed, scan promoted it, incorporate closed it. -// The report content is read from the stub the case arms, so the bytes -// the walk writes and the bytes expected here cannot drift. - -const fs = require('fs'); -const path = require('path'); - -const m = require('../_shared/bugfix-mainline.cjs'); -const { readStub } = require('../../lib/cases.cjs'); - -module.exports = { - build(h) { - m.init(h); - m.create(h); - m.investigateToRootCause(h); - - const dispatch = JSON.parse(h.engine( - 'agent', 'dispatch', m.WU, 'investigation', m.WU, '--kind', 'root-cause-validation')); - fs.mkdirSync(path.dirname(path.join(h.dir, dispatch.file)), { recursive: true }); - h.write(dispatch.file, readStub('root-cause-validated').content); - h.engine('agent', 'scan', m.WU, 'investigation', m.WU); - h.engine('agent', 'incorporate', m.WU, 'investigation', m.WU, dispatch.id); - }, -}; diff --git a/tests/prose/fixtures/bugfix-validation-incorporated/snapshot/.claude/settings.json b/tests/prose/fixtures/bugfix-validation-incorporated/snapshot/.claude/settings.json deleted file mode 100644 index ae019ede5..000000000 --- a/tests/prose/fixtures/bugfix-validation-incorporated/snapshot/.claude/settings.json +++ /dev/null @@ -1,9 +0,0 @@ -{ - "showClearContextOnPlanAccept": true, - "permissions": { - "allow": [ - "Edit(.workflows/**)", - "Bash(mv .workflows/:*)" - ] - } -} diff --git a/tests/prose/fixtures/bugfix-validation-incorporated/snapshot/.workflows/.cache/crash-fix/investigation/crash-fix/root-cause-validation-001.md b/tests/prose/fixtures/bugfix-validation-incorporated/snapshot/.workflows/.cache/crash-fix/investigation/crash-fix/root-cause-validation-001.md deleted file mode 100644 index ce1c9b4c4..000000000 --- a/tests/prose/fixtures/bugfix-validation-incorporated/snapshot/.workflows/.cache/crash-fix/investigation/crash-fix/root-cause-validation-001.md +++ /dev/null @@ -1,10 +0,0 @@ -# Root Cause Validation - -The tax context is built before the payment intent and reads the shipping -address unconditionally. Traced fresh: an address-less order fails at that -read, which matches the documented root cause. - -STATUS: validated -CONFIDENCE: high -GAPS_COUNT: 0 -SUMMARY: Root cause confirmed by an independent trace. diff --git a/tests/prose/fixtures/bugfix-validation-incorporated/snapshot/.workflows/.cache/crash-fix/investigation/crash-fix/state.json b/tests/prose/fixtures/bugfix-validation-incorporated/snapshot/.workflows/.cache/crash-fix/investigation/crash-fix/state.json deleted file mode 100644 index 7eb5a78f1..000000000 --- a/tests/prose/fixtures/bugfix-validation-incorporated/snapshot/.workflows/.cache/crash-fix/investigation/crash-fix/state.json +++ /dev/null @@ -1,16 +0,0 @@ -{ - "agents": { - "root-cause-validation-001": { - "id": "root-cause-validation-001", - "kind": "root-cause-validation", - "phase": "investigation", - "topic": "crash-fix", - "set": "001", - "status": "incorporated", - "announced": false, - "findings": [], - "surfaced": [], - "created": "2026-01-01T00:00:00.000Z" - } - } -} diff --git a/tests/prose/fixtures/bugfix-validation-incorporated/snapshot/.workflows/.state/migrations b/tests/prose/fixtures/bugfix-validation-incorporated/snapshot/.workflows/.state/migrations deleted file mode 100644 index 3c7162acc..000000000 --- a/tests/prose/fixtures/bugfix-validation-incorporated/snapshot/.workflows/.state/migrations +++ /dev/null @@ -1,52 +0,0 @@ -001 -002 -003 -004 -005 -006 -007 -008 -009 -010 -011 -012 -013 -014 -015 -016 -017 -018 -019 -020 -021 -022 -023 -024 -025 -026 -027 -028 -029 -030 -031 -032 -033 -034 -035 -036 -037 -038 -039 -040 -041 -042 -043 -044 -045 -046 -047 -048 -049 -050 -051 -052 diff --git a/tests/prose/fixtures/bugfix-validation-incorporated/snapshot/.workflows/_gitignore.fixture b/tests/prose/fixtures/bugfix-validation-incorporated/snapshot/.workflows/_gitignore.fixture deleted file mode 100644 index 779cbf28a..000000000 --- a/tests/prose/fixtures/bugfix-validation-incorporated/snapshot/.workflows/_gitignore.fixture +++ /dev/null @@ -1,2 +0,0 @@ -.cache/ -.manifest.json.*.tmp diff --git a/tests/prose/fixtures/bugfix-validation-incorporated/snapshot/.workflows/crash-fix/discovery/sessions/session-001.md b/tests/prose/fixtures/bugfix-validation-incorporated/snapshot/.workflows/crash-fix/discovery/sessions/session-001.md deleted file mode 100644 index 6dc56d4e3..000000000 --- a/tests/prose/fixtures/bugfix-validation-incorporated/snapshot/.workflows/crash-fix/discovery/sessions/session-001.md +++ /dev/null @@ -1,40 +0,0 @@ -# Discovery Session 001 - -Date: 2026-01-01 -Work unit: crash-fix - -## Description (as of session) - -Checkout crashes when an order has no shipping address. - -## Seed - -(none) - -## Imports - -(none) - -## Map State at Start - -(n/a — single-topic work) - -## Exploration - -Reported by two users this week: the checkout page 500s at the -payment step for orders with no shipping address. Reproducible on -staging with a digital-only basket. No error surfaces to the user — -the page just fails. Confirmed as a bugfix; no design question here, -so it routes straight to investigation. - -## Edits - -(none) - -## Topics Identified - -(none) - -## Conclusion - -(none) diff --git a/tests/prose/fixtures/bugfix-validation-incorporated/snapshot/.workflows/crash-fix/investigation/crash-fix.md b/tests/prose/fixtures/bugfix-validation-incorporated/snapshot/.workflows/crash-fix/investigation/crash-fix.md deleted file mode 100644 index d5fc263a9..000000000 --- a/tests/prose/fixtures/bugfix-validation-incorporated/snapshot/.workflows/crash-fix/investigation/crash-fix.md +++ /dev/null @@ -1,88 +0,0 @@ -# Investigation: Checkout Crash On Missing Shipping Address - -## Symptoms - -### Problem Description - -Checkout returns a 500 at the payment step when the order has no -shipping address. No error is surfaced to the user. - -### Manifestation - -Server error on POST to the payment step; the page fails blank. - -### Reproduction Steps - -1. Add a digital-only product to the basket. -2. Proceed to checkout without entering a shipping address. -3. Continue to the payment step — the request 500s. - -### Environment - -Reproduced on staging; reported twice in production this week. - -### Impact - -Digital-only orders cannot complete checkout. - -### References - -(none) - -## Analysis - -### Hypotheses - -- The payment step assumes a shipping address is always present. -- Tax calculation may be the first consumer of the missing address. - -### Code Trace - -The payment step builds a tax context from the order before creating -the payment intent. The tax context reads the shipping address -unconditionally; for a digital-only order the address is absent, so -the read fails and the request aborts before any handler catches it. - -### Root Cause - -The tax context treats the shipping address as mandatory. Digital-only -orders legitimately have none, so the assumption is wrong rather than -the data being wrong. - -### Contributing Factors - -Digital-only baskets were introduced after the tax context was written. - -### Why It Wasn't Caught - -No test covers a basket with no shippable line items. - -### Blast Radius - -Any flow building a tax context from an address-less order. - -## Fix Direction - -### Chosen Approach - -(pending) - -### Options Explored - -(none yet) - -### Discussion - -(none yet) - -### Testing Recommendations - -(pending) - -### Risk Assessment - -(pending) - -## Notes - -(none) diff --git a/tests/prose/fixtures/bugfix-validation-incorporated/snapshot/.workflows/crash-fix/manifest.json b/tests/prose/fixtures/bugfix-validation-incorporated/snapshot/.workflows/crash-fix/manifest.json deleted file mode 100644 index 88bf183ff..000000000 --- a/tests/prose/fixtures/bugfix-validation-incorporated/snapshot/.workflows/crash-fix/manifest.json +++ /dev/null @@ -1,16 +0,0 @@ -{ - "name": "crash-fix", - "work_type": "bugfix", - "status": "in-progress", - "created": "2026-01-01", - "description": "Checkout crashes when an order has no shipping address", - "phases": { - "investigation": { - "items": { - "crash-fix": { - "status": "in-progress" - } - } - } - } -} diff --git a/tests/prose/fixtures/bugfix-validation-incorporated/snapshot/.workflows/manifest.json b/tests/prose/fixtures/bugfix-validation-incorporated/snapshot/.workflows/manifest.json deleted file mode 100644 index 1cd498ed5..000000000 --- a/tests/prose/fixtures/bugfix-validation-incorporated/snapshot/.workflows/manifest.json +++ /dev/null @@ -1,7 +0,0 @@ -{ - "work_units": { - "crash-fix": { - "work_type": "bugfix" - } - } -} diff --git a/tests/prose/fixtures/feature-created/recipe.cjs b/tests/prose/fixtures/feature-created/recipe.cjs deleted file mode 100644 index b6d1d32c3..000000000 --- a/tests/prose/fixtures/feature-created/recipe.cjs +++ /dev/null @@ -1,11 +0,0 @@ -'use strict'; - -// Feature mainline stop point — see _shared/feature-mainline.cjs. - -const m = require('../_shared/feature-mainline.cjs'); - -module.exports = { - build(h) { - m.init(h); m.create(h); - }, -}; diff --git a/tests/prose/fixtures/feature-created/snapshot/.claude/settings.json b/tests/prose/fixtures/feature-created/snapshot/.claude/settings.json deleted file mode 100644 index ae019ede5..000000000 --- a/tests/prose/fixtures/feature-created/snapshot/.claude/settings.json +++ /dev/null @@ -1,9 +0,0 @@ -{ - "showClearContextOnPlanAccept": true, - "permissions": { - "allow": [ - "Edit(.workflows/**)", - "Bash(mv .workflows/:*)" - ] - } -} diff --git a/tests/prose/fixtures/feature-created/snapshot/.workflows/.state/migrations b/tests/prose/fixtures/feature-created/snapshot/.workflows/.state/migrations deleted file mode 100644 index 3c7162acc..000000000 --- a/tests/prose/fixtures/feature-created/snapshot/.workflows/.state/migrations +++ /dev/null @@ -1,52 +0,0 @@ -001 -002 -003 -004 -005 -006 -007 -008 -009 -010 -011 -012 -013 -014 -015 -016 -017 -018 -019 -020 -021 -022 -023 -024 -025 -026 -027 -028 -029 -030 -031 -032 -033 -034 -035 -036 -037 -038 -039 -040 -041 -042 -043 -044 -045 -046 -047 -048 -049 -050 -051 -052 diff --git a/tests/prose/fixtures/feature-created/snapshot/.workflows/_gitignore.fixture b/tests/prose/fixtures/feature-created/snapshot/.workflows/_gitignore.fixture deleted file mode 100644 index 779cbf28a..000000000 --- a/tests/prose/fixtures/feature-created/snapshot/.workflows/_gitignore.fixture +++ /dev/null @@ -1,2 +0,0 @@ -.cache/ -.manifest.json.*.tmp diff --git a/tests/prose/fixtures/feature-created/snapshot/.workflows/manifest.json b/tests/prose/fixtures/feature-created/snapshot/.workflows/manifest.json deleted file mode 100644 index 987f6c236..000000000 --- a/tests/prose/fixtures/feature-created/snapshot/.workflows/manifest.json +++ /dev/null @@ -1,7 +0,0 @@ -{ - "work_units": { - "pay": { - "work_type": "feature" - } - } -} diff --git a/tests/prose/fixtures/feature-created/snapshot/.workflows/pay/discovery/sessions/session-001.md b/tests/prose/fixtures/feature-created/snapshot/.workflows/pay/discovery/sessions/session-001.md deleted file mode 100644 index 2aab96961..000000000 --- a/tests/prose/fixtures/feature-created/snapshot/.workflows/pay/discovery/sessions/session-001.md +++ /dev/null @@ -1,39 +0,0 @@ -# Discovery Session 001 - -Date: 2026-01-01 -Work unit: pay - -## Description (as of session) - -Accept card payments at checkout. - -## Seed - -(none) - -## Imports - -(none) - -## Map State at Start - -(n/a — single-topic work) - -## Exploration - -Shaped as a single feature: accept card payments at checkout using -the existing gateway account. Card-only for v1 came up early and was -softly agreed; wallet support was noted as a likely deferral. No -research need surfaced — routed straight to discussion. - -## Edits - -(none) - -## Topics Identified - -(none) - -## Conclusion - -(none) diff --git a/tests/prose/fixtures/feature-created/snapshot/.workflows/pay/manifest.json b/tests/prose/fixtures/feature-created/snapshot/.workflows/pay/manifest.json deleted file mode 100644 index 1cf8afb0e..000000000 --- a/tests/prose/fixtures/feature-created/snapshot/.workflows/pay/manifest.json +++ /dev/null @@ -1,8 +0,0 @@ -{ - "name": "pay", - "work_type": "feature", - "status": "in-progress", - "created": "2026-01-01", - "description": "Accept card payments at checkout", - "phases": {} -} diff --git a/tests/prose/fixtures/feature-discussed/recipe.cjs b/tests/prose/fixtures/feature-discussed/recipe.cjs deleted file mode 100644 index 0385917fa..000000000 --- a/tests/prose/fixtures/feature-discussed/recipe.cjs +++ /dev/null @@ -1,11 +0,0 @@ -'use strict'; - -// Feature mainline stop point — see _shared/feature-mainline.cjs. - -const m = require('../_shared/feature-mainline.cjs'); - -module.exports = { - build(h) { - m.init(h); m.create(h); m.discuss(h); - }, -}; diff --git a/tests/prose/fixtures/feature-discussed/snapshot/.claude/settings.json b/tests/prose/fixtures/feature-discussed/snapshot/.claude/settings.json deleted file mode 100644 index ae019ede5..000000000 --- a/tests/prose/fixtures/feature-discussed/snapshot/.claude/settings.json +++ /dev/null @@ -1,9 +0,0 @@ -{ - "showClearContextOnPlanAccept": true, - "permissions": { - "allow": [ - "Edit(.workflows/**)", - "Bash(mv .workflows/:*)" - ] - } -} diff --git a/tests/prose/fixtures/feature-discussed/snapshot/.workflows/.state/migrations b/tests/prose/fixtures/feature-discussed/snapshot/.workflows/.state/migrations deleted file mode 100644 index 3c7162acc..000000000 --- a/tests/prose/fixtures/feature-discussed/snapshot/.workflows/.state/migrations +++ /dev/null @@ -1,52 +0,0 @@ -001 -002 -003 -004 -005 -006 -007 -008 -009 -010 -011 -012 -013 -014 -015 -016 -017 -018 -019 -020 -021 -022 -023 -024 -025 -026 -027 -028 -029 -030 -031 -032 -033 -034 -035 -036 -037 -038 -039 -040 -041 -042 -043 -044 -045 -046 -047 -048 -049 -050 -051 -052 diff --git a/tests/prose/fixtures/feature-discussed/snapshot/.workflows/_gitignore.fixture b/tests/prose/fixtures/feature-discussed/snapshot/.workflows/_gitignore.fixture deleted file mode 100644 index 779cbf28a..000000000 --- a/tests/prose/fixtures/feature-discussed/snapshot/.workflows/_gitignore.fixture +++ /dev/null @@ -1,2 +0,0 @@ -.cache/ -.manifest.json.*.tmp diff --git a/tests/prose/fixtures/feature-discussed/snapshot/.workflows/manifest.json b/tests/prose/fixtures/feature-discussed/snapshot/.workflows/manifest.json deleted file mode 100644 index 987f6c236..000000000 --- a/tests/prose/fixtures/feature-discussed/snapshot/.workflows/manifest.json +++ /dev/null @@ -1,7 +0,0 @@ -{ - "work_units": { - "pay": { - "work_type": "feature" - } - } -} diff --git a/tests/prose/fixtures/feature-discussed/snapshot/.workflows/pay/discovery/sessions/session-001.md b/tests/prose/fixtures/feature-discussed/snapshot/.workflows/pay/discovery/sessions/session-001.md deleted file mode 100644 index 2aab96961..000000000 --- a/tests/prose/fixtures/feature-discussed/snapshot/.workflows/pay/discovery/sessions/session-001.md +++ /dev/null @@ -1,39 +0,0 @@ -# Discovery Session 001 - -Date: 2026-01-01 -Work unit: pay - -## Description (as of session) - -Accept card payments at checkout. - -## Seed - -(none) - -## Imports - -(none) - -## Map State at Start - -(n/a — single-topic work) - -## Exploration - -Shaped as a single feature: accept card payments at checkout using -the existing gateway account. Card-only for v1 came up early and was -softly agreed; wallet support was noted as a likely deferral. No -research need surfaced — routed straight to discussion. - -## Edits - -(none) - -## Topics Identified - -(none) - -## Conclusion - -(none) diff --git a/tests/prose/fixtures/feature-discussed/snapshot/.workflows/pay/discussion/pay.md b/tests/prose/fixtures/feature-discussed/snapshot/.workflows/pay/discussion/pay.md deleted file mode 100644 index ec0cfc62f..000000000 --- a/tests/prose/fixtures/feature-discussed/snapshot/.workflows/pay/discussion/pay.md +++ /dev/null @@ -1,15 +0,0 @@ -# Discussion — pay - -## Context - -Accept card payments at checkout using the existing gateway account. - -## Decisions - -- Use the existing gateway account; no new provider onboarding. -- Card-only for v1 — wallet support is deferred. -- Webhooks confirm capture; the checkout never polls. - -## Deferred - -- Wallet support (Apple/Google Pay) — revisit after v1. diff --git a/tests/prose/fixtures/feature-discussed/snapshot/.workflows/pay/manifest.json b/tests/prose/fixtures/feature-discussed/snapshot/.workflows/pay/manifest.json deleted file mode 100644 index d4a3fca6c..000000000 --- a/tests/prose/fixtures/feature-discussed/snapshot/.workflows/pay/manifest.json +++ /dev/null @@ -1,16 +0,0 @@ -{ - "name": "pay", - "work_type": "feature", - "status": "in-progress", - "created": "2026-01-01", - "description": "Accept card payments at checkout", - "phases": { - "discussion": { - "items": { - "pay": { - "status": "completed" - } - } - } - } -} diff --git a/tests/prose/fixtures/feature-implementation-started/recipe.cjs b/tests/prose/fixtures/feature-implementation-started/recipe.cjs deleted file mode 100644 index 6aa1cf8a2..000000000 --- a/tests/prose/fixtures/feature-implementation-started/recipe.cjs +++ /dev/null @@ -1,27 +0,0 @@ -'use strict'; - -// Expected end state for the implementation-pickup walk: the environment -// question answered "none" (which the prose records so it is never asked -// again), implementation tracking initialised by `task init`, and the -// first task started. No task work done — the walk stops at the point the -// prose directs implementation to begin. - -const m = require('../_shared/feature-mainline.cjs'); - -module.exports = { - build(h) { - m.init(h); - m.create(h); - m.discuss(h); - m.specify(h); - m.plan(h); - - // Prose order: resume detection (task init) and its commit first, then - // environment setup records the answer so it is never asked again. - h.engine('task', 'init', m.WU, m.WU); - h.engine('commit', m.WU, '-m', `impl(${m.WU}): start implementation`); - h.write('.workflows/.state/environment-setup.md', 'No special setup required.\n'); - h.engine('commit', '--workflows', '-m', `impl(${m.WU}): record environment setup`); - h.engine('task', 'start', m.WU, m.WU, `${m.WU}-1-1`); - }, -}; diff --git a/tests/prose/fixtures/feature-implementation-started/snapshot/.claude/settings.json b/tests/prose/fixtures/feature-implementation-started/snapshot/.claude/settings.json deleted file mode 100644 index ae019ede5..000000000 --- a/tests/prose/fixtures/feature-implementation-started/snapshot/.claude/settings.json +++ /dev/null @@ -1,9 +0,0 @@ -{ - "showClearContextOnPlanAccept": true, - "permissions": { - "allow": [ - "Edit(.workflows/**)", - "Bash(mv .workflows/:*)" - ] - } -} diff --git a/tests/prose/fixtures/feature-implementation-started/snapshot/.workflows/.state/environment-setup.md b/tests/prose/fixtures/feature-implementation-started/snapshot/.workflows/.state/environment-setup.md deleted file mode 100644 index 56b13047e..000000000 --- a/tests/prose/fixtures/feature-implementation-started/snapshot/.workflows/.state/environment-setup.md +++ /dev/null @@ -1 +0,0 @@ -No special setup required. diff --git a/tests/prose/fixtures/feature-implementation-started/snapshot/.workflows/.state/migrations b/tests/prose/fixtures/feature-implementation-started/snapshot/.workflows/.state/migrations deleted file mode 100644 index 3c7162acc..000000000 --- a/tests/prose/fixtures/feature-implementation-started/snapshot/.workflows/.state/migrations +++ /dev/null @@ -1,52 +0,0 @@ -001 -002 -003 -004 -005 -006 -007 -008 -009 -010 -011 -012 -013 -014 -015 -016 -017 -018 -019 -020 -021 -022 -023 -024 -025 -026 -027 -028 -029 -030 -031 -032 -033 -034 -035 -036 -037 -038 -039 -040 -041 -042 -043 -044 -045 -046 -047 -048 -049 -050 -051 -052 diff --git a/tests/prose/fixtures/feature-implementation-started/snapshot/.workflows/_gitignore.fixture b/tests/prose/fixtures/feature-implementation-started/snapshot/.workflows/_gitignore.fixture deleted file mode 100644 index 779cbf28a..000000000 --- a/tests/prose/fixtures/feature-implementation-started/snapshot/.workflows/_gitignore.fixture +++ /dev/null @@ -1,2 +0,0 @@ -.cache/ -.manifest.json.*.tmp diff --git a/tests/prose/fixtures/feature-implementation-started/snapshot/.workflows/manifest.json b/tests/prose/fixtures/feature-implementation-started/snapshot/.workflows/manifest.json deleted file mode 100644 index 987f6c236..000000000 --- a/tests/prose/fixtures/feature-implementation-started/snapshot/.workflows/manifest.json +++ /dev/null @@ -1,7 +0,0 @@ -{ - "work_units": { - "pay": { - "work_type": "feature" - } - } -} diff --git a/tests/prose/fixtures/feature-implementation-started/snapshot/.workflows/pay/discovery/sessions/session-001.md b/tests/prose/fixtures/feature-implementation-started/snapshot/.workflows/pay/discovery/sessions/session-001.md deleted file mode 100644 index 2aab96961..000000000 --- a/tests/prose/fixtures/feature-implementation-started/snapshot/.workflows/pay/discovery/sessions/session-001.md +++ /dev/null @@ -1,39 +0,0 @@ -# Discovery Session 001 - -Date: 2026-01-01 -Work unit: pay - -## Description (as of session) - -Accept card payments at checkout. - -## Seed - -(none) - -## Imports - -(none) - -## Map State at Start - -(n/a — single-topic work) - -## Exploration - -Shaped as a single feature: accept card payments at checkout using -the existing gateway account. Card-only for v1 came up early and was -softly agreed; wallet support was noted as a likely deferral. No -research need surfaced — routed straight to discussion. - -## Edits - -(none) - -## Topics Identified - -(none) - -## Conclusion - -(none) diff --git a/tests/prose/fixtures/feature-implementation-started/snapshot/.workflows/pay/discussion/pay.md b/tests/prose/fixtures/feature-implementation-started/snapshot/.workflows/pay/discussion/pay.md deleted file mode 100644 index ec0cfc62f..000000000 --- a/tests/prose/fixtures/feature-implementation-started/snapshot/.workflows/pay/discussion/pay.md +++ /dev/null @@ -1,15 +0,0 @@ -# Discussion — pay - -## Context - -Accept card payments at checkout using the existing gateway account. - -## Decisions - -- Use the existing gateway account; no new provider onboarding. -- Card-only for v1 — wallet support is deferred. -- Webhooks confirm capture; the checkout never polls. - -## Deferred - -- Wallet support (Apple/Google Pay) — revisit after v1. diff --git a/tests/prose/fixtures/feature-implementation-started/snapshot/.workflows/pay/manifest.json b/tests/prose/fixtures/feature-implementation-started/snapshot/.workflows/pay/manifest.json deleted file mode 100644 index 23023a119..000000000 --- a/tests/prose/fixtures/feature-implementation-started/snapshot/.workflows/pay/manifest.json +++ /dev/null @@ -1,70 +0,0 @@ -{ - "name": "pay", - "work_type": "feature", - "status": "in-progress", - "created": "2026-01-01", - "description": "Accept card payments at checkout", - "phases": { - "discussion": { - "items": { - "pay": { - "status": "completed" - } - } - }, - "specification": { - "items": { - "pay": { - "status": "completed", - "sources": { - "pay": { - "status": "incorporated" - } - } - } - } - }, - "planning": { - "items": { - "pay": { - "status": "completed", - "format": "local-markdown", - "task_list_gate_mode": "gated", - "author_gate_mode": "gated", - "finding_gate_mode": "gated", - "review_cycle": 0, - "phase": 1, - "task": null, - "task_map": { - "pay-1-1": "pay-1-1", - "pay-1-2": "pay-1-2" - }, - "storage_paths": [], - "approvals": { - "structure": "2026-01-01", - "tasks": { - "p1": "2026-01-01" - } - } - } - } - }, - "implementation": { - "items": { - "pay": { - "status": "in-progress", - "task_gate_mode": "gated", - "fix_gate_mode": "gated", - "analysis_gate_mode": "gated", - "fix_attempts": 0, - "analysis_cycle_total": 0, - "analysis_cycle_session": 0, - "linters": [], - "project_skills": [], - "current_phase": 1, - "current_task": "pay-1-1" - } - } - } - } -} diff --git a/tests/prose/fixtures/feature-implementation-started/snapshot/.workflows/pay/planning/pay/planning.md b/tests/prose/fixtures/feature-implementation-started/snapshot/.workflows/pay/planning/pay/planning.md deleted file mode 100644 index 9131d98c7..000000000 --- a/tests/prose/fixtures/feature-implementation-started/snapshot/.workflows/pay/planning/pay/planning.md +++ /dev/null @@ -1,8 +0,0 @@ -# Plan — pay - -## Phase 1: Payment core - -| Task | Title | -|------|-------| -| pay-1-1 | Create Payment Intent | -| pay-1-2 | Handle Capture Webhooks | diff --git a/tests/prose/fixtures/feature-implementation-started/snapshot/.workflows/pay/planning/pay/tasks/pay-1-1.md b/tests/prose/fixtures/feature-implementation-started/snapshot/.workflows/pay/planning/pay/tasks/pay-1-1.md deleted file mode 100644 index f7d43a082..000000000 --- a/tests/prose/fixtures/feature-implementation-started/snapshot/.workflows/pay/planning/pay/tasks/pay-1-1.md +++ /dev/null @@ -1,10 +0,0 @@ ---- -id: pay-1-1 -phase: 1 -status: pending -created: 2026-01-01 ---- - -# Create Payment Intent - -Create a gateway payment intent when checkout begins and attach it to the order. diff --git a/tests/prose/fixtures/feature-implementation-started/snapshot/.workflows/pay/planning/pay/tasks/pay-1-2.md b/tests/prose/fixtures/feature-implementation-started/snapshot/.workflows/pay/planning/pay/tasks/pay-1-2.md deleted file mode 100644 index 8fb22697d..000000000 --- a/tests/prose/fixtures/feature-implementation-started/snapshot/.workflows/pay/planning/pay/tasks/pay-1-2.md +++ /dev/null @@ -1,10 +0,0 @@ ---- -id: pay-1-2 -phase: 1 -status: pending -created: 2026-01-01 ---- - -# Handle Capture Webhooks - -Consume gateway capture webhooks and mark the order paid; no polling path. diff --git a/tests/prose/fixtures/feature-implementation-started/snapshot/.workflows/pay/specification/pay/specification.md b/tests/prose/fixtures/feature-implementation-started/snapshot/.workflows/pay/specification/pay/specification.md deleted file mode 100644 index 29a5f4322..000000000 --- a/tests/prose/fixtures/feature-implementation-started/snapshot/.workflows/pay/specification/pay/specification.md +++ /dev/null @@ -1,11 +0,0 @@ -# Specification — pay - -## Requirements - -- Checkout creates a payment intent against the existing gateway account. -- Card payments only; wallet flows are out of scope for v1. -- Capture is confirmed by gateway webhook, never by polling. - -## Out of scope - -- Wallet support (deferred by discussion). diff --git a/tests/prose/fixtures/feature-implemented/recipe.cjs b/tests/prose/fixtures/feature-implemented/recipe.cjs deleted file mode 100644 index 9daef50c8..000000000 --- a/tests/prose/fixtures/feature-implemented/recipe.cjs +++ /dev/null @@ -1,11 +0,0 @@ -'use strict'; - -// Feature mainline stop point — see _shared/feature-mainline.cjs. - -const m = require('../_shared/feature-mainline.cjs'); - -module.exports = { - build(h) { - m.init(h); m.create(h); m.discuss(h); m.specify(h); m.plan(h); m.implement(h); - }, -}; diff --git a/tests/prose/fixtures/feature-implemented/snapshot/.claude/settings.json b/tests/prose/fixtures/feature-implemented/snapshot/.claude/settings.json deleted file mode 100644 index ae019ede5..000000000 --- a/tests/prose/fixtures/feature-implemented/snapshot/.claude/settings.json +++ /dev/null @@ -1,9 +0,0 @@ -{ - "showClearContextOnPlanAccept": true, - "permissions": { - "allow": [ - "Edit(.workflows/**)", - "Bash(mv .workflows/:*)" - ] - } -} diff --git a/tests/prose/fixtures/feature-implemented/snapshot/.workflows/.state/migrations b/tests/prose/fixtures/feature-implemented/snapshot/.workflows/.state/migrations deleted file mode 100644 index 3c7162acc..000000000 --- a/tests/prose/fixtures/feature-implemented/snapshot/.workflows/.state/migrations +++ /dev/null @@ -1,52 +0,0 @@ -001 -002 -003 -004 -005 -006 -007 -008 -009 -010 -011 -012 -013 -014 -015 -016 -017 -018 -019 -020 -021 -022 -023 -024 -025 -026 -027 -028 -029 -030 -031 -032 -033 -034 -035 -036 -037 -038 -039 -040 -041 -042 -043 -044 -045 -046 -047 -048 -049 -050 -051 -052 diff --git a/tests/prose/fixtures/feature-implemented/snapshot/.workflows/_gitignore.fixture b/tests/prose/fixtures/feature-implemented/snapshot/.workflows/_gitignore.fixture deleted file mode 100644 index 779cbf28a..000000000 --- a/tests/prose/fixtures/feature-implemented/snapshot/.workflows/_gitignore.fixture +++ /dev/null @@ -1,2 +0,0 @@ -.cache/ -.manifest.json.*.tmp diff --git a/tests/prose/fixtures/feature-implemented/snapshot/.workflows/manifest.json b/tests/prose/fixtures/feature-implemented/snapshot/.workflows/manifest.json deleted file mode 100644 index 987f6c236..000000000 --- a/tests/prose/fixtures/feature-implemented/snapshot/.workflows/manifest.json +++ /dev/null @@ -1,7 +0,0 @@ -{ - "work_units": { - "pay": { - "work_type": "feature" - } - } -} diff --git a/tests/prose/fixtures/feature-implemented/snapshot/.workflows/pay/discovery/sessions/session-001.md b/tests/prose/fixtures/feature-implemented/snapshot/.workflows/pay/discovery/sessions/session-001.md deleted file mode 100644 index 2aab96961..000000000 --- a/tests/prose/fixtures/feature-implemented/snapshot/.workflows/pay/discovery/sessions/session-001.md +++ /dev/null @@ -1,39 +0,0 @@ -# Discovery Session 001 - -Date: 2026-01-01 -Work unit: pay - -## Description (as of session) - -Accept card payments at checkout. - -## Seed - -(none) - -## Imports - -(none) - -## Map State at Start - -(n/a — single-topic work) - -## Exploration - -Shaped as a single feature: accept card payments at checkout using -the existing gateway account. Card-only for v1 came up early and was -softly agreed; wallet support was noted as a likely deferral. No -research need surfaced — routed straight to discussion. - -## Edits - -(none) - -## Topics Identified - -(none) - -## Conclusion - -(none) diff --git a/tests/prose/fixtures/feature-implemented/snapshot/.workflows/pay/discussion/pay.md b/tests/prose/fixtures/feature-implemented/snapshot/.workflows/pay/discussion/pay.md deleted file mode 100644 index ec0cfc62f..000000000 --- a/tests/prose/fixtures/feature-implemented/snapshot/.workflows/pay/discussion/pay.md +++ /dev/null @@ -1,15 +0,0 @@ -# Discussion — pay - -## Context - -Accept card payments at checkout using the existing gateway account. - -## Decisions - -- Use the existing gateway account; no new provider onboarding. -- Card-only for v1 — wallet support is deferred. -- Webhooks confirm capture; the checkout never polls. - -## Deferred - -- Wallet support (Apple/Google Pay) — revisit after v1. diff --git a/tests/prose/fixtures/feature-implemented/snapshot/.workflows/pay/manifest.json b/tests/prose/fixtures/feature-implemented/snapshot/.workflows/pay/manifest.json deleted file mode 100644 index 269cab533..000000000 --- a/tests/prose/fixtures/feature-implemented/snapshot/.workflows/pay/manifest.json +++ /dev/null @@ -1,77 +0,0 @@ -{ - "name": "pay", - "work_type": "feature", - "status": "in-progress", - "created": "2026-01-01", - "description": "Accept card payments at checkout", - "phases": { - "discussion": { - "items": { - "pay": { - "status": "completed" - } - } - }, - "specification": { - "items": { - "pay": { - "status": "completed", - "sources": { - "pay": { - "status": "incorporated" - } - } - } - } - }, - "planning": { - "items": { - "pay": { - "status": "completed", - "format": "local-markdown", - "task_list_gate_mode": "gated", - "author_gate_mode": "gated", - "finding_gate_mode": "gated", - "review_cycle": 0, - "phase": 1, - "task": null, - "task_map": { - "pay-1-1": "pay-1-1", - "pay-1-2": "pay-1-2" - }, - "storage_paths": [], - "approvals": { - "structure": "2026-01-01", - "tasks": { - "p1": "2026-01-01" - } - } - } - } - }, - "implementation": { - "items": { - "pay": { - "status": "completed", - "task_gate_mode": "gated", - "fix_gate_mode": "gated", - "analysis_gate_mode": "gated", - "fix_attempts": 0, - "analysis_cycle_total": 0, - "analysis_cycle_session": 0, - "linters": [], - "project_skills": [], - "current_phase": 1, - "current_task": null, - "completed_tasks": [ - "pay-1-1", - "pay-1-2" - ], - "completed_phases": [ - 1 - ] - } - } - } - } -} diff --git a/tests/prose/fixtures/feature-implemented/snapshot/.workflows/pay/planning/pay/planning.md b/tests/prose/fixtures/feature-implemented/snapshot/.workflows/pay/planning/pay/planning.md deleted file mode 100644 index 9131d98c7..000000000 --- a/tests/prose/fixtures/feature-implemented/snapshot/.workflows/pay/planning/pay/planning.md +++ /dev/null @@ -1,8 +0,0 @@ -# Plan — pay - -## Phase 1: Payment core - -| Task | Title | -|------|-------| -| pay-1-1 | Create Payment Intent | -| pay-1-2 | Handle Capture Webhooks | diff --git a/tests/prose/fixtures/feature-implemented/snapshot/.workflows/pay/planning/pay/tasks/pay-1-1.md b/tests/prose/fixtures/feature-implemented/snapshot/.workflows/pay/planning/pay/tasks/pay-1-1.md deleted file mode 100644 index 8ceaac9af..000000000 --- a/tests/prose/fixtures/feature-implemented/snapshot/.workflows/pay/planning/pay/tasks/pay-1-1.md +++ /dev/null @@ -1,10 +0,0 @@ ---- -id: pay-1-1 -phase: 1 -status: completed -created: 2026-01-01 ---- - -# Create Payment Intent - -Create a gateway payment intent when checkout begins and attach it to the order. diff --git a/tests/prose/fixtures/feature-implemented/snapshot/.workflows/pay/planning/pay/tasks/pay-1-2.md b/tests/prose/fixtures/feature-implemented/snapshot/.workflows/pay/planning/pay/tasks/pay-1-2.md deleted file mode 100644 index 23c692d2c..000000000 --- a/tests/prose/fixtures/feature-implemented/snapshot/.workflows/pay/planning/pay/tasks/pay-1-2.md +++ /dev/null @@ -1,10 +0,0 @@ ---- -id: pay-1-2 -phase: 1 -status: completed -created: 2026-01-01 ---- - -# Handle Capture Webhooks - -Consume gateway capture webhooks and mark the order paid; no polling path. diff --git a/tests/prose/fixtures/feature-implemented/snapshot/.workflows/pay/specification/pay/specification.md b/tests/prose/fixtures/feature-implemented/snapshot/.workflows/pay/specification/pay/specification.md deleted file mode 100644 index 29a5f4322..000000000 --- a/tests/prose/fixtures/feature-implemented/snapshot/.workflows/pay/specification/pay/specification.md +++ /dev/null @@ -1,11 +0,0 @@ -# Specification — pay - -## Requirements - -- Checkout creates a payment intent against the existing gateway account. -- Card payments only; wallet flows are out of scope for v1. -- Capture is confirmed by gateway webhook, never by polling. - -## Out of scope - -- Wallet support (deferred by discussion). diff --git a/tests/prose/fixtures/feature-planned/recipe.cjs b/tests/prose/fixtures/feature-planned/recipe.cjs deleted file mode 100644 index d2a02e75b..000000000 --- a/tests/prose/fixtures/feature-planned/recipe.cjs +++ /dev/null @@ -1,11 +0,0 @@ -'use strict'; - -// Feature mainline stop point — see _shared/feature-mainline.cjs. - -const m = require('../_shared/feature-mainline.cjs'); - -module.exports = { - build(h) { - m.init(h); m.create(h); m.discuss(h); m.specify(h); m.plan(h); - }, -}; diff --git a/tests/prose/fixtures/feature-planned/snapshot/.claude/settings.json b/tests/prose/fixtures/feature-planned/snapshot/.claude/settings.json deleted file mode 100644 index ae019ede5..000000000 --- a/tests/prose/fixtures/feature-planned/snapshot/.claude/settings.json +++ /dev/null @@ -1,9 +0,0 @@ -{ - "showClearContextOnPlanAccept": true, - "permissions": { - "allow": [ - "Edit(.workflows/**)", - "Bash(mv .workflows/:*)" - ] - } -} diff --git a/tests/prose/fixtures/feature-planned/snapshot/.workflows/.state/migrations b/tests/prose/fixtures/feature-planned/snapshot/.workflows/.state/migrations deleted file mode 100644 index 3c7162acc..000000000 --- a/tests/prose/fixtures/feature-planned/snapshot/.workflows/.state/migrations +++ /dev/null @@ -1,52 +0,0 @@ -001 -002 -003 -004 -005 -006 -007 -008 -009 -010 -011 -012 -013 -014 -015 -016 -017 -018 -019 -020 -021 -022 -023 -024 -025 -026 -027 -028 -029 -030 -031 -032 -033 -034 -035 -036 -037 -038 -039 -040 -041 -042 -043 -044 -045 -046 -047 -048 -049 -050 -051 -052 diff --git a/tests/prose/fixtures/feature-planned/snapshot/.workflows/_gitignore.fixture b/tests/prose/fixtures/feature-planned/snapshot/.workflows/_gitignore.fixture deleted file mode 100644 index 779cbf28a..000000000 --- a/tests/prose/fixtures/feature-planned/snapshot/.workflows/_gitignore.fixture +++ /dev/null @@ -1,2 +0,0 @@ -.cache/ -.manifest.json.*.tmp diff --git a/tests/prose/fixtures/feature-planned/snapshot/.workflows/manifest.json b/tests/prose/fixtures/feature-planned/snapshot/.workflows/manifest.json deleted file mode 100644 index 987f6c236..000000000 --- a/tests/prose/fixtures/feature-planned/snapshot/.workflows/manifest.json +++ /dev/null @@ -1,7 +0,0 @@ -{ - "work_units": { - "pay": { - "work_type": "feature" - } - } -} diff --git a/tests/prose/fixtures/feature-planned/snapshot/.workflows/pay/discovery/sessions/session-001.md b/tests/prose/fixtures/feature-planned/snapshot/.workflows/pay/discovery/sessions/session-001.md deleted file mode 100644 index 2aab96961..000000000 --- a/tests/prose/fixtures/feature-planned/snapshot/.workflows/pay/discovery/sessions/session-001.md +++ /dev/null @@ -1,39 +0,0 @@ -# Discovery Session 001 - -Date: 2026-01-01 -Work unit: pay - -## Description (as of session) - -Accept card payments at checkout. - -## Seed - -(none) - -## Imports - -(none) - -## Map State at Start - -(n/a — single-topic work) - -## Exploration - -Shaped as a single feature: accept card payments at checkout using -the existing gateway account. Card-only for v1 came up early and was -softly agreed; wallet support was noted as a likely deferral. No -research need surfaced — routed straight to discussion. - -## Edits - -(none) - -## Topics Identified - -(none) - -## Conclusion - -(none) diff --git a/tests/prose/fixtures/feature-planned/snapshot/.workflows/pay/discussion/pay.md b/tests/prose/fixtures/feature-planned/snapshot/.workflows/pay/discussion/pay.md deleted file mode 100644 index ec0cfc62f..000000000 --- a/tests/prose/fixtures/feature-planned/snapshot/.workflows/pay/discussion/pay.md +++ /dev/null @@ -1,15 +0,0 @@ -# Discussion — pay - -## Context - -Accept card payments at checkout using the existing gateway account. - -## Decisions - -- Use the existing gateway account; no new provider onboarding. -- Card-only for v1 — wallet support is deferred. -- Webhooks confirm capture; the checkout never polls. - -## Deferred - -- Wallet support (Apple/Google Pay) — revisit after v1. diff --git a/tests/prose/fixtures/feature-planned/snapshot/.workflows/pay/manifest.json b/tests/prose/fixtures/feature-planned/snapshot/.workflows/pay/manifest.json deleted file mode 100644 index 91f99aef4..000000000 --- a/tests/prose/fixtures/feature-planned/snapshot/.workflows/pay/manifest.json +++ /dev/null @@ -1,53 +0,0 @@ -{ - "name": "pay", - "work_type": "feature", - "status": "in-progress", - "created": "2026-01-01", - "description": "Accept card payments at checkout", - "phases": { - "discussion": { - "items": { - "pay": { - "status": "completed" - } - } - }, - "specification": { - "items": { - "pay": { - "status": "completed", - "sources": { - "pay": { - "status": "incorporated" - } - } - } - } - }, - "planning": { - "items": { - "pay": { - "status": "completed", - "format": "local-markdown", - "task_list_gate_mode": "gated", - "author_gate_mode": "gated", - "finding_gate_mode": "gated", - "review_cycle": 0, - "phase": 1, - "task": null, - "task_map": { - "pay-1-1": "pay-1-1", - "pay-1-2": "pay-1-2" - }, - "storage_paths": [], - "approvals": { - "structure": "2026-01-01", - "tasks": { - "p1": "2026-01-01" - } - } - } - } - } - } -} diff --git a/tests/prose/fixtures/feature-planned/snapshot/.workflows/pay/planning/pay/planning.md b/tests/prose/fixtures/feature-planned/snapshot/.workflows/pay/planning/pay/planning.md deleted file mode 100644 index 9131d98c7..000000000 --- a/tests/prose/fixtures/feature-planned/snapshot/.workflows/pay/planning/pay/planning.md +++ /dev/null @@ -1,8 +0,0 @@ -# Plan — pay - -## Phase 1: Payment core - -| Task | Title | -|------|-------| -| pay-1-1 | Create Payment Intent | -| pay-1-2 | Handle Capture Webhooks | diff --git a/tests/prose/fixtures/feature-planned/snapshot/.workflows/pay/planning/pay/tasks/pay-1-1.md b/tests/prose/fixtures/feature-planned/snapshot/.workflows/pay/planning/pay/tasks/pay-1-1.md deleted file mode 100644 index f7d43a082..000000000 --- a/tests/prose/fixtures/feature-planned/snapshot/.workflows/pay/planning/pay/tasks/pay-1-1.md +++ /dev/null @@ -1,10 +0,0 @@ ---- -id: pay-1-1 -phase: 1 -status: pending -created: 2026-01-01 ---- - -# Create Payment Intent - -Create a gateway payment intent when checkout begins and attach it to the order. diff --git a/tests/prose/fixtures/feature-planned/snapshot/.workflows/pay/planning/pay/tasks/pay-1-2.md b/tests/prose/fixtures/feature-planned/snapshot/.workflows/pay/planning/pay/tasks/pay-1-2.md deleted file mode 100644 index 8fb22697d..000000000 --- a/tests/prose/fixtures/feature-planned/snapshot/.workflows/pay/planning/pay/tasks/pay-1-2.md +++ /dev/null @@ -1,10 +0,0 @@ ---- -id: pay-1-2 -phase: 1 -status: pending -created: 2026-01-01 ---- - -# Handle Capture Webhooks - -Consume gateway capture webhooks and mark the order paid; no polling path. diff --git a/tests/prose/fixtures/feature-planned/snapshot/.workflows/pay/specification/pay/specification.md b/tests/prose/fixtures/feature-planned/snapshot/.workflows/pay/specification/pay/specification.md deleted file mode 100644 index 29a5f4322..000000000 --- a/tests/prose/fixtures/feature-planned/snapshot/.workflows/pay/specification/pay/specification.md +++ /dev/null @@ -1,11 +0,0 @@ -# Specification — pay - -## Requirements - -- Checkout creates a payment intent against the existing gateway account. -- Card payments only; wallet flows are out of scope for v1. -- Capture is confirmed by gateway webhook, never by polling. - -## Out of scope - -- Wallet support (deferred by discussion). diff --git a/tests/prose/fixtures/feature-specified/recipe.cjs b/tests/prose/fixtures/feature-specified/recipe.cjs deleted file mode 100644 index d046654fc..000000000 --- a/tests/prose/fixtures/feature-specified/recipe.cjs +++ /dev/null @@ -1,11 +0,0 @@ -'use strict'; - -// Feature mainline stop point — see _shared/feature-mainline.cjs. - -const m = require('../_shared/feature-mainline.cjs'); - -module.exports = { - build(h) { - m.init(h); m.create(h); m.discuss(h); m.specify(h); - }, -}; diff --git a/tests/prose/fixtures/feature-specified/snapshot/.claude/settings.json b/tests/prose/fixtures/feature-specified/snapshot/.claude/settings.json deleted file mode 100644 index ae019ede5..000000000 --- a/tests/prose/fixtures/feature-specified/snapshot/.claude/settings.json +++ /dev/null @@ -1,9 +0,0 @@ -{ - "showClearContextOnPlanAccept": true, - "permissions": { - "allow": [ - "Edit(.workflows/**)", - "Bash(mv .workflows/:*)" - ] - } -} diff --git a/tests/prose/fixtures/feature-specified/snapshot/.workflows/.state/migrations b/tests/prose/fixtures/feature-specified/snapshot/.workflows/.state/migrations deleted file mode 100644 index 3c7162acc..000000000 --- a/tests/prose/fixtures/feature-specified/snapshot/.workflows/.state/migrations +++ /dev/null @@ -1,52 +0,0 @@ -001 -002 -003 -004 -005 -006 -007 -008 -009 -010 -011 -012 -013 -014 -015 -016 -017 -018 -019 -020 -021 -022 -023 -024 -025 -026 -027 -028 -029 -030 -031 -032 -033 -034 -035 -036 -037 -038 -039 -040 -041 -042 -043 -044 -045 -046 -047 -048 -049 -050 -051 -052 diff --git a/tests/prose/fixtures/feature-specified/snapshot/.workflows/_gitignore.fixture b/tests/prose/fixtures/feature-specified/snapshot/.workflows/_gitignore.fixture deleted file mode 100644 index 779cbf28a..000000000 --- a/tests/prose/fixtures/feature-specified/snapshot/.workflows/_gitignore.fixture +++ /dev/null @@ -1,2 +0,0 @@ -.cache/ -.manifest.json.*.tmp diff --git a/tests/prose/fixtures/feature-specified/snapshot/.workflows/manifest.json b/tests/prose/fixtures/feature-specified/snapshot/.workflows/manifest.json deleted file mode 100644 index 987f6c236..000000000 --- a/tests/prose/fixtures/feature-specified/snapshot/.workflows/manifest.json +++ /dev/null @@ -1,7 +0,0 @@ -{ - "work_units": { - "pay": { - "work_type": "feature" - } - } -} diff --git a/tests/prose/fixtures/feature-specified/snapshot/.workflows/pay/discovery/sessions/session-001.md b/tests/prose/fixtures/feature-specified/snapshot/.workflows/pay/discovery/sessions/session-001.md deleted file mode 100644 index 2aab96961..000000000 --- a/tests/prose/fixtures/feature-specified/snapshot/.workflows/pay/discovery/sessions/session-001.md +++ /dev/null @@ -1,39 +0,0 @@ -# Discovery Session 001 - -Date: 2026-01-01 -Work unit: pay - -## Description (as of session) - -Accept card payments at checkout. - -## Seed - -(none) - -## Imports - -(none) - -## Map State at Start - -(n/a — single-topic work) - -## Exploration - -Shaped as a single feature: accept card payments at checkout using -the existing gateway account. Card-only for v1 came up early and was -softly agreed; wallet support was noted as a likely deferral. No -research need surfaced — routed straight to discussion. - -## Edits - -(none) - -## Topics Identified - -(none) - -## Conclusion - -(none) diff --git a/tests/prose/fixtures/feature-specified/snapshot/.workflows/pay/discussion/pay.md b/tests/prose/fixtures/feature-specified/snapshot/.workflows/pay/discussion/pay.md deleted file mode 100644 index ec0cfc62f..000000000 --- a/tests/prose/fixtures/feature-specified/snapshot/.workflows/pay/discussion/pay.md +++ /dev/null @@ -1,15 +0,0 @@ -# Discussion — pay - -## Context - -Accept card payments at checkout using the existing gateway account. - -## Decisions - -- Use the existing gateway account; no new provider onboarding. -- Card-only for v1 — wallet support is deferred. -- Webhooks confirm capture; the checkout never polls. - -## Deferred - -- Wallet support (Apple/Google Pay) — revisit after v1. diff --git a/tests/prose/fixtures/feature-specified/snapshot/.workflows/pay/manifest.json b/tests/prose/fixtures/feature-specified/snapshot/.workflows/pay/manifest.json deleted file mode 100644 index 029755ae0..000000000 --- a/tests/prose/fixtures/feature-specified/snapshot/.workflows/pay/manifest.json +++ /dev/null @@ -1,28 +0,0 @@ -{ - "name": "pay", - "work_type": "feature", - "status": "in-progress", - "created": "2026-01-01", - "description": "Accept card payments at checkout", - "phases": { - "discussion": { - "items": { - "pay": { - "status": "completed" - } - } - }, - "specification": { - "items": { - "pay": { - "status": "completed", - "sources": { - "pay": { - "status": "incorporated" - } - } - } - } - } - } -} diff --git a/tests/prose/fixtures/feature-specified/snapshot/.workflows/pay/specification/pay/specification.md b/tests/prose/fixtures/feature-specified/snapshot/.workflows/pay/specification/pay/specification.md deleted file mode 100644 index 29a5f4322..000000000 --- a/tests/prose/fixtures/feature-specified/snapshot/.workflows/pay/specification/pay/specification.md +++ /dev/null @@ -1,11 +0,0 @@ -# Specification — pay - -## Requirements - -- Checkout creates a payment intent against the existing gateway account. -- Card payments only; wallet flows are out of scope for v1. -- Capture is confirmed by gateway webhook, never by polling. - -## Out of scope - -- Wallet support (deferred by discussion). From d1287d5efd43b98edff2b52bc8fbbba6213d0dd9 Mon Sep 17 00:00:00 2001 From: Lee Overy Date: Sat, 25 Jul 2026 19:06:17 +0100 Subject: [PATCH 08/11] =?UTF-8?q?feat(prose-tests):=20nested=20agents=20?= =?UTF-8?q?=E2=80=94=20orchestrator=20per=20case,=20prose=20out=20of=20cod?= =?UTF-8?q?e?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Lee's design, taken properly this time: /prose-test dispatches one prose-orchestrator per case, which owns the world, dispatches prose-walker and prose-asserter, escalates a FAIL to Opus, and returns a verdict alone. Transcripts never reach the main session — the reason this scales past a handful of cases. Standing instructions move to .claude/agents/prose-*.md; run.cjs emits only the per-case payload from tests/prose/prompts/. That removes the last 52 lines of prose living as JS string literals in the runner. Every agent is told in prose that its job is to run the test and report — never to fix, never to work out why. The first real runs showed an asserter wandering into engine source to explain a failure; tool restrictions can't be relied on to stop that, so the instruction is explicit in each definition. Also from those runs: the walker must quote every block the prose directs it to emit, since a skipped emission was otherwise invisible. Co-Authored-By: Claude Opus 5 --- .claude/agents/prose-asserter.md | 55 +++++++++++ .claude/agents/prose-orchestrator.md | 75 +++++++++++++++ .claude/agents/prose-walker.md | 76 +++++++++++++++ .claude/skills/prose-test/SKILL.md | 68 +++++-------- design/prose-tests.md | 44 ++++++++- tests/prose/lib/prompts.cjs | 86 +++++++++++++++++ tests/prose/prompts/asserter.md | 26 +++++ tests/prose/prompts/walker.md | 56 +++++++++++ tests/prose/run.cjs | 137 ++++----------------------- 9 files changed, 463 insertions(+), 160 deletions(-) create mode 100644 .claude/agents/prose-asserter.md create mode 100644 .claude/agents/prose-orchestrator.md create mode 100644 .claude/agents/prose-walker.md create mode 100644 tests/prose/lib/prompts.cjs create mode 100644 tests/prose/prompts/asserter.md create mode 100644 tests/prose/prompts/walker.md diff --git a/.claude/agents/prose-asserter.md b/.claude/agents/prose-asserter.md new file mode 100644 index 000000000..ddda5fb7d --- /dev/null +++ b/.claude/agents/prose-asserter.md @@ -0,0 +1,55 @@ +--- +name: prose-asserter +description: Judges a prose-test walk against the case's expected path and the code-computed world delta, returning a four-part verdict. Dispatched by prose-orchestrator. +model: sonnet +--- + +# Prose Asserter + +You judge the result of a prose-test case. You did not perform the walk. + +**Your job is to judge the evidence and report. It is not to fix +anything, and it is not to work out why.** Both are out of scope and +strictly forbidden. A failure is a result, complete in itself — the +person reading your verdict decides what it means and what to do. + +**Use no tools.** Everything you are entitled to consider is already in +your prompt: the expected path, the world delta, and the transcript. Do +not read the repository, the engine, the case directory, or anything +else — not to check a claim, not to confirm a suspicion, not to enrich +your answer. If it is not in the prompt, it is not evidence, and its +absence is itself something to report. This holds regardless of what +tools you appear to have or what mode you are running in. + +## Rules + +- **Never explain a failure.** Report what the evidence shows and stop. + Diagnosing why the prose behaved as it did, tracing an implementation + to account for it, or proposing what should change — all forbidden. + "The step is missing from the transcript" is the finding, whole. +- A PASS on any path step **requires a quote** from the transcript that + shows it. A PASS without a quote is invalid — mark it FAIL. +- Judge the path against what the transcript records, and the world + against the delta. Never infer one from the other. +- A world can match while the walk still went wrong: engine calls that + converge on the same end state hide a skipped step. An empty or + volatile-only delta never stands in for the path being correct. +- Volatile values — timestamps, git SHAs, engine-allocated ids — differ + by nature and are immaterial. Differences in shape, field presence, + status vocabulary, or content are material. + +## Verdict + +Return exactly this and nothing else: + +1. **Path** — one line per expected step: PASS or FAIL, each PASS quoting + the transcript line that shows it, each FAIL stating what is missing + or contradicting. +2. **World** — PASS or FAIL. Enumerate EVERY difference in the delta and + classify each volatile or material. Any material difference fails. An + empty delta passes. +3. **Markers** — list every `UNSCRIPTED QUESTION`, `AMBIGUOUS`, and + `DEVIATION` in the transcript. Each is a finding in its own right, + even when everything else passed. +4. **VERDICT** — PASS or FAIL, then one sentence on what it means for the + prose. diff --git a/.claude/agents/prose-orchestrator.md b/.claude/agents/prose-orchestrator.md new file mode 100644 index 000000000..a0a55fa09 --- /dev/null +++ b/.claude/agents/prose-orchestrator.md @@ -0,0 +1,75 @@ +--- +name: prose-orchestrator +description: Runs one prose-test case end to end — builds the world, dispatches the walker, computes the delta, dispatches the asserter, escalates a failure to Opus, destroys the world — and returns just the verdict. Dispatched by the /prose-test skill, one per case. +tools: Bash, Read, Agent +model: sonnet +--- + +# Prose Orchestrator + +You run **one** prose-test case from end to end and return its verdict. +Your caller gives you a single case id. Everything below runs from the +repository root. + +**Your job is to run the test as stated and report the result. It is not +to fix anything, and it is not to work out why a case failed.** Both are +out of scope and strictly forbidden — for you and for the agents you +dispatch. A failing case is a finished result, not a problem to solve. + +The transcripts and prompts stay with you — your caller receives only the +verdict. That is the point of this agent: the main session never carries +a walk. + +## Rules + +- **Never compose a prompt yourself.** `run.cjs` emits the walker and + asserter payloads; you pass them through verbatim. Writing your own, or + adding a hint, breaks the boundary that makes the result meaningful. +- **Never read the case's `assert.md`,** and never let its content reach + the walker. The walker must not know what is expected. +- **Never edit anything** — not prose, not the case, not a snapshot. A + failure is a result, not a task. +- **Never investigate a failure.** Do not read the skill or engine source + to explain a verdict, and do not add your own analysis to what the + asserter returned. Pass the verdict through as given. +- Destroy every world you build. On a failure, destroy it too but report + the case id so it can be rebuilt for inspection. + +## Steps + +1. **Build the world** — `node tests/prose/run.cjs world `. The + response carries the path, or `world: null` for a structure-only case + (skip `--world` everywhere below when so). + +2. **Walk** — `node tests/prose/run.cjs prompt --world `. + Dispatch the **prose-walker** agent with that output as its prompt, + verbatim and unmodified. Keep the transcript it returns. + +3. **Assert** — `node tests/prose/run.cjs assert --world `. + Dispatch the **prose-asserter** agent with that output, followed by + the walk transcript under a `=== TRANSCRIPT ===` line. Keep its + verdict. + +4. **Escalate a failure** — if the verdict is FAIL, destroy the world, + build a fresh one, and repeat steps 2 and 3 with **model: opus** on + both agents. Two outcomes: + - Opus also FAILs → a confirmed finding. + - Opus PASSes → a disagreement. Report both, quoting the lines where + they diverge. Never resolve it yourself. + +5. **Destroy the world** — `node tests/prose/run.cjs destroy --world `. + +## What you return + +Return exactly this and nothing else: + +``` +CASE: +VERDICT: PASS | FAIL | DISAGREEMENT +PATH: / steps passed +WORLD: PASS | FAIL +MARKERS: +EVIDENCE: +``` + +No transcripts, no prompts, no commentary, no recommendations. diff --git a/.claude/agents/prose-walker.md b/.claude/agents/prose-walker.md new file mode 100644 index 000000000..1edf112de --- /dev/null +++ b/.claude/agents/prose-walker.md @@ -0,0 +1,76 @@ +--- +name: prose-walker +description: Executes workflow prose exactly as a live session would, against a disposable test world, and returns a transcript of what it did. Dispatched by prose-orchestrator during a prose-test run. +tools: Read, Write, Edit, Bash, Glob, Grep +model: sonnet +--- + +# Prose Walker + +You execute this project's workflow prose exactly as a live session +would, and report what happened. + +**Your job is to run the walk as stated and report it. It is not to fix +anything, and it is not to work out why anything went wrong.** You are a +probe. When the prose misbehaves, that misbehaviour *is* the result you +were sent to collect — record it and carry on. Repairing it, or +investigating its cause, destroys the very thing being measured. + +Your caller supplies the case payload: the project directory, the +situation, the task, the prose in scope, the scripted user answers, and +any harness substitutions. Follow it exactly. + +## Rules + +- Follow the prose literally, step by step, arm by arm. Where it names an + engine or knowledge call, run it from the project directory and use the + real response to decide which arm applies. Never predict a response. +- You also play the user, from the fixed script in the payload. When the + prose presents a menu or question, consume the next scripted answer, in + order. +- **Never silently repair, reinterpret, or improve the prose.** Execute + what is written, even where it looks wrong. A broken instruction is the + finding — the single most damaging thing you can do is quietly do the + sensible thing instead. +- **Never investigate.** Do not diagnose why something failed, read + engine or skill source to explain behaviour, or hunt for causes. + Record what happened and move on. This holds however tempting the + explanation looks and however capable you are of finding it. +- **Never fix.** Not the prose, not the world, not a command that + errored. Use only the tools the walk itself requires. +- Do not read the case directory (`tests/prose/cases/…`). It holds the + expected result, and seeing it invalidates the run. + +## Markers + +Record these inline, exactly as named, the moment they occur: + +- `UNSCRIPTED QUESTION:` — the prose asked something the script has no + next answer for. Record the question verbatim and STOP. +- `AMBIGUOUS:` — two arms both appear to match. Name both, then follow + the one the prose's own ordering or guard rules select. +- `DEVIATION:` — the prose cannot be followed literally: a step that + contradicts the state, a missing file it assumes, an instruction that + cannot be executed. Record what you could not do, then continue as best + you can. +- `SUBSTITUTED:` — a harness substitution fired. Name it. + +## Stopping + +Stop at the task's stop condition, the end of the flow, an +`UNSCRIPTED QUESTION`, or a hard error — whichever comes first. + +## Transcript + +Your entire final output is the transcript, in order of events. Return +nothing else — no preamble, no summary, no assessment. + +1. Every prose section or arm entered: `file.md § Heading`, plus the + quoted guard line that selected it. +2. Every command run, and the first line of its output. +3. Every block the prose directed you to emit, quoted — a skipped + emission must be visible as an absence. +4. Every menu or question encountered, verbatim, and the scripted answer + used. +5. Every file written or edited (path only), and every marker above. +6. Finally: `STOPPED: `. diff --git a/.claude/skills/prose-test/SKILL.md b/.claude/skills/prose-test/SKILL.md index 280eae39b..97b3709fc 100644 --- a/.claude/skills/prose-test/SKILL.md +++ b/.claude/skills/prose-test/SKILL.md @@ -1,15 +1,17 @@ --- name: prose-test -description: Run prose-logic test cases — a walker agent executes workflow prose against a materialised fixture world, an asserting agent judges the path taken and the world delta. Scope by diff (default), case ids, or --all. +description: Run prose-logic test cases — one orchestrator agent per case walks the real prose against a materialised world and asserts the result, reporting back a verdict. Scope by diff (default), case ids, or --all. --- # Prose Test Run prose-test cases (design: `design/prose-tests.md`; runner: -`tests/prose/run.cjs`). Cases are Given-When-Then: a world is arranged -from a committed fixture, an agent acts by walking the real prose, and a -second agent asserts the resulting path and world. Walks cost tokens — -this skill runs on command only, never as part of a routine gate. +`tests/prose/run.cjs`). Each case is dispatched to a **prose-orchestrator** +agent that owns the whole run — world, walk, delta, assertion, cleanup — +and returns only a verdict. Transcripts never enter this session. + +Walks cost tokens: this skill runs on command only, never as part of a +routine gate. ## Step 1: Select @@ -23,49 +25,31 @@ Parse the arguments: If the selection is empty: report that no cases intersect and stop. Otherwise show the selected case ids and proceed. -## Step 2: Arrange and act - -Cases are independent — run up to 4 concurrently. Per case: - -1. **World** (skip when the case has no world): `node tests/prose/run.cjs world ` — note the returned path. -2. **Walk**: `node tests/prose/run.cjs prompt [--world ]`, then dispatch a subagent (model: **sonnet**) with that prompt **verbatim and unmodified**. Never hand-assemble a walker prompt, and never mention any expectation — the `prompt` command's output is the whole contract, and it cannot contain assert.md. Save the returned transcript to a scratch file. - -## Step 3: Assert - -`node tests/prose/run.cjs assert [--world ]` produces the -asserting agent's prompt: the expected path, any further claims, and the -code-computed world delta. Dispatch a second subagent (model: **sonnet**) -with that prompt plus the walk transcript. - -It returns a verdict in four parts: path (per step, each PASS quoting the -transcript line that shows it), world (every difference in the delta -enumerated and classified volatile or material), markers, and the overall -verdict. A path PASS without a quote is invalid — treat as FAIL. - -`UNSCRIPTED QUESTION`, `AMBIGUOUS`, or `DEVIATION` markers in a transcript -are findings in their own right — carry them into the report even when -every claim passed. +## Step 2: Dispatch one orchestrator per case -## Step 4: Escalate failures +Cases are independent — dispatch up to 4 concurrently, each a +**prose-orchestrator** agent whose prompt is the case id and the +instruction to run it end to end. -Any FAIL: +Never run the walker or asserter yourself, and never read a case's +`assert.md` — the orchestrator owns that boundary, and anything you learn +about an expected result can leak into a later dispatch. -1. Build a **fresh** world and re-run the walk on model: **opus**, then re-assert. -2. Both fail → confirmed finding. -3. Sonnet-fail / Opus-pass → disagreement: report both walks with the divergent lines quoted. Never auto-resolve. +## Step 3: Collate -## Step 5: Report and clean up +Report a verdict table from what the orchestrators returned: case id, +verdict, path steps passed, world, markers. Quote the evidence line for +every failure. -Report a verdict table: case id, path passed/total, world verdict, -markers, escalations. Quote the evidence for every failure. Findings are -findings — never edit prose, cases, or snapshots mid-run; report and stop. +A `DISAGREEMENT` verdict means Sonnet and Opus reached different +conclusions on the same case — surface both, resolve neither. -Destroy every world (`node tests/prose/run.cjs destroy --world `) -except one you are actively citing in a failure report; name any world -kept so it can be removed later. +`UNSCRIPTED QUESTION`, `AMBIGUOUS` and `DEVIATION` markers are findings +in their own right; carry them into the report even when every case +passed. ## Rules -- The walker never sees `assert.md` — only the `prompt` command's output reaches it. -- A failing case is a finding either way: broken prose, or a stale case. The report says which is suspected; the user decides. -- Never regenerate snapshots or edit cases to make a run pass. +- Findings are findings. Never edit prose, cases, or snapshots to make a run pass — report and stop. +- A failing case is a finding either way: broken prose, or a stale case. Say which is suspected; the user decides. +- Never regenerate a snapshot as part of a run. diff --git a/design/prose-tests.md b/design/prose-tests.md index 4ccfe21cb..d920d9b0c 100644 --- a/design/prose-tests.md +++ b/design/prose-tests.md @@ -101,9 +101,23 @@ testing. this PR's prose changes"), by hand-picked ids, or `--all`. An engine-only PR intersects nothing and suggests nothing. - **P6 — tiered walkers.** Sonnet walks by default; any FAIL re-runs - on Opus before it is believed. Persistent disagreement surfaces to - Lee with both walks quoted — failures are findings, never + on Opus before it is believed (the agent definitions set the default, + the dispatch overrides it on retry). Persistent disagreement surfaces + to Lee with both walks quoted — failures are findings, never auto-resolved gates. +- **P6a — a nested agent per case.** The `/prose-test` skill dispatches + one **prose-orchestrator** per case, which builds the world, dispatches + **prose-walker**, computes the delta, dispatches **prose-asserter**, + escalates a failure, destroys the world, and returns a verdict alone. + Transcripts never reach the main session — the reason the design + scales past a handful of cases. Standing instructions live in + `.claude/agents/prose-*.md`; only the per-case payload comes from + `tests/prose/prompts/`, so no agent ever reads words composed in code. +- **P6b — run the test, nothing else.** Every agent in the chain is + forbidden from fixing anything and from working out why a case failed. + A failure is a finished result; diagnosing it is a separate, human-led + act. This is stated in prose in each definition rather than relied on + from tool restrictions, which an agent may hold regardless. - **P7 — a failing case is a finding either way.** Either the prose broke, or the world/design moved and the case is stale. The adjudication is the point; only the typo-class staleness is @@ -142,6 +156,32 @@ testing. ## Log +- 2026-07-25 — First real runs, and the nested-agent shape. The framework + executed end to end for the first time: a Sonnet walker followed + root-cause-validation.md against a live world — real engine calls, the + scripted answer, the stub firing at its trigger — and a Sonnet asserter + graded five path steps with quoted evidence and classified the lone + world difference (an agent-row timestamp) as volatile. PASS, ~107k + tokens, 2.5 minutes. + **The negative test is the more valuable result.** With `agent scan` + deleted from the world's own copy of the prose, the run correctly + FAILED — the asserter quoted the missing call and raised a DEVIATION. + Crucially **the world delta was byte-identical to the passing run**: + `incorporate` sets `incorporated` from any prior status, so skipping + `scan` converges on the same end state. Whole-world assertion alone + would have passed a real defect; only the granular expected path caught + it. Lee's argument for granularity, proven by evidence rather than + reasoning. + Two changes came out of the runs: the walker must quote every block the + prose directs it to emit (a skipped emission was invisible), and the + agent layer moved into `.claude/agents/` as prose-orchestrator → + prose-walker + prose-asserter, so transcripts stay out of the main + session (P6a) and standing instructions stop being JS string literals + in the runner. Noted, not actioned: `incorporate` never requires a row + to have reached `pending` — forgiving by design, since recovery paths + close rows that never produced a report, but it is why this class of + skipped step is invisible to state. + - 2026-07-25 — A case becomes a directory (Lee, still reviewing in the TUI). Three faults in the flat-markdown shape, all his: **(1)** code regex-parsed values out of markdown — the exact pattern this project diff --git a/tests/prose/lib/prompts.cjs b/tests/prose/lib/prompts.cjs new file mode 100644 index 000000000..1651e7a8d --- /dev/null +++ b/tests/prose/lib/prompts.cjs @@ -0,0 +1,86 @@ +'use strict'; + +// Prompt assembly: every word an agent reads comes from prose, never +// from a string in code. +// +// Standing instructions — how a walker or asserter behaves — live in +// .claude/agents/prose-*.md and are loaded by the harness when the agent +// is dispatched. What varies per case lives in tests/prose/prompts/*.md, +// assembled here. +// +// A template file is sections delimited by `=== name ===` lines, with +// `{{placeholder}}` slots. This module only splits and substitutes — it +// holds no wording of its own, so changing what an agent is told is a +// prose edit, reviewable as a diff, and identical on every run. +// +// An unfilled placeholder is a hard error rather than something an agent +// silently receives. + +const fs = require('fs'); +const path = require('path'); + +const PROMPTS_DIR = path.join(__dirname, '../prompts'); + +function loadTemplate(name) { + const raw = fs.readFileSync(path.join(PROMPTS_DIR, `${name}.md`), 'utf8'); + const sections = {}; + let current = null; + for (const line of raw.split('\n')) { + const header = line.match(/^=== ([a-z-]+) ===$/); + if (header) { + current = header[1]; + sections[current] = []; + continue; + } + if (current) sections[current].push(line); + } + for (const key of Object.keys(sections)) { + sections[key] = sections[key].join('\n').replace(/^\n+|\n+$/g, ''); + } + return sections; +} + +function fill(text, values) { + const filled = text.replace(/\{\{([a-z_]+)\}\}/g, (match, key) => { + if (!(key in values)) return match; + return values[key]; + }); + const leftover = filled.match(/\{\{[a-z_]+\}\}/); + if (leftover) throw new Error(`prompt placeholder ${leftover[0]} was never filled`); + return filled; +} + +/** Indent a block so it reads as a nested unit inside the prompt. */ +function indent(text, by = ' ') { + return text.split('\n').map((l) => (l.length ? by + l : l)).join('\n'); +} + +function walkerPrompt({ worldDir, root, situation, task, scope, stubs, answers }) { + const t = loadTemplate('walker'); + const parts = [ + worldDir ? fill(t.world, { world_dir: worldDir }) : fill(t.structural, { root }), + ]; + if (situation) parts.push(fill(t.situation, { situation })); + parts.push(fill(t.task, { task, scope })); + if (answers) parts.push(fill(t.answers, { answers })); + if (stubs.length) { + const entries = stubs.map((s) => fill(t['stub-entry'], { + name: s.name, + trigger: s.trigger, + description: s.description, + content: indent(s.content), + })).join('\n'); + parts.push(fill(t.stubs, { entries })); + } + return `${parts.join('\n\n')}\n`; +} + +function asserterPrompt({ expected, world }) { + const t = loadTemplate('asserter'); + const parts = [fill(t.main, { expected })]; + if (world) parts.push(fill(t.world, { expecting: world.expecting, delta: world.delta })); + parts.push(t.transcript); + return `${parts.join('\n\n')}\n`; +} + +module.exports = { PROMPTS_DIR, loadTemplate, fill, walkerPrompt, asserterPrompt }; diff --git a/tests/prose/prompts/asserter.md b/tests/prose/prompts/asserter.md new file mode 100644 index 000000000..da1a122ed --- /dev/null +++ b/tests/prose/prompts/asserter.md @@ -0,0 +1,26 @@ +The asserter's per-case payload. Sections are delimited by `=== name ===` +lines and assembled by lib/prompts.cjs; `{{placeholders}}` are filled +from the case and the computed world delta. + +Only what varies per case lives here. How an asserter behaves — quote or +it didn't happen, never explain a failure, volatile versus material, the +verdict format — is standing instruction and lives in +`.claude/agents/prose-asserter.md`. + +=== main === +WHAT THE PROSE SHOULD HAVE DONE: + +{{expected}} + +=== world === + +EXPECTED WORLD: {{expecting}}. + +WORLD DELTA — the factual difference between the world after the walk and +the expected world, computed by code: + +{{delta}} + +=== transcript === + +The walk's transcript follows under `=== TRANSCRIPT ===`. diff --git a/tests/prose/prompts/walker.md b/tests/prose/prompts/walker.md new file mode 100644 index 000000000..693c2d4ac --- /dev/null +++ b/tests/prose/prompts/walker.md @@ -0,0 +1,56 @@ +The walker's per-case payload. Sections are delimited by `=== name ===` +lines and assembled by lib/prompts.cjs; `{{placeholders}}` are filled +from the case. + +Only what varies per case lives here. How a walker behaves — follow +literally, never repair, never investigate, the markers, the transcript +format — is standing instruction and lives in +`.claude/agents/prose-walker.md`. + +Nothing from `assert.md` may ever appear in this file or in anything +assembled from it. That is the boundary the design rests on. + +=== world === +Project directory — your cwd for EVERY command: {{world_dir}} +The workflow skills are installed at .claude/skills/ inside that project. +Mutations are expected and safe: the project is a disposable test world. + +=== structural === +Structure-only walk: read the named prose and trace the logic. Execute +nothing — no commands, no writes. + +Repository root: {{root}} + +=== situation === + +SITUATION — where the project stands as you begin: +{{situation}} + +=== task === + +TASK +{{task}} + +SCOPE — the prose under walk: +{{scope}} + +=== answers === + +SCRIPTED USER ANSWERS — consume in order, one per question the prose asks: +{{answers}} + +=== stubs === + +HARNESS SUBSTITUTIONS — these are NOT part of the process you are walking. +They stand in for steps this framework deliberately does not simulate. +Apply each only at the moment stated, then resume the prose exactly where +you left it. +{{entries}} + +=== stub-entry === + +### {{name}} +WHEN: {{trigger}} +WHAT IT IS: {{description}} +CONTENT (write these exact bytes where the substitution calls for a file): +{{content}} diff --git a/tests/prose/run.cjs b/tests/prose/run.cjs index 26872856f..327777d57 100644 --- a/tests/prose/run.cjs +++ b/tests/prose/run.cjs @@ -21,6 +21,7 @@ const { execFileSync } = require('child_process'); const cases = require('./lib/cases.cjs'); const worlds = require('./lib/worlds.cjs'); +const prompts = require('./lib/prompts.cjs'); const ROOT = cases.ROOT; @@ -113,84 +114,17 @@ function cmdPrompt(argv) { const c = getCase(argv[0]); const worldDir = c.hasFixtureState ? requireWorld(argv, c) : null; - const setting = worldDir - ? [ - "You are executing this project's workflow prose exactly as a live session would.", - '', - `Project directory — your cwd for EVERY command: ${worldDir}`, - 'The workflow skills are installed at .claude/skills/ inside that project.', - 'Mutations are expected and safe: the project is a disposable test world.', - ] - : [ - 'You are walking workflow prose structurally: read the named files and trace the logic.', - 'Execute nothing — no commands, no writes. This is a read-only walk.', - '', - `Repository root: ${ROOT}`, - ]; - - const answers = c.answers.length - ? c.answers.map((a, i) => ` ${i + 1}. ${a}`).join('\n') - : ' (none — the walk should reach its stop condition without user questions)'; - - const substitutions = c.stubs.length ? [ - '', - 'HARNESS SUBSTITUTIONS — these are NOT part of the process you are walking.', - 'They stand in for steps this framework deliberately does not simulate.', - 'Apply each only at the moment stated, then resume the prose exactly where', - 'you left it, and record `SUBSTITUTED: ` in the transcript.', - '', - ...c.stubs.flatMap((s) => { - const stub = cases.readStub(s.name); - return [ - `### ${s.name}`, - `WHEN: ${s.trigger}`, - `WHAT IT IS: ${stub.description}`, - 'CONTENT (write these exact bytes where the substitution calls for a file):', - ...stub.content.split('\n').map((l) => ` ${l}`), - '', - ]; - }), - ] : []; - - process.stdout.write(`${[ - ...setting, - ...(c.situation ? ['', 'SITUATION — where the project stands as you begin:', c.situation] : []), - '', - 'TASK', - c.act, - '', - 'SCOPE — the prose under walk:', - ...c.files.map((f) => ` - ${f.path}${f.anchor ? ` (start at the heading containing "${f.anchor}")` : ''}`), - ...substitutions, - '', - 'RULES', - '- Follow the prose literally, step by step, arm by arm. Where it names an', - ' engine or knowledge call, run it from the project directory and use the', - ' real response to decide which arm applies.', - '- You also play the user, from a fixed script. When the prose presents a', - ' menu or question, consume the next scripted answer, in order:', - answers, - '- If the prose asks a question and the script has no next answer: STOP and', - ' record `UNSCRIPTED QUESTION:` with the exact question text.', - '- If two arms both appear to match, record `AMBIGUOUS:` naming both, then', - " follow the one the prose's own ordering/guard rules select.", - '- If the prose cannot be followed literally — a step that contradicts the', - ' state, a missing file it assumes, an instruction that cannot be executed —', - ' record `DEVIATION:` with what you could not do, then continue as best you', - ' can. NEVER silently repair, reinterpret, or improve the prose. You are a', - ' probe, not a reviewer: a broken instruction is the finding.', - "- Stop at the TASK's stop condition, the end of the flow, an UNSCRIPTED", - ' QUESTION, or a hard error — whichever comes first.', - '', - 'TRANSCRIPT — your entire final output, in order of events:', - '1. Every prose section/arm entered: `file.md § Heading` plus the quoted', - ' guard line that selected it.', - '2. Every command run and the first line of its output.', - '3. Every menu/question encountered (verbatim) and the scripted answer used.', - '4. Every file written or edited (path only), and every SUBSTITUTED marker.', - '5. Finally: `STOPPED: `.', - 'Return nothing but the transcript.', - ].join('\n')}\n`); + process.stdout.write(prompts.walkerPrompt({ + worldDir, + root: ROOT, + situation: c.situation, + task: c.act, + scope: c.files + .map((f) => ` - ${f.path}${f.anchor ? ` (start at the heading containing "${f.anchor}")` : ''}`) + .join('\n'), + stubs: c.stubs.map((s) => ({ ...s, ...cases.readStub(s.name) })), + answers: c.answers.map((a, i) => ` ${i + 1}. ${a}`).join('\n'), + })); } // --- diff (the facts) ----------------------------------------------------- @@ -208,47 +142,18 @@ function cmdDiff(argv) { function cmdAssert(argv) { const c = getCase(argv[0]); - const lines = [ - 'You are asserting the result of a prose-test case. You did not perform', - 'the walk. Judge only what the evidence shows.', - '', - 'WHAT THE PROSE SHOULD HAVE DONE:', - '', - c.assert, - ]; - + let world = null; if (c.hasFixtureState) { const delta = worlds.diffWorld(c.id, requireWorld(argv, c)); - lines.push( - '', - `EXPECTED WORLD: ${delta.expecting}.`, - '', - 'WORLD DELTA — the factual difference between the world after the walk and', - 'the expected world, computed by code. Volatile values (timestamps, git', - 'SHAs, engine-allocated ids) are NOT normalised: a difference in one of', - 'those is expected and immaterial. A difference in shape, field presence,', - 'status vocabulary, or content is material.', - '', - JSON.stringify({ added: delta.added, removed: delta.removed, identical: delta.identical }, null, 2), - ...delta.changed, - ); + world = { + expecting: delta.expecting, + delta: [ + JSON.stringify({ added: delta.added, removed: delta.removed, identical: delta.identical }, null, 2), + ...delta.changed, + ].join('\n'), + }; } - - lines.push( - '', - 'THE TRANSCRIPT of the walk is supplied separately by the caller.', - '', - 'VERDICT — return exactly this, nothing else:', - '1. Path: one line per expected step — PASS or FAIL, each PASS quoting the', - ' transcript line that shows it. A PASS with no quote is invalid.', - '2. World: PASS or FAIL. Enumerate EVERY difference in the delta and classify', - ' each as volatile (immaterial) or material. Any material difference fails.', - ' An empty delta passes.', - '3. Markers: list every UNSCRIPTED QUESTION, AMBIGUOUS, and DEVIATION in the', - ' transcript. Each is a finding in its own right, even if everything passed.', - '4. VERDICT: PASS or FAIL, then one sentence on what it means for the prose.', - ); - process.stdout.write(`${lines.join('\n')}\n`); + process.stdout.write(prompts.asserterPrompt({ expected: c.assert, world })); } // --- snap / verify -------------------------------------------------------- From e20cc01f14814c3c2beb82d6791ba49b6fb9cfe0 Mon Sep 17 00:00:00 2001 From: Lee Overy Date: Sat, 25 Jul 2026 19:30:36 +0100 Subject: [PATCH 09/11] =?UTF-8?q?fix(prose-tests):=20both=20agents=20run?= =?UTF-8?q?=20on=20Opus=20=E2=80=94=20a=20result=20you=20can't=20rely=20on?= =?UTF-8?q?=20is=20worth=20nothing?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Three runs of evidence: a Sonnet walker performed the walk correctly every time but narrated it in summary, omitting the quoted evidence the asserter requires — even with a worked transcript example in front of it. The asserter was right to fail it; the tiering was wrong. Tiered models produced a DISAGREEMENT on every run, which is noise rather than signal, so the cheaper tier was a false economy. Walker and asserter both run on Opus, and the orchestrator never overrides a model — each definition names the model its result is trusted at. A FAIL now confirms by repetition from a fresh world rather than by promotion to a bigger model; a failure that doesn't reproduce reports as FLAKY. Also: renames test-prose-fixtures.cjs to test-prose-snapshots.cjs (it tests snapshots), and clears two stale fragments left in the contract by the Given-When-Then reshape. Co-Authored-By: Claude Opus 5 --- .claude/agents/prose-asserter.md | 2 +- .claude/agents/prose-orchestrator.md | 21 +++++---- .claude/agents/prose-walker.md | 44 ++++++++++++++++--- .claude/skills/prose-test/SKILL.md | 4 +- design/prose-tests.md | 31 +++++++------ package.json | 2 +- ...-fixtures.cjs => test-prose-snapshots.cjs} | 0 7 files changed, 74 insertions(+), 30 deletions(-) rename tests/scripts/{test-prose-fixtures.cjs => test-prose-snapshots.cjs} (100%) diff --git a/.claude/agents/prose-asserter.md b/.claude/agents/prose-asserter.md index ddda5fb7d..72439d1f7 100644 --- a/.claude/agents/prose-asserter.md +++ b/.claude/agents/prose-asserter.md @@ -1,7 +1,7 @@ --- name: prose-asserter description: Judges a prose-test walk against the case's expected path and the code-computed world delta, returning a four-part verdict. Dispatched by prose-orchestrator. -model: sonnet +model: opus --- # Prose Asserter diff --git a/.claude/agents/prose-orchestrator.md b/.claude/agents/prose-orchestrator.md index a0a55fa09..ae53106e7 100644 --- a/.claude/agents/prose-orchestrator.md +++ b/.claude/agents/prose-orchestrator.md @@ -50,12 +50,17 @@ a walk. the walk transcript under a `=== TRANSCRIPT ===` line. Keep its verdict. -4. **Escalate a failure** — if the verdict is FAIL, destroy the world, - build a fresh one, and repeat steps 2 and 3 with **model: opus** on - both agents. Two outcomes: - - Opus also FAILs → a confirmed finding. - - Opus PASSes → a disagreement. Report both, quoting the lines where - they diverge. Never resolve it yourself. +Never pass a `model` when dispatching either agent — each definition +names the model the result is trusted at, and overriding it makes a +verdict unreliable. + +4. **Confirm a failure** — if the verdict is FAIL, destroy the world, + build a fresh one, and repeat steps 2 and 3 once. A defect in the + prose reproduces; a one-off does not. Two outcomes: + - The second run also FAILs → a confirmed finding. Report the + evidence from the second run. + - The second run PASSes → a non-reproducing failure. Report it as + `FLAKY`, quoting both, and resolve nothing yourself. 5. **Destroy the world** — `node tests/prose/run.cjs destroy --world `. @@ -65,11 +70,11 @@ Return exactly this and nothing else: ``` CASE: -VERDICT: PASS | FAIL | DISAGREEMENT +VERDICT: PASS | FAIL | FLAKY PATH: / steps passed WORLD: PASS | FAIL MARKERS: -EVIDENCE: +EVIDENCE: ``` No transcripts, no prompts, no commentary, no recommendations. diff --git a/.claude/agents/prose-walker.md b/.claude/agents/prose-walker.md index 1edf112de..46dad2afa 100644 --- a/.claude/agents/prose-walker.md +++ b/.claude/agents/prose-walker.md @@ -2,7 +2,7 @@ name: prose-walker description: Executes workflow prose exactly as a live session would, against a disposable test world, and returns a transcript of what it did. Dispatched by prose-orchestrator during a prose-test run. tools: Read, Write, Edit, Bash, Glob, Grep -model: sonnet +model: opus --- # Prose Walker @@ -62,15 +62,49 @@ Stop at the task's stop condition, the end of the flow, an ## Transcript -Your entire final output is the transcript, in order of events. Return -nothing else — no preamble, no summary, no assessment. +Your entire final output is the transcript. Return nothing else — no +preamble, no assessment, no closing summary. + +**It is a log, not a recollection.** Write one entry at the moment each +event happens, in the order they happen. Never compress several steps +into a sentence, never describe a stretch of the walk in the past tense, +and never leave out a step because a later one implies it. A reader who +has seen nothing but your transcript must be able to tell exactly what +occurred, in order, with the words that were on screen. + +**If you did it, it is in the log.** Every command you ran, every block +you emitted, every question you were asked. An event you performed but +did not log is indistinguishable from one you skipped, and will be +counted as skipped. + +Log these, each as its own entry: 1. Every prose section or arm entered: `file.md § Heading`, plus the quoted guard line that selected it. 2. Every command run, and the first line of its output. -3. Every block the prose directed you to emit, quoted — a skipped - emission must be visible as an absence. +3. Every block the prose directed you to emit, quoted in full. 4. Every menu or question encountered, verbatim, and the scripted answer used. 5. Every file written or edited (path only), and every marker above. 6. Finally: `STOPPED: `. + +The shape, abbreviated: + +``` +ENTERED: some-reference.md § A. Offer Something + guard: (start of file) +EMITTED: + > An independent agent can trace the code fresh… +EMITTED (menu): + Do the thing? + - **`y`/`yes`** — Do it + - **`s`/`skip`** — Skip it +ANSWERED: yes — do the thing (scripted answer 1) +ENTERED: some-reference.md § A — #### If `yes` + guard: "#### If `yes`" +RAN: node .claude/skills/workflow-engine/scripts/engine.cjs thing do wu topic + → {"ok":true,"id":"thing-001","file":".workflows/.cache/…/thing-001.md"} +WROTE: .workflows/.cache/…/thing-001.md +SUBSTITUTED: the-stub-name +STOPPED: the reference returned to its caller +``` diff --git a/.claude/skills/prose-test/SKILL.md b/.claude/skills/prose-test/SKILL.md index 97b3709fc..4ebab0ca3 100644 --- a/.claude/skills/prose-test/SKILL.md +++ b/.claude/skills/prose-test/SKILL.md @@ -41,8 +41,8 @@ Report a verdict table from what the orchestrators returned: case id, verdict, path steps passed, world, markers. Quote the evidence line for every failure. -A `DISAGREEMENT` verdict means Sonnet and Opus reached different -conclusions on the same case — surface both, resolve neither. +A `FLAKY` verdict means a failure did not reproduce on a second run from +a fresh world — surface both runs, resolve neither. `UNSCRIPTED QUESTION`, `AMBIGUOUS` and `DEVIATION` markers are findings in their own right; carry them into the report even when every case diff --git a/design/prose-tests.md b/design/prose-tests.md index d920d9b0c..abab376cd 100644 --- a/design/prose-tests.md +++ b/design/prose-tests.md @@ -47,9 +47,6 @@ testing. the prose's behaviour changes. Worlds are per-case; duplication is accepted, because a case readable in one directory beats a deduplicated fixture library chased across the tree. - **Then**: `world_after` (a fixture, or `unchanged`), a granular - `trace` of the path the prose should have taken, and any further - `notes`. - **P1b — coarse act, granular assertion.** `act.md` never scripts the workflow's own steps: if the case says which commands to run, the walker stops deriving the path from the prose and the case would pass @@ -81,11 +78,11 @@ testing. (so a world the engine moved goes red at the gate and lands as a reviewable snapshot diff in the PR that moved it), and every case's paths, anchors, worlds, stubs, and trace resolve. -- **P4 — walker/asserter separation.** The walking agent never sees the - `then` block; an agent that knows the expected answer will find it. +- **P4 — walker/asserter separation.** The walking agent never sees + `assert.md`; an agent that knows the expected answer will find it. The walker gets the world, the coarse instruction, the answer script and the stubs, and returns a transcript. The asserter sees the - transcript, the expected trace, and the world delta — never the + transcript, the expected path, and the world delta — never the reverse. - **P4a — substitutions are declared and marked.** A stub is named content; the case arming it owns the trigger, so one stub serves @@ -100,12 +97,20 @@ testing. computation powers the PR-end suggestion — "these N cases intersect this PR's prose changes"), by hand-picked ids, or `--all`. An engine-only PR intersects nothing and suggests nothing. -- **P6 — tiered walkers.** Sonnet walks by default; any FAIL re-runs - on Opus before it is believed (the agent definitions set the default, - the dispatch overrides it on retry). Persistent disagreement surfaces - to Lee with both walks quoted — failures are findings, never - auto-resolved gates. -- **P6a — a nested agent per case.** The `/prose-test` skill dispatches +- **P6 — one model, chosen for trust.** Walker and asserter both run on + **Opus**. Measured, not assumed: across three runs a Sonnet walker + performed the walk correctly every time but narrated it in summary, + omitting the quoted evidence the asserter requires — even with a + worked transcript example in front of it. A tiered arrangement then + produces a disagreement on every case, which is noise, not a signal. + A result you cannot rely on is worth nothing, so the cheaper tier is + a false economy. The orchestrator never overrides the model: each + definition names the model its result is trusted at. +- **P6a — a FAIL is confirmed by repetition, not by promotion.** A + failing case re-runs once from a fresh world at the same models. A + defect in the prose reproduces; a one-off does not, and is reported + as FLAKY with both runs quoted. Nothing is auto-resolved. +- **P6b — a nested agent per case.** The `/prose-test` skill dispatches one **prose-orchestrator** per case, which builds the world, dispatches **prose-walker**, computes the delta, dispatches **prose-asserter**, escalates a failure, destroys the world, and returns a verdict alone. @@ -113,7 +118,7 @@ testing. scales past a handful of cases. Standing instructions live in `.claude/agents/prose-*.md`; only the per-case payload comes from `tests/prose/prompts/`, so no agent ever reads words composed in code. -- **P6b — run the test, nothing else.** Every agent in the chain is +- **P6c — run the test, nothing else.** Every agent in the chain is forbidden from fixing anything and from working out why a case failed. A failure is a finished result; diagnosing it is a separate, human-led act. This is stated in prose in each definition rather than relied on diff --git a/package.json b/package.json index 482711aca..a2be82ef1 100644 --- a/package.json +++ b/package.json @@ -7,7 +7,7 @@ "build": "node build/knowledge.build.js", "knowledge:repl": "node scripts/knowledge-repl.js", "typecheck": "tsc -p tsconfig.json", - "test": "node --test tests/scripts/test-render.cjs tests/scripts/test-engine-gateway.cjs tests/scripts/test-engine-epic-projections.cjs tests/scripts/test-engine-start-projections.cjs tests/scripts/test-engine-workunit-projections.cjs tests/scripts/test-engine-discussion-map.cjs tests/scripts/test-engine-discovery-map.cjs tests/scripts/test-engine-tasks.cjs tests/scripts/test-engine-render-surfaces.cjs tests/scripts/test-engine-transactions.cjs tests/scripts/test-engine-workunit-create.cjs tests/scripts/test-engine-workunit-pivot.cjs tests/scripts/test-engine-workunit-absorb.cjs tests/scripts/test-engine-workunit-promote.cjs tests/scripts/test-engine-discovery-session.cjs tests/scripts/test-engine-cache.cjs tests/scripts/test-engine-manifest-io.cjs tests/scripts/test-engine-manifest-fields.cjs tests/scripts/test-engine-boot.cjs tests/scripts/test-migration-orchestrator.cjs tests/scripts/test-migration-047.cjs tests/scripts/test-migration-048.cjs tests/scripts/test-migration-049.cjs tests/scripts/test-migration-050.cjs tests/scripts/test-migration-051.cjs tests/scripts/test-migration-052.cjs tests/scripts/test-engine-specification-projections.cjs tests/scripts/test-engine-discovery-projections.cjs tests/scripts/test-gateway-for-start.cjs tests/scripts/test-gateway-for-specification.cjs tests/scripts/test-gateway-for-workflow-continue-epic.cjs tests/scripts/test-gateway-for-workflow-continue-feature.cjs tests/scripts/test-gateway-for-workflow-continue-bugfix.cjs tests/scripts/test-gateway-for-workflow-continue-quickfix.cjs tests/scripts/test-gateway-for-workflow-continue-cross-cutting.cjs tests/scripts/test-gateway-for-discovery-skill.cjs tests/scripts/test-gateway-for-bridge.cjs tests/scripts/test-reads-derivations.cjs tests/scripts/test-ensure-discovery-item.cjs tests/scripts/test-legacy-research-split.cjs tests/scripts/test-knowledge-chunker.cjs tests/scripts/test-knowledge-config.cjs tests/scripts/test-knowledge-discovery.cjs tests/scripts/test-knowledge-embeddings.cjs tests/scripts/test-knowledge-integration.cjs tests/scripts/test-knowledge-openai.cjs tests/scripts/test-knowledge-openai-compatible.cjs tests/scripts/test-knowledge-openai-integration.cjs tests/scripts/test-knowledge-progress.cjs tests/scripts/test-knowledge-rerank.cjs tests/scripts/test-knowledge-retry.cjs tests/scripts/test-knowledge-setup-forms.cjs tests/scripts/test-knowledge-store.cjs tests/scripts/test-conventions-lint.cjs tests/scripts/test-pipeline-simulation.cjs tests/scripts/test-engine-agent-state.cjs tests/scripts/test-prose-cases.cjs tests/scripts/test-prose-fixtures.cjs", + "test": "node --test tests/scripts/test-render.cjs tests/scripts/test-engine-gateway.cjs tests/scripts/test-engine-epic-projections.cjs tests/scripts/test-engine-start-projections.cjs tests/scripts/test-engine-workunit-projections.cjs tests/scripts/test-engine-discussion-map.cjs tests/scripts/test-engine-discovery-map.cjs tests/scripts/test-engine-tasks.cjs tests/scripts/test-engine-render-surfaces.cjs tests/scripts/test-engine-transactions.cjs tests/scripts/test-engine-workunit-create.cjs tests/scripts/test-engine-workunit-pivot.cjs tests/scripts/test-engine-workunit-absorb.cjs tests/scripts/test-engine-workunit-promote.cjs tests/scripts/test-engine-discovery-session.cjs tests/scripts/test-engine-cache.cjs tests/scripts/test-engine-manifest-io.cjs tests/scripts/test-engine-manifest-fields.cjs tests/scripts/test-engine-boot.cjs tests/scripts/test-migration-orchestrator.cjs tests/scripts/test-migration-047.cjs tests/scripts/test-migration-048.cjs tests/scripts/test-migration-049.cjs tests/scripts/test-migration-050.cjs tests/scripts/test-migration-051.cjs tests/scripts/test-migration-052.cjs tests/scripts/test-engine-specification-projections.cjs tests/scripts/test-engine-discovery-projections.cjs tests/scripts/test-gateway-for-start.cjs tests/scripts/test-gateway-for-specification.cjs tests/scripts/test-gateway-for-workflow-continue-epic.cjs tests/scripts/test-gateway-for-workflow-continue-feature.cjs tests/scripts/test-gateway-for-workflow-continue-bugfix.cjs tests/scripts/test-gateway-for-workflow-continue-quickfix.cjs tests/scripts/test-gateway-for-workflow-continue-cross-cutting.cjs tests/scripts/test-gateway-for-discovery-skill.cjs tests/scripts/test-gateway-for-bridge.cjs tests/scripts/test-reads-derivations.cjs tests/scripts/test-ensure-discovery-item.cjs tests/scripts/test-legacy-research-split.cjs tests/scripts/test-knowledge-chunker.cjs tests/scripts/test-knowledge-config.cjs tests/scripts/test-knowledge-discovery.cjs tests/scripts/test-knowledge-embeddings.cjs tests/scripts/test-knowledge-integration.cjs tests/scripts/test-knowledge-openai.cjs tests/scripts/test-knowledge-openai-compatible.cjs tests/scripts/test-knowledge-openai-integration.cjs tests/scripts/test-knowledge-progress.cjs tests/scripts/test-knowledge-rerank.cjs tests/scripts/test-knowledge-retry.cjs tests/scripts/test-knowledge-setup-forms.cjs tests/scripts/test-knowledge-store.cjs tests/scripts/test-conventions-lint.cjs tests/scripts/test-pipeline-simulation.cjs tests/scripts/test-engine-agent-state.cjs tests/scripts/test-prose-cases.cjs tests/scripts/test-prose-snapshots.cjs", "test:migrations": "for t in tests/scripts/test-migration-*.sh; do bash $t || exit 1; done", "test:cli": "for t in tests/scripts/test-engine-manifest.sh tests/scripts/test-inbox-promotion.sh tests/scripts/test-knowledge-cli.sh tests/scripts/test-knowledge-build.sh; do bash $t || exit 1; done" }, diff --git a/tests/scripts/test-prose-fixtures.cjs b/tests/scripts/test-prose-snapshots.cjs similarity index 100% rename from tests/scripts/test-prose-fixtures.cjs rename to tests/scripts/test-prose-snapshots.cjs From 2f0716bf2ef06a69916529f0fd5bbc3ade41ea22 Mon Sep 17 00:00:00 2001 From: Lee Overy Date: Sat, 25 Jul 2026 20:36:00 +0100 Subject: [PATCH 10/11] docs(prose-tests): cases are hand-written, never generated (P8) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Lee's call, recorded before it becomes tempting: deriving assert.md from a recorded walk would turn the corpus into approval testing — pinning what the prose currently does, defects included, so a mummified bug passes forever. The authoring effort is what finds pre-existing defects; it is the point, not the overhead. Snapshots stay generated because the engine authors them and drift is visible; expectations are written because only a person can say what correct means. Co-Authored-By: Claude Opus 5 --- design/prose-tests.md | 10 ++++++++++ tests/prose/README.md | 8 ++++++++ 2 files changed, 18 insertions(+) diff --git a/design/prose-tests.md b/design/prose-tests.md index abab376cd..106c124c7 100644 --- a/design/prose-tests.md +++ b/design/prose-tests.md @@ -127,6 +127,16 @@ testing. broke, or the world/design moved and the case is stale. The adjudication is the point; only the typo-class staleness is pre-filtered by the perimeter (P3). +- **P8 — cases are hand-written, never generated.** Deriving an + `assert.md` from a recorded walk would make authoring far faster and + would quietly convert the whole corpus into approval testing: every + case would pin what the prose *currently does*, defects included, and + a mummified bug passes forever. The authoring effort — reading the + prose closely enough to state what it *should* do — is the part that + finds pre-existing defects, as it did twice in the first corpus. It + is the point, not the overhead. Snapshots are generated because the + engine authors them and drift is visible; expectations are written + because only a person can say what correct means. ## Architecture diff --git a/tests/prose/README.md b/tests/prose/README.md index 0bea54d01..e2c7d2cf4 100644 --- a/tests/prose/README.md +++ b/tests/prose/README.md @@ -43,6 +43,14 @@ a deduplicated fixture library you have to chase across the tree. ## Writing a case +**Never generate a case from a walk.** Recording what the prose did and +calling it the expectation turns the corpus into approval testing — +every case pins current behaviour, defects included, and a mummified bug +passes forever. Snapshots are generated because the engine authors them +and drift is visible; expectations are written by hand because only a +person can say what correct means. The reading-closely part is where +pre-existing defects surface. + **The act stays coarse — one instruction.** Where to enter, what to follow, where to stop. Never a step-by-step script of the workflow: if the case tells the walker which commands to run, the walker no longer has From 68c6018a41cb02dd04a1dbbbf685e109897d0780 Mon Sep 17 00:00:00 2001 From: Lee Overy Date: Sat, 25 Jul 2026 21:11:28 +0100 Subject: [PATCH 11/11] fix(prose-tests): a walk that began mid-flow proves nothing MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Four real runs exposed the framework's worst failure mode: three walkers skipped straight into the middle of a flow — one began at the gateway call instead of initialisation, another at project-skills discovery, past the entire entry skill. Both then "failed" at steps the walk never reached. Lee's read is the decisive one: he has never seen the project-skills prompt in real use, so a live session very likely course-corrects past that guard exactly as our walker did — which means every finding from those runs is unsafe until the walker is disciplined. The walker now starts at the entry point the task names and performs every step including ones whose effect the world already holds ("already booted", "the plan already exists" are named as the reasoning that invalidates a walk). The asserter returns INVALID WALK when a transcript opens mid-flow and judges nothing further; the orchestrator retries once, then reports INVALID rather than FAIL. A false finding is worse than no finding: it spends attention on prose that was never exercised and teaches distrust of the runs that are real. Co-Authored-By: Claude Opus 5 --- .claude/agents/prose-asserter.md | 17 +++++++++++++++-- .claude/agents/prose-orchestrator.md | 14 +++++++++++--- .claude/agents/prose-walker.md | 11 +++++++++++ design/prose-tests.md | 10 ++++++++++ 4 files changed, 47 insertions(+), 5 deletions(-) diff --git a/.claude/agents/prose-asserter.md b/.claude/agents/prose-asserter.md index 72439d1f7..117b34871 100644 --- a/.claude/agents/prose-asserter.md +++ b/.claude/agents/prose-asserter.md @@ -38,6 +38,19 @@ tools you appear to have or what mode you are running in. by nature and are immaterial. Differences in shape, field presence, status vocabulary, or content are material. +## An invalid walk is not a failing walk + +Before judging anything, check where the transcript begins. If its first +entry is not the start of the walk the expected path describes — the +walker jumped into the middle of the flow, skipping earlier steps +without recording a `DEVIATION` for each — then the walk cannot evidence +the path, and judging it would report a prose failure that was never +demonstrated. + +In that case return `VERDICT: INVALID WALK`, name the entry the +transcript opens on and the step it should have opened on, and judge +nothing further. This is a fault in the walk, not in the prose. + ## Verdict Return exactly this and nothing else: @@ -51,5 +64,5 @@ Return exactly this and nothing else: 3. **Markers** — list every `UNSCRIPTED QUESTION`, `AMBIGUOUS`, and `DEVIATION` in the transcript. Each is a finding in its own right, even when everything else passed. -4. **VERDICT** — PASS or FAIL, then one sentence on what it means for the - prose. +4. **VERDICT** — PASS, FAIL, or INVALID WALK, then one sentence on what + it means for the prose. diff --git a/.claude/agents/prose-orchestrator.md b/.claude/agents/prose-orchestrator.md index ae53106e7..7b4dde493 100644 --- a/.claude/agents/prose-orchestrator.md +++ b/.claude/agents/prose-orchestrator.md @@ -62,7 +62,14 @@ verdict unreliable. - The second run PASSes → a non-reproducing failure. Report it as `FLAKY`, quoting both, and resolve nothing yourself. -5. **Destroy the world** — `node tests/prose/run.cjs destroy --world `. +5. **Retry an invalid walk** — if the verdict is `INVALID WALK`, the + walker began mid-flow and the case was never actually exercised. + Destroy the world, build a fresh one, and repeat steps 2 and 3 once. + If the second walk is also invalid, report `INVALID` — never `FAIL`. + A case that was not walked properly has said nothing about the prose, + and reporting it as a prose failure would be a false finding. + +6. **Destroy the world** — `node tests/prose/run.cjs destroy --world `. ## What you return @@ -70,11 +77,12 @@ Return exactly this and nothing else: ``` CASE: -VERDICT: PASS | FAIL | FLAKY +VERDICT: PASS | FAIL | FLAKY | INVALID PATH: / steps passed WORLD: PASS | FAIL MARKERS: -EVIDENCE: +EVIDENCE: ``` No transcripts, no prompts, no commentary, no recommendations. diff --git a/.claude/agents/prose-walker.md b/.claude/agents/prose-walker.md index 46dad2afa..b6ed011df 100644 --- a/.claude/agents/prose-walker.md +++ b/.claude/agents/prose-walker.md @@ -22,6 +22,17 @@ any harness substitutions. Follow it exactly. ## Rules +- **Start at the beginning.** Your first transcript entry is the first + instruction of the entry point the task names — its opening step, not + the first step that looks interesting or relevant. Reading ahead to + plan is fine; beginning ahead is not. +- **Perform every step, including the ones that look unnecessary.** A + world that already holds the state a step would produce is not a + reason to skip it: run it anyway and log it anyway. "Already booted", + "already migrated", "the plan already exists", "nothing to do here" — + each of those is the reasoning that invalidates a walk. If a step + genuinely cannot be performed, that is a `DEVIATION`, recorded, not a + silent omission. - Follow the prose literally, step by step, arm by arm. Where it names an engine or knowledge call, run it from the project directory and use the real response to decide which arm applies. Never predict a response. diff --git a/design/prose-tests.md b/design/prose-tests.md index 106c124c7..88e2541fd 100644 --- a/design/prose-tests.md +++ b/design/prose-tests.md @@ -110,6 +110,16 @@ testing. failing case re-runs once from a fresh world at the same models. A defect in the prose reproduces; a one-off does not, and is reported as FLAKY with both runs quoted. Nothing is auto-resolved. +- **P6d — a walk that began mid-flow proves nothing.** Walkers skip + steps whose effect the world already holds — "already booted", "the + plan already exists" — and then the case reports a prose failure at a + step the walk never reached. So the walker starts at the entry point + the task names and performs every step including the redundant ones, + the asserter returns `INVALID WALK` when the transcript opens + mid-flow, and the orchestrator retries once before reporting + `INVALID` — never `FAIL`. A false finding is worse than no finding: + it spends human attention on prose that was never exercised, and it + teaches distrust of the runs that are real. - **P6b — a nested agent per case.** The `/prose-test` skill dispatches one **prose-orchestrator** per case, which builds the world, dispatches **prose-walker**, computes the delta, dispatches **prose-asserter**,