Skip to content

Commit 5909307

Browse files
os-zhuangclaude
andauthored
docs(qa): rule 2 carries the access-control disclosure carve-out at its source (#9506)
RUNNER.md rule 2 defined a completed `fail` verdict as one whose issue carries a reproduction rule, with no exception. The checklist-test skill guardrail forbids publishing a reproduction for an authentication or authorization hole anywhere on GitHub. A runner obeying both had to choose between a complete verdict and not publishing an exploit, and the conflict resolved only by precedence asserted from the skill side. Rule 2 now states the carve-out itself: an auth/authz `fail` is complete when it records the item id, the clause and `detail withheld pending maintainer`, with the reproduction held in the runner session. Force is unchanged for every other failure class. Wording is semantically the same rule the merged skill guardrail states, so neither document depends on the other's precedence. Three later restatements of the same obligation in this file now point at the carve-out instead of repeating it unconditionally: the run-record report contents, the issue-body composition line, and the extraction obligation — the last being the public-card shape the original disclosure travelled through. Maintainer disclosure ruling, 2026-08-18, recorded on #9387. Claude-Session: https://claude.ai/code/session_01Rn7aaamsR99FXRqLcpL99q Co-authored-by: Claude <noreply@anthropic.com>
1 parent 6ce8feb commit 5909307

1 file changed

Lines changed: 22 additions & 6 deletions

File tree

docs/qa/platform-checklist/RUNNER.md

Lines changed: 22 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -55,6 +55,18 @@ test-run output the clause's `evidence` field names.
5555
issue. A `fail` with no reproduction rule in its issue is not a completed verdict.
5656
(The screenshot that convinced you is a live judgment aid, not report content —
5757
describe what it showed in one line; never attach it.)
58+
- **Carve-out — authentication and authorization findings only.** ⛔ Never publish a
59+
reproduction for an authentication or authorization hole **anywhere on GitHub**: not
60+
the run issue, not a tracking or extracted card, not a comment. Relocating it is not a
61+
mitigation — a tracking card is a public issue in a public repo just the same. Such a
62+
`fail` **is a completed verdict** when it records the item id, the clause, and `detail
63+
withheld pending maintainer`; the reproduction stays in the runner session, and the
64+
runner stops there and waits for the maintainer. Existence published, recipe withheld,
65+
is a complete and actionable report — getting a defect fixed never requires handing
66+
anyone a working exploit. (Maintainer disclosure ruling, 2026-08-18, recorded on
67+
#9387; the `checklist-test` skill states the same guardrail in the same terms — one
68+
rule written in both places, not a precedence claim by either.) **No other failure
69+
class is softened by this**: everything else owes its reproduction rule in full.
5870
3. **Classify blockers honestly.** Missing seed/persona/fixture → `blocked(fixture)`,
5971
and *record the gap on the item* (`fixtures.knownGaps` or `blocked`) so the next
6072
sweep doesn't rediscover it. A defect in the fixture itself (seed silently failing,
@@ -213,7 +225,9 @@ in the repo** — not the JSON, not screenshots; `runs/` is git-ignored except i
213225
reach a verdict — never report artifacts. What the report carries for a defect is the
214226
**reproduction rule**: ordered steps / API calls (method · path · body) / the
215227
ref-targeted selector path + expected-vs-actual from the oracle, enough to re-hit it on a
216-
fresh boot with no picture. A clause whose oracle was a screenshot is recorded as a
228+
fresh boot with no picture — **except under rule 2's authentication/authorization
229+
carve-out**, where the report carries the item, the clause and `detail withheld pending
230+
maintainer`, and nothing else. A clause whose oracle was a screenshot is recorded as a
217231
one-line text description of what it showed, not a link.
218232

219233
The in-environment JSON scratch (RUNNER shape, never committed):
@@ -246,10 +260,10 @@ The in-environment JSON scratch (RUNNER shape, never committed):
246260
```
247261

248262
The issue body is: env fingerprint · scope (selector + per-item `revision`) · the
249-
per-clause verdict table (text oracle evidence) · a reproduction rule per `fail` ·
250-
derived item verdicts + fixture gaps. The durable, version-controlled truth is still the
251-
checklist under `areas/`; a run is a dated assertion about one build, and it lives in its
252-
issue, not the tree.
263+
per-clause verdict table (text oracle evidence) · a reproduction rule per `fail` (rule 2's
264+
carve-out excepted) · derived item verdicts + fixture gaps. The durable,
265+
version-controlled truth is still the checklist under `areas/`; a run is a dated assertion
266+
about one build, and it lives in its issue, not the tree.
253267

254268
### Extraction obligation — the run record is a protocol carrier, not work
255269

@@ -264,7 +278,9 @@ closing out the report includes the extraction, owed by the runner:
264278
- **Product defects found during the run**: at close-out, extract each one into its own
265279
standalone issue — self-contained title, reproduction and mechanism itemized in the
266280
card, a pointer back to the run record for the full evidence chain. A defect card does
267-
not carry `qa-run`; it enters triage first-touch normally.
281+
not carry `qa-run`; it enters triage first-touch normally. ⛔ An authentication or
282+
authorization defect is extracted under rule 2's carve-out — item, clause, `detail
283+
withheld pending maintainer`, no recipe. The extracted card is public too.
268284
- **Checklist-accuracy findings and fixture gaps** close out through the wave's anchor
269285
card (the sweep's tracking issue) — ⛔ not extracted.
270286
- **Environment blockers**: recorded in the run record is enough.

0 commit comments

Comments
 (0)