From 5a04f92a5613844d6363efba57181149f5378cfc Mon Sep 17 00:00:00 2001 From: firaen22 Date: Fri, 24 Jul 2026 23:23:46 +0800 Subject: [PATCH] ground-truth-gates: a gate over hardcoded facts asserts the facts, not just the shape MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit New rule 6 under 'What makes a gate real'. When code embeds domain constants (holiday dates, rates, valid-state sets), a structure-only suite passes whether the values are right or a later edit corrupted one. The values were cross-checked once — but in the conversation, which evaporates at session end. Anchor the load-bearing values as assertions from the authority you already consulted, so a future silent constant change fails. This is item 2's change-detector carve-out used correctly (the value IS the contract). Class-distilled from a mining pass over the owner's own sessions (no code taken): a 'tests check shape, not date-truth' cross-model finding on hardcoded holiday dates. Co-Authored-By: Claude Fable 5 --- skills/ground-truth-gates/SKILL.md | 34 ++++++++++++++++++++++++++++++ 1 file changed, 34 insertions(+) diff --git a/skills/ground-truth-gates/SKILL.md b/skills/ground-truth-gates/SKILL.md index f69a1b9..53452a8 100644 --- a/skills/ground-truth-gates/SKILL.md +++ b/skills/ground-truth-gates/SKILL.md @@ -269,6 +269,32 @@ A generic green test is not proof. A gate is real only if: elsewhere — is legitimate as-is and needs no anchor, as does an intrinsic gate like a syntax or forbidden-character scan. The rule bites only when agreement is made to stand in for an external fact.) +8. **A gate over hardcoded facts asserts the facts, not just the shape — and + the cross-check that established them belongs IN the fixture, not in the + chat** (`unprobed` — private incident as shape; see Provenance). When code + embeds domain constants (holiday dates, a tax rate, a fee schedule, the set + of valid states), a suite that checks structure — the array is non-empty, + each entry parses, the shape is right — passes identically whether the + values are correct or a later edit corrupted one. Those values were usually + cross-checked once, against an authority or several independent sources or a + reviewer's recall — but that check happened in the conversation and + evaporates when the session ends, so the next bad edit sails through a + shape-only gate. Anchor the fact: assert the specific load-bearing values + (a fixed holiday falls on its known date, the standard rate equals the + published number), from the authority you already consulted, so a future + silent change to a constant fails. This *extends* item 2's carve-out — + from an output-interface string to an embedded input constant — and is NOT + item 3's "hardcoded expected value" fake pass: the anchor's value comes from + an external authority, not copied back from the code's own output. Item 3's + tautology asserts the source is the source; this asserts the source matches + the world. It shares rule 7's remedy — an external anchor — but not its + trigger: rule 7 bites where *agreement between artifacts* is made to stand + in for an external fact, this one where *structure* is. A fact that + legitimately changes gets the value versioned (the regenerate-and-diff rule + below), an always-fixed one is cheap to anchor permanently. + ❌ "the holiday tests pass" — they assert the list has the right count and + types, never that any date is the right day; a fat-fingered edit to one + date stays green. **A red result is not automatically a real defect** — but ruling one "environmental" is a gate change, not the worker's call (rule 4): quarantine it @@ -446,5 +472,13 @@ unless the project opted in (MIT, ideas only), corroborated by NYCU-Chung/my-claude-devteam's bypassPermissions-hook framing (MIT) — it matches this pack's own no-auto-registered-hooks invariant. All three ship `unprobed` per the covenant; their probes join the private round-5 queue. +Rule 8 (fact-anchoring; 2026-07-24) is class-distilled from a mining pass +over the owner's own sessions (no code taken): hardcoded public-holiday dates +were cross-checked against four independent sources, but a cross-model +reviewer noted the suite validated structure only — "tests check shape, not +date-truth" — so the truth evidence lived in the conversation, not the gate; +the fix added anchor-date assertions as a regression guard against a future +bad edit. Ships `unprobed` per the covenant; its probe joins the private +round-5 queue. `template/` scripts are self-contained (Node + bash, zero deps) and were run green on 2026-07-06 with Node v23; re-verify with `bash template/run-all.sh`.