From 4844c2ef02d024edd4af1d5941d80790de5de9a0 Mon Sep 17 00:00:00 2001 From: Derek Gulbranson Date: Thu, 3 Sep 2026 00:34:44 -0700 Subject: [PATCH 01/23] docs(design)+tooling(differential): the worker pass is sub-second, not multi-minute "The multi-minute worker pass" entered on 2026-08-05 in 7767ba2 at one site, with no measurement recorded beside it, and was copied outward until eight sites carried it -- four in compare.py, three in test_differential.py, and a Declined bullet in decisions.md that gave it as "the whole of the reason" for declining precise per-name contest detection. Measured 2026-09-03 over the whole corpus: a run is 0.33s wall at baseline 1.4.0 and 0.57s at 2.2.0, the default, of which the worker pass is 0.10s and 0.29s; all four baselines back to back total 1.97s; a run with uv's cache emptied first, so the pinned wheel is downloaded, is still under a second. Wrong by orders of magnitude. The three compare.py comments and the two test docstrings that used the phrase were making an ORDERING argument -- a refusal before the worker beats one after it. That argument stands and is kept; it is restated on what is actually true, that a refusal after the worker lands below the run's own `baseline:` line, having already published a comparison the run was about to disown. The Declined bullet is rewritten to rest on COVERAGE, which is true: a run-time check sees one ledger and only when somebody invokes the tool, while the static predicate covers a rule from the moment it is written. Every figure carries its date and a recompute. Prose only -- no code, rule or test behavior changed. Refs #497 Co-Authored-By: Claude Opus 5 --- docs/design/decisions.md | 2 +- tests/v2/test_differential.py | 27 +++++++++++++--------- tools/differential/compare.py | 42 ++++++++++++++++++++++++++--------- 3 files changed, 50 insertions(+), 21 deletions(-) diff --git a/docs/design/decisions.md b/docs/design/decisions.md index 9516d56d..c2a6620f 100644 --- a/docs/design/decisions.md +++ b/docs/design/decisions.md @@ -783,7 +783,7 @@ Declined: - Reordering the eleven (2026-09-02) — the obvious way to make the check unnecessary, and wrong twice. Reordering moves which rule classifies a name, which is a behavior change to the ledger and breaks `_CROSS_RULE_WINNERS`; and no pair is even a reorder CANDIDATE, because in all eleven the narrower rule's own prose would be FALSE or MERELY PARTIAL of the co-matched names if it won, and either disqualifies it as classifier-of-record. Partiality is the commoner half and the arc's load-bearing distinction — compound versus component, which the `fix(#400/#274)`/`fix(#400)` exemption calls "the canonical compound-versus-component shape". Read the eleven `why` texts for it: `fix(#400)` "says nothing about a maiden marker"; a widened `fix(comma-family) lone post-comma piece routes to suffix/title` "would take the union and report only half of it"; `fix(cjk-glued-honorific-peel)` says "nothing about a division"; `feat(#273)`'s prose "has nothing to say about" the remainder rejoining as a name; and two of the three regex-accident pairs are partial rather than false, the `fix(suffix-routing)` jr and numeral rules genuinely naming the `{family, suffix}` half of `김민준씨 Jr.` and of `田中さん II` and describing neither the peel nor the segmentation that moves `given`. Falsity is the minority, and one of the eleven says so of itself: the exemption `fix(comma-family) a comma followed only by titles keeps the given/family split` carries over `fix(comma-precomma-family)` calls its own pair "the one where the narrower rule would be actively WRONG rather than merely partial", and adds that the claim "is about THIS name and is deliberately not generalised". `马丁·路德·金씨` is the clearest compound, and `王先生, V.` the reverse case, where the winner is already the least wrong reading available — and stays so, since the family-side twin that would describe it was weighed as #496 and declined (the next bullet). - A family-side twin of `fix(cjk-comma-honorific-peel)` for a PRE-comma glued honorific (2026-09-02, [#496](https://github.com/derek73/python-nameparser/issues/496), closed as not planned) — the rule that would honestly describe `王先生, V.`, `田中さん, V.` and `김민준씨, V.`, whose diff is the glued peel alone while the label they carry says comma compound (the finding bullet above). Declined on two grounds. FIRST, no gate can demand it: all 17 names the compound/peel pair is contested across measure radar — but by two different warrants, and an audit that assumes one will find the count short. Sixteen sit in `corpus_cjk_tolerated.jsonl`, the file #488 created by demoting the composed comma/Latin-wrapper CJK forms to tolerated input, `김, 민준씨` — the name #382 was filed over — among them. The seventeenth, `Dr 田中さん, V.`, is in `corpus_issues.jsonl` alone: harvested and append-only, radar since #468, and untouched by #488. (`Dr 김민준씨, Jr.` is in both files, and reads radar from either.) Radar means an unmatched diff is reported rather than fatal for want of a RULE; it is not an unconditional never-fatal, since a `[[never]]` exclusion outranks the tier and stays fatal on a radar name — `_CORPUS_TIERS`' own note, and `main()`'s two-reasons comment, which routes an excluded name to `unexplained` rather than to `radar`. No exclusion refuses these, so nothing here can demand the twin. SECOND, [#495](https://github.com/derek73/python-nameparser/issues/495) points the other way over the same corner: it asks whether the radar-only rules already in the ledger still earn their place and records `fix(cjk-comma-compound)` as having zero contract-tier reach, so #496 proposed a SECOND rule for the very names #495 is weighing a first one away from. Fewer rules for this corner is the coherent direction, and that is what makes the decline decisive rather than merely permissive. REOPEN it if a pre-comma glued-honorific name ever reads contract — and watch BOTH routes, because the two files promote by different mechanisms. A `corpus_cjk_tolerated.jsonl` name is promoted by clearing `tolerated` on its case rows, which moves the text into `corpus_cjk.jsonl` (`_CORPUS_TIERS`, and build_cjk_corpus.py's split). `Dr 田中さん, V.` has no case row at all, so there is no flag to clear: it changes tier only by being CHOSEN — a new unmarked row, or a rules.md example — which puts the text in a contract corpus, and the (name, order) dedup loads contract files first and keeps that reading. Read the tier off `_CORPUS_TIERS` either way, never off a hand-built file→tier map (the roster caution above). -- Precise per-name contest detection at differential-run time (2026-09-02) — it would replace the static predicate's over-reporting with the measured nine-of-eleven split, and it needs the pinned-wheel worker pass to do it. That puts the check behind a multi-minute run, so a rule added by a later bundle would go unchecked at pytest speed — which is the whole point of #382, and is the whole of the reason. Do NOT restate this as an error-direction argument: within the NESTED pairs computing real diffs can only remove them, but the predicate is not a superset of the contests, since nesting is sufficient for one and not necessary (the bullet above, and #498). What the static check buys is cheapness and coverage of a rule nobody has run the wheel against, not a guarantee of refusing everything it should. +- Precise per-name contest detection at differential-run time (2026-09-02; its stated reason replaced 2026-09-03, [#497](https://github.com/derek73/python-nameparser/issues/497)) — it would replace the static predicate's over-reporting with the measured nine-of-eleven split, and it needs the pinned-wheel worker pass to do it. The decline stands, on COVERAGE alone: a run-time check sees one ledger — the one for the baseline that run was invoked with — and only once somebody invokes the tool, whereas the static predicate covers a rule from the moment it is written, including a rule nobody has ever run the wheel against, which is the whole point of #382. Nor can the unit suite simply adopt the precise check: `tests/v2/test_differential.py` is deliberately hermetic — its own header says "nothing here spawns `uv` or the network", and the tests that need a baseline fake `_run_worker` or `Popen` outright — so moving it to pytest speed means spawning a subprocess and, on a cold cache, reaching PyPI. WITHDRAWN, and named here so it is not re-argued: this bullet used to say the check would sit "behind a multi-minute run" and gave that as "the whole of the reason". No measurement was recorded beside it: the phrase entered on 2026-08-05 in `7767ba2` at a single site in `tools/differential/compare.py`, and was copied outward until eight sites carried it — four there, three in `tests/v2/test_differential.py`, and this bullet. Measured 2026-09-03 over the corpus population named above, a whole run is 0.33s wall at baseline 1.4.0 and 0.57s at 2.2.0, the default; the worker pass is 0.10s and 0.29s of those, timed inside `main()`. All four baselines back to back total 1.97s, and a run with `uv`'s cache emptied first, so the pinned wheel is downloaded rather than reused, is still under a second. Read the class and not the digits: the worker pass is a fraction of a second and the whole gate is seconds at most, where the withdrawn phrase said minutes — so no argument here may be built on what the worker pass costs. RECOMPUTE: `time (uv run python tools/differential/compare.py --baseline 1.4.0 >/dev/null)`. For the worker's share, wrap `compare._run_worker` in a timer and then call `main()` — calling `_run_worker` directly on every loaded entry aborts at a 1.4.0 baseline, since the seven order-bearing shape-4/5 entries have to be dropped by the baseline-minimum skip first. Do NOT restate the decline as an error-direction argument either: within the NESTED pairs computing real diffs can only remove them, but the predicate is not a superset of the contests, since nesting is sufficient for one and not necessary (the bullet above, and #498). What the static check buys is cheapness and coverage of a rule nobody has run the wheel against, not a guarantee of refusing everything it should. - Giving the peel rule a predicate the compound rule fails (2026-09-02, #382 option 1) — narrowing `fix(cjk-comma-honorific-peel)`'s `name_regex` to the honorific-bearing shapes, whose stated effect in #382 was that the pair "becomes order-independent and the original contract holds again". Declined because that effect is UNREACHABLE, and measured rather than argued. The two rules ship a BYTE-IDENTICAL `name_regex` (verified 2026-09-02 by comparing the two strings), and `_rule_reach` computes each rule's names from its OWN pattern, so narrowing the peel rule's regex narrows the peel rule's reach and nothing else: the compound rule goes on reaching every one of the names. The general form, which no regex edit escapes — where the earlier rule's `fields` are a SUBSET of the later one's, order-independence requires the LATER rule to stop reaching the name, and an edit to the earlier rule's own regex cannot cause that. Here the peel rule's `{given, suffix}` is a strict subset of the compound rule's `{family, given, suffix, title}`, so every diff the peel rule admits the compound rule admits too. Implemented to check it: narrowing the peel regex to a `GLUED_HONORIFICS` alternation keeps all seven of the co-matched names whose 1.4.0 diff is exactly `{given, suffix}`, and swapping the two rules in the NARROWED ledger still reattributes all seven. So the pair keeps the arrangement the arc settled on, and that is the answer rather than a cost trade: it is narrow-first, the declaration-free default, so it appears in none of the eleven contests `order_contests` reports and owes no `precedes_narrower` block, and all seven names are pinned by name in `_CROSS_RULE_WINNERS` (23 corpus names reach both regexes; the seven are the ones #375's reorder mutation moves). The cost stands as a second reason and not the first: the narrowing would hand-copy more honorific vocabulary into the pattern and grow the `_HONORIFIC_SOURCES` sync-roster surface (mechanisms.md#CURATED-VOCABULARY-ALTERNATION's second half). Option 2's convention plus a check on the wide-first exceptions to it is what the arc took, and option 3 it REFUSED on a false premise (the finding above). One thing this does not settle, and the ledger comment on the peel rule overstates it: `fields` separates the two only for the UNION rows whose diff includes `family`; for those seven it is file order that decides. - Scoping the check to contract-tier contests only (2026-09-02) — five of the eleven are contested only over radar names, so this would have cut the file's exemptions by nearly half. Declined because `_CROSS_RULE_WINNERS` already pins radar names — measured, most of the names it pins are radar-tier (22 of 33 today) — so the repo would be inconsistent with itself about whether a radar contest matters. Whether the radar-only CJK comma rules still earn their place after #488's demotion is a real question and is filed as #495; it is a question about those rules, not about the check. - A shared name-population function for both callers (2026-09-02, the spec's own first sketch) — see the two-populations decision above. `main()` and the unit guard must read different populations, so one function would break `--corpus`; they agree by a docstring that names the function it mirrors instead. diff --git a/tests/v2/test_differential.py b/tests/v2/test_differential.py index 1eabbf89..976e5583 100644 --- a/tests/v2/test_differential.py +++ b/tests/v2/test_differential.py @@ -1759,8 +1759,11 @@ def test_main_checks_contests_against_the_names_it_loaded() -> None: The unit guard is what catches a new rule at pytest speed; this is the belt for a run over a --corpus the guard never sees, and it has - to fire early -- after the multi-minute worker pass, the reader has - already paid for the answer. + to fire early -- a refusal raised after the worker pass prints + below the run's own `baseline:` line, so the reader has been shown + work done under a ledger the run was about to reject. The ordering + is the argument; the "multi-minute" cost this docstring used to + give for it was withdrawn as unmeasured (#497). """ src = (compare.HERE / "compare.py").read_text(encoding="utf-8") body = src[src.index("def main("):] @@ -2144,10 +2147,10 @@ def test_object_corpus_lines_are_read_and_labels_printed( def test_a_malformed_tests_label_is_a_hard_error( tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None: """Caught at load time, not report time: a 'tests' label is read - only when printing the radar block, after the multi-minute worker - pass, so a bad one left unchecked would crash there instead -- - the failure mode validate_rules' compile-at-startup paragraph - exists to avoid.""" + only when printing the radar block, at the end of the run and well + past the worker pass, so a bad one left unchecked would crash there + instead -- the failure mode validate_rules' compile-at-startup + paragraph exists to avoid.""" import json as _json corpus = tmp_path / "corpus_x.jsonl" corpus.write_text(_json.dumps( @@ -2563,10 +2566,14 @@ def test_run_worker_sends_the_name_and_resolved_order_on_the_wire( @pytest.mark.parametrize("want_v2", [True, False]) def test_worker_source_compiles(want_v2: bool) -> None: - """A syntax error in the rendered template currently surfaces only - as 'worker exited 1' after a multi-minute uv install; this catches - it at test time instead, for both renderings (WANT_V2 gates a - def-inside-if that is easy to misindent).""" + """A syntax error in the rendered template surfaces only as the + opaque 'worker exited 1', and by the time _run_worker raises it the + TemporaryDirectory holding the rendered source is already gone, so + there is nothing left to open; this catches it at test time + instead, for both renderings (WANT_V2 gates a def-inside-if that is + easy to misindent). The install this used to call "multi-minute" is + sub-second even on a cold uv cache -- withdrawn as unmeasured by + #497; the opacity is the reason, not the wait.""" compile(compare._worker_source("2.2.0", want_v2=want_v2), "", "exec") diff --git a/tools/differential/compare.py b/tools/differential/compare.py index 39c5edc9..8d88758e 100644 --- a/tools/differential/compare.py +++ b/tools/differential/compare.py @@ -709,8 +709,9 @@ def validate_rules(rules: list[dict[str, object]], ledger: str) -> None: Rules are also compiled here, not at match time. `classify` returns on the first match, so an uncompilable pattern at position k only raises once a diff gets past rules 1..k-1: a ledger can run green - for months and then explode mid-run, after the multi-minute worker - pass, in a traceback naming neither the file nor the rule. + for months and then explode mid-run, past the worker pass and deep + into the comparison, in a traceback naming neither the file nor + the rule. `ledger` is named rather than hardcoded because there is one per baseline now: a message naming the wrong file sends the reader to @@ -1618,10 +1619,11 @@ def _load_entries(path: Path) -> list[dict[str, object]]: lines, the other three are still bare strings, and both shapes stay legal everywhere a corpus line is read. - A "tests" label is read only when the radar block prints, well - after the multi-minute worker pass, so a malformed one left - unchecked would crash there rather than here -- exactly what - validate_rules' compile-at-startup paragraph exists to prevent. + A "tests" label is read only when the radar block prints, at the + very end of the run and well past the worker pass, so a malformed + one left unchecked would crash there rather than here -- exactly + what validate_rules' compile-at-startup paragraph exists to + prevent. "shape" is checked for a different hazard. main() resolves it against shapes.py into the "order" the worker protocol sends, and @@ -1816,8 +1818,14 @@ def main() -> int: # before any corpus is read: whether two rules CONTEST a diff is a # question about NAMES -- both regexes have to reach one -- and the # names arrive at this line. Before the worker pass, deliberately: - # a ledger refused after the multi-minute wait is a ledger refused - # too late (#382). + # a ledger refused after the worker runs is a ledger refused below + # its own `baseline:` line, so the run has already reported work + # done under rules it was about to call inadmissible (#382). It is + # the ORDER of the two that earns this placement and not the wait + # -- the worker pass is a tenth of a second, and the "multi-minute" + # this comment used to argue from was withdrawn as unmeasured + # (#497; the figures and their recompute are at the tree check + # further down this file). # # The names are the LOADED entries, not the corpus*.jsonl glob the # unit guard in tests/v2/test_ledger_guards.py reads: `--corpus` @@ -1962,8 +1970,22 @@ def main() -> int: # The tree is checked BEFORE the worker runs. It depends on nothing # the worker produces, and validate_rules' own reasoning applies: a - # misconfiguration that aborts after a full uv-install-plus-751-name - # pass costs minutes to learn what costs a second here. + # misconfiguration that aborts after the whole install-and-compare + # pass is one the run reports below its own `baseline:` line, + # having already published a comparison it was about to disown. It + # is the ORDER that earns the placement, not the clock. This + # comment used to say that pass "costs minutes"; the claim was + # never measured, entered on 2026-08-05, and had been copied to + # seven other sites -- three more here, three in + # tests/v2/test_differential.py, and a Declined bullet in + # docs/design/decisions.md that rested a decline on it -- before + # #497 withdrew it. Measured 2026-09-03 over the whole corpus: a + # run is 0.33s wall at baseline 1.4.0 and 0.57s at 2.2.0, the + # default, of which the worker pass is 0.10s and 0.29s; all four + # baselines back to back total 1.97s; and a run with uv's cache + # emptied first, so the pinned wheel is downloaded rather than + # reused, is still under a second. RECOMPUTE: time (uv run python + # tools/differential/compare.py --baseline 1.4.0 >/dev/null) import nameparser # the working tree -- verified, not assumed from nameparser import HumanName tree_at = _check_tree(nameparser.__file__) From 40c82a704b7a752d2f0fe0fab442e351e315b823 Mon Sep 17 00:00:00 2001 From: Derek Gulbranson Date: Thu, 3 Sep 2026 00:50:47 -0700 Subject: [PATCH 02/23] docs(design)+tooling(differential): one dated home for the run figures, not two Review of 4844c2e caught four instances of the defect that commit exists to remove. The figures lived verbatim in both compare.py and decisions.md, both dated, with nothing linking them -- the ledger's own "two copies means one gets updated and the other does not". decisions.md keeps the full set, both recompute recipes and the _run_worker timing trap, since AGENTS.md puts dated measurements there by convention; the compare.py block is reduced to the argument (order, not clock), the class claim, and a citation by heading, as compare.py already cites decisions.md#H1 and "the initials view" elsewhere. The same block claimed to be where "multi-minute" entered. git log -S says otherwise: 7767ba2 seeded it in validate_rules' docstring and 016fead copied it here the same day, which the block's own opening concedes ("validate_rules' own reasoning applies"). Claim dropped rather than restated. "The worker pass is a tenth of a second" was the 1.4.0 figure quoted at a site that never names a baseline; at the 2.2.0 default it is 0.29s. Now "a fraction of a second", which is true at every baseline. The bullet's closing sentence still sold the static check on "cheapness" -- the very axis the bullet had just retired. Dropped. Also: two sites described the counterfactual late refusal as one that had published a comparison. At that position only corpora:/tree:/ baseline: have printed and the classify loop has not run. What is really spent is a wheel installed and the whole corpus parsed for a comparison that will never be made, under a published baseline: header. The twin at the tree check was accurate and is left alone. Gate re-run at all four baselines: unexplained 0, exit 0 at each. Refs #497 Co-Authored-By: Claude Opus 5 --- docs/design/decisions.md | 2 +- tests/v2/test_differential.py | 11 +++++----- tools/differential/compare.py | 40 +++++++++++++++-------------------- 3 files changed, 24 insertions(+), 29 deletions(-) diff --git a/docs/design/decisions.md b/docs/design/decisions.md index c2a6620f..b8304e9a 100644 --- a/docs/design/decisions.md +++ b/docs/design/decisions.md @@ -783,7 +783,7 @@ Declined: - Reordering the eleven (2026-09-02) — the obvious way to make the check unnecessary, and wrong twice. Reordering moves which rule classifies a name, which is a behavior change to the ledger and breaks `_CROSS_RULE_WINNERS`; and no pair is even a reorder CANDIDATE, because in all eleven the narrower rule's own prose would be FALSE or MERELY PARTIAL of the co-matched names if it won, and either disqualifies it as classifier-of-record. Partiality is the commoner half and the arc's load-bearing distinction — compound versus component, which the `fix(#400/#274)`/`fix(#400)` exemption calls "the canonical compound-versus-component shape". Read the eleven `why` texts for it: `fix(#400)` "says nothing about a maiden marker"; a widened `fix(comma-family) lone post-comma piece routes to suffix/title` "would take the union and report only half of it"; `fix(cjk-glued-honorific-peel)` says "nothing about a division"; `feat(#273)`'s prose "has nothing to say about" the remainder rejoining as a name; and two of the three regex-accident pairs are partial rather than false, the `fix(suffix-routing)` jr and numeral rules genuinely naming the `{family, suffix}` half of `김민준씨 Jr.` and of `田中さん II` and describing neither the peel nor the segmentation that moves `given`. Falsity is the minority, and one of the eleven says so of itself: the exemption `fix(comma-family) a comma followed only by titles keeps the given/family split` carries over `fix(comma-precomma-family)` calls its own pair "the one where the narrower rule would be actively WRONG rather than merely partial", and adds that the claim "is about THIS name and is deliberately not generalised". `马丁·路德·金씨` is the clearest compound, and `王先生, V.` the reverse case, where the winner is already the least wrong reading available — and stays so, since the family-side twin that would describe it was weighed as #496 and declined (the next bullet). - A family-side twin of `fix(cjk-comma-honorific-peel)` for a PRE-comma glued honorific (2026-09-02, [#496](https://github.com/derek73/python-nameparser/issues/496), closed as not planned) — the rule that would honestly describe `王先生, V.`, `田中さん, V.` and `김민준씨, V.`, whose diff is the glued peel alone while the label they carry says comma compound (the finding bullet above). Declined on two grounds. FIRST, no gate can demand it: all 17 names the compound/peel pair is contested across measure radar — but by two different warrants, and an audit that assumes one will find the count short. Sixteen sit in `corpus_cjk_tolerated.jsonl`, the file #488 created by demoting the composed comma/Latin-wrapper CJK forms to tolerated input, `김, 민준씨` — the name #382 was filed over — among them. The seventeenth, `Dr 田中さん, V.`, is in `corpus_issues.jsonl` alone: harvested and append-only, radar since #468, and untouched by #488. (`Dr 김민준씨, Jr.` is in both files, and reads radar from either.) Radar means an unmatched diff is reported rather than fatal for want of a RULE; it is not an unconditional never-fatal, since a `[[never]]` exclusion outranks the tier and stays fatal on a radar name — `_CORPUS_TIERS`' own note, and `main()`'s two-reasons comment, which routes an excluded name to `unexplained` rather than to `radar`. No exclusion refuses these, so nothing here can demand the twin. SECOND, [#495](https://github.com/derek73/python-nameparser/issues/495) points the other way over the same corner: it asks whether the radar-only rules already in the ledger still earn their place and records `fix(cjk-comma-compound)` as having zero contract-tier reach, so #496 proposed a SECOND rule for the very names #495 is weighing a first one away from. Fewer rules for this corner is the coherent direction, and that is what makes the decline decisive rather than merely permissive. REOPEN it if a pre-comma glued-honorific name ever reads contract — and watch BOTH routes, because the two files promote by different mechanisms. A `corpus_cjk_tolerated.jsonl` name is promoted by clearing `tolerated` on its case rows, which moves the text into `corpus_cjk.jsonl` (`_CORPUS_TIERS`, and build_cjk_corpus.py's split). `Dr 田中さん, V.` has no case row at all, so there is no flag to clear: it changes tier only by being CHOSEN — a new unmarked row, or a rules.md example — which puts the text in a contract corpus, and the (name, order) dedup loads contract files first and keeps that reading. Read the tier off `_CORPUS_TIERS` either way, never off a hand-built file→tier map (the roster caution above). -- Precise per-name contest detection at differential-run time (2026-09-02; its stated reason replaced 2026-09-03, [#497](https://github.com/derek73/python-nameparser/issues/497)) — it would replace the static predicate's over-reporting with the measured nine-of-eleven split, and it needs the pinned-wheel worker pass to do it. The decline stands, on COVERAGE alone: a run-time check sees one ledger — the one for the baseline that run was invoked with — and only once somebody invokes the tool, whereas the static predicate covers a rule from the moment it is written, including a rule nobody has ever run the wheel against, which is the whole point of #382. Nor can the unit suite simply adopt the precise check: `tests/v2/test_differential.py` is deliberately hermetic — its own header says "nothing here spawns `uv` or the network", and the tests that need a baseline fake `_run_worker` or `Popen` outright — so moving it to pytest speed means spawning a subprocess and, on a cold cache, reaching PyPI. WITHDRAWN, and named here so it is not re-argued: this bullet used to say the check would sit "behind a multi-minute run" and gave that as "the whole of the reason". No measurement was recorded beside it: the phrase entered on 2026-08-05 in `7767ba2` at a single site in `tools/differential/compare.py`, and was copied outward until eight sites carried it — four there, three in `tests/v2/test_differential.py`, and this bullet. Measured 2026-09-03 over the corpus population named above, a whole run is 0.33s wall at baseline 1.4.0 and 0.57s at 2.2.0, the default; the worker pass is 0.10s and 0.29s of those, timed inside `main()`. All four baselines back to back total 1.97s, and a run with `uv`'s cache emptied first, so the pinned wheel is downloaded rather than reused, is still under a second. Read the class and not the digits: the worker pass is a fraction of a second and the whole gate is seconds at most, where the withdrawn phrase said minutes — so no argument here may be built on what the worker pass costs. RECOMPUTE: `time (uv run python tools/differential/compare.py --baseline 1.4.0 >/dev/null)`. For the worker's share, wrap `compare._run_worker` in a timer and then call `main()` — calling `_run_worker` directly on every loaded entry aborts at a 1.4.0 baseline, since the seven order-bearing shape-4/5 entries have to be dropped by the baseline-minimum skip first. Do NOT restate the decline as an error-direction argument either: within the NESTED pairs computing real diffs can only remove them, but the predicate is not a superset of the contests, since nesting is sufficient for one and not necessary (the bullet above, and #498). What the static check buys is cheapness and coverage of a rule nobody has run the wheel against, not a guarantee of refusing everything it should. +- Precise per-name contest detection at differential-run time (2026-09-02; its stated reason replaced 2026-09-03, [#497](https://github.com/derek73/python-nameparser/issues/497)) — it would replace the static predicate's over-reporting with the measured nine-of-eleven split, and it needs the pinned-wheel worker pass to do it. The decline stands, on COVERAGE alone: a run-time check sees one ledger — the one for the baseline that run was invoked with — and only once somebody invokes the tool, whereas the static predicate covers a rule from the moment it is written, including a rule nobody has ever run the wheel against, which is the whole point of #382. Nor can the unit suite simply adopt the precise check: `tests/v2/test_differential.py` is deliberately hermetic — its own header says "nothing here spawns `uv` or the network", and the tests that need a baseline fake `_run_worker` or `Popen` outright — so moving it to pytest speed means spawning a subprocess and, on a cold cache, reaching PyPI. WITHDRAWN, and named here so it is not re-argued: this bullet used to say the check would sit "behind a multi-minute run" and gave that as "the whole of the reason". No measurement was recorded beside it: the phrase entered on 2026-08-05 in `7767ba2` at a single site in `tools/differential/compare.py`, and was copied outward until eight sites carried it — four there, three in `tests/v2/test_differential.py`, and this bullet. Measured 2026-09-03 over the corpus population named above, a whole run is 0.33s wall at baseline 1.4.0 and 0.57s at 2.2.0, the default; the worker pass is 0.10s and 0.29s of those, timed inside `main()`. All four baselines back to back total 1.97s, and a run with `uv`'s cache emptied first, so the pinned wheel is downloaded rather than reused, is still under a second. Read the class and not the digits: the worker pass is a fraction of a second and the whole gate is seconds at most, where the withdrawn phrase said minutes — so no argument here may be built on what the worker pass costs. RECOMPUTE: `time (uv run python tools/differential/compare.py --baseline 1.4.0 >/dev/null)`. For the worker's share, wrap `compare._run_worker` in a timer and then call `main()` — calling `_run_worker` directly on every loaded entry aborts at a 1.4.0 baseline, since the seven order-bearing shape-4/5 entries have to be dropped by the baseline-minimum skip first. Do NOT restate the decline as an error-direction argument either: within the NESTED pairs computing real diffs can only remove them, but the predicate is not a superset of the contests, since nesting is sufficient for one and not necessary (the bullet above, and #498). What the static check buys is coverage of a rule nobody has run the wheel against, not a guarantee of refusing everything it should. - Giving the peel rule a predicate the compound rule fails (2026-09-02, #382 option 1) — narrowing `fix(cjk-comma-honorific-peel)`'s `name_regex` to the honorific-bearing shapes, whose stated effect in #382 was that the pair "becomes order-independent and the original contract holds again". Declined because that effect is UNREACHABLE, and measured rather than argued. The two rules ship a BYTE-IDENTICAL `name_regex` (verified 2026-09-02 by comparing the two strings), and `_rule_reach` computes each rule's names from its OWN pattern, so narrowing the peel rule's regex narrows the peel rule's reach and nothing else: the compound rule goes on reaching every one of the names. The general form, which no regex edit escapes — where the earlier rule's `fields` are a SUBSET of the later one's, order-independence requires the LATER rule to stop reaching the name, and an edit to the earlier rule's own regex cannot cause that. Here the peel rule's `{given, suffix}` is a strict subset of the compound rule's `{family, given, suffix, title}`, so every diff the peel rule admits the compound rule admits too. Implemented to check it: narrowing the peel regex to a `GLUED_HONORIFICS` alternation keeps all seven of the co-matched names whose 1.4.0 diff is exactly `{given, suffix}`, and swapping the two rules in the NARROWED ledger still reattributes all seven. So the pair keeps the arrangement the arc settled on, and that is the answer rather than a cost trade: it is narrow-first, the declaration-free default, so it appears in none of the eleven contests `order_contests` reports and owes no `precedes_narrower` block, and all seven names are pinned by name in `_CROSS_RULE_WINNERS` (23 corpus names reach both regexes; the seven are the ones #375's reorder mutation moves). The cost stands as a second reason and not the first: the narrowing would hand-copy more honorific vocabulary into the pattern and grow the `_HONORIFIC_SOURCES` sync-roster surface (mechanisms.md#CURATED-VOCABULARY-ALTERNATION's second half). Option 2's convention plus a check on the wide-first exceptions to it is what the arc took, and option 3 it REFUSED on a false premise (the finding above). One thing this does not settle, and the ledger comment on the peel rule overstates it: `fields` separates the two only for the UNION rows whose diff includes `family`; for those seven it is file order that decides. - Scoping the check to contract-tier contests only (2026-09-02) — five of the eleven are contested only over radar names, so this would have cut the file's exemptions by nearly half. Declined because `_CROSS_RULE_WINNERS` already pins radar names — measured, most of the names it pins are radar-tier (22 of 33 today) — so the repo would be inconsistent with itself about whether a radar contest matters. Whether the radar-only CJK comma rules still earn their place after #488's demotion is a real question and is filed as #495; it is a question about those rules, not about the check. - A shared name-population function for both callers (2026-09-02, the spec's own first sketch) — see the two-populations decision above. `main()` and the unit guard must read different populations, so one function would break `--corpus`; they agree by a docstring that names the function it mirrors instead. diff --git a/tests/v2/test_differential.py b/tests/v2/test_differential.py index 976e5583..45cc6430 100644 --- a/tests/v2/test_differential.py +++ b/tests/v2/test_differential.py @@ -1759,11 +1759,12 @@ def test_main_checks_contests_against_the_names_it_loaded() -> None: The unit guard is what catches a new rule at pytest speed; this is the belt for a run over a --corpus the guard never sees, and it has - to fire early -- a refusal raised after the worker pass prints - below the run's own `baseline:` line, so the reader has been shown - work done under a ledger the run was about to reject. The ordering - is the argument; the "multi-minute" cost this docstring used to - give for it was withdrawn as unmeasured (#497). + to fire early -- a refusal raised after the worker pass has already + installed the pinned wheel and parsed the whole corpus for a + comparison that will never be made, and it prints below the run's + own published `baseline:` header. The ordering is the argument; the + "multi-minute" cost this docstring used to give for it was + withdrawn as unmeasured (#497). """ src = (compare.HERE / "compare.py").read_text(encoding="utf-8") body = src[src.index("def main("):] diff --git a/tools/differential/compare.py b/tools/differential/compare.py index 8d88758e..41587b03 100644 --- a/tools/differential/compare.py +++ b/tools/differential/compare.py @@ -1818,14 +1818,14 @@ def main() -> int: # before any corpus is read: whether two rules CONTEST a diff is a # question about NAMES -- both regexes have to reach one -- and the # names arrive at this line. Before the worker pass, deliberately: - # a ledger refused after the worker runs is a ledger refused below - # its own `baseline:` line, so the run has already reported work - # done under rules it was about to call inadmissible (#382). It is - # the ORDER of the two that earns this placement and not the wait - # -- the worker pass is a tenth of a second, and the "multi-minute" - # this comment used to argue from was withdrawn as unmeasured - # (#497; the figures and their recompute are at the tree check - # further down this file). + # a ledger refused after the worker runs has already installed the + # pinned wheel and parsed the whole corpus for a comparison that + # will never be made, and it refuses below its own published + # `baseline:` header (#382). It is the ORDER of the two that earns + # this placement and not the wait -- the worker pass is a fraction + # of a second, and the "multi-minute" this comment used to argue + # from was withdrawn as unmeasured (#497, and decisions.md, "the + # rule-order arc", which carries the dated figures). # # The names are the LOADED entries, not the corpus*.jsonl glob the # unit guard in tests/v2/test_ledger_guards.py reads: `--corpus` @@ -1971,21 +1971,15 @@ def main() -> int: # The tree is checked BEFORE the worker runs. It depends on nothing # the worker produces, and validate_rules' own reasoning applies: a # misconfiguration that aborts after the whole install-and-compare - # pass is one the run reports below its own `baseline:` line, - # having already published a comparison it was about to disown. It - # is the ORDER that earns the placement, not the clock. This - # comment used to say that pass "costs minutes"; the claim was - # never measured, entered on 2026-08-05, and had been copied to - # seven other sites -- three more here, three in - # tests/v2/test_differential.py, and a Declined bullet in - # docs/design/decisions.md that rested a decline on it -- before - # #497 withdrew it. Measured 2026-09-03 over the whole corpus: a - # run is 0.33s wall at baseline 1.4.0 and 0.57s at 2.2.0, the - # default, of which the worker pass is 0.10s and 0.29s; all four - # baselines back to back total 1.97s; and a run with uv's cache - # emptied first, so the pinned wheel is downloaded rather than - # reused, is still under a second. RECOMPUTE: time (uv run python - # tools/differential/compare.py --baseline 1.4.0 >/dev/null) + # pass is one the run reports having already published a + # comparison it is about to disown. It is the ORDER that earns the + # placement, not the clock. This comment used to say that pass + # "costs minutes" -- a magnitude nobody had measured, and wrong: + # every baseline runs in well under a second. Withdrawn by #497. + # The figures, both recompute recipes, and the trap in timing + # _run_worker directly are in decisions.md, "the rule-order arc", + # kept in that one place because they carry a date there and a + # second copy is the copy that does not get updated. import nameparser # the working tree -- verified, not assumed from nameparser import HumanName tree_at = _check_tree(nameparser.__file__) From 9cf544b92c3a44c676d9cb59cc1b591fbdc777f6 Mon Sep 17 00:00:00 2001 From: Derek Gulbranson Date: Thu, 3 Sep 2026 00:56:54 -0700 Subject: [PATCH 03/23] tooling(differential): restore the baseline: anchor the twin comments keep Dropping the clause also dropped the comma that made the trailing participle work, so the sentence read as 'reports the fact of having published' rather than 'reports below its own baseline: line'. The anchor is true at that position -- baseline: prints after _run_worker and before the classify loop -- and both twin comments keep it. Refs #497 Co-Authored-By: Claude Opus 5 --- tools/differential/compare.py | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/tools/differential/compare.py b/tools/differential/compare.py index 41587b03..5b7300d7 100644 --- a/tools/differential/compare.py +++ b/tools/differential/compare.py @@ -1971,9 +1971,10 @@ def main() -> int: # The tree is checked BEFORE the worker runs. It depends on nothing # the worker produces, and validate_rules' own reasoning applies: a # misconfiguration that aborts after the whole install-and-compare - # pass is one the run reports having already published a - # comparison it is about to disown. It is the ORDER that earns the - # placement, not the clock. This comment used to say that pass + # pass is one the run reports below its own `baseline:` line, + # having already published a comparison it is about to disown. It + # is the ORDER that earns the placement, not the clock. This + # comment used to say that pass # "costs minutes" -- a magnitude nobody had measured, and wrong: # every baseline runs in well under a second. Withdrawn by #497. # The figures, both recompute recipes, and the trap in timing From 6db7cd6d9bbb025b287eb34ae8fbb84474e2c626 Mon Sep 17 00:00:00 2001 From: Derek Gulbranson Date: Thu, 3 Sep 2026 01:01:42 -0700 Subject: [PATCH 04/23] tooling(differential): a recorded diff shape the run can contradict The roster states a shape to ask which rule wins, then feeds that shape to classify() as an input -- so a guessed one agreed with itself forever. This is the half only a run can do. Pure function here; the data and the wiring follow. Two of the tests are written against a mutation that survived their first draft: the order-bearing row goes SECOND (a last-wins read of every row would otherwise agree by accident) and the matching-shape row is asserted twice, once with the recorded tuple spelled out of order, since nothing else pinned that both sides are sorted before they are compared. Refs #497 Co-Authored-By: Claude Opus 5 --- tests/v2/test_differential.py | 60 +++++++++++++++++++++++++++++++++++ tools/differential/compare.py | 52 ++++++++++++++++++++++++++++++ 2 files changed, 112 insertions(+) diff --git a/tests/v2/test_differential.py b/tests/v2/test_differential.py index 45cc6430..42c95783 100644 --- a/tests/v2/test_differential.py +++ b/tests/v2/test_differential.py @@ -3228,6 +3228,66 @@ def test_validate_rules_rejects_two_rules_sharing_an_issue() -> None: "expected_since_1.4.0.toml") +def test_a_recorded_shape_matching_the_run_is_no_mismatch() -> None: + diffing = [("Smith, Jr.", {"family", "suffix"}, None)] + assert compare.recorded_diff_mismatches( + {"Smith, Jr.": ("family", "suffix")}, diffing, + {"Smith, Jr."}) == [] + + # ... and the recorded side is a hand-written tuple, so both sides + # are sorted before they are compared: a row spelled in any other + # order is the same shape, not a mismatch. + assert compare.recorded_diff_mismatches( + {"Smith, Jr.": ("suffix", "family")}, diffing, + {"Smith, Jr."}) == [] + + +def test_a_recorded_shape_the_run_contradicts_is_reported() -> None: + """The check the roster could not do for itself. + + _CROSS_RULE_WINNERS feeds its recorded shape into classify() and + asserts the winner, so a guessed shape agrees with itself forever -- + which is how '田中さん II' sat recorded as {given, suffix} when the + measured diff is {family, given, suffix}. + """ + diffing = [("田中さん II", {"family", "given", "suffix"}, None)] + got = compare.recorded_diff_mismatches( + {"田中さん II": ("given", "suffix")}, diffing, {"田中さん II"}) + assert [(m.name, m.recorded, m.measured) for m in got] == [ + ("田中さん II", ("given", "suffix"), ("family", "given", "suffix"))] + + +def test_a_recorded_name_that_stops_diffing_is_reported() -> None: + """Recorded means it diffed. If it no longer does, that is a finding + about the parser, not a row to delete.""" + got = compare.recorded_diff_mismatches( + {"Smith, Jr.": ("family", "suffix")}, [], {"Smith, Jr."}) + assert [(m.name, m.measured) for m in got] == [("Smith, Jr.", None)] + + +def test_a_recorded_name_outside_this_run_is_skipped() -> None: + """--corpus narrows the name set, and absence is not a finding -- + the same asymmetry the vacancy check reads (#382).""" + assert compare.recorded_diff_mismatches( + {"Smith, Jr.": ("family", "suffix")}, [], set()) == [] + + +def test_only_the_order_none_comparison_is_read() -> None: + """The roster calls classify() with no order, so the shape it records + is the default-order comparison's. An order-bearing comparison of the + same string is a different question. + + The order-bearing row is written SECOND on purpose. A check that + read every row and let the last one win would agree with the + recorded shape by accident if the order-None row came last -- + mutation-tested, and the reason this row is written the way it is. + """ + diffing = [("Kim, Jr.", {"family", "suffix"}, None), + ("Kim, Jr.", {"family"}, "FAMILY_FIRST")] + assert compare.recorded_diff_mismatches( + {"Kim, Jr.": ("family", "suffix")}, diffing, {"Kim, Jr."}) == [] + + harvester = load_tool("build_issues_corpus") diff --git a/tools/differential/compare.py b/tools/differential/compare.py index 5b7300d7..32bc1f72 100644 --- a/tools/differential/compare.py +++ b/tools/differential/compare.py @@ -1335,6 +1335,58 @@ def vacant_exemptions(rules: list[dict[str, object]], if (str(rule.get("issue", "")), later) not in live] +class _ShapeMismatch(NamedTuple): + """A recorded diff shape the run disagrees with (#497). + + Named rather than a bare triple for _Vacancy's reason: the caller + formats these into a message, and `m.recorded`/`m.measured` says + which side is which where `m[1]`/`m[2]` would not. + """ + name: str + #: the shape _RECORDED_DIFFS carries, sorted + recorded: tuple[str, ...] + #: what this run measured, sorted; None when the name did not diff + measured: tuple[str, ...] | None + + +def recorded_diff_mismatches( + recorded: dict[str, tuple[str, ...]], + diffing: list[tuple[str, set[str], str | None]], + compared: set[str]) -> list[_ShapeMismatch]: + """Recorded shapes this run contradicts (#497). + + The roster in tests/v2/test_ledger_guards.py pins WHICH RULE wins a + contested name, and to ask that question it must state the diff + shape -- which it then feeds to classify() as an input. Nothing + checked the shape itself, so a guessed one agreed with itself + forever. This is the half only a run can do: main() has already + measured every name's real diff by the time it calls this. + + Only the order-None comparison is read. The roster calls classify() + with no order, so the shape it records is the default-order one; a + string also compared under a declared order is a different question + and its own row would be needed to ask it. + + A name absent from `compared` is SKIPPED rather than reported. Under + `--corpus` the name set is narrowed, and absence is then a fact + about the run rather than about the roster -- the same asymmetry the + vacancy check reads (#382). A name that is compared and does NOT + diff is reported, with `measured` None: recorded means it diffed + once, so its stopping is a finding about the parser. + """ + measured = {name: tuple(sorted(diff)) + for name, diff, order in diffing if order is None} + out: list[_ShapeMismatch] = [] + for name, shape in recorded.items(): + if name not in compared: + continue + got = measured.get(name) + want = tuple(sorted(shape)) + if got != want: + out.append(_ShapeMismatch(name, want, got)) + return out + + def _entry_matches(rule: dict[str, object], name: str, diff_fields: set[str], order: str | None = None) -> bool: """Does this entry's narrowing admit this diff? From a82f73e9aa68795127d79333065909fe1af240ab Mon Sep 17 00:00:00 2001 From: Derek Gulbranson Date: Thu, 3 Sep 2026 01:13:11 -0700 Subject: [PATCH 05/23] tooling(differential): pin the order filter in both directions Review found a survivor: the first fixture pins that a LAST-wins read of every row breaks, and the plan's pinned that a FIRST-wins one does, so neither alone refuses a check that simply drops the order filter. Both are reachable -- `diffing` is appended in corpus file order, so a name's two entries arrive either way round. The second fixture is the order-bearing row alone: recorded, compared, and no default-order comparison of it, where the honest answer is `measured` None. The measured side's sort now has an instrument too. It cannot be refuted outright, since a set CAN iterate alphabetically -- the two-role set does under 85 of PYTHONHASHSEED 0..199 -- so the row uses all seven roles, which did under none of them. Also: the `measured` field comment named _RECORDED_DIFFS, which no line in the tree defines yet, and says `the roster` instead. Refs #497 Co-Authored-By: Claude Opus 5 --- tests/v2/test_differential.py | 35 +++++++++++++++++++++++++++++++---- tools/differential/compare.py | 2 +- 2 files changed, 32 insertions(+), 5 deletions(-) diff --git a/tests/v2/test_differential.py b/tests/v2/test_differential.py index 42c95783..018c31c7 100644 --- a/tests/v2/test_differential.py +++ b/tests/v2/test_differential.py @@ -3241,6 +3241,20 @@ def test_a_recorded_shape_matching_the_run_is_no_mismatch() -> None: {"Smith, Jr.": ("suffix", "family")}, diffing, {"Smith, Jr."}) == [] + # ... and so is the measured side, which arrives as a SET: its + # iteration order is this interpreter's hash seed, so an unsorted + # reading would agree with the roster on some runs and report a + # mismatch on others. All seven roles rather than the two above + # because an unsorted reading cannot be refuted outright -- a set + # CAN iterate alphabetically, and the two-role set above does under + # 85 of PYTHONHASHSEED 0..199. The seven-role set did under none of + # them, so the size is what makes this row an instrument rather + # than a coin flip. + assert compare.recorded_diff_mismatches( + {"Smith, Jr.": tuple(sorted(compare.V2_FIELDS))}, + [("Smith, Jr.", set(compare.V2_FIELDS), None)], + {"Smith, Jr."}) == [] + def test_a_recorded_shape_the_run_contradicts_is_reported() -> None: """The check the roster could not do for itself. @@ -3277,16 +3291,29 @@ def test_only_the_order_none_comparison_is_read() -> None: is the default-order comparison's. An order-bearing comparison of the same string is a different question. - The order-bearing row is written SECOND on purpose. A check that - read every row and let the last one win would agree with the - recorded shape by accident if the order-None row came last -- - mutation-tested, and the reason this row is written the way it is. + Two fixtures, pinning the two ways a reading of every row survives, + since either one alone leaves the other's mutant alive. Both are + reachable: `diffing` is appended in corpus file order, so a name's + two entries can arrive either way round. Neither is a measurement -- + the shapes here are illustrative, chosen to separate the readings, + where _CROSS_RULE_WINNERS records 'Kim, Jr.' on four roles. + + FIRST: the order-bearing row written LAST, which kills a reading + where the last row wins. SECOND: the order-bearing row ALONE, which + kills a reading where the first row wins -- there the shape is + recorded, the name is compared, and no order-None comparison of it + exists, so the honest answer is `measured` None. """ diffing = [("Kim, Jr.", {"family", "suffix"}, None), ("Kim, Jr.", {"family"}, "FAMILY_FIRST")] assert compare.recorded_diff_mismatches( {"Kim, Jr.": ("family", "suffix")}, diffing, {"Kim, Jr."}) == [] + got = compare.recorded_diff_mismatches( + {"Kim, Jr.": ("family",)}, + [("Kim, Jr.", {"family"}, "FAMILY_FIRST")], {"Kim, Jr."}) + assert [(m.name, m.measured) for m in got] == [("Kim, Jr.", None)] + harvester = load_tool("build_issues_corpus") diff --git a/tools/differential/compare.py b/tools/differential/compare.py index 32bc1f72..d7d34dc9 100644 --- a/tools/differential/compare.py +++ b/tools/differential/compare.py @@ -1343,7 +1343,7 @@ class _ShapeMismatch(NamedTuple): which side is which where `m[1]`/`m[2]` would not. """ name: str - #: the shape _RECORDED_DIFFS carries, sorted + #: the shape the roster records, sorted recorded: tuple[str, ...] #: what this run measured, sorted; None when the name did not diff measured: tuple[str, ...] | None From 255ef7a287637ee694fb9385d64f67b891d3e808 Mon Sep 17 00:00:00 2001 From: Derek Gulbranson Date: Thu, 3 Sep 2026 01:25:53 -0700 Subject: [PATCH 06/23] tooling(differential): say what `compared` holds, and what it does not check Three promises the docstring was making loosely, where Task 4 has to read it as a contract. `compared` is now named: the entry list AFTER the baseline-minimum skip, not the pre-skip `corpus_names` the contest checks read on purpose. The nearest wrong variable sits two screens up, and taking it would report a name skipped at an old baseline as a name that stopped diffing. Figures re-measured today at --baseline 1.4.0: 1120 load, 7 skip, 1113 compare. The #382 citation claimed more than it did. vacant_exemptions' caller NOTEs under --corpus and REFUSES under a full run; this function is silent in both, so a recorded name no corpus holds any more is forgiven forever -- the shape #497 is itself about. Said plainly, with the missing half named (`set(recorded) - compared` under `full_corpus`) and left to the caller. `measured` None has two states behind it -- the parser stopped diffing, or the name is compared under a declared order alone -- and they are collapsed on purpose: separating them means reading the order-bearing rows this function ignores by design. Documented as a collapse, with what a message may claim and what it may not, and _Dormant's `kind` named as the shape to copy if it ever has to be carried. Also: the seed figure gets its recompute line, and the pair moves down beside dormant_rules and over_declared_rules -- the other two run-fed checks -- rather than sitting among the static ledger ones. Refs #497 Co-Authored-By: Claude Opus 5 --- tests/v2/test_differential.py | 16 +++- tools/differential/compare.py | 136 +++++++++++++++++++++------------- 2 files changed, 97 insertions(+), 55 deletions(-) diff --git a/tests/v2/test_differential.py b/tests/v2/test_differential.py index 018c31c7..256740db 100644 --- a/tests/v2/test_differential.py +++ b/tests/v2/test_differential.py @@ -3249,7 +3249,11 @@ def test_a_recorded_shape_matching_the_run_is_no_mismatch() -> None: # CAN iterate alphabetically, and the two-role set above does under # 85 of PYTHONHASHSEED 0..199. The seven-role set did under none of # them, so the size is what makes this row an instrument rather - # than a coin flip. + # than a coin flip. Recompute either count with + # for s in $(seq 0 199); do PYTHONHASHSEED=$s python3 -c \ + # 'print(list({"family","suffix"})==sorted({"family","suffix"}))' + # done | grep -c True + # substituting compare.V2_FIELDS for the two-role set. assert compare.recorded_diff_mismatches( {"Smith, Jr.": tuple(sorted(compare.V2_FIELDS))}, [("Smith, Jr.", set(compare.V2_FIELDS), None)], @@ -3280,8 +3284,14 @@ def test_a_recorded_name_that_stops_diffing_is_reported() -> None: def test_a_recorded_name_outside_this_run_is_skipped() -> None: - """--corpus narrows the name set, and absence is not a finding -- - the same asymmetry the vacancy check reads (#382).""" + """--corpus narrows the name set, so absence is a fact about the run. + + Only the SUBSET half of the asymmetry the vacancy check's caller + reads (#382): that caller REFUSES under a full run, and this + function is silent in both cases. Refusing a recorded name no + corpus holds any more is the caller's half -- `set(recorded) - + compared` under `full_corpus` -- and nothing here does it. + """ assert compare.recorded_diff_mismatches( {"Smith, Jr.": ("family", "suffix")}, [], set()) == [] diff --git a/tools/differential/compare.py b/tools/differential/compare.py index d7d34dc9..eed5a6b9 100644 --- a/tools/differential/compare.py +++ b/tools/differential/compare.py @@ -1335,58 +1335,6 @@ def vacant_exemptions(rules: list[dict[str, object]], if (str(rule.get("issue", "")), later) not in live] -class _ShapeMismatch(NamedTuple): - """A recorded diff shape the run disagrees with (#497). - - Named rather than a bare triple for _Vacancy's reason: the caller - formats these into a message, and `m.recorded`/`m.measured` says - which side is which where `m[1]`/`m[2]` would not. - """ - name: str - #: the shape the roster records, sorted - recorded: tuple[str, ...] - #: what this run measured, sorted; None when the name did not diff - measured: tuple[str, ...] | None - - -def recorded_diff_mismatches( - recorded: dict[str, tuple[str, ...]], - diffing: list[tuple[str, set[str], str | None]], - compared: set[str]) -> list[_ShapeMismatch]: - """Recorded shapes this run contradicts (#497). - - The roster in tests/v2/test_ledger_guards.py pins WHICH RULE wins a - contested name, and to ask that question it must state the diff - shape -- which it then feeds to classify() as an input. Nothing - checked the shape itself, so a guessed one agreed with itself - forever. This is the half only a run can do: main() has already - measured every name's real diff by the time it calls this. - - Only the order-None comparison is read. The roster calls classify() - with no order, so the shape it records is the default-order one; a - string also compared under a declared order is a different question - and its own row would be needed to ask it. - - A name absent from `compared` is SKIPPED rather than reported. Under - `--corpus` the name set is narrowed, and absence is then a fact - about the run rather than about the roster -- the same asymmetry the - vacancy check reads (#382). A name that is compared and does NOT - diff is reported, with `measured` None: recorded means it diffed - once, so its stopping is a finding about the parser. - """ - measured = {name: tuple(sorted(diff)) - for name, diff, order in diffing if order is None} - out: list[_ShapeMismatch] = [] - for name, shape in recorded.items(): - if name not in compared: - continue - got = measured.get(name) - want = tuple(sorted(shape)) - if got != want: - out.append(_ShapeMismatch(name, want, got)) - return out - - def _entry_matches(rule: dict[str, object], name: str, diff_fields: set[str], order: str | None = None) -> bool: """Does this entry's narrowing admit this diff? @@ -1661,6 +1609,90 @@ def over_declared_rules( return tuple(found) +class _ShapeMismatch(NamedTuple): + """A recorded diff shape the run disagrees with (#497). + + Named rather than a bare triple for _Vacancy's reason: the caller + formats these into a message, and `m.recorded`/`m.measured` says + which side is which where `m[1]`/`m[2]` would not. + """ + name: str + #: the shape the roster records, sorted + recorded: tuple[str, ...] + #: what this run measured, sorted; None when the run produced no + #: default-order diff of the name at all -- TWO states reach that, + #: deliberately collapsed, and the docstring says what a message + #: over one may and may not claim + measured: tuple[str, ...] | None + + +def recorded_diff_mismatches( + recorded: dict[str, tuple[str, ...]], + diffing: list[tuple[str, set[str], str | None]], + compared: set[str]) -> list[_ShapeMismatch]: + """Recorded shapes this run contradicts (#497). + + The roster in tests/v2/test_ledger_guards.py pins WHICH RULE wins a + contested name, and to ask that question it must state the diff + shape -- which it then feeds to classify() as an input. Nothing + checked the shape itself, so a guessed one agreed with itself + forever. This is the half only a run can do: main() has already + measured every name's real diff by the time it calls this. + + Only the order-None comparison is read. The roster calls classify() + with no order, so the shape it records is the default-order one; a + string also compared under a declared order is a different question + and its own row would be needed to ask it. + + `compared` is the names this run actually compared: the entry list + AFTER the baseline-minimum skip -- main()'s `corpus`, NOT the + `corpus_names` the contest checks read, which is built before that + skip runs. The two differ by exactly the entries an old baseline + cannot honor -- 7 of them at 1.4.0, where 1120 load and 1113 + compare; re-measure by running this file with `--baseline 1.4.0` + and reading its `skipped` and `corpus:` lines. decisions.md, "the + rule-order arc", records the same trap for the contest checks, + which read the pre-skip list on purpose. Pass that list here and a + roster row whose only entry was skipped reports as a name that + stopped diffing, when it was never compared at all. + + A name absent from `compared` is SKIPPED rather than reported. Under + `--corpus` the name set is narrowed, and absence is then a fact + about the run rather than about the roster. That is only the SUBSET + half of what the vacancy check's caller does with the same + asymmetry (#382): under a FULL run that caller refuses a + declaration whose pair is gone, and nothing here refuses a recorded + name no corpus holds any more -- forgiving forever the very shape + #497 is about. `set(recorded) - compared` under `full_corpus` is + the missing half, and it belongs to the caller, which is the only + side that knows whether this run read every corpus. + + A name that IS compared and produces no default-order diff is + reported with `measured` None, and two states reach that: the + parser stopped diffing a name recorded as diffing, or the name is + compared under a declared order ALONE, so no default-order + comparison of it exists to have a shape. They are collapsed on + purpose -- telling them apart means reading the order-bearing rows + this function otherwise ignores, and the result would stop being a + statement about the default-order comparison alone. So a message + over these rows may say the run measured no default-order diff, and + must NOT say the name stopped diffing. A caller needing the + distinction holds `diffing` and can take it from there; _Dormant's + `kind` is the shape to copy if it ever has to be carried here. + """ + measured = {name: tuple(sorted(diff)) + for name, diff, order in diffing if order is None} + out: list[_ShapeMismatch] = [] + for name, shape in recorded.items(): + if name not in compared: + continue + got = measured.get(name) + want = tuple(sorted(shape)) + if got != want: + out.append(_ShapeMismatch(name, want, got)) + return out + + def _load_entries(path: Path) -> list[dict[str, object]]: """Corpus lines as entry dicts. A line is either a bare JSON string (the original format) or an object with a "name" plus From 9302dac021d7ade9cc0ad14544c7d2e55152939b Mon Sep 17 00:00:00 2001 From: Derek Gulbranson Date: Thu, 3 Sep 2026 01:35:30 -0700 Subject: [PATCH 07/23] tooling(differential): the caller's half reads the PRE-skip list MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two corrections to what the docstring tells Task 4. `set(recorded) - compared` was the wrong expression for "a recorded name no corpus holds any more", and wrong by the trap the paragraph above it warns about: `compared` is post-skip, so that difference also holds names every corpus still has which this baseline skipped. Measured at 1.4.0, three of them -- 'de Mesnil Jean, Dr.', 'de la Cruz Juan Carlos, Dr.' and 'de la Cruz née Vega', all order-bearing with no order-None twin, all in corpus_shapes.jsonl throughout. `full_corpus` does not screen them: it is _CORPUS_FLOORS against the files on disk and knows nothing about the skip. A caller taking the prescription verbatim would refuse at the compat baseline and tell a contributor to delete a live row -- the earlier-draft `vacant` bug this file already records. The expression is `set(recorded) - set(corpus_names)`, and the docstring now says why the pre-skip list is right there while the post-skip one is right for `compared`: one question is about the run, the other about the files. The escape hatch offered for the collapsed `measured` None did not work. main() appends to `diffing` only where a comparison DIFFED, so a name compared under the default order with no diff leaves no row: a name compared under both orders and one compared under a declared order alone hand this function byte-identical arguments -- constructed and run. The collapse stands and its reason is stronger, but "a caller holds `diffing` and can take it from there" is deleted: separating the states needs the orders each name was compared under, which is a fourth argument, and a Task 4 author filling _Dormant-style `kind` off `diffing` would print a confident wrong label. Refs #497 Co-Authored-By: Claude Opus 5 --- tests/v2/test_differential.py | 6 +++-- tools/differential/compare.py | 42 ++++++++++++++++++++++++++--------- 2 files changed, 35 insertions(+), 13 deletions(-) diff --git a/tests/v2/test_differential.py b/tests/v2/test_differential.py index 256740db..70b3d879 100644 --- a/tests/v2/test_differential.py +++ b/tests/v2/test_differential.py @@ -3289,8 +3289,10 @@ def test_a_recorded_name_outside_this_run_is_skipped() -> None: Only the SUBSET half of the asymmetry the vacancy check's caller reads (#382): that caller REFUSES under a full run, and this function is silent in both cases. Refusing a recorded name no - corpus holds any more is the caller's half -- `set(recorded) - - compared` under `full_corpus` -- and nothing here does it. + corpus holds any more is the caller's half, and it asks + `set(recorded) - set(corpus_names)` -- the PRE-skip list, since the + baseline-minimum skip takes a name out of the RUN and out of no + file. Nothing here does either. """ assert compare.recorded_diff_mismatches( {"Smith, Jr.": ("family", "suffix")}, [], set()) == [] diff --git a/tools/differential/compare.py b/tools/differential/compare.py index eed5a6b9..3a017714 100644 --- a/tools/differential/compare.py +++ b/tools/differential/compare.py @@ -1663,22 +1663,42 @@ def recorded_diff_mismatches( asymmetry (#382): under a FULL run that caller refuses a declaration whose pair is gone, and nothing here refuses a recorded name no corpus holds any more -- forgiving forever the very shape - #497 is about. `set(recorded) - compared` under `full_corpus` is - the missing half, and it belongs to the caller, which is the only - side that knows whether this run read every corpus. + #497 is about. That missing half belongs to the caller, the only + side that knows whether this run read every corpus, and it asks + `set(recorded) - set(corpus_names)`: the PRE-skip list, which is + the opposite of what `compared` takes in the paragraph above, and + deliberately so. + Two different questions. "Was this name compared?" is about the + RUN, and an entry the baseline skipped was not; "does any corpus + still hold this name?" is about the FILES, and the skip removes a + name from no file. Measured at --baseline 1.4.0, the two lists + differ by three names -- 'de Mesnil Jean, Dr.', 'de la Cruz Juan + Carlos, Dr.' and 'de la Cruz née Vega', each order-bearing with no + order-None twin to keep it in `compared`, and each sitting in + corpus_shapes.jsonl the whole time. Refusing off the post-skip list + would tell a contributor to delete a roster row for a name that is + right there, at the compat baseline. `full_corpus` does not screen + that: it is _CORPUS_FLOORS against the files on disk and knows + nothing about the skip. It is the earlier-draft `vacant` bug + recorded below, one baseline over. A name that IS compared and produces no default-order diff is reported with `measured` None, and two states reach that: the parser stopped diffing a name recorded as diffing, or the name is compared under a declared order ALONE, so no default-order - comparison of it exists to have a shape. They are collapsed on - purpose -- telling them apart means reading the order-bearing rows - this function otherwise ignores, and the result would stop being a - statement about the default-order comparison alone. So a message - over these rows may say the run measured no default-order diff, and - must NOT say the name stopped diffing. A caller needing the - distinction holds `diffing` and can take it from there; _Dormant's - `kind` is the shape to copy if it ever has to be carried here. + comparison of it exists to have a shape. They are collapsed because + nothing in these arguments separates them. main() appends to + `diffing` only where a comparison DIFFED, so a name compared under + the default order that produces no diff leaves no row at all, and + the two states hand this function byte-identical `diffing` and + `compared` -- constructed and run, not reasoned about. The + distinction lives in the ORDERS each name was compared under, which + is main()'s `entries` and would be a fourth argument here. So a + message over these rows may say the run measured no default-order + diff, and must NOT say the name stopped diffing; and a caller must + not try to label them off `diffing`, which cannot answer it. + _Dormant's `kind` is the shape to copy if that fourth argument is + ever added and the two are told apart. """ measured = {name: tuple(sorted(diff)) for name, diff, order in diffing if order is None} From 962ae43e20731509a1ba42658e966d2142f91b97 Mon Sep 17 00:00:00 2001 From: Derek Gulbranson Date: Thu, 3 Sep 2026 01:47:46 -0700 Subject: [PATCH 08/23] tooling(differential): the recorded shapes move where the run can see them MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit _CROSS_RULE_WINNERS keeps what it is for -- which rule wins a contested name -- and takes the shape from compare._RECORDED_DIFFS, which the run can verify. Thirty-five hand-written shapes leave the test. No winner moves: the pair (name, shape) -> winner reconstructs the old dict exactly. Re-measuring them to write the provenance note found four that a run contradicts, all four added by #452 and all four in the 2.x ledgers. 'Nguyen, Van' diffs at no baseline at all, so the {family} recorded for it is a shape no run makes; 'Jane née and Jones Smith' diffs {family, given, maiden, middle} at both 2.x baselines, not the three recorded (fix(#445) wins it either way). The 31 rows at 1.4.0 all still agree. Left standing here, where nothing but the shapes' address changes; the note beside _RECORDED_DIFFS records the measurements and the roster's 2.1.0 block stops claiming it was measured. Refs #497 Co-Authored-By: Claude Opus 5 --- tests/v2/test_ledger_guards.py | 145 ++++++++++++++++++++------------- tools/differential/compare.py | 100 +++++++++++++++++++++-- 2 files changed, 183 insertions(+), 62 deletions(-) diff --git a/tests/v2/test_ledger_guards.py b/tests/v2/test_ledger_guards.py index e744efea..2e0fff27 100644 --- a/tests/v2/test_ledger_guards.py +++ b/tests/v2/test_ledger_guards.py @@ -2423,7 +2423,8 @@ def test_every_rule_claims_the_recorded_share_of_the_corpus() -> None: #: Which rule classify() actually picks, for names several rules could -#: claim. Keyed by (name, the diff it produces against that baseline). +#: claim. Keyed by name; the diff shape each one produces against that +#: baseline is compare._RECORDED_DIFFS' half (#497). #: #: Every other guard here measures a rule ALONE: _CORPUS_CLAIMS records #: what a regex reaches, and the gate's total counts names. Neither can @@ -2444,35 +2445,33 @@ def test_every_rule_claims_the_recorded_share_of_the_corpus() -> None: #: peels -- only the CJK lookahead separates them, so it is the row #: that fails if that lookahead is ever dropped. #: -#: The diff shapes are measured against the 1.4.0 wheel, not guessed. -#: Re-measure rather than adjust them if a parser change moves one: -#: a diff shape that shifted is a finding, not a number to update. -#: -#: Blind spot since `orders` (#468): every row is recomputed at -#: comparison order None, so a contest that exists only under a -#: declared order is not recorded here -- true today because no -#: recorded winner is order-scoped, and it stops being true the day one -#: is. -_CROSS_RULE_WINNERS: dict[str, dict[tuple[str, tuple[str, ...]], str]] = { +#: The shapes these rows are classified with used to sit in the keys +#: here and moved to compare._RECORDED_DIFFS (#497), because a shape is +#: something only a RUN can measure: fed to classify() as an input and +#: checked by nothing, a guessed one agreed with itself forever. That +#: comment carries what this one used to say about them -- how each was +#: measured and where the claim did not hold, that a shifted shape is a +#: finding rather than a number to update, and that they are +#: default-order only, the blind spot `orders` (#468) opened. One copy +#: of each fact, since two means one of them goes quietly stale. +_CROSS_RULE_WINNERS: dict[str, dict[str, str]] = { # open cycle: one rule, so nothing for a second one to contest "expected_since_2.2.0.toml": {}, "expected_since_1.4.0.toml": { - ("Andrews, M.D.", ("given", "suffix")): "fix(comma-family)", - ("田中, 太郎さん", ("given", "suffix")): "fix(cjk-comma-honorific-peel)", - ("김, 민준씨", ("given", "suffix")): "fix(cjk-comma-honorific-peel)", - ("김, 민준씨 (Jimmy)", ("given", "suffix")): "fix(cjk-comma-honorific-peel)", - ("김민준, 씨", ("given", "suffix")): "fix(cjk-comma-honorific-peel)", - ("김민준, 씨.", ("given", "suffix")): "fix(cjk-comma-honorific-peel)", - ("선생님, J.씨", ("given", "suffix")): "fix(cjk-comma-honorific-peel)", - ("이, J.씨", ("given", "suffix")): "fix(cjk-comma-honorific-peel)", + "Andrews, M.D.": "fix(comma-family)", + "田中, 太郎さん": "fix(cjk-comma-honorific-peel)", + "김, 민준씨": "fix(cjk-comma-honorific-peel)", + "김, 민준씨 (Jimmy)": "fix(cjk-comma-honorific-peel)", + "김민준, 씨": "fix(cjk-comma-honorific-peel)", + "김민준, 씨.": "fix(cjk-comma-honorific-peel)", + "선생님, J.씨": "fix(cjk-comma-honorific-peel)", + "이, J.씨": "fix(cjk-comma-honorific-peel)", # the union rows: they move `family`, so the peel rule's fields # exclude them and they must stay on the compound rule - ("Dr 김민준씨, V.", ("family", "given", "suffix")): - "fix(cjk-comma-compound)", + "Dr 김민준씨, V.": "fix(cjk-comma-compound)", # since #296's audit the title moves too (PhD is the postnominal) - ("田中さん, PhD", ("family", "given", "suffix", "title")): - "fix(cjk-comma-compound)", - ("田中さん, V.", ("family", "suffix")): "fix(cjk-comma-compound)", + "田中さん, PhD": "fix(cjk-comma-compound)", + "田中さん, V.": "fix(cjk-comma-compound)", # #372's suffix-routing split. 'Bob Jones, author' moves NO # suffix, which is what disqualifies it from the routing rule; # 'Smith Jr.' is the Latin shape that rule is named for; the two @@ -2483,16 +2482,16 @@ def test_every_rule_claims_the_recorded_share_of_the_corpus() -> None: # keeps its split here, and the rule written for that shape is # ahead of the precomma merge in the file; 'MD, PHD' has one # pre-comma piece, no split to keep, and stays merged - ("Bob Jones, author", ("family", "given")): + "Bob Jones, author": "fix(comma-family) a comma followed only by titles keeps " "the given/family split", # since #296's audit 'PHD' is a postnominal here and the string # is credentials only; the rule written for that shape is ahead # of the precomma merge in the file - ("MD, PHD", ("family", "given", "suffix", "title")): + "MD, PHD": "fix(#296) a credential-only comma string reads a name and " "its postnominal", - ("Smith Jr.", ("family", "suffix")): + "Smith Jr.": "fix(suffix-routing) a two-token name ending in the suffix " "word jr keeps it in `suffix`", # #451's two contests with the numeral rule, whose regex reaches @@ -2507,10 +2506,10 @@ def test_every_rule_claims_the_recorded_share_of_the_corpus() -> None: # because a later edit that moves either rule, or widens the # numeral rule's `fields`, would take one silently -- exactly the # absorption #451 exists to end. - ("Carod i", ("family", "suffix")): + "Carod i": "fix(#397) NOT WANTED: a trailing Catalan/Polish linking " "'i' is read as a generation marker and the family is lost", - ("田中さん II", ("family", "given", "suffix")): + "田中さん II": "fix(cjk-glued-honorific-peel) glued honorific peels into " "suffix", # #484's three `_initials` contests with a literal rule on one @@ -2522,11 +2521,11 @@ def test_every_rule_claims_the_recorded_share_of_the_corpus() -> None: # diff on initials, so narrowing or deleting a literal rule # would hand its name to the per-word rule rather than surface # it: the handover this row exists to catch. - ("de los Santos", ("_initials",)): + "de los Santos": "fix(#360) los joined the particles, so it no longer initials", - ("van Berg Jan de", ("_initials",)): + "van Berg Jan de": "fix(#385/#402) an all-particle name part initials its words (R2)", - ("van ma van", ("_initials",)): + "van ma van": "fix(#385/#402) an all-particle name part initials its words (R2)", # the glued/spaced boundary. 'Andersonさん' and '김민준씨' left # suffix-routing for a rule that names them; '김민준 씨.' is @@ -2544,19 +2543,16 @@ def test_every_rule_claims_the_recorded_share_of_the_corpus() -> None: # that catch-all, so there is no last-resort tier left in any # ledger and the residual cost went with it: all four rules that # replaced it carry a name_regex. - ("Andersonさん", ("given", "suffix")): - "fix(cjk-glued-honorific-peel)", - ("김민준씨", ("family", "given", "suffix")): - "fix(cjk-glued-honorific-peel)", - ("김민준 씨.", ("family", "given", "suffix")): - "fix(cjk-honorific-suffix)", + "Andersonさん": "fix(cjk-glued-honorific-peel)", + "김민준씨": "fix(cjk-glued-honorific-peel)", + "김민준 씨.": "fix(cjk-honorific-suffix)", # '.,' moved off `fix(comma-family) lone post-comma piece # routes to suffix/title, not first`, whose Latin-range comma # regex reaches it, onto the A2 rule that describes it (#451). # Recorded because only file order separates the two rules -- # they are in the same tier and both reach the name -- and a # later edit that moves either one silently hands it back. - (".,", ("given",)): "fix(A2) content-free input names nobody, so every role empties", + ".,": "fix(A2) content-free input names nobody, so every role empties", # The one exception the cjk-comma-compound rule's `middle` # argument turns on, added by #452's review. That comment says # ten of the eleven names it explains have a single-token @@ -2569,7 +2565,7 @@ def test_every_rule_claims_the_recorded_share_of_the_corpus() -> None: # saying so. Shape measured against the 1.4.0 tag: v1 reads # first '田中さん', suffix 'Ph. D.'; the tree reads family # '田中', suffix 'さん, Ph. D.'. - ("田中さん, Ph. D.", ("family", "given", "suffix")): + "田中さん, Ph. D.": "fix(cjk-comma-compound) comma routing compounds with the CJK order flip", # The jr rule's surplus, added by the #453 review. Its regex # reaches these three and does not explain them; `fields` is @@ -2583,11 +2579,11 @@ def test_every_rule_claims_the_recorded_share_of_the_corpus() -> None: # 1.4.0 wheel, not guessed. 'Doe,, Jr.' is the fourth name the # regex reaches and has no row: it does not diff at this # baseline, so there is no winner to pin. - ("Kim, Jr.", ("family", "given", "suffix", "title")): + "Kim, Jr.": "fix(#296) a lone post-comma credential is a suffix", - ("Smith, Jr.", ("family", "given", "suffix", "title")): + "Smith, Jr.": "fix(#296) a lone post-comma credential is a suffix", - ("김민준씨 Jr.", ("family", "given", "suffix")): + "김민준씨 Jr.": "fix(cjk-glued-honorific-peel) glued honorific peels into suffix", # #484's per-word rules. Measured over the corpus, exactly ONE # name is reached by two of them AND diffs on `_initials`: @@ -2601,7 +2597,7 @@ def test_every_rule_claims_the_recorded_share_of_the_corpus() -> None: # Jones') and none of them has an `_initials` diff to pin: # three move ROLES, so the pseudo-field never enters, and two # do not diff at all. - ("de la Vega y Santos Juan", ("_initials",)): + "de la Vega y Santos Juan": "fix(initials-per-word) a connective run", # Not contested today -- the bound-given rule is the only one # of the four whose regex reaches these three -- and recorded @@ -2611,12 +2607,11 @@ def test_every_rule_claims_the_recorded_share_of_the_corpus() -> None: # or a particle rule grown to reach a trailing 'van' takes # these names silently: reach is per-rule and the gate's total # is per-corpus, so nothing else here would say so. - ("Abu Bakr Al Baghdadi, MD", ("_initials",)): - "fix(initials-per-word) a bound-given run", - ("abu bakr al baghdadi", ("_initials",)): + "Abu Bakr Al Baghdadi, MD": "fix(initials-per-word) a bound-given run", - ("Berg, abdul van", ("_initials",)): + "abu bakr al baghdadi": "fix(initials-per-word) a bound-given run", + "Berg, abdul van": "fix(initials-per-word) a bound-given run", }, # The two 2.x ledgers had NO section here until #452, and the # coverage assertion below was `<=`, so their absence read as "no @@ -2627,28 +2622,36 @@ def test_every_rule_claims_the_recorded_share_of_the_corpus() -> None: # Neither _CORPUS_CLAIMS nor the gate's totals can see a handover -- # reach is regex-only and the total is per-corpus -- so these rows # are the only thing that would. + # All four rows below are the ones #497 caught: their SHAPES are + # the four compare._RECORDED_DIFFS records a run contradicting, so + # read the argument each comment makes as an argument about + # classify() -- which rule takes which shape, still true -- and not + # as a claim about what these two names diff today. The provenance + # note beside _RECORDED_DIFFS has the measurements. "expected_since_2.0.0.toml": { # fix(#296) lost `family` and `given`; {family} on this name is # one of the shapes it stopped admitting, and fix(#379) takes it. # The right home -- a tussenvoegsel attaching to the family is # exactly what that rule is about -- which is not the point: the # point is that a later edit sends it somewhere else in silence. - ("Nguyen, Van", ("family",)): + "Nguyen, Van": "fix(#379) a tussenvoegsel after a family comma attaches to the family", # fix(#412) lost `middle`; this shape went to fix(#445), which # sits BEHIND it in file order, so the handover was decided by # the narrowing rather than by position. - ("Jane née and Jones Smith", ("family", "maiden", "middle")): + "Jane née and Jones Smith": "fix(#445) the lone name word beside a marker a connective join no longer absorbs", }, "expected_since_2.1.0.toml": { - # The same two handovers, measured against this baseline's own - # run rather than copied from the 2.0.0 rows -- the ledgers - # differ, and #452's own lesson is that a claim true in one file - # is not thereby true in its siblings. - ("Nguyen, Van", ("family",)): + # The same two handovers, recorded against this baseline rather + # than copied from the 2.0.0 rows -- the ledgers differ, and + # #452's own lesson is that a claim true in one file is not + # thereby true in its siblings. This block said "measured + # against this baseline's own run" until #497 ran one: both + # shapes are among the four that disagree with it. + "Nguyen, Van": "fix(#379) a tussenvoegsel after a family comma attaches to the family", - ("Jane née and Jones Smith", ("family", "maiden", "middle")): + "Jane née and Jones Smith": "fix(#445) the lone name word beside a marker a connective join no longer absorbs", }, } @@ -2679,7 +2682,9 @@ def test_the_recorded_rule_still_wins_each_contested_name() -> None: ledger = by_name[ledger_name] rules = compare._sorted_rules(_rules(ledger)) never = _exclusions(ledger) - for (name, fields), expected in winners.items(): + shapes = compare._RECORDED_DIFFS[ledger_name] + for name, expected in winners.items(): + fields = shapes[name] got = compare.classify(name, set(fields), rules, never) assert got is not None and got.startswith(expected), ( f"{ledger_name}: {name!r} diffing {list(fields)} is now " @@ -2702,6 +2707,32 @@ def test_the_recorded_rule_still_wins_each_contested_name() -> None: f"was the odd one out.") +def test_every_pinned_winner_has_a_recorded_shape() -> None: + """The roster names a contested name; _RECORDED_DIFFS says what it + diffs. A name in one and not the other is a half-recorded pin -- + the winner cannot be checked without the shape, and a shape nothing + pins a winner for is unverified by the run for no purpose.""" + compare = load_tool("compare") + for ledger_name, winners in _CROSS_RULE_WINNERS.items(): + shapes = compare._RECORDED_DIFFS.get(ledger_name, {}) + assert set(winners) == set(shapes), ( + f"{ledger_name}: winners without a recorded shape " + f"{sorted(set(winners) - set(shapes))}; shapes with no " + f"pinned winner {sorted(set(shapes) - set(winners))}") + # Per-ledger equality above iterates the ROSTER's keys, so a + # _RECORDED_DIFFS section for a ledger the roster does not name is + # invisible to it -- and to the run, which reads shapes per ledger + # too. The sibling test asserts the ROSTER's keys are exactly the + # ledgers on disk; this carries that to _RECORDED_DIFFS, which has + # no such check of its own. + assert set(compare._RECORDED_DIFFS) == set(_CROSS_RULE_WINNERS), ( + f"_RECORDED_DIFFS and _CROSS_RULE_WINNERS name different " + f"ledgers: only in _RECORDED_DIFFS " + f"{sorted(set(compare._RECORDED_DIFFS) - set(_CROSS_RULE_WINNERS))}; " + f"only in _CROSS_RULE_WINNERS " + f"{sorted(set(_CROSS_RULE_WINNERS) - set(compare._RECORDED_DIFFS))}") + + def test_the_family_first_fold_is_not_explained_under_the_default_order( ) -> None: """The hazard the `orders` key exists for, pinned against the diff --git a/tools/differential/compare.py b/tools/differential/compare.py index 3a017714..24f9c1c8 100644 --- a/tools/differential/compare.py +++ b/tools/differential/compare.py @@ -1626,6 +1626,96 @@ class _ShapeMismatch(NamedTuple): measured: tuple[str, ...] | None +#: The recorded diff shape of every name _CROSS_RULE_WINNERS pins +#: (tests/v2/test_ledger_guards.py), per ledger (#497). +#: +#: Baseline-relative by construction, which is why it is keyed per +#: ledger rather than once by name even though the two names appearing +#: in two ledgers carry the same shape in both: a string moves a +#: different set of roles against different baselines, and decisions.md +#: (2026-08-28 #452) records `fix(#296) a lone post-comma credential is +#: a suffix` moving four roles at 1.4.0 and two at both 2.x baselines. +#: +#: A recorded shape that MOVES is a FINDING, not a number to update: +#: the parser changed what that name does, and the winner pinned beside +#: it in _CROSS_RULE_WINNERS was recorded for the OLD shape, so both +#: want reading before either is edited. +#: +#: HERE rather than beside the roster because this is where the +#: measurement happens. main() computes every name's real diff, so a +#: run can check these; the unit suite cannot, and deliberately -- it +#: spawns no uv and no network, and every test in +#: tests/v2/test_differential.py monkeypatches _run_worker. Same +#: direction as _CORPUS_FLOORS above, which lives here and is read from +#: there. +#: +#: Default-order shapes only, because the roster classifies with no +#: order. recorded_diff_mismatches below says what that leaves out. +#: +#: PROVENANCE, and it is not uniform. Re-measured 2026-09-03 by driving +#: main() at all four baselines and feeding its `diffing` and its +#: post-skip corpus to recorded_diff_mismatches: the 31 rows at 1.4.0 +#: are measured against the 1.4.0 wheel, as the roster always claimed, +#: and all 31 still agree. The four 2.x rows, which #452 added under +#: the same claim, do NOT: +#: 'Nguyen, Van' produces no diff at ANY of the four baselines. It is +#: compared under the default order out of corpus_rules.jsonl every +#: time, and the tree agrees with all four wheels on it, so {family} +#: is a shape no run makes. That is the position 'Doe,, Jr.' is in, +#: which the roster gives no row precisely because it does not diff. +#: 'Jane née and Jones Smith' diffs {family, given, maiden, middle} +#: at both 2.x baselines, not the {family, maiden, middle} recorded. +#: fix(#445) wins it under either shape, measured, so correcting it +#: moves no pin. +#: Left as they stand by the commit that moved them, which changes no +#: pin. Wiring this into main() is what forces the decision. +_RECORDED_DIFFS: dict[str, dict[str, tuple[str, ...]]] = { + # open cycle: one rule, so nothing for a second one to contest + "expected_since_2.2.0.toml": {}, + "expected_since_1.4.0.toml": { + "Andrews, M.D.": ("given", "suffix"), + "田中, 太郎さん": ("given", "suffix"), + "김, 민준씨": ("given", "suffix"), + "김, 민준씨 (Jimmy)": ("given", "suffix"), + "김민준, 씨": ("given", "suffix"), + "김민준, 씨.": ("given", "suffix"), + "선생님, J.씨": ("given", "suffix"), + "이, J.씨": ("given", "suffix"), + "Dr 김민준씨, V.": ("family", "given", "suffix"), + "田中さん, PhD": ("family", "given", "suffix", "title"), + "田中さん, V.": ("family", "suffix"), + "Bob Jones, author": ("family", "given"), + "MD, PHD": ("family", "given", "suffix", "title"), + "Smith Jr.": ("family", "suffix"), + "Carod i": ("family", "suffix"), + "田中さん II": ("family", "given", "suffix"), + "de los Santos": ("_initials",), + "van Berg Jan de": ("_initials",), + "van ma van": ("_initials",), + "Andersonさん": ("given", "suffix"), + "김민준씨": ("family", "given", "suffix"), + "김민준 씨.": ("family", "given", "suffix"), + ".,": ("given",), + "田中さん, Ph. D.": ("family", "given", "suffix"), + "Kim, Jr.": ("family", "given", "suffix", "title"), + "Smith, Jr.": ("family", "given", "suffix", "title"), + "김민준씨 Jr.": ("family", "given", "suffix"), + "de la Vega y Santos Juan": ("_initials",), + "Abu Bakr Al Baghdadi, MD": ("_initials",), + "abu bakr al baghdadi": ("_initials",), + "Berg, abdul van": ("_initials",), + }, + "expected_since_2.0.0.toml": { + "Nguyen, Van": ("family",), + "Jane née and Jones Smith": ("family", "maiden", "middle"), + }, + "expected_since_2.1.0.toml": { + "Nguyen, Van": ("family",), + "Jane née and Jones Smith": ("family", "maiden", "middle"), + }, +} + + def recorded_diff_mismatches( recorded: dict[str, tuple[str, ...]], diffing: list[tuple[str, set[str], str | None]], @@ -1633,11 +1723,11 @@ def recorded_diff_mismatches( """Recorded shapes this run contradicts (#497). The roster in tests/v2/test_ledger_guards.py pins WHICH RULE wins a - contested name, and to ask that question it must state the diff - shape -- which it then feeds to classify() as an input. Nothing - checked the shape itself, so a guessed one agreed with itself - forever. This is the half only a run can do: main() has already - measured every name's real diff by the time it calls this. + contested name, and to ask that question it needs the diff shape -- + which it reads from _RECORDED_DIFFS above and feeds to classify() as + an input. Nothing checks the shape itself, so a guessed one agrees + with itself forever. This is the half only a run can do: main() has + already measured every name's real diff by the time it calls this. Only the order-None comparison is read. The roster calls classify() with no order, so the shape it records is the default-order one; a From b793ebc1b1dc3bd1145563d16011c14c7e823cc6 Mon Sep 17 00:00:00 2001 From: Derek Gulbranson Date: Thu, 3 Sep 2026 02:05:37 -0700 Subject: [PATCH 09/23] tooling(differential): four rows pinned a race with one runner MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The two 2.x sections of _CROSS_RULE_WINNERS and _RECORDED_DIFFS held 'Nguyen, Van' and 'Jane née and Jones Smith'. Both recorded a diff shape a run contradicts, and the wrong shape is the symptom: at the shape each name really produces, exactly one rule admits it, so classify() was never adjudicating a contest and the recorded shape only ever had to agree with itself. 'Nguyen, Van' produces no diff at ANY of the four baselines -- compared under the default order every time, the tree agreeing with all four wheels -- so no run ever asks classify() about it. That is 'Doe,, Jr.'s position, which the 1.4.0 roster gives no row for the same reason. 'Jane née and Jones Smith' diffs {family, given, maiden, middle} at 1.4.0/2.0.0/2.1.0 and not at all at 2.2.0, never the {family, maiden, middle} recorded; its shape was correctable and the rows went anyway, the string being a malformed harvest from corpus_issues.jsonl (radar) whose right parse nobody can state. Both sections stay present as explicit empty mappings -- the completeness guards require it, and the comments now say emptiness is a position, not an oversight. They also say what it is NOT: measured 2026-09-03, 5 of the 247 diffs at 2.0.0 and 1 of the 155 at 2.1.0 do move a shape two or more rules admit. Nobody has argued one of those boundaries here yet. And "only one rule admits it" is not on its own grounds to delete a row: 13 of the 31 surviving 1.4.0 rows are in that position too and stay, because the shapes they pin are shapes runs actually make. No corpus changes: deleting a row removes a pin, not a name. recorded_diff_mismatches now reports zero at all four baselines; the gate is unchanged at 352 / 247 / 155 / 14 intentional, 0 unexplained, 0 radar-unclassified. Co-Authored-By: Claude Opus 5 --- tests/v2/test_ledger_guards.py | 75 ++++++++++++++++------------------ tools/differential/compare.py | 53 ++++++++++++++---------- 2 files changed, 67 insertions(+), 61 deletions(-) diff --git a/tests/v2/test_ledger_guards.py b/tests/v2/test_ledger_guards.py index 2e0fff27..ee0b8dc3 100644 --- a/tests/v2/test_ledger_guards.py +++ b/tests/v2/test_ledger_guards.py @@ -2615,45 +2615,42 @@ def test_every_rule_claims_the_recorded_share_of_the_corpus() -> None: }, # The two 2.x ledgers had NO section here until #452, and the # coverage assertion below was `<=`, so their absence read as "no - # contest to pin" rather than "nobody looked". The #452 narrowings - # are what made that expensive: shrinking a rule's `fields` hands - # every shape it no longer admits to whatever claims it next, and - # measured across the fourteen, that moved shapes in both files. - # Neither _CORPUS_CLAIMS nor the gate's totals can see a handover -- - # reach is regex-only and the total is per-corpus -- so these rows - # are the only thing that would. - # All four rows below are the ones #497 caught: their SHAPES are - # the four compare._RECORDED_DIFFS records a run contradicting, so - # read the argument each comment makes as an argument about - # classify() -- which rule takes which shape, still true -- and not - # as a claim about what these two names diff today. The provenance - # note beside _RECORDED_DIFFS has the measurements. - "expected_since_2.0.0.toml": { - # fix(#296) lost `family` and `given`; {family} on this name is - # one of the shapes it stopped admitting, and fix(#379) takes it. - # The right home -- a tussenvoegsel attaching to the family is - # exactly what that rule is about -- which is not the point: the - # point is that a later edit sends it somewhere else in silence. - "Nguyen, Van": - "fix(#379) a tussenvoegsel after a family comma attaches to the family", - # fix(#412) lost `middle`; this shape went to fix(#445), which - # sits BEHIND it in file order, so the handover was decided by - # the narrowing rather than by position. - "Jane née and Jones Smith": - "fix(#445) the lone name word beside a marker a connective join no longer absorbs", - }, - "expected_since_2.1.0.toml": { - # The same two handovers, recorded against this baseline rather - # than copied from the 2.0.0 rows -- the ledgers differ, and - # #452's own lesson is that a claim true in one file is not - # thereby true in its siblings. This block said "measured - # against this baseline's own run" until #497 ran one: both - # shapes are among the four that disagree with it. - "Nguyen, Van": - "fix(#379) a tussenvoegsel after a family comma attaches to the family", - "Jane née and Jones Smith": - "fix(#445) the lone name word beside a marker a connective join no longer absorbs", - }, + # contest to pin" rather than "nobody looked". They are back to + # empty, and it is now the other thing: a stated position, which is + # what the equality assertion below makes sayable. Do not restore a + # row to fill them. + # + # #452 gave each two rows -- 'Nguyen, Van' and 'Jane née and Jones + # Smith', the handovers its narrowings caused -- and #497 deleted + # all four rather than correct them. Each recorded a diff shape a + # run contradicts (compare._RECORDED_DIFFS' provenance note has the + # measurements and the recompute, in one copy, including why the one + # correctable shape was not corrected), and at the shape each name + # really produces, exactly one rule admits it. That is the defect + # under the wrong shapes rather than beside them: a row pinning a + # race with one runner is never exercised as a contest, so its shape + # only ever had to agree with itself. + # + # EMPTY IS NOT "these ledgers hold no contested name". Measured + # 2026-09-03 over the diffs each baseline's own run produces, 5 of + # the 247 at 2.0.0 and 1 of the 155 at 2.1.0 move a shape two or + # more rules admit, file order picking the winner ('MD, PHD' is the + # one both have). Recompute: drive compare.main() at the baseline, + # capture its `diffing` by wrapping dormant_rules, and count the + # names for which more than one rule satisfies + # compare._entry_matches at the measured shape. None of those + # boundaries is argued about in this file, and this roster pins the + # arguments this file makes -- so a row is owed when someone argues + # one, not before. + # + # Nor is "only one rule admits it" grounds on its own to delete a + # row: 13 of the 31 above are in that position too, and say so (the + # jr rule's surplus, the bound-given trio). They stay because the + # shapes they pin are shapes runs actually make, so widening a + # `fields` or moving a rule hands the name over and this test says + # so. The four deleted rows could not do that work at any edit. + "expected_since_2.0.0.toml": {}, + "expected_since_2.1.0.toml": {}, } diff --git a/tools/differential/compare.py b/tools/differential/compare.py index 24f9c1c8..288c4120 100644 --- a/tools/differential/compare.py +++ b/tools/differential/compare.py @@ -1652,23 +1652,38 @@ class _ShapeMismatch(NamedTuple): #: Default-order shapes only, because the roster classifies with no #: order. recorded_diff_mismatches below says what that leaves out. #: -#: PROVENANCE, and it is not uniform. Re-measured 2026-09-03 by driving -#: main() at all four baselines and feeding its `diffing` and its -#: post-skip corpus to recorded_diff_mismatches: the 31 rows at 1.4.0 -#: are measured against the 1.4.0 wheel, as the roster always claimed, -#: and all 31 still agree. The four 2.x rows, which #452 added under -#: the same claim, do NOT: +#: PROVENANCE. The 31 rows at 1.4.0 are measured against the 1.4.0 +#: wheel, as the roster always claimed, and all 31 still agree -- +#: re-measured 2026-09-03 by driving main() at all four baselines and +#: feeding its `diffing` and its post-skip corpus to +#: recorded_diff_mismatches, which is the recompute: wrap _run_worker to +#: capture the post-skip entries and dormant_rules to capture `diffing`, +#: since both receive exactly what main() built. +#: +#: Both 2.x sections are EMPTY, and that is a position rather than an +#: omission -- _CROSS_RULE_WINNERS carries why no 2.x contest is pinned, +#: including the measured fact that contests DO exist there. Each held +#: two rows, added by #452, until #497 ran that recompute against them. +#: All four recorded a shape no run makes: #: 'Nguyen, Van' produces no diff at ANY of the four baselines. It is #: compared under the default order out of corpus_rules.jsonl every #: time, and the tree agrees with all four wheels on it, so {family} -#: is a shape no run makes. That is the position 'Doe,, Jr.' is in, -#: which the roster gives no row precisely because it does not diff. -#: 'Jane née and Jones Smith' diffs {family, given, maiden, middle} -#: at both 2.x baselines, not the {family, maiden, middle} recorded. -#: fix(#445) wins it under either shape, measured, so correcting it -#: moves no pin. -#: Left as they stand by the commit that moved them, which changes no -#: pin. Wiring this into main() is what forces the decision. +#: is a shape no run makes and no run ever asked classify() about the +#: name at all. That is the position 'Doe,, Jr.' is in, which the +#: roster gives no row precisely because it does not diff. +#: 'Jane née and Jones Smith' diffs {family, given, maiden, middle} at +#: 1.4.0, 2.0.0 and 2.1.0, and not at all at 2.2.0 -- never the +#: {family, maiden, middle} recorded. Its shape WAS correctable, and +#: the rows went anyway: the string is a malformed harvest from +#: corpus_issues.jsonl (radar), the tree reads it family 'Jane' / +#: maiden 'and Jones Smith', and nobody can state what it ought to +#: parse to, so a pin on it defends no boundary anyone would argue +#: for. In both 2.x ledgers fix(#445) is the only rule admitting it at +#: either shape, measured, so no pin moved either way. +#: Deleting a row removes a PIN, not a name: corpus_issues.jsonl is +#: append-only and both strings are still compared, and classified, on +#: every run. 'Nguyen, Van' is classified by nothing only because it +#: diffs from nothing. _RECORDED_DIFFS: dict[str, dict[str, tuple[str, ...]]] = { # open cycle: one rule, so nothing for a second one to contest "expected_since_2.2.0.toml": {}, @@ -1705,14 +1720,8 @@ class _ShapeMismatch(NamedTuple): "abu bakr al baghdadi": ("_initials",), "Berg, abdul van": ("_initials",), }, - "expected_since_2.0.0.toml": { - "Nguyen, Van": ("family",), - "Jane née and Jones Smith": ("family", "maiden", "middle"), - }, - "expected_since_2.1.0.toml": { - "Nguyen, Van": ("family",), - "Jane née and Jones Smith": ("family", "maiden", "middle"), - }, + "expected_since_2.0.0.toml": {}, + "expected_since_2.1.0.toml": {}, } From 95727aa8ac71aa06623c288fd1d52a90f40fafc9 Mon Sep 17 00:00:00 2001 From: Derek Gulbranson Date: Thu, 3 Sep 2026 02:09:24 -0700 Subject: [PATCH 10/23] tests(ledger-guards): point the empty 2.x sections at the open question The comment states a position -- a row is owed when someone argues a boundary, not before -- and #501 asks whether that should change now that the six contested 2.x diffs are measured rather than unexamined. Cite it so a reader meets the question where the position is stated. Refs #497, #501 Co-Authored-By: Claude Opus 5 --- tests/v2/test_ledger_guards.py | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/tests/v2/test_ledger_guards.py b/tests/v2/test_ledger_guards.py index ee0b8dc3..b41a5861 100644 --- a/tests/v2/test_ledger_guards.py +++ b/tests/v2/test_ledger_guards.py @@ -2641,7 +2641,10 @@ def test_every_rule_claims_the_recorded_share_of_the_corpus() -> None: # compare._entry_matches at the measured shape. None of those # boundaries is argued about in this file, and this roster pins the # arguments this file makes -- so a row is owed when someone argues - # one, not before. + # one, not before. Whether that position should change now that the + # six are measured rather than merely unexamined is #501; the answer + # there decides whether these two sections stay empty, and nothing + # in this file presumes it. # # Nor is "only one rule admits it" grounds on its own to delete a # row: 13 of the 31 above are in that position too, and say so (the From a5b87c09aa5d85996afad0feced0678e2ec02459 Mon Sep 17 00:00:00 2001 From: Derek Gulbranson Date: Thu, 3 Sep 2026 02:40:18 -0700 Subject: [PATCH 11/23] tooling(differential): the run refuses a recorded shape it contradicts Closes the half _CROSS_RULE_WINNERS could not check for itself: its shapes were inputs to classify(), so a guessed one stayed green forever. A moved shape is a finding, and the message says so. Two name sets, one line apart and not interchangeable. The shape check reads the POST-skip corpus, because "was this name compared?" is about the run. The absent-name half -- a roster row naming a string no corpus holds, which would otherwise be forgiven forever under the full gate -- reads the PRE-skip corpus_names, because "does any corpus still hold this name?" is about the files, and the baseline-minimum skip empties no file. Measured at 1.4.0 the two differ by three names, all in corpus_shapes.jsonl; the post-skip reading would wait for the first roster row on a shape-tagged name and then tell a contributor to delete a row for a name the run just read past. Like the vacancy check it refuses only over a full corpus, and unlike it stays silent under --corpus, where there is no count to misread. A row with no measured shape has two causes -- the parser stopped moving the name, or it is compared under a declared order alone -- and the two hand the check byte-identical arguments, so the message names both and asserts neither. Refs #497 Co-Authored-By: Claude Opus 5 --- tests/v2/test_differential.py | 183 +++++++++++++++++++++++++++++++++- tools/differential/compare.py | 83 ++++++++++++++- 2 files changed, 260 insertions(+), 6 deletions(-) diff --git a/tests/v2/test_differential.py b/tests/v2/test_differential.py index 70b3d879..02b5e5cb 100644 --- a/tests/v2/test_differential.py +++ b/tests/v2/test_differential.py @@ -2025,6 +2025,183 @@ def test_a_corpus_narrowing_does_not_refuse_the_shipped_1_4_ledger( assert code == 1 +#: A ledger that explains the fixture's own family-role diff, so a run +#: over _DIFFERS is quiet except for whatever the test under it puts +#: into _RECORDED_DIFFS. +_CLAIMS_FAMILY = ('[[change]]\nissue = "claimed"\nname_regex = "Smith"\n' + 'fields = ["family"]\n') + + +def test_main_refuses_a_recorded_shape_the_run_contradicts( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None: + """The roster's shapes are checked where they are measured (#497). + + _CROSS_RULE_WINNERS feeds a recorded shape into classify() as an + INPUT, so a guessed one agrees with itself forever; the shape is + only falsifiable against a run. recorded_diff_mismatches has its + own unit tests above -- this pins that main() calls it, which is + the half that can go silently permissive. + """ + monkeypatch.setitem(compare._RECORDED_DIFFS, + "expected_since_1.4.0.toml", + {"John Smith": ("nickname",)}) + with pytest.raises(SystemExit) as exc: + _run_main(tmp_path, monkeypatch, _CLAIMS_FAMILY, _DIFFERS) + message = str(exc.value) + assert "John Smith" in message + # both sides, because a message naming only one leaves the reader + # unable to see which way the shape moved + assert "nickname" in message and "family" in message + # a FINDING, not a number to update: the winner recorded beside the + # shape was recorded for the OLD shape + assert "FINDING" in message + assert "_CROSS_RULE_WINNERS" in message + + +def test_the_shape_refusal_over_an_unmeasured_name_claims_no_cause( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None: + """`measured` None has two reachable causes and the message must + not assert one of them. + + The parser may have stopped moving a name recorded as moving, or + the name may be compared under a declared order ALONE, so no + default-order comparison of it exists to have a shape. The two hand + recorded_diff_mismatches byte-identical arguments -- main() appends + to `diffing` only where a comparison DIFFED -- so this check cannot + tell them apart, and a message saying the name "stopped diffing" + would send half its readers to the parser over a corpus entry. + Written in the shape of test_the_vacancy_refusal_names_both_of_its_causes. + """ + monkeypatch.setitem(compare._RECORDED_DIFFS, + "expected_since_1.4.0.toml", + {"John Smith": ("family",)}) + with pytest.raises(SystemExit) as exc: + _run_main(tmp_path, monkeypatch, _CLAIMS_FAMILY, _SAME_FACADE) + message = str(exc.value) + assert "no default-order diff" in message + assert "declared order" in message + assert "stopped diffing" not in message + + +def test_a_full_run_refuses_a_recorded_name_no_corpus_holds( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None: + """The caller's half of the skip (#497 addendum). + + recorded_diff_mismatches SKIPS a recorded name it did not compare, + which is right under `--corpus` and forgiving forever under the + full gate: a roster row naming a name no corpus holds is a row that + agrees with itself because nothing measures it, which is the very + defect this arc is about. Same asymmetry as the vacancy check's + caller (#382), and `corpus_flag=False` is the only run whose name + set can tell a departed name from one this run did not reach. + """ + monkeypatch.setitem(compare._RECORDED_DIFFS, + "expected_since_1.4.0.toml", + {"Nobody Here, Esq.": ("family",)}) + with pytest.raises(SystemExit) as exc: + _run_main(tmp_path, monkeypatch, _CLAIMS_FAMILY, _DIFFERS, + corpus_flag=False) + message = str(exc.value) + assert "Nobody Here, Esq." in message + # both repairs, because the row may be stale OR the corpus may be + # what regressed, and they are not interchangeable + assert "restore" in message and "delete" in message + assert "_CROSS_RULE_WINNERS" in message + + +def test_a_corpus_run_is_silent_about_a_recorded_name_outside_it( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None: + """The inversion, and the reason the half above is not in the + function: `--corpus` narrows the name set on purpose, so absence is + a fact about the run. Refusing here would tell a contributor to + delete a roster row over a name the full gate compares -- the + earlier-draft `vacant` bug, one check over. + """ + monkeypatch.setitem(compare._RECORDED_DIFFS, + "expected_since_1.4.0.toml", + {"Nobody Here, Esq.": ("family",)}) + code, out = _run_main(tmp_path, monkeypatch, _CLAIMS_FAMILY, _DIFFERS) + assert "Nobody Here, Esq." not in out + assert code == 0, out + + +def test_naming_every_corpus_refuses_a_recorded_name_no_corpus_holds( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None: + """`--corpus` is not the question; the NAME SET is. + + The flag is `action="append"`, so a run listing every corpus + compares exactly what the flagless gate compares. Keying the + silence on `args.corpus` rather than on `full_corpus` would let a + genuinely departed roster name pass by naming the roster. + """ + monkeypatch.setitem(compare._RECORDED_DIFFS, + "expected_since_1.4.0.toml", + {"Nobody Here, Esq.": ("family",)}) + with pytest.raises(SystemExit) as exc: + _run_main(tmp_path, monkeypatch, _CLAIMS_FAMILY, _DIFFERS, + names_every_corpus=True) + assert "Nobody Here, Esq." in str(exc.value) + + +def test_a_name_this_baseline_skipped_is_not_a_name_the_corpus_lost( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None: + """The two name sets the check reads, one line apart, are opposite. + + "Was this name compared?" is about the RUN, and an order-bearing + entry an old baseline cannot honor was not -- so the shape half + reads the POST-skip list and stays quiet about it. "Does any corpus + still hold this name?" is about the FILES, and the skip removes a + name from no file -- so the absent-name half reads the PRE-skip + `corpus_names`. Swap them and this full run refuses, telling a + contributor to delete a roster row for a name sitting in the corpus + it just read. Measured on the shipped tree at --baseline 1.4.0 the + two lists differ by three names, all in corpus_shapes.jsonl, so the + hazard is live and only unarmed because none of the three carries a + roster row yet. + + Hand-rolled rather than via _run_main, which writes bare-string + corpus lines and so cannot produce a skipped entry at all; same + construction as + test_the_contest_check_reads_names_this_baseline_will_not_compare. + Its control is the sibling above: a name in NO corpus is refused by + this same run shape. + """ + import json as _json + import sys + name = "Ménil Christophe du" + corpus = tmp_path / "corpus_x.jsonl" + corpus.write_text( + _json.dumps({"name": name, "shape": 4}, ensure_ascii=False) + "\n" + + _json.dumps("John Smith") + "\n", encoding="utf-8") + (tmp_path / "expected_since_1.4.0.toml").write_text( + _CLAIMS_FAMILY, encoding="utf-8") + monkeypatch.setitem(compare._RECORDED_DIFFS, + "expected_since_1.4.0.toml", {name: ("family",)}) + # wholesale, so the flagless glob below sees a full corpus roster + monkeypatch.setattr(compare, "_CORPUS_FLOORS", {corpus.name: 1}) + monkeypatch.setitem(compare._CORPUS_TIERS, corpus.name, "contract") + monkeypatch.setattr(compare, "HERE", tmp_path) + _WORKER_CALL.clear() + + def _fake(v: str, w: bool, n: list[dict]) -> tuple[dict, list[dict]]: + _WORKER_CALL.update(version=v, names=[e["name"] for e in n]) + return ({"__version__": v, + "__file__": "/wheel/nameparser/__init__.py"}, + [{"facade": dict(_DIFFERS)}]) + + monkeypatch.setattr(compare, "_run_worker", _fake) + monkeypatch.setattr(sys, "argv", ["compare.py", "--baseline", "1.4.0"]) + import contextlib + import io + buf = io.StringIO() + with contextlib.redirect_stdout(buf): + code = compare.main() + out = buf.getvalue() + assert _WORKER_CALL["names"] == ["John Smith"], out + assert name not in out, out + assert code == 0, out + + def test_radar_diff_with_no_rule_exits_0_and_is_reported( tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None: """The tier split's entire point (#468): a harvested name's diff @@ -3292,7 +3469,11 @@ def test_a_recorded_name_outside_this_run_is_skipped() -> None: corpus holds any more is the caller's half, and it asks `set(recorded) - set(corpus_names)` -- the PRE-skip list, since the baseline-minimum skip takes a name out of the RUN and out of no - file. Nothing here does either. + file. Nothing here does either; + test_a_full_run_refuses_a_recorded_name_no_corpus_holds pins the + caller's side of it, and + test_a_name_this_baseline_skipped_is_not_a_name_the_corpus_lost + pins that it reads the other list. """ assert compare.recorded_diff_mismatches( {"Smith, Jr.": ("family", "suffix")}, [], set()) == [] diff --git a/tools/differential/compare.py b/tools/differential/compare.py index 288c4120..dda11319 100644 --- a/tools/differential/compare.py +++ b/tools/differential/compare.py @@ -1762,11 +1762,11 @@ def recorded_diff_mismatches( asymmetry (#382): under a FULL run that caller refuses a declaration whose pair is gone, and nothing here refuses a recorded name no corpus holds any more -- forgiving forever the very shape - #497 is about. That missing half belongs to the caller, the only - side that knows whether this run read every corpus, and it asks - `set(recorded) - set(corpus_names)`: the PRE-skip list, which is - the opposite of what `compared` takes in the paragraph above, and - deliberately so. + #497 is about. That half belongs to the caller, the only side that + knows whether this run read every corpus; main() does it, under a + full run alone, asking `set(recorded) - set(corpus_names)`: the + PRE-skip list, which is the opposite of what `compared` takes in + the paragraph above, and deliberately so. Two different questions. "Was this name compared?" is about the RUN, and an entry the baseline skipped was not; "does any corpus still hold this name?" is about the FILES, and the skip removes a @@ -2342,6 +2342,79 @@ def _tree_parse(name: str, order: str | None) -> object: for issue, name, tagged in order_blind: print(f" {issue!r} explained {name!r}{_order_tag(tagged)}") print() + # Checked HERE because this is the only place both halves exist: + # the roster's recorded shape, and the diff this run actually + # measured. validate_rules runs before any corpus is read, and the + # unit suite spawns no worker -- deliberately -- so neither can ask + # this (#497). + # + # TWO NAME SETS, one line apart, and they are not interchangeable. + # `corpus` is POST-skip, and recorded_diff_mismatches takes it + # because its question is "was this name compared?" -- an + # order-bearing entry a baseline cannot honor was not, and a shape + # cannot be read off a comparison that never ran. `corpus_names` is + # PRE-skip, and the absent-name half below takes it because its + # question is "does any corpus still hold this name?" -- the skip + # takes a name out of the RUN and out of no file. Measured at + # --baseline 1.4.0 the two differ by three names -- 'de Mesnil + # Jean, Dr.', 'de la Cruz Juan Carlos, Dr.' and 'de la Cruz née + # Vega', each order-bearing with no order-None twin, each sitting + # in corpus_shapes.jsonl throughout. None of the three carries a + # roster row today, so reading the post-skip list here would refuse + # nothing YET: it would wait for the first row on a shape-tagged + # name and then tell a contributor to delete a row for a name the + # run just read past. That is the earlier-draft `vacant` bug + # recorded above, one check over. Recompute both lists with the + # recipe in recorded_diff_mismatches' docstring. + # + # The absent-name half is the CALLER's because only the caller + # knows whether this run read every corpus, and it splits on + # `full_corpus` for the vacancy check's reason: a roster row for a + # name outside a narrowed run reads exactly like one for a name + # that left. Under `--corpus` it is silent rather than a NOTE -- + # unlike `vacant` there is no count anyone could misread into a + # repair, since recorded_diff_mismatches has already skipped those + # names without reporting them. + compared = {str(n) for n in corpus} + recorded = _RECORDED_DIFFS.get(ledger.name, {}) + shape_bad = recorded_diff_mismatches(recorded, diffing, compared) + if shape_bad: + raise SystemExit("\n".join( + [f"{ledger.name}: {len(shape_bad)} recorded diff shape(s) in " + f"_RECORDED_DIFFS disagree with this run. A moved shape is " + f"a FINDING, not a number to update -- the parser changed " + f"what the name does, and the winner pinned beside it in " + f"_CROSS_RULE_WINNERS (tests/v2/test_ledger_guards.py) was " + f"recorded for the OLD shape, so read both before editing " + f"either:"] + + [f" {m.name!r} recorded {list(m.recorded)}, " + + (f"measured {list(m.measured)}" if m.measured is not None + else "and this run measured no default-order diff of it") + for m in shape_bad] + # Named for the vacancy refusal's reason: a message that + # offers repair advice for one of two reachable causes + # sends half its readers to the wrong file. The two are + # collapsed on purpose (recorded_diff_mismatches' docstring + # says why), so the message must not pick one. + + (["A row with no measured shape has TWO causes and this " + "check cannot tell them apart: the parser may have " + "stopped moving the name, or the name may be compared " + "under a declared order alone, leaving no default-order " + "comparison to have a shape. Read the corpus entry " + "before you read the parser."] + if any(m.measured is None for m in shape_bad) else []))) + gone = sorted(set(recorded) - set(corpus_names)) + if gone and full_corpus: + raise SystemExit("\n".join( + [f"{ledger.name}: {len(gone)} recorded diff shape(s) name a " + f"string no corpus holds any more, over the FULL corpus. " + f"Nothing measures such a row, so it agrees with itself " + f"forever -- the shape of the defect #497 is about. Two " + f"repairs, and they are not interchangeable: restore the " + f"name to a corpus if the contest it pins is still worth " + f"pinning, or delete the row AND the _CROSS_RULE_WINNERS " + f"entry beside it if the name left on purpose:"] + + [f" {n!r}" for n in gone])) dormancy = dormant_rules(rules, set(by_issue), diffing, exclusions) for dormant in dormancy.undeclared: print(f"EXPLAINED NOTHING {dormant.issue!r}\n " From e1f81c63d2505a181fc153b7230458160803701c Mon Sep 17 00:00:00 2001 From: Derek Gulbranson Date: Thu, 3 Sep 2026 02:54:11 -0700 Subject: [PATCH 12/23] tooling(differential): a stale roster row must not eat the report Two placements, both wrong for the same reason the file already argues elsewhere. The absent-name half needs the ledger and the loaded names and nothing the worker produces, so it belongs beside `vacant`, pre-worker -- a refusal after the wheel is installed and the corpus compared prints below the run's own `baseline:` header for a comparison it is about to disown (#382). Moved. The post-skip list is not even in scope there any more, which is the placement paying for itself. The measured half is the opposite case and stays after the comparison, but it cannot raise there: the raise landed MID-report and took dormancy, OVER-DECLARED, UNEXPLAINED and the radar block with it. Measured, a `--corpus corpus.jsonl` run at 1.4.0 prints 62 EXPLAINED NOTHING lines and 0 of them with one _RECORDED_DIFFS row corrupted -- a stale roster row hiding an unexplained diff, which is the gate's primary output. It now prints and feeds the exit code, like over_declared_rules, its structural sibling. Also: the header sentence claimed "the parser changed what the name does", which is false for the second cause of an unmeasured row. Dropped; the FINDING instruction, true either way, stays. Refs #497 Co-Authored-By: Claude Opus 5 --- tests/v2/test_differential.py | 80 +++++++++++---- tools/differential/compare.py | 178 ++++++++++++++++++++-------------- 2 files changed, 169 insertions(+), 89 deletions(-) diff --git a/tests/v2/test_differential.py b/tests/v2/test_differential.py index 02b5e5cb..0735934e 100644 --- a/tests/v2/test_differential.py +++ b/tests/v2/test_differential.py @@ -2032,7 +2032,7 @@ def test_a_corpus_narrowing_does_not_refuse_the_shipped_1_4_ledger( 'fields = ["family"]\n') -def test_main_refuses_a_recorded_shape_the_run_contradicts( +def test_main_reports_a_recorded_shape_the_run_contradicts( tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None: """The roster's shapes are checked where they are measured (#497). @@ -2045,22 +2045,53 @@ def test_main_refuses_a_recorded_shape_the_run_contradicts( monkeypatch.setitem(compare._RECORDED_DIFFS, "expected_since_1.4.0.toml", {"John Smith": ("nickname",)}) - with pytest.raises(SystemExit) as exc: - _run_main(tmp_path, monkeypatch, _CLAIMS_FAMILY, _DIFFERS) - message = str(exc.value) - assert "John Smith" in message - # both sides, because a message naming only one leaves the reader + code, out = _run_main(tmp_path, monkeypatch, _CLAIMS_FAMILY, _DIFFERS) + assert "MOVED SHAPE" in out + assert "John Smith" in out + # both sides, because a report naming only one leaves the reader # unable to see which way the shape moved - assert "nickname" in message and "family" in message + assert "nickname" in out and "family" in out # a FINDING, not a number to update: the winner recorded beside the # shape was recorded for the OLD shape - assert "FINDING" in message - assert "_CROSS_RULE_WINNERS" in message + assert "FINDING" in out + assert "_CROSS_RULE_WINNERS" in out + # The verdict, not just the print. A block that reports and exits 0 + # is read by CI as silence -- the same trap + # test_main_exits_1_and_reports_an_unclassified_diff exists for. + # Ledger and fixture are otherwise clean, so 1 here is this check's + # alone. + assert code == 1, out -def test_the_shape_refusal_over_an_unmeasured_name_claims_no_cause( +def test_a_moved_shape_does_not_truncate_the_report( tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None: - """`measured` None has two reachable causes and the message must + """A stale roster row must not hide an unexplained diff. + + This check runs after the worker, so unlike the two pre-worker + refusals it cannot raise: a SystemExit here lands MID-report and + takes dormancy, OVER-DECLARED, UNEXPLAINED and the radar block down + with it -- measured on the shipped tree, a `--corpus corpus.jsonl` + run at 1.4.0 prints 62 EXPLAINED NOTHING lines and 0 of them with + one _RECORDED_DIFFS row corrupted. So it prints and feeds the exit + code, like over_declared_rules. Pinned with all three in one run, + because a mismatch alone cannot show what a raise would have eaten. + """ + monkeypatch.setitem(compare._RECORDED_DIFFS, + "expected_since_1.4.0.toml", + {"John Smith": ("nickname",)}) + code, out = _run_main( + tmp_path, monkeypatch, + '[[change]]\nissue = "unrelated"\nname_regex = "ZZZ"\n' + 'fields = ["family"]\n', _DIFFERS) + assert "MOVED SHAPE" in out + assert "EXPLAINED NOTHING 'unrelated'" in out + assert "UNEXPLAINED 'John Smith'" in out + assert code == 1, out + + +def test_the_shape_report_over_an_unmeasured_name_claims_no_cause( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None: + """`measured` None has two reachable causes and the report must not assert one of them. The parser may have stopped moving a name recorded as moving, or @@ -2068,19 +2099,23 @@ def test_the_shape_refusal_over_an_unmeasured_name_claims_no_cause( default-order comparison of it exists to have a shape. The two hand recorded_diff_mismatches byte-identical arguments -- main() appends to `diffing` only where a comparison DIFFED -- so this check cannot - tell them apart, and a message saying the name "stopped diffing" + tell them apart, and a line saying the name "stopped diffing" would send half its readers to the parser over a corpus entry. Written in the shape of test_the_vacancy_refusal_names_both_of_its_causes. """ monkeypatch.setitem(compare._RECORDED_DIFFS, "expected_since_1.4.0.toml", {"John Smith": ("family",)}) - with pytest.raises(SystemExit) as exc: - _run_main(tmp_path, monkeypatch, _CLAIMS_FAMILY, _SAME_FACADE) - message = str(exc.value) - assert "no default-order diff" in message - assert "declared order" in message - assert "stopped diffing" not in message + code, out = _run_main(tmp_path, monkeypatch, _CLAIMS_FAMILY, + _SAME_FACADE) + assert "no default-order diff" in out + assert "declared order" in out + # Every line of the block, not just the one carrying the shape: the + # header sentence is the one that used to assert the parser had + # changed, which is false for the second cause. + assert "stopped diffing" not in out + assert "the parser changed" not in out + assert code == 1, out def test_a_full_run_refuses_a_recorded_name_no_corpus_holds( @@ -2094,6 +2129,12 @@ def test_a_full_run_refuses_a_recorded_name_no_corpus_holds( defect this arc is about. Same asymmetry as the vacancy check's caller (#382), and `corpus_flag=False` is the only run whose name set can tell a departed name from one this run did not reach. + + Pre-worker, like both refusals it sits beside: it reads the ledger + and the loaded names and nothing the comparison produces, and a + refusal raised afterwards prints below the run's own `baseline:` + header for a comparison it is about to disown. Its measured half + cannot do that, and prints instead. """ monkeypatch.setitem(compare._RECORDED_DIFFS, "expected_since_1.4.0.toml", @@ -2107,6 +2148,8 @@ def test_a_full_run_refuses_a_recorded_name_no_corpus_holds( # what regressed, and they are not interchangeable assert "restore" in message and "delete" in message assert "_CROSS_RULE_WINNERS" in message + assert not _WORKER_CALL, ( + "main() spawned the worker before refusing the roster row") def test_a_corpus_run_is_silent_about_a_recorded_name_outside_it( @@ -2141,6 +2184,7 @@ def test_naming_every_corpus_refuses_a_recorded_name_no_corpus_holds( _run_main(tmp_path, monkeypatch, _CLAIMS_FAMILY, _DIFFERS, names_every_corpus=True) assert "Nobody Here, Esq." in str(exc.value) + assert not _WORKER_CALL def test_a_name_this_baseline_skipped_is_not_a_name_the_corpus_lost( diff --git a/tools/differential/compare.py b/tools/differential/compare.py index dda11319..8a506b45 100644 --- a/tools/differential/compare.py +++ b/tools/differential/compare.py @@ -2120,6 +2120,64 @@ def main() -> int: f"exactly like one for a hazard that is live:"] + [f" {v.earlier!r}\n declares precedence over {v.later!r}" for v in vacant])) + # A recorded diff shape naming a string no corpus holds any more + # (#497). _RECORDED_DIFFS pins the shape _CROSS_RULE_WINNERS feeds + # to classify() as an input, and a row nothing measures agrees with + # itself forever -- the exact shape of the defect that roster's + # shapes were found in. recorded_diff_mismatches cannot ask it: it + # skips a name it did not compare, which is right under `--corpus` + # and forgiving forever under the full gate, so this half is the + # caller's -- the only side that knows whether this run read every + # corpus. + # + # HERE, beside `vacant`, for the reason the block above gives: it + # reads the ledger and the loaded names and nothing the worker + # produces, and a refusal raised after the worker has installed the + # pinned wheel and compared the whole corpus prints below the run's + # own `baseline:` header, disowning a comparison it just published + # (#382). The measured half of this check is the opposite case and + # sits after the comparison, where the diff it reads exists. + # + # PRE-skip `corpus_names`, where the measured half takes the + # POST-skip `corpus`, and the two are not interchangeable. "Does + # any corpus still hold this name?" is about the FILES, and the + # baseline-minimum skip below takes a name out of the RUN and out + # of no file; "was this name compared?" is the other question, and + # an entry an old baseline cannot honor was not. Measured at + # --baseline 1.4.0 the two lists differ by three names -- 'de + # Mesnil Jean, Dr.', 'de la Cruz Juan Carlos, Dr.' and 'de la Cruz + # née Vega', each order-bearing with no order-None twin, each + # sitting in corpus_shapes.jsonl throughout. None of the three + # carries a roster row today, so the post-skip reading would refuse + # nothing YET: it would wait for the first row on a shape-tagged + # name and then tell a contributor to delete a row for a name the + # run had just read past. That is the earlier-draft `vacant` bug + # above, one check over. Recompute both lists with the recipe in + # recorded_diff_mismatches' docstring. + # + # `full_corpus`, not `args.corpus`, for the same reason `vacant` + # reads it. Under a narrowing this is SILENT rather than a NOTE, + # which is where the two checks part: `vacant` prints a count a + # reader might act on, and here recorded_diff_mismatches has + # already dropped those names without reporting one, so a NOTE + # would add noise and no information. Measured, it would name 18 to + # 30 of the 31 rows depending on which corpus was asked for (30 for + # corpus_shapes.jsonl) against `vacant`'s 5 to 11 of 11 -- recompute + # with `set(_RECORDED_DIFFS[ledger]) - {e["name"] for e in + # _load_entries(path)}` per corpus file. + recorded = _RECORDED_DIFFS.get(ledger.name, {}) + gone = sorted(set(recorded) - set(corpus_names)) + if gone and full_corpus: + raise SystemExit("\n".join( + [f"{ledger.name}: {len(gone)} recorded diff shape(s) name a " + f"string no corpus holds any more, over the FULL corpus. " + f"Nothing measures such a row, so it agrees with itself " + f"forever -- the shape of the defect #497 is about. Two " + f"repairs, and they are not interchangeable: restore the " + f"name to a corpus if the contest it pins is still worth " + f"pinning, or delete the row AND the _CROSS_RULE_WINNERS " + f"entry beside it if the name left on purpose:"] + + [f" {n!r}" for n in gone])) # an ORDER-BEARING entry must never reach a worker whose baseline # cannot honor it (no Policy below 2.0.0) -- skip it and say so, # rather than shrink the comparison silently. An order-NONE @@ -2342,79 +2400,54 @@ def _tree_parse(name: str, order: str | None) -> object: for issue, name, tagged in order_blind: print(f" {issue!r} explained {name!r}{_order_tag(tagged)}") print() - # Checked HERE because this is the only place both halves exist: - # the roster's recorded shape, and the diff this run actually - # measured. validate_rules runs before any corpus is read, and the - # unit suite spawns no worker -- deliberately -- so neither can ask - # this (#497). + # The measured half of the recorded-shape check (#497): the shape + # _CROSS_RULE_WINNERS pins against the diff this run actually made. + # HERE because this is the only place both exist -- validate_rules + # runs before any corpus is read, and the unit suite spawns no + # worker, deliberately, so neither can ask it. Its absent-name half + # needs none of that and refuses up beside `vacant`, pre-worker. # - # TWO NAME SETS, one line apart, and they are not interchangeable. - # `corpus` is POST-skip, and recorded_diff_mismatches takes it - # because its question is "was this name compared?" -- an - # order-bearing entry a baseline cannot honor was not, and a shape - # cannot be read off a comparison that never ran. `corpus_names` is - # PRE-skip, and the absent-name half below takes it because its - # question is "does any corpus still hold this name?" -- the skip - # takes a name out of the RUN and out of no file. Measured at - # --baseline 1.4.0 the two differ by three names -- 'de Mesnil - # Jean, Dr.', 'de la Cruz Juan Carlos, Dr.' and 'de la Cruz née - # Vega', each order-bearing with no order-None twin, each sitting - # in corpus_shapes.jsonl throughout. None of the three carries a - # roster row today, so reading the post-skip list here would refuse - # nothing YET: it would wait for the first row on a shape-tagged - # name and then tell a contributor to delete a row for a name the - # run just read past. That is the earlier-draft `vacant` bug - # recorded above, one check over. Recompute both lists with the - # recipe in recorded_diff_mismatches' docstring. + # PRINTS and feeds the exit code rather than raising, like + # over_declared_rules below -- its structural sibling, the other + # post-worker check on recorded data. A raise lands MID-REPORT and + # takes the rest of the run's output with it: measured, a + # `--corpus corpus.jsonl` run at 1.4.0 prints 62 EXPLAINED NOTHING + # lines, and 0 of them with one _RECORDED_DIFFS row corrupted, + # because the raise preceded dormancy, OVER-DECLARED, UNEXPLAINED + # and the radar block. A stale roster row must not hide an + # unexplained diff, which is the gate's primary output. The two + # pre-worker refusals raise before anything has printed, which is + # why they may; this one cannot. # - # The absent-name half is the CALLER's because only the caller - # knows whether this run read every corpus, and it splits on - # `full_corpus` for the vacancy check's reason: a roster row for a - # name outside a narrowed run reads exactly like one for a name - # that left. Under `--corpus` it is silent rather than a NOTE -- - # unlike `vacant` there is no count anyone could misread into a - # repair, since recorded_diff_mismatches has already skipped those - # names without reporting them. + # `compared` is the POST-skip name set -- main()'s `corpus` -- + # because the question is "was this name compared?", and a shape + # cannot be read off a comparison that never ran. The absent-name + # half reads the PRE-skip list, for the opposite reason spelled out + # there. compared = {str(n) for n in corpus} - recorded = _RECORDED_DIFFS.get(ledger.name, {}) shape_bad = recorded_diff_mismatches(recorded, diffing, compared) + for m in shape_bad: + print(f"MOVED SHAPE {ledger.name}: {m.name!r}\n " + f"_RECORDED_DIFFS records {list(m.recorded)}; this run " + + (f"measured {list(m.measured)}. " + if m.measured is not None else + # No cause is named, here or in the line above, + # because the check cannot tell the two apart -- + # recorded_diff_mismatches' docstring says why they + # are collapsed. Naming one would send half the + # readers of this line to the wrong file. + "measured no default-order diff of it. TWO causes " + "reach that and this check cannot separate them: the " + "parser may have stopped moving the name, or the name " + "may be compared under a declared order alone, " + "leaving no default-order comparison to have a shape " + "-- read the corpus entry before you read the parser. ") + + "A moved shape is a FINDING, not a number to update: " + "the winner pinned beside it in _CROSS_RULE_WINNERS " + "(tests/v2/test_ledger_guards.py) was recorded for the " + "OLD shape, so read both before editing either.") if shape_bad: - raise SystemExit("\n".join( - [f"{ledger.name}: {len(shape_bad)} recorded diff shape(s) in " - f"_RECORDED_DIFFS disagree with this run. A moved shape is " - f"a FINDING, not a number to update -- the parser changed " - f"what the name does, and the winner pinned beside it in " - f"_CROSS_RULE_WINNERS (tests/v2/test_ledger_guards.py) was " - f"recorded for the OLD shape, so read both before editing " - f"either:"] - + [f" {m.name!r} recorded {list(m.recorded)}, " - + (f"measured {list(m.measured)}" if m.measured is not None - else "and this run measured no default-order diff of it") - for m in shape_bad] - # Named for the vacancy refusal's reason: a message that - # offers repair advice for one of two reachable causes - # sends half its readers to the wrong file. The two are - # collapsed on purpose (recorded_diff_mismatches' docstring - # says why), so the message must not pick one. - + (["A row with no measured shape has TWO causes and this " - "check cannot tell them apart: the parser may have " - "stopped moving the name, or the name may be compared " - "under a declared order alone, leaving no default-order " - "comparison to have a shape. Read the corpus entry " - "before you read the parser."] - if any(m.measured is None for m in shape_bad) else []))) - gone = sorted(set(recorded) - set(corpus_names)) - if gone and full_corpus: - raise SystemExit("\n".join( - [f"{ledger.name}: {len(gone)} recorded diff shape(s) name a " - f"string no corpus holds any more, over the FULL corpus. " - f"Nothing measures such a row, so it agrees with itself " - f"forever -- the shape of the defect #497 is about. Two " - f"repairs, and they are not interchangeable: restore the " - f"name to a corpus if the contest it pins is still worth " - f"pinning, or delete the row AND the _CROSS_RULE_WINNERS " - f"entry beside it if the name left on purpose:"] - + [f" {n!r}" for n in gone])) + print() dormancy = dormant_rules(rules, set(by_issue), diffing, exclusions) for dormant in dormancy.undeclared: print(f"EXPLAINED NOTHING {dormant.issue!r}\n " @@ -2477,10 +2510,13 @@ def _tree_parse(name: str, order: str | None) -> object: # code does. A rule explaining LESS than it declares is the third # way that happens -- it still matches, so nothing here looks # broken, but the `fields` it names are no longer what the code - # moves. Same exit code for all three, so none of them is the one - # nobody noticed. + # moves. A recorded shape the run contradicts is the fourth, and it + # is the one that reaches furthest: _CROSS_RULE_WINNERS feeds that + # shape to classify() as an input, so a wrong one takes the roster's + # verdict with it. Same exit code for all four, so none of them is + # the one nobody noticed. return 1 if unexplained or dormancy.undeclared or dormancy.awake \ - or overwide else 0 + or overwide or shape_bad else 0 if __name__ == "__main__": From afa7f140e00917dee13db2ea0f2a655a23459dde Mon Sep 17 00:00:00 2001 From: Derek Gulbranson Date: Thu, 3 Sep 2026 03:06:42 -0700 Subject: [PATCH 13/23] tooling(differential): one home for the measurement, both files in the message Review fixes, all convention rather than design. The recompute one-liner beside `gone` did not run: `_RECORDED_DIFFS[ledger]` keys a dict of file names with a Path, one line above code that spells it `ledger.name`. Corrected and run; the figures it prints (18 to 30 of 31, 30 for corpus_shapes.jsonl) are unchanged. The three-name measurement had reached three copies and had already drifted -- the docstring ends the analogy "one baseline over" and the new comment said "one check over". This file argues against exactly that 60 lines below, so the docstring keeps it and the other two point there. One sentence stays in main() because it exists nowhere else and is an argument about the placement rather than about the function: none of the three names carries a roster row today, so the post-skip reading would refuse nothing yet. The departed-name refusal named neither file, while telling the reader to delete a row "beside" another in a different one, under a `{ledger.name}:` prefix pointing at a .toml where no repair happens. It now names compare.py and test_ledger_guards.py, and leads with the question that picks the repair -- did the name leave deliberately, `git log -S` over the corpora -- since both repairs can be right for one name. "THREE CHECKS READ THE NARROWING AT THREE DIFFERENT STRENGTHS" was four, in compare.py and in README.md, with the fourth arguing its strength 95 lines below the count. Both amended, and the README gains the MOVED SHAPE section it had for every peer check. MOVED SHAPE hoists its invariant text into a one-time block header instead of reprinting ~45 words per row, following the Role-vocabulary legend rather than OVER-DECLARED's one-sentence NOTE. The two-cause disclaimer is one-time AND conditional: a block whose rows all carry a measured shape should not print advice for a case that did not occur. Nits: full_corpus serves two checks now, not one; the exit-code comment counted four ways where the return has five terms, and never named NO LONGER DORMANT; `recorded`'s binding site is now half a clause away. Refs #497 Co-Authored-By: Claude Opus 5 --- tests/v2/test_differential.py | 32 ++++--- tools/differential/README.md | 20 ++-- tools/differential/compare.py | 169 +++++++++++++++++++++------------- 3 files changed, 134 insertions(+), 87 deletions(-) diff --git a/tests/v2/test_differential.py b/tests/v2/test_differential.py index 0735934e..1bfdb0d2 100644 --- a/tests/v2/test_differential.py +++ b/tests/v2/test_differential.py @@ -2055,6 +2055,13 @@ def test_main_reports_a_recorded_shape_the_run_contradicts( # shape was recorded for the OLD shape assert "FINDING" in out assert "_CROSS_RULE_WINNERS" in out + # The FINDING instruction leads the block once rather than riding + # every row, and the two-cause disclaimer beside it is conditional: + # every row here carries a measured shape, so advice for the + # unmeasured case would be advice for something that did not + # happen. Its sibling below pins the other branch. + assert out.count("FINDING") == 1 + assert "TWO" not in out # The verdict, not just the print. A block that reports and exits 0 # is read by CI as silence -- the same trap # test_main_exits_1_and_reports_an_unclassified_diff exists for. @@ -2110,6 +2117,9 @@ def test_the_shape_report_over_an_unmeasured_name_claims_no_cause( _SAME_FACADE) assert "no default-order diff" in out assert "declared order" in out + # ... and once, not once per row: the disclaimer is a property of + # the check, so it leads the block rather than riding every line. + assert out.count("TWO reach it") == 1 # Every line of the block, not just the one carrying the shape: the # header sentence is the one that used to assert the parser had # changed, which is false for the second cause. @@ -2189,19 +2199,15 @@ def test_naming_every_corpus_refuses_a_recorded_name_no_corpus_holds( def test_a_name_this_baseline_skipped_is_not_a_name_the_corpus_lost( tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None: - """The two name sets the check reads, one line apart, are opposite. - - "Was this name compared?" is about the RUN, and an order-bearing - entry an old baseline cannot honor was not -- so the shape half - reads the POST-skip list and stays quiet about it. "Does any corpus - still hold this name?" is about the FILES, and the skip removes a - name from no file -- so the absent-name half reads the PRE-skip - `corpus_names`. Swap them and this full run refuses, telling a - contributor to delete a roster row for a name sitting in the corpus - it just read. Measured on the shipped tree at --baseline 1.4.0 the - two lists differ by three names, all in corpus_shapes.jsonl, so the - hazard is live and only unarmed because none of the three carries a - roster row yet. + """The two name sets the check reads are opposite, and this is the + only instrument that can tell which one it read. + + Swap them and this full run refuses, telling a contributor to + delete a roster row for a name sitting in the corpus it just read. + The two questions, and the three names the lists differ by at + 1.4.0, are recorded once in recorded_diff_mismatches' docstring; + the shipped rosters cannot show the swap, because none of those + three carries a row yet. Hand-rolled rather than via _run_main, which writes bare-string corpus lines and so cannot produce a skipped entry at all; same diff --git a/tools/differential/README.md b/tools/differential/README.md index b580a91d..0188edd9 100644 --- a/tools/differential/README.md +++ b/tools/differential/README.md @@ -726,14 +726,18 @@ would delete an exemption the full gate needs and then fail the full run for the undeclared contest that reappears. So a vacancy is a hard failure on a full run and a printed NOTE under `--corpus`. -Three checks now read the flag differently, and the differences are -deliberate rather than untidy -- read them together before making any -of them uniform. The corpus-floor roster is SKIPPED entirely under -`--corpus`, because narrowing is the point of the flag. -`over_declared_rules` still FAILS the run and appends a NOTE saying -the union it computed is over a subset, so its repair advice is not -followed blindly. The vacancy check does not fail at all, because it -is the only one of the three whose verdict INVERTS under narrowing. +Four checks now read the flag differently, and the differences are deliberate rather than untidy -- read them together before making any of them uniform. The corpus-floor roster is SKIPPED entirely under `--corpus`, because narrowing is the point of the flag. `over_declared_rules` still FAILS the run and appends a NOTE saying the union it computed is over a subset, so its repair advice is not followed blindly. The vacancy check does not fail at all, because its verdict INVERTS under narrowing rather than merely its evidence. The departed-name half of the recorded-shape check (below) inverts the same way and goes one step further, printing nothing at all under `--corpus`: a NOTE there would name most of the roster and tell the reader nothing, where the vacancy NOTE names a handful. + +### `MOVED SHAPE`: the roster's recorded diffs, checked against a run + +`tests/v2/test_ledger_guards.py` carries `_CROSS_RULE_WINNERS`, a roster pinning which rule should win a contested name. To ask that question it needs the name's diff SHAPE, which it feeds to `classify()` as an input -- so the shape is never itself checked, and a guessed one agrees with itself forever. That is how `田中さん II` sat recorded as `{given, suffix}` under a docstring promising the shapes were measured; the real diff is `{family, given, suffix}` (#497). The unit suite cannot catch it: it spawns no worker, deliberately, so it has no measured diff to compare against. + +So the shapes live in `_RECORDED_DIFFS` in `compare.py`, keyed per ledger (a string moves a different set of roles against different baselines), and every run checks them. Two findings come out of it, and they behave differently on purpose. + +- **`MOVED SHAPE`** -- a recorded shape the run contradicts. Printed after the comparison, alongside `EXPLAINED NOTHING` and `OVER-DECLARED`, and it feeds the exit code the same way. It does not raise: a refusal there would land mid-report and take the `UNEXPLAINED` block down with it, and a stale roster row must never hide an unexplained diff. A moved shape is a FINDING, not a number to update -- the parser changed what the name does, and the winner pinned beside it in `_CROSS_RULE_WINNERS` was recorded for the OLD shape, so read both before editing either. Where the run measured no default-order diff at all, the report says so and names no cause: two states reach it (the parser stopped moving the name, or the name is compared only under a declared order) and the check cannot separate them. +- **A row naming a name no corpus holds** -- refused, before the worker runs, on a FULL run only. Nothing measures such a row, so it agrees with itself forever, which is the defect above in a second form. The repair is not mechanical: ask first whether the name left deliberately (`git log -S'' -- tools/differential/corpus*.jsonl`), then either delete the row and its `_CROSS_RULE_WINNERS` partner, or restore the name to the corpus that lost it. Under `--corpus` this prints nothing, since a narrowed run legitimately holds almost none of the roster. + +The two read opposite name lists, and swapping them is the live hazard. "Was this name compared?" is about the RUN, so the shape check reads the list AFTER the baseline-minimum shape skip. "Does any corpus still hold this name?" is about the FILES, and the skip empties no file, so the departed-name check reads the list BEFORE it. `recorded_diff_mismatches`' docstring in `compare.py` carries the measurement and the recompute. ### Shapes that must never be explained (`[[never]]`) diff --git a/tools/differential/compare.py b/tools/differential/compare.py index 8a506b45..020168e3 100644 --- a/tools/differential/compare.py +++ b/tools/differential/compare.py @@ -1934,10 +1934,12 @@ def main() -> int: # ask them. Skipped under --corpus, where narrowing is the point. # # The same question, asked of the names rather than of the flag, - # answers "is this run over the FULL corpus" for the vacancy check - # below -- which is what that check needs, and not the same as - # "was --corpus omitted": the flag is `action="append"`, so naming - # every corpus explicitly narrows nothing. + # answers "is this run over the FULL corpus" for the two checks + # below whose verdict inverts under narrowing -- the vacancy check + # and the departed-name half of the recorded-shape check (#497). + # That is what both need, and it is not the same as "was --corpus + # omitted": the flag is `action="append"`, so naming every corpus + # explicitly narrows nothing. missing = sorted(set(_CORPUS_FLOORS) - {p.name for p in paths}) full_corpus = not missing if not args.corpus: @@ -2060,17 +2062,22 @@ def main() -> int: # corpus_shapes.jsonl, and 8, 7 and 5 for the other three. So a # partial run NOTES that count and does not act on it. # - # THREE CHECKS READ THE NARROWING AT THREE DIFFERENT STRENGTHS, and + # FOUR CHECKS READ THE NARROWING AT FOUR DIFFERENT STRENGTHS, and # the differences are the point rather than an inconsistency to # tidy. The corpus-floor roster above is SKIPPED entirely, because # narrowing is what the flag is for. over_declared_rules still # FAILS the run -- `overwide` feeds the exit code on every run -- # and only appends a NOTE that the union it computed is over a # subset, so its repair advice is not followed blindly. `vacant` - # alone does not fail, because it is the only one of the three - # whose VERDICT inverts under narrowing rather than merely its - # evidence. Do not fold the two branches below back into one - # shape, and do not level the three checks onto one strength: an + # does not fail, because its VERDICT inverts under narrowing rather + # than merely its evidence. `gone` below -- the roster rows naming + # a departed name (#497) -- inverts the same way and goes one step + # further, staying SILENT: `vacant` prints a count a reader might + # act on, and there recorded_diff_mismatches has already dropped + # those names without reporting one, so a NOTE would carry noise + # and no information. Its own comment measures how much. + # Do not fold the two branches below back into one + # shape, and do not level the four checks onto one strength: an # earlier draft of `vacant` refused under `--corpus` and told the # contributor to delete legitimate exemptions. # @@ -2139,44 +2146,53 @@ def main() -> int: # sits after the comparison, where the diff it reads exists. # # PRE-skip `corpus_names`, where the measured half takes the - # POST-skip `corpus`, and the two are not interchangeable. "Does - # any corpus still hold this name?" is about the FILES, and the - # baseline-minimum skip below takes a name out of the RUN and out - # of no file; "was this name compared?" is the other question, and - # an entry an old baseline cannot honor was not. Measured at - # --baseline 1.4.0 the two lists differ by three names -- 'de - # Mesnil Jean, Dr.', 'de la Cruz Juan Carlos, Dr.' and 'de la Cruz - # née Vega', each order-bearing with no order-None twin, each - # sitting in corpus_shapes.jsonl throughout. None of the three - # carries a roster row today, so the post-skip reading would refuse - # nothing YET: it would wait for the first row on a shape-tagged - # name and then tell a contributor to delete a row for a name the - # run had just read past. That is the earlier-draft `vacant` bug - # above, one check over. Recompute both lists with the recipe in - # recorded_diff_mismatches' docstring. + # POST-skip `corpus`. The two questions, why they take opposite + # lists, the three names they differ by at 1.4.0 and the recompute + # are recorded ONCE, in recorded_diff_mismatches' docstring -- read + # it before touching either line. One thing is not there and lives + # here, because it is an argument about THIS placement and not + # about that function: none of those three names carries a roster + # row today, so the post-skip reading would refuse nothing YET. It + # would wait for the first row on a shape-tagged name and then tell + # a contributor to delete a row for a name the run had just read + # past, which is why the hazard is invisible to the gate and has to + # be argued rather than measured. # # `full_corpus`, not `args.corpus`, for the same reason `vacant` # reads it. Under a narrowing this is SILENT rather than a NOTE, # which is where the two checks part: `vacant` prints a count a # reader might act on, and here recorded_diff_mismatches has # already dropped those names without reporting one, so a NOTE - # would add noise and no information. Measured, it would name 18 to - # 30 of the 31 rows depending on which corpus was asked for (30 for - # corpus_shapes.jsonl) against `vacant`'s 5 to 11 of 11 -- recompute - # with `set(_RECORDED_DIFFS[ledger]) - {e["name"] for e in - # _load_entries(path)}` per corpus file. + # would add noise and no information. Measured 2026-09-03, it would + # name 18 to 30 of the 31 rows depending on which corpus was asked + # for (30 for corpus_shapes.jsonl) against `vacant`'s 5 to 11 of + # 11. Recompute, from the worktree root: + # uv run python -c "import sys;sys.path.insert(0,'tools/\ + # differential');import compare,pathlib;r=set(compare.\ + # _RECORDED_DIFFS['expected_since_1.4.0.toml']);[print(p.name,\ + # len(r-{str(e['name']) for e in compare._load_entries(p)}),\ + # 'of',len(r)) for p in sorted(pathlib.Path('tools/differential')\ + # .glob('corpus*.jsonl'))]" + # `ledger` is a Path, so the key is `ledger.name` -- as the line + # below has it, and as an earlier draft of this recipe did not. recorded = _RECORDED_DIFFS.get(ledger.name, {}) gone = sorted(set(recorded) - set(corpus_names)) if gone and full_corpus: raise SystemExit("\n".join( - [f"{ledger.name}: {len(gone)} recorded diff shape(s) name a " - f"string no corpus holds any more, over the FULL corpus. " - f"Nothing measures such a row, so it agrees with itself " - f"forever -- the shape of the defect #497 is about. Two " - f"repairs, and they are not interchangeable: restore the " - f"name to a corpus if the contest it pins is still worth " - f"pinning, or delete the row AND the _CROSS_RULE_WINNERS " - f"entry beside it if the name left on purpose:"] + [f"tools/differential/compare.py: {len(gone)} row(s) in " + f"_RECORDED_DIFFS[{ledger.name!r}] name a string no corpus " + f"holds any more, over the FULL corpus. Nothing measures " + f"such a row, so it agrees with itself forever -- the shape " + f"of the defect #497 is about. Settle the question that " + f"picks the repair first: did the name leave DELIBERATELY? " + f"`git log -S'' -- tools/differential/corpus*.jsonl` " + f"answers it. If it did, delete the row here AND its " + f"partner in _CROSS_RULE_WINNERS " + f"(tests/v2/test_ledger_guards.py), which pins a winner " + f"for a contest nothing raises now. If it did not, restore " + f"the name to a corpus -- the corpus is what regressed. " + f"Both can be true of one name, which is why the question " + f"comes before the edit:"] + [f" {n!r}" for n in gone])) # an ORDER-BEARING entry must never reach a worker whose baseline # cannot honor it (no Policy below 2.0.0) -- skip it and say so, @@ -2423,29 +2439,49 @@ def _tree_parse(name: str, order: str | None) -> object: # because the question is "was this name compared?", and a shape # cannot be read off a comparison that never ran. The absent-name # half reads the PRE-skip list, for the opposite reason spelled out - # there. + # there, and `recorded` is the per-ledger row dict both halves read, + # bound up beside it. compared = {str(n) for n in corpus} shape_bad = recorded_diff_mismatches(recorded, diffing, compared) + if shape_bad: + # The instruction and the disclaimer are properties of the + # CHECK, not of a row, so they lead the block once instead of + # riding every line -- the shape the Role-vocabulary legend + # below uses for the two blocks that share it. A real parser + # move lands on many of these rows at once, and per-row + # repetition would bury the names under its own advice. + # (OVER-DECLARED repeats per row, but its shared text is one + # sentence.) No cause is named for a row with no measured + # shape: the check cannot tell the two apart -- + # recorded_diff_mismatches' docstring says why they are + # collapsed -- and naming one would send half the readers to + # the wrong file. + print(f"MOVED SHAPE {ledger.name}: {len(shape_bad)} recorded " + f"diff shape(s) disagree with this run. Each is a " + f"FINDING, not a number to update: the winner pinned " + f"beside the shape in _CROSS_RULE_WINNERS " + f"(tests/v2/test_ledger_guards.py) was recorded for the " + f"OLD shape, so read both before editing either." + # The disclaimer is one-time like the sentence above it, + # but conditional as well: a block whose rows all carry a + # measured shape would otherwise print repair advice for + # a case that did not occur, which is what + # OVER-DECLARED's `--corpus` NOTE is conditional to + # avoid. + + ("\n A row reading 'measured no default-order diff' " + "names no cause because TWO reach it and this check " + "cannot separate them: the parser may have stopped " + "moving the name, or the name may be compared under a " + "declared order alone, leaving no default-order " + "comparison to have a shape. Read the corpus entry " + "before you read the parser." + if any(m.measured is None for m in shape_bad) else "")) for m in shape_bad: - print(f"MOVED SHAPE {ledger.name}: {m.name!r}\n " - f"_RECORDED_DIFFS records {list(m.recorded)}; this run " - + (f"measured {list(m.measured)}. " - if m.measured is not None else - # No cause is named, here or in the line above, - # because the check cannot tell the two apart -- - # recorded_diff_mismatches' docstring says why they - # are collapsed. Naming one would send half the - # readers of this line to the wrong file. - "measured no default-order diff of it. TWO causes " - "reach that and this check cannot separate them: the " - "parser may have stopped moving the name, or the name " - "may be compared under a declared order alone, " - "leaving no default-order comparison to have a shape " - "-- read the corpus entry before you read the parser. ") - + "A moved shape is a FINDING, not a number to update: " - "the winner pinned beside it in _CROSS_RULE_WINNERS " - "(tests/v2/test_ledger_guards.py) was recorded for the " - "OLD shape, so read both before editing either.") + measured = (f"measured {list(m.measured)}" + if m.measured is not None + else "measured no default-order diff of it") + print(f" {m.name!r}\n _RECORDED_DIFFS records " + f"{list(m.recorded)}; this run {measured}") if shape_bad: print() dormancy = dormant_rules(rules, set(by_issue), diffing, exclusions) @@ -2506,15 +2542,16 @@ def _tree_parse(name: str, order: str | None) -> object: _print_field_diffs(old_facade, new, old_v2, new_v2, order, initials_only=initials_only) # A rule explaining nothing is as much a broken contract as an - # unexplained diff: both mean the ledger no longer describes what the - # code does. A rule explaining LESS than it declares is the third - # way that happens -- it still matches, so nothing here looks - # broken, but the `fields` it names are no longer what the code - # moves. A recorded shape the run contradicts is the fourth, and it - # is the one that reaches furthest: _CROSS_RULE_WINNERS feeds that - # shape to classify() as an input, so a wrong one takes the roster's - # verdict with it. Same exit code for all four, so none of them is - # the one nobody noticed. + # unexplained diff: both mean the ledger no longer describes what + # the code does. A rule that has STOPPED explaining nothing is the + # same statement inverted -- its `dormant` reason is now false. A + # rule explaining LESS than it declares is a fourth way, and the + # quietest: it still matches, so nothing looks broken, but the + # `fields` it names are no longer what the code moves. A recorded + # shape the run contradicts is the fifth, and it reaches furthest: + # _CROSS_RULE_WINNERS feeds that shape to classify() as an input, so + # a wrong one takes the roster's verdict with it. One exit code for + # all five terms below, so none of them is the one nobody noticed. return 1 if unexplained or dormancy.undeclared or dormancy.awake \ or overwide or shape_bad else 0 From 22d6b9d515f71bce5bf1b7a49fb3eb2ed8520386 Mon Sep 17 00:00:00 2001 From: Derek Gulbranson Date: Thu, 3 Sep 2026 03:08:30 -0700 Subject: [PATCH 14/23] docs(differential): a moved shape names no cause in the README either The code header dropped 'the parser changed what the name does' because two causes reach a moved shape and the check cannot separate them. The README bullet kept it -- and the claim is false in the case that motivated the check: the four rows #497 deleted recorded shapes no run makes, wrong when written rather than overtaken by a parser change. Refs #497 Co-Authored-By: Claude Opus 5 --- tools/differential/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tools/differential/README.md b/tools/differential/README.md index 0188edd9..66b9fb08 100644 --- a/tools/differential/README.md +++ b/tools/differential/README.md @@ -734,7 +734,7 @@ Four checks now read the flag differently, and the differences are deliberate ra So the shapes live in `_RECORDED_DIFFS` in `compare.py`, keyed per ledger (a string moves a different set of roles against different baselines), and every run checks them. Two findings come out of it, and they behave differently on purpose. -- **`MOVED SHAPE`** -- a recorded shape the run contradicts. Printed after the comparison, alongside `EXPLAINED NOTHING` and `OVER-DECLARED`, and it feeds the exit code the same way. It does not raise: a refusal there would land mid-report and take the `UNEXPLAINED` block down with it, and a stale roster row must never hide an unexplained diff. A moved shape is a FINDING, not a number to update -- the parser changed what the name does, and the winner pinned beside it in `_CROSS_RULE_WINNERS` was recorded for the OLD shape, so read both before editing either. Where the run measured no default-order diff at all, the report says so and names no cause: two states reach it (the parser stopped moving the name, or the name is compared only under a declared order) and the check cannot separate them. +- **`MOVED SHAPE`** -- a recorded shape the run contradicts. Printed after the comparison, alongside `EXPLAINED NOTHING` and `OVER-DECLARED`, and it feeds the exit code the same way. It does not raise: a refusal there would land mid-report and take the `UNEXPLAINED` block down with it, and a stale roster row must never hide an unexplained diff. A moved shape is a FINDING, not a number to update, and it names no cause because it cannot: the parser may have changed what the name does, or the row may have been wrong when it was recorded -- which is what #497 found, four rows recording a shape no run makes under a roster promising the shapes were measured. Either way the winner pinned beside it in `_CROSS_RULE_WINNERS` was recorded against the OLD shape, so read both before editing either. Where the run measured no default-order diff at all, the report says so and names no cause: two states reach it (the parser stopped moving the name, or the name is compared only under a declared order) and the check cannot separate them. - **A row naming a name no corpus holds** -- refused, before the worker runs, on a FULL run only. Nothing measures such a row, so it agrees with itself forever, which is the defect above in a second form. The repair is not mechanical: ask first whether the name left deliberately (`git log -S'' -- tools/differential/corpus*.jsonl`), then either delete the row and its `_CROSS_RULE_WINNERS` partner, or restore the name to the corpus that lost it. Under `--corpus` this prints nothing, since a narrowed run legitimately holds almost none of the roster. The two read opposite name lists, and swapping them is the live hazard. "Was this name compared?" is about the RUN, so the shape check reads the list AFTER the baseline-minimum shape skip. "Does any corpus still hold this name?" is about the FILES, and the skip empties no file, so the departed-name check reads the list BEFORE it. `recorded_diff_mismatches`' docstring in `compare.py` carries the measurement and the recompute. From 22e3b2414d566e8a989ae2069f16d1d5d25757dd Mon Sep 17 00:00:00 2001 From: Derek Gulbranson Date: Thu, 3 Sep 2026 03:20:24 -0700 Subject: [PATCH 15/23] tooling(differential): keep the findings, drop the digits that rot Sorted by decisions.md's own criterion: a past-tense count records a fixed state and stays; a count standing beside the names it counts is redundant with a self-checking list; a live exhaustiveness claim with nothing behind it is regrounded. 'This rule now claims exactly one name' measured eight. Also fixes the three "all five corpora" claims routed here by the review of Task 1: there are six corpus*.jsonl on disk, and a run globs them all, so the corpus count was never the argument. Refs #497 Co-Authored-By: Claude Opus 5 --- tests/v2/test_differential.py | 3 +- tests/v2/test_ledger_guards.py | 34 +++++++----- tools/differential/README.md | 4 +- tools/differential/build_issues_corpus.py | 6 +-- tools/differential/expected_since_1.4.0.toml | 56 ++++++++++++-------- tools/differential/expected_since_2.0.0.toml | 14 +++-- tools/differential/expected_since_2.1.0.toml | 14 +++-- tools/differential/expected_since_2.2.0.toml | 7 ++- 8 files changed, 85 insertions(+), 53 deletions(-) diff --git a/tests/v2/test_differential.py b/tests/v2/test_differential.py index 1bfdb0d2..1d4cbc0d 100644 --- a/tests/v2/test_differential.py +++ b/tests/v2/test_differential.py @@ -2394,7 +2394,8 @@ def test_a_misspelled_corpus_key_is_a_hard_error( silently did not happen: the line compares under the default order while its author believes they declared a family-first one. The message names the FILE, like every other loader error here, because - five corpora are read in one run.""" + one run reads every corpus in the directory and a message without a + filename says nothing about which one to open.""" import json as _json corpus = tmp_path / "corpus_x.jsonl" corpus.write_text(_json.dumps( diff --git a/tests/v2/test_ledger_guards.py b/tests/v2/test_ledger_guards.py index b41a5861..5f6b3b7d 100644 --- a/tests/v2/test_ledger_guards.py +++ b/tests/v2/test_ledger_guards.py @@ -445,9 +445,10 @@ def test_every_span_bearing_rule_matches_the_script_ranges( # written in none of them -- and unlike a depth test, this does # not care how the widening is spelled. "(?:CJK|[A-Za-z])" # hides the pipe at depth 1 where the check above stops - # looking, and claims 644 of the 654 unclassified corpus names; - # this sees - # it. Both are kept: the depth test gives the clearer message + # looking, and its Latin half claims all but a handful of + # _UNCLASSIFIED_NAMES -- the whole population this assertion is + # over, whatever that population's size that day; this sees it. + # Both are kept: the depth test gives the clearer message # for the naive spelling, and catches a widening toward a # script the corpora happen not to contain. unclassified = _UNCLASSIFIED_NAMES.intersection(_claimed(regex)) @@ -761,8 +762,11 @@ def test_case_shape_ids_exist_in_the_inventory() -> None: "fix(#400/#274)": ("abd Berg née Jones", "abd Allah Smith", "Jane Smith née Jones"), # The literal-anchored rules #413 added. Each claims exactly one - # corpus name, so _CORPUS_CLAIMS cannot see a widening that reaches - # only names the corpora lack -- these probes are the only wall. + # corpus name -- _CORPUS_CLAIMS carries that reach for both, so the + # number is pinned there rather than asserted here -- and a reach + # that small is one _CORPUS_CLAIMS cannot use: a widening reaching + # only names the corpora lack leaves it unmoved. These probes are + # the only wall. "fix(#399) a maiden marker bounds the particle chain: the geb. spelling": ("Berg, Ursula von der geb. Albrecht",), "fix(credential-pair-order) a split credential and a suffix render in written order": @@ -1421,10 +1425,11 @@ class _LatinCopy(NamedTuple): # fields and share the first set; the eighth ('John née Jones # Smith Ma') moves those plus `suffix`, and the ninth ('Smith # (née Jones)') keeps the rule it already had. At 2.0.0/2.1.0 the - # second set holds six that move two fields: five of the 1.4 - # seven, plus that acronym name, whose acronym is no part of the - # diff at those baselines. The two the second set drops are the - # compounds with #412 and #424, each with a rule of its own. + # second set holds the ones that move two fields: the first set + # less the two compounds it drops -- with #412 and with #424, each + # with a rule of its own -- plus that acronym name, whose acronym + # is no part of the diff at those baselines. Read the composition + # off the two literals below rather than off a count here. # # Spelled out rather than written as the shape -- one word, then a # marker -- because that shape also matches rules.md#M4's two @@ -2209,15 +2214,15 @@ def _claim(rule: dict) -> _Claim: _Claim(27, ('_initials',), "6b242c287db8", ('DEFAULT',)), "fix(#360) los joined the particles, so it no longer initials": _Claim(1, ('_initials',), "cd721215f463", ('DEFAULT',)), - # fix(#462)'s reach is 18 where it explains 14. Its regex is a - # letter SHAPE rather than a name list, and the four extra -- + # fix(#462) reaches more than it explains -- the reach is the + # _Claim below, and the gap is the names named here. Its regex + # is a letter SHAPE rather than a name list, and the extra -- # 'E Anne D', 'E Jones', 'E Maria', 'Y. L.' -- carry the E/Y in # the GIVEN group, which has always initialed every word it # holds whatever the vocabulary says, so the fix does not move # them. The discriminator is the GROUP and not the position: # 'E Anne D,Leonardo' also leads with the E, but its comma - # re-roles the whole run into the FAMILY group, and it is one - # of the 14 that move. + # re-roles the whole run into the FAMILY group, and it moves. # The digest is the same in all three 2.x ledgers because the # regex is the same string in each. "fix(#462) the facade keeps an initial-shaped conjunction letter": @@ -2225,7 +2230,8 @@ def _claim(rule: dict) -> _Claim: }, # The 2.3 cycle's first rule, and a facade-only render fix: every # role is identical, so `_initials` alone. Reach and digest as in - # the 2.0.0 mapping above, the same regex classifying the same 14. + # the 2.0.0 mapping above, the same regex classifying the same + # names. "expected_since_2.2.0.toml": { "fix(#462) the facade keeps an initial-shaped conjunction letter": _Claim(18, ('_initials',), "3dd0e0276be6", ('DEFAULT',)), diff --git a/tools/differential/README.md b/tools/differential/README.md index 66b9fb08..beedd6f6 100644 --- a/tools/differential/README.md +++ b/tools/differential/README.md @@ -315,8 +315,8 @@ the moment a ` Ph. D.` name appeared. Backticks prompted a second look at prose, and the same change added two screens neither branch had: `:` joins the structural characters (it -appears in no name across all five corpora, and accounts for three -error messages and a PyPI trove classifier), and a short list of English function words +appears in no name in any corpus, and accounted for three error +messages and a PyPI trove classifier), and a short list of English function words rejects capitalized sentences the character screen cannot see — `What this gate does not cover` is well-formed as a phrase. That list is narrow on purpose: `and`, `the`, `of`, `will`, `can` and `do` all diff --git a/tools/differential/build_issues_corpus.py b/tools/differential/build_issues_corpus.py index b78234a4..20cf996a 100644 --- a/tools/differential/build_issues_corpus.py +++ b/tools/differential/build_issues_corpus.py @@ -74,9 +74,9 @@ # strings a quoted phrase rarely produces -- 'Constants.__init__(self, # **state)', 'DEVIATION #364', 'SUFFIX_ACRONYMS ∩ SUFFIX_NOT_ACRONYMS', # 'TypeError: a bytes-like object is required'. None of '*#~;_:' -# appears in a name; the colon alone accounts for three error messages -# and a PyPI trove classifier, and none of the six characters occurs in -# any name across all five corpora. The one existing entry this drops, +# appears in a name; the colon alone accounted for three error +# messages and a PyPI trove classifier, and none of them occurs in any +# name in any corpus. The one existing entry this drops, # 'St. ___', is a placeholder, not a name. _NOT_A_NAME = set('{}<>=/\\|*#~;_:') diff --git a/tools/differential/expected_since_1.4.0.toml b/tools/differential/expected_since_1.4.0.toml index 3160cc7c..a3937ef7 100644 --- a/tools/differential/expected_since_1.4.0.toml +++ b/tools/differential/expected_since_1.4.0.toml @@ -189,8 +189,10 @@ issue = "fix(#274) maiden markers consumed" # meant: a marker with a name word BEFORE it and one AFTER it. Reach # 16 -> 11 when they were added, and every name it explained then was # inside the eleven -- figures from that measurement rather than from -# today's corpus, which has grown since. Today it explains 12 -# (measured 2026-08-27, after #445). +# today's corpus, which has grown since and keeps growing. For today's, +# read the `## fix(#274) maiden markers consumed (N)` heading off +# `compare.py --baseline 1.4.0`; a number written here instead would +# only be that heading's value on the day someone typed it. # # Five claimed names still do not move, recorded rather than # excluded: 'Jane Smith, née Jones' (post-comma, M2's remaining @@ -512,13 +514,21 @@ issue = "fix(comma-family) lone post-comma piece routes to suffix/title, not fir # and file order broke the tie -- in favour of the one written first, # which was this one. # -# Measured, narrowing it: corpus reach 236 -> 215, and this rule now -# claims exactly one name, 'Andrews, M.D.', which is the shape named -# above. All seven moved to fix(cjk-comma-compound), which grew 10 -> -# 17 and is the rule named for that shape. The gate is unmoved at 108 +# Measured at the narrowing: corpus reach 236 -> 215, and all seven +# CJK names moved to fix(cjk-comma-compound), which grew 10 -> 17 and +# is the rule named for that shape. The gate was unmoved at 108 # intentional / 0 unexplained -- nothing became unexplained, so no name # lost an explanation, it only changed which rule gave it. # +# Those are that narrowing's figures and none of them is today's. What +# the narrowing pinned is the SCRIPT bound, not a count: this rule goes +# on claiming the Latin comma-names, and their number grows with the +# corpus. The sentence that stood here said the rule "now claims +# exactly one name, 'Andrews, M.D.'"; re-measured 2026-09-03 it +# explained eight, 'Andrews, M.D.' among them. For today's, read the +# `## fix(comma-family) lone post-comma piece routes to suffix/title, +# not first (N)` heading off `compare.py --baseline 1.4.0`. +# # The cut is at U+0250, the threshold _is_latin_only uses in # compare.py, matching the trailing-'Ph. D.' exclusion. Anchored at # both ends because a name is Latin or it is not: an unanchored class @@ -790,11 +800,13 @@ issue = "fix(emoji-boundary) an emoji inside a token divides it" # # The emoji must stand BETWEEN two non-space characters, which is the # behavior: it divides a token. A bare character class was the first -# draft and is rejected -- measured, it reaches four corpus names, and -# the three it adds carry a spaced or trailing emoji, are byte-identical -# on both sides today, and would be pre-excused by a rule that claims -# them. Pre-claiming shapes it has never seen is the #372 failure mode -# the fix(#379) comment names. +# draft and is rejected -- measured, it reaches past 'John😀Smith' to +# corpus names carrying a spaced or trailing emoji, which are +# byte-identical on both sides today and would be pre-excused by a rule +# that claims them. How many such names the corpus holds is not the +# argument and grows with the corpus; that the draft admits the class +# at all is. Pre-claiming shapes it has never seen is the #372 failure +# mode the fix(#379) comment names. # # The class is the ASTRAL HALF of _tokenize.py's _EMOJI_RANGES, hand # copied and pinned by test_the_emoji_boundary_rule_copies_the_dividing_ranges. @@ -2472,7 +2484,8 @@ issue = "fix(suffix-routing) a two-token name ending in a roman numeral keeps it # two rules claiming one token would leave file order to decide which # explains it. # -# The regex also reaches two corpus names this rule does not explain. +# The regex also reaches corpus names this rule does not explain, and +# the list is the count: # 'Carod i' is fix(#397)'s NOT-WANTED rule above -- a real contest, # same fields and both regexes matching, decided by nothing but file # order -- and '田中さん II' is fix(cjk-glued-honorific-peel)'s, where @@ -2504,21 +2517,22 @@ issue = "fix(suffix-routing) a two-token name ending in the suffix word jr keeps # become an alternation and be enrolled against SUFFIX_WORDS then, not # be smuggled in as another literal. # -# The regex also reaches four corpus names this rule does not explain. -# 'Kim, Jr.' and 'Smith, Jr.' are fix(#296)'s lone post-comma +# The regex also reaches corpus names this rule does not explain, and +# the list is the count. Three of them diff and belong to two other +# rules: 'Kim, Jr.' and 'Smith, Jr.' are fix(#296)'s lone post-comma # credential and '김민준씨 Jr.' is fix(cjk-glued-honorific-peel)'s. -# Three names, two rules, and what keeps them there is `fields`, not -# file order: measured, they diff {family, given, suffix, title} and +# What keeps them there is `fields`, not file order: measured, they +# diff {family, given, suffix, title} and # {family, given, suffix}, and neither is a subset of the two roles # below, so this rule is INELIGIBLE for all three wherever it sits. # Order is the second line of defence, not the first -- a widening of # `fields` alone would still lose to the rules above, and it would # take a widening AND a move to hand any of them over. -# _CROSS_RULE_WINNERS pins all three against that pair of edits. The -# fourth, 'Doe,, Jr.', does not diff against this baseline at all, so -# it has no winner to lose and no row. That reach is real either way, so none of the four is -# a _MUST_NOT_MATCH probe here: that roster tests the regex, not -# classify(). +# _CROSS_RULE_WINNERS pins all three against that pair of edits. +# 'Doe,, Jr.' is reached too and does not diff against this baseline +# at all, so it has no winner to lose and no row. That reach is real +# either way, so none of the reached names is a _MUST_NOT_MATCH probe +# here: that roster tests the regex, not classify(). name_regex = "(?i)^\\S+\\s+jr\\.?$" fields = ["family", "suffix"] diff --git a/tools/differential/expected_since_2.0.0.toml b/tools/differential/expected_since_2.0.0.toml index b1b55354..7d0f4915 100644 --- a/tools/differential/expected_since_2.0.0.toml +++ b/tools/differential/expected_since_2.0.0.toml @@ -1253,9 +1253,10 @@ issue = "fix(#445) the lone name word beside a marker a connective join no longe # 'and Jones Smith'. # # Its own rule rather than a widening of the fix(#412) rule above, -# which claims two names: the other one keeps a given name, and one -# rule holding both would have to declare `given` for both and -# pre-excuse its loss where losing it would be a regression. At 1.4.0 +# whose regex reaches this name too: the OTHER name it claims -- the +# one it explains, 'Jane van der Berg née y Jones' -- keeps a given +# name, and one rule holding both would have to declare `given` for +# both and pre-excuse its loss where losing it would be a regression. At 1.4.0 # the fix(#445) rule carries this name with the same four fields, v1 # having read the marker as a middle word. name_regex = "(?i)^Jane n[ée]e and Jones Smith$" @@ -1420,8 +1421,11 @@ issue = "fix(#462) the facade keeps an initial-shaped conjunction letter" # dotted E./Y. of either case, standing as its own word. Lowercase # bare e/y is the connective and does not move. The regex reaches 18 # corpus names, of which 14 diffed when the rule was written -# (2026-09-02). The four it reaches and does not move are what the -# gap IS, and the invariant behind it is a GROUP rather than a count: +# (2026-09-02); _CORPUS_CLAIMS in tests/v2/test_ledger_guards.py pins +# that reach and fails if it moves, so the digit here is checked +# rather than remembered. The names it reaches and does not move are +# what the gap IS, and the invariant behind it is a GROUP rather than +# a count: # they carry the E/Y in the GIVEN group, which has always initialed # every word it holds whatever the vocabulary says, so there was # nothing for the fix to restore. diff --git a/tools/differential/expected_since_2.1.0.toml b/tools/differential/expected_since_2.1.0.toml index f60f7148..6d2840e7 100644 --- a/tools/differential/expected_since_2.1.0.toml +++ b/tools/differential/expected_since_2.1.0.toml @@ -1202,9 +1202,10 @@ issue = "fix(#445) the lone name word beside a marker a connective join no longe # 'and Jones Smith'. # # Its own rule rather than a widening of the fix(#412) rule above, -# which claims two names: the other one keeps a given name, and one -# rule holding both would have to declare `given` for both and -# pre-excuse its loss where losing it would be a regression. At 1.4.0 +# whose regex reaches this name too: the OTHER name it claims -- the +# one it explains, 'Jane van der Berg née y Jones' -- keeps a given +# name, and one rule holding both would have to declare `given` for +# both and pre-excuse its loss where losing it would be a regression. At 1.4.0 # the fix(#445) rule carries this name with the same four fields, v1 # having read the marker as a middle word. name_regex = "(?i)^Jane n[ée]e and Jones Smith$" @@ -1369,8 +1370,11 @@ issue = "fix(#462) the facade keeps an initial-shaped conjunction letter" # dotted E./Y. of either case, standing as its own word. Lowercase # bare e/y is the connective and does not move. The regex reaches 18 # corpus names, of which 14 diffed when the rule was written -# (2026-09-02). The four it reaches and does not move are what the -# gap IS, and the invariant behind it is a GROUP rather than a count: +# (2026-09-02); _CORPUS_CLAIMS in tests/v2/test_ledger_guards.py pins +# that reach and fails if it moves, so the digit here is checked +# rather than remembered. The names it reaches and does not move are +# what the gap IS, and the invariant behind it is a GROUP rather than +# a count: # they carry the E/Y in the GIVEN group, which has always initialed # every word it holds whatever the vocabulary says, so there was # nothing for the fix to restore. diff --git a/tools/differential/expected_since_2.2.0.toml b/tools/differential/expected_since_2.2.0.toml index 8194499f..ffa11422 100644 --- a/tools/differential/expected_since_2.2.0.toml +++ b/tools/differential/expected_since_2.2.0.toml @@ -49,8 +49,11 @@ issue = "fix(#462) the facade keeps an initial-shaped conjunction letter" # dotted E./Y. of either case, standing as its own word. Lowercase # bare e/y is the connective and does not move. The regex reaches 18 # corpus names, of which 14 diffed when the rule was written -# (2026-09-02). The four it reaches and does not move are what the -# gap IS, and the invariant behind it is a GROUP rather than a count: +# (2026-09-02); _CORPUS_CLAIMS in tests/v2/test_ledger_guards.py pins +# that reach and fails if it moves, so the digit here is checked +# rather than remembered. The names it reaches and does not move are +# what the gap IS, and the invariant behind it is a GROUP rather than +# a count: # they carry the E/Y in the GIVEN group, which has always initialed # every word it holds whatever the vocabulary says, so there was # nothing for the fix to restore. From 8593f3a9376bdc0083044489e79836a63072c9e9 Mon Sep 17 00:00:00 2001 From: Derek Gulbranson Date: Thu, 3 Sep 2026 03:26:26 -0700 Subject: [PATCH 16/23] tooling(differential): the three the first sweep identified and skipped Follow-up to 22e3b24, which flagged these and left them. They are the issue's own class and leaving them shipped a sweep that skipped known instances. _reaches_non_vocabulary's docstring: figures taken against a 751-name corpus (634/751, 542, 592, 783 lines) against 1116 distinct names from 1251 lines today. Past-tensed the demonstration and dropped the digits: the finding is that a member unconstrained outside one 3-character window walks past every hand-picked probe, which no count strengthens, and the eight-string tuple it describes no longer exists. The rule's own claim figure is not restated because it could not be reproduced -- a replacement digit nobody can re-derive is the defect, not the fix. _MUST_NOT_MATCH's preamble said "Six such widenings" over a list of four. The enumeration is the evidence and is self-checking; the count was the only part that could be wrong, and nobody can recover which two are missing. fix(#274)'s ledger comment said "Five claimed names" with five enumerated beside it, and walked "all 31 the regex claims" where the regex claims 32 today -- a live count inside the very paragraph that names stale counts as this file's standing hazard. Three corrections to the sweep's own scope, for the record: - the step-1 scan's 19 matches included TWO false positives -- a version number at expected_since_1.4.0.toml's fix(leading-credential) note ("explained 2.0 routing"), and a hypothesis rather than a measurement at _MUST_NOT_MATCH ("a rule ... that claims exactly ONE corpus name CAN BE widened"). The real machine-readable population was 17. - the scan's file list omitted expected_since_2.2.0.toml, which carried a 20th copy of the fix(#462) claim. Treated with its two siblings. - the scan's predicate matched only lines starting with '#', so every docstring claim was invisible to it. That is how the _reaches_non_vocabulary figures survived. A docstring pass over both guard files found one further stale digit (_carries said the isascii() split covers "12 of the 16 entries" where MAIDEN_MARKERS ships 17), fixed here by naming the predicate instead of counting it. Three other docstring counts measured correct and stay: _carries' own two/eight, which already carry a runnable recompute; the 256 shapes, which is 2**8 - 1 role subsets plus the standalone _initials that validate_rules refuses beside any other field; and order_contests' 1350/657/2, dated and carrying a recompute recipe. Refs #497 Co-Authored-By: Claude Opus 5 --- tests/v2/test_ledger_guards.py | 41 +++++++++++++------- tools/differential/expected_since_1.4.0.toml | 16 ++++---- 2 files changed, 36 insertions(+), 21 deletions(-) diff --git a/tests/v2/test_ledger_guards.py b/tests/v2/test_ledger_guards.py index 5f6b3b7d..9ac46ad0 100644 --- a/tests/v2/test_ledger_guards.py +++ b/tests/v2/test_ledger_guards.py @@ -727,12 +727,14 @@ def test_case_shape_ids_exist_in_the_inventory() -> None: #: catches a widening that changes corpus reach or roles -- but a rule #: whose regex is literal-anchored and claims exactly ONE corpus name #: can be widened with the count unmoved, because the names it newly -#: reaches are not in the corpora. Six such widenings were demonstrated -#: on this file's own rules with the whole suite green: `^mc\s+\S+$` +#: reaches are not in the corpora. Such widenings were demonstrated on +#: this file's own rules with the whole suite green: `^mc\s+\S+$` #: to `^mc` (every leading Mc*), the vd rule to a bare `\bvd\b`, #: `^sir\s+de\b` without its anchor, and the nakaguro rule to `·.*씨`. #: Each of those then stood ready to explain exactly what its own -#: comment promises will arrive UNEXPLAINED. +#: comment promises will arrive UNEXPLAINED. The list is the evidence; +#: a count here said six over four items and was the only part anyone +#: could get wrong, since the widenings themselves are named. #: #: The probes are taken from the names those comments already argue #: about, so this roster records an answer someone already wrote in @@ -1522,15 +1524,23 @@ def _reaches_non_vocabulary(member: str, vocabulary: frozenset[str]) -> list[str fullmatch against the vocabulary bounds what a member matches WITHIN those entries and says nothing about what it matches in a NAME. That gap was first filled with a tuple of eight hand-picked - probe strings, and the tuple was defeated by a wider rule than the - one it was added to stop: every entry this rule needs is three - characters, so a member must accept some 3-character string and is - unconstrained everywhere else -- "[acdf-uw-z]{3,}" covers `roz`, - dodges all eight probes, and reaches 634 of the 751 corpus names - as a fourth alternative (the rule carrying it claims 542). - Measured with the IGNORECASE this function applies; the - case-sensitive figure, 592, is not what runs. Counts here and below are - against _CORPUS_NAMES, which deduplicates the 783 corpus lines. + probe strings, and the tuple WAS DEFEATED by a wider rule than the + one it was added to stop: every entry fix(#274) needs is three + characters (`geb`, `nee`, `née`), so a member must accept some + 3-character string and is unconstrained everywhere else -- + "[acdf-uw-z]{3,}" covers `roz`, dodged all eight probes, and as a + fourth alternative reached most of the corpus, far past what the + rule carrying it claimed. The figures that stood here were taken + against a corpus of 751 names and are not restated: the corpus has + since grown by half, the eight-string tuple this replaced is gone, + and the finding is the SHAPE -- a member unconstrained outside one + 3-character window walks past every probe anyone hand-picks -- + which is what the paragraph below acts on and which no count + strengthens. If a number is wanted, drive the member over + _CORPUS_NAMES with the IGNORECASE this function applies; the + case-sensitive figure is smaller and is not what runs. Counts here + and below are against _CORPUS_NAMES, which deduplicates the corpus + lines rather than counting them. Eight strings could never be more than a spot check. The corpus is the whole population the rule will ever be asked about, so ask it @@ -1690,8 +1700,11 @@ def _carries(name: str, vocabulary: frozenset[str]) -> bool: marker like 旧姓 is written against the name it marks rather than spaced off it. - Note what the isascii() split actually covers: 12 of the 16 - entries, not only the CJK one. `né` is two characters, so the + Note what the isascii() split actually covers: EVERY non-ASCII + entry, not only the CJK one -- the great majority of + MAIDEN_MARKERS, and the predicate below is the enumeration, so + read it there rather than from a count here, which said 12 of 16 + while the vocabulary shipped 17. `né` is two characters, so the substring branch reads `René` as carrying a marker. Every over-match here SHRINKS the set of unexplained names and so weakens the guard -- the direction this module exists to close -- diff --git a/tools/differential/expected_since_1.4.0.toml b/tools/differential/expected_since_1.4.0.toml index a3937ef7..63e1ae67 100644 --- a/tools/differential/expected_since_1.4.0.toml +++ b/tools/differential/expected_since_1.4.0.toml @@ -194,19 +194,21 @@ issue = "fix(#274) maiden markers consumed" # `compare.py --baseline 1.4.0`; a number written here instead would # only be that heading's value on the day someone typed it. # -# Five claimed names still do not move, recorded rather than +# These claimed names still do not move, recorded rather than # excluded: 'Jane Smith, née Jones' (post-comma, M2's remaining # Accepted limit), 'Jane van der Berg née PhD' and 'Jane Smith née V' # (only a suffix follows, so the consumer declines), and # 'Jane née Jr y Jones' and 'Jane van der Berg née Jr Jones' (the # connective and bound-given joins, #412). Each would need the regex # to ask a vocabulary question -- suffix or connective membership -- -# a second alternation this rule cannot spend. The count said four -# until the review round re-measured it: the corpus grew and the -# sentence did not, which is this file's standing hazard and the -# reason every count in it carries the date it was taken. Measured -# 2026-08-27 against the 1.4.0 wheel, name by name over all 31 the -# regex claims. +# a second alternation this rule cannot spend. A count stood at the +# head of that list and said four until a review round re-measured +# it: the corpus grew and the sentence did not, which is this file's +# standing hazard. The list is the count now, and it is self-checking +# where a digit was not. Measured 2026-08-27 against the 1.4.0 wheel, +# name by name over everything the regex claims rather than over a +# sample of it -- a walk whose length is whatever `_claimed()` returns +# that day. # # #445 landed 2026-08-27, and the prediction this note used to carry # was wrong in a way worth keeping on the record. It reasoned that a From d9c3bd8c1617ea80c103f0db01bfb2d13c11ff9b Mon Sep 17 00:00:00 2001 From: Derek Gulbranson Date: Thu, 3 Sep 2026 03:56:37 -0700 Subject: [PATCH 17/23] docs(design): the recorded-shapes arc MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The seventh differential-ledger arc entry: recorded diff shapes move into the tool and the run verifies them; why a shape may be pinned where a count may not; the four rows that pinned a race with one runner; the prose sweep and what its own scan got wrong; three declines. The headline finding is the arc's own subject arriving in its own record: "the multi-minute worker pass" was never measured, and it was under TWO decisions rather than one. The declined run-time contest check kept its conclusion on a new argument (coverage); the pre-worker placement of the absent-name refusal kept its ORDER while losing its reason, which is re-argued here on the published-header ground. The entry says which support was doing the work in each case. One home per fact throughout. The timing figures, the eight-site census, the 62-lines-to-0 placement measurement, the 5-of-247 / 1-of-155 contested 2.x diffs, the 13-of-31 one-runner survivors, the per-name deletion measurements and the corpus tier split are CITED where they already live -- the rule-order arc's Declined bullet, compare.py's _RECORDED_DIFFS provenance note and recorded_diff_mismatches' docstring, and _CROSS_RULE_WINNERS' 2.x section note. #451's rule applied to the claim that demonstrates it. mechanisms.md#RECORDED-ROSTERS gains the limit this arc found and amends its Lives-in list. The contract statement forbids re-deriving the EXPECTATION from the inputs the check reads; _CROSS_RULE_WINNERS obeyed that and still went wrong on the other side of the assertion, because a roster row can hold a recorded INPUT and only the expectation half is asserted. The test for a recorded input is not "is it checked" but "would the consumer answer differently if it were wrong". rules.md is untouched -- no parsing rule changed. Review corrections, all measured, listed because several are this arc's own class: "carried wrong rows for months" was five days (the roster itself is 22 days old); "every test monkeypatches _run_worker" is false at 168 test functions, 20 of which mention it and six of which run the real one under a faked Popen -- the entry now states the hermeticity claim in the form that survives measurement and says so, the loose form standing in compare.py today; "no number at all" in _reaches_non_vocabulary's docstring is no LIVE number; "one of the four had a correctable shape" is one of the two NAMES; the eight-site census and the 1116/326/790 split were restated where this file already owns them, and now point at the owner. And the 31 rows at 1.4.0 holding is not evidence the 1.4.0 half was ever sound -- 田中さん II was guessed on a genuinely contested row and was fixed by hand in e5e4955, which is the case #497 was filed off. Verified: 419 doc-citation/rules-doc tests, 6480 passed / 156 skipped / 5 xfailed, ruff and mypy clean; the gate at all four baselines 352 / 247 / 155 / 14 intentional, 0 unexplained, 0 radar-unclassified, no MOVED SHAPE block. Refs #497 Co-Authored-By: Claude Opus 5 --- docs/design/decisions.md | 26 ++++++++++++++++++++++++++ docs/design/mechanisms.md | 2 +- 2 files changed, 27 insertions(+), 1 deletion(-) diff --git a/docs/design/decisions.md b/docs/design/decisions.md index b8304e9a..8e224e9a 100644 --- a/docs/design/decisions.md +++ b/docs/design/decisions.md @@ -789,6 +789,32 @@ Declined: - A shared name-population function for both callers (2026-09-02, the spec's own first sketch) — see the two-populations decision above. `main()` and the unit guard must read different populations, so one function would break `--corpus`; they agree by a docstring that names the function it mirrors instead. - Deleting the rules whose explained names are all radar (2026-09-02, [#495](https://github.com/derek73/python-nameparser/issues/495), closed as not planned) — filed off #488's demotion on the reasoning that a rule nothing can demand is a rule to remove, and refused on what deleting one actually does. Measured at baseline 1.4.0, 19 of the 72 explaining rules explain only radar names, between them 48 of the 352 intentional diffs — and that set is a SUPERSET of what the issue proposed deleting, not a different set. Three of the five rules #495 tabulated are in it — `fix(#296) a credential-only comma string reads a name and its postnominal`, `fix(cjk-comma-compound)`, and the roman-numeral `fix(suffix-routing)` — and they are exactly the three its option 2 named as having zero contract reach. The other two are outside it for the reason that disqualified them as candidates there too: they explain contract names, fourteen of them for `fix(cjk-glued-honorific-peel)` and `'Smith Jr.'` (corpus_rules.jsonl) for the jr `fix(suffix-routing)`. So the criterion reaches everything the issue wanted gone and more, and what refuses it is the fate of those 48 diffs. Measured by driving `classify()` over each one with its own rule removed: 25 report UNCLASSIFIED and 23 are claimed by ANOTHER rule (26 and 22 with all 19 deleted at once). For nearly half the population, deleting does not leave the diff watched-but-unexplained — it leaves it explained, by a rule whose comment is about something else. Nine of `fix(cjk-comma-compound)`'s eleven land on `fix(cjk-glued-honorific-peel)`; all seven of `fix(cjk-comma-honorific-peel)`'s land on `fix(cjk-comma-compound)`; `'Carod i'` lands on the roman-numeral `fix(suffix-routing)`, and `'Bob Jones, author'` on `fix(comma-precomma-family)`. That is the rule-broader-than-the-diff absorption this file already names twice — `fix(#424)` at decisions.md#H1, going on classifying a diff that shrank underneath it while the fields-only catch-all's heading stepped 14 -> 17 with nothing announcing either, and the fields-only arc's "a fields-only rule silently falsifies OTHER comments" — reached this time on purpose rather than by accident, which is not an improvement. Applied, it deletes `fix(#342)` and `fix(#397)`, the two NOT WANTED rules whose whole job is classifying a reading nobody wants so the gate stays green while the bug is open, and which carry the delete-when-fixed instruction the dormancy check enforces once the fix lands (the fields-only arc above). The cost there is NOT a red gate, and a first draft of this bullet said it was: both names sit in corpus_issues.jsonl, which is radar, and on a radar corpus an unmatched diff prints UNCLASSIFIED (radar) while the run keeps exiting 0 — UNEXPLAINED is the contract tier's word (tools/differential/README.md). Measured with both rules gone, the run reports `unexplained: 0; radar unclassified: 1` and exits 0. The one is `'Aishwarya Rai'`; `'Carod i'` is not reported at all, the numeral rule having taken it — so one of the two open bugs loses its marker not to silence but to a wrong label. Verified end to end against a ledger with `fix(#397)` removed: exit 0, 352 intentional diffs still, no UNCLASSIFIED, no EXPLAINED NOTHING, and `'Carod i'` printed under `fix(suffix-routing) a two-token name ending in a roman numeral`. It also deletes `feat(#273)`, `feat(#269) Arabic بن prefix chains onto family (non-Latin new-recognition)` and `fix(#360) ste moved into the never-given particles with mc` — records of new recognition behavior and of a vocabulary move — and the last two have to be named to the RULE rather than to the issue, as the ledger holds sibling rules under both strings. The `feat(#269)` sibling is the sharp one: `feat(#269) non-Latin titles/conjunctions recognized` does not qualify, explaining the contract-tier `'Хосе И Мария Сантос'` (corpus_rules.jsonl), so a delete by issue prefix takes both and sends that name UNEXPLAINED — measured, and a red gate. The criterion that does justify deleting a rule is that it explains NOTHING, which `dormant` already names and the dormancy check already enforces; a rule explaining eleven radar names is not dormant. What the radar tier is FOR answers the rest: decisions.md#cjk-comma-demotion puts it as giving the differential "somewhere to WATCH a name without promising it", and a rule explaining a radar name IS the watching. The five exemptions citing #495 are rewritten in the same diff rather than left standing: four of them put it as a LIVE question — the two under `fix(#296) a credential-only comma string`, the one on the `fix(cjk-comma-compound)` block, and the one on the `'田中さん II'` numeral contest — and the fifth, on the jr contest, called neither rule a candidate in the present tense. Each records the answer now and keeps every measurement it carried. One of those measurements was wrong and is corrected with them: `fix(comma-family) lone post-comma piece routes to suffix/title, not first` explains eight names of which FIVE are contract, not six — the six having shipped in #499. It is not a mis-map: six is what tiering by FILE MEMBERSHIP gives from a CORRECT `_CORPUS_TIERS`, and the single name it gets wrong is `'John Smith, Dr.'`, the third RECOMPUTE trap below. #496's decline was argued inside the `fix(cjk-comma-compound)` one and stands without it: the twin rule was refused because nothing can demand it, which is untouched by what #495 answered. RECOMPUTE: do not join names to files by hand. `_CORPUS_TIERS` maps FILES, the report prints names with no file, and 24 of the 352 explained names sit in files of BOTH tiers, so a hand join has to reproduce `main()`'s contract-first dedup AND its baseline-minimum skip to get them right. Let the run do it: call `compare.main()` unchanged under two wrappers — one around `_run_worker`, whose `entries` argument is the final list with `tier` and `file` already stamped on every entry, and one around `classify()`, recording (name, order, diff, issue) for each of its 352 calls — then reclassify offline from those records against `rules` minus whichever rule is under test, having first confirmed the wrapped run reproduced the shipped one (352 intentional, 0 unexplained, 0 radar unclassified). Parse no printed repr by stripping quotes: three of the 352 names do not survive it — `"'Smitty' Dr. Jones"`, `"'Smitty' Jones Jr."`, and the spelling of محمد بن سلمان wrapped in RIGHT-TO-LEFT MARKs — and dropping them silently is what produced the wrong count of 20 on the first pass, the first of the three being `fix(#410)`'s only contract name. `ast.literal_eval` reads the printout correctly. The ten-name truncation is NOT the hazard it looks: contract files load first, so any contract name a rule explains sits in the printed prefix, and tiering from the first ten alone gives the same 19. THIRD, and the only one of the three that changes an answer on this tree: a name's tier, in any claim scoped to a BASELINE, is a property of the ENTRY that run compared and not of the files the string appears in. The two part where a string sits in files of both tiers AND its contract copy carries a shape tag, because the dedup key is `(name, order)` — a shape declaring an order leaves TWO entries rather than one contract winner, and the baseline-minimum skip can then drop the contract one. Measured, 2 of those 24 names carry a shape tag and exactly 1 bites at 1.4.0: `'John Smith, Dr.'` is in corpus_issues.jsonl (radar, no shape) and corpus_shapes.jsonl (contract, shape 4, FAMILY_FIRST, min_baseline 2.0.0); both entries survive the dedup, which the 2.0.0 run shows by comparing both, and at 1.4.0 the shape-4 one is among the seven skipped, leaving the radar entry as the only comparison. `'Beethoven, Ludwig van'`'s shape 2 is order-None with a 1.4.0 minimum, so the dedup collapses it and contract wins at every baseline. File-membership tiering gets exactly 1 of the 352 wrong, and that one is what `fix(comma-family)`'s six-versus-five turned on; `entry["tier"]` gets it right for free. The radar-only count of 19 is the same under either tiering. To exercise a ledger edit, point `_allowlist_for` at a scratch file rather than copying the tree — the corpus glob resolves from `HERE` and `_check_tree` refuses a nameparser outside `REPO_ROOT`, so a copy elsewhere cannot run at all. +### differential-ledger, the recorded-shapes arc (2026-09-03, #497) + +The seventh ledger arc, and the one that makes a recorded INPUT measurable. The arc above closed on the finding that `test_the_recorded_rule_still_wins_each_contested_name` feeds `classify()` a diff shape nothing ever checks, so a shape that is wrong but still routes to the same rule passes forever; this is what was done about it, and about the second thing that search turned up — that the cost the same arc declined a check on had never been measured at all. Mechanics — where the shapes live, which name list each half of the check reads, what a `MOVED SHAPE` block may and may not claim — are owned by `tools/differential/compare.py` (`_RECORDED_DIFFS`' provenance note, `recorded_diff_mismatches`' docstring, and the two placement comments in `main()`), by `tests/v2/test_ledger_guards.py` (`_CROSS_RULE_WINNERS`' header and the note over its two empty 2.x sections) and by tools/differential/README.md; these are the decisions. Every figure below was measured or recomputed 2026-09-03 on this branch; each one that can drift carries its recipe, and the two that cannot be re-derived from the tree at all — the sweep scan's own hit counts — say so where they appear. + +Decisions that landed: + +- 2026-09-03 [#497](https://github.com/derek73/python-nameparser/issues/497) — the recorded shapes live in the TOOL, as `compare._RECORDED_DIFFS`, and the RUN verifies them; `_CROSS_RULE_WINNERS` keeps only which rule wins a contested name and reads the shape from there. There because it is the one place both halves exist: `main()` has measured every name's real diff by the time it asks, where `validate_rules` runs before any corpus is read and the unit suite installs no wheel at all — `tests/v2/test_differential.py`'s own header says "nothing here spawns `uv` or the network", and the tests that need a baseline fake `_run_worker` or fake `Popen` under it. State it that way and not as "every test monkeypatches `_run_worker`", which is the short form the same claim takes in compare.py's `_RECORDED_DIFFS` provenance note and is false: measured 2026-09-03, of 168 test functions in that module 20 mention `_run_worker` at all and six exercise the real one with `Popen` faked beneath it. RECOMPUTE by walking the module's `def test_*` bodies for the name. The hermeticity claim is unharmed and the loose paraphrase of it is what fails, which is this arc's subject arriving inside its own evidence. Read HERMETICITY as the reason and not speed. The rival placement is the unit suite, and the reason it is refused is that a check there would have to install a wheel — not that installing one is slow. The timing finding below retires every cost argument in this file and moves this decision not at all, which is the test such a placement argument has to pass and the reason to say out loud which ground carries it. `_CORPUS_FLOORS` is the precedent — tool-owned, corpus-shaped data that the guards read and monkeypatch — and this is the same direction. +- 2026-09-03 #497 — ONE roster, TWO checks, TWO PLACEMENTS, and the asymmetry is the decision rather than where the code happened to fit. The ABSENT-NAME half — a recorded row naming a string no corpus holds any more, which nothing else would ever report — refuses PRE-worker, beside `vacant`, because it reads the ledger and the loaded names and nothing the worker produces. It takes the PLACEMENT #382 chose for its own pre-worker refusal and NOT #382's reason for it, which was the cost claim below — `compare.py`'s comment at that placement is one of the sites the timing finding retires. The ground it stands on was rewritten on this branch: a refusal raised after the wheel is installed and the corpus compared prints below the run's own `baseline:` header, disowning a comparison it has just published. Recorded because it is the second decision the false number was under, and the two came apart differently — the declined check below kept its conclusion on a new argument, and here the ORDER was right all along while the reason for it was not. Only re-deriving a claim says which of its supports was doing the work; a placement that reads unchanged is not thereby unaffected. The MOVED-SHAPE half is the opposite case — it needs the diffs, so it stays after the comparison — and there it may not RAISE. It PRINTS and feeds the exit code, like `over_declared_rules`, its structural sibling and the other post-worker check on recorded data: a raise at that point lands MID-report and takes dormancy, OVER-DECLARED, UNEXPLAINED and the radar block with it, which compare.py measures on a narrowed 1.4.0 run beside the check. A stale roster row must never hide an unexplained diff, that being the gate's primary output. The two pre-worker refusals may raise because nothing has printed yet; this one cannot, and the difference is PRINT ORDER rather than severity. The two halves also read opposite name lists — pre-skip for the files question, post-skip for the run question — which is `recorded_diff_mismatches`' docstring's to state and is not restated here. +- 2026-09-03 #497 — a SHAPE may be pinned where a COUNT may not, and that line is the whole of what keeps this roster affordable. A diff shape is one name under two parser versions, so adding a name to a corpus cannot move it; an explained-name count moves on every corpus edit. Shapes drift only when the PARSER changes those names — so the drift and the reason to re-read the roster are the same event, which is exactly what a recorded number usually fails to be. That is why #452's churn objection to `explains = N` (the fields-only arc above, 2026-08-28) does not reach this roster, and why pinning shapes here does not weaken it: the four ledgers carry 201 rules today, one `_CORPUS_CLAIMS` entry each, every one liable to move whenever a corpus does, against 31 recorded shapes that move only with the parser. Read the two CAUSES and not the two totals — the argument is that one set is coupled to corpus edits and the other is not, and it survives either number moving. RECOMPUTE: `sum(len(v) for v in _CORPUS_CLAIMS.values())` in tests/v2/test_ledger_guards.py, and `sum(len(v) for v in _RECORDED_DIFFS.values())` in compare.py. +- 2026-09-03 #497 — four of the 35 rows the roster carried were DELETED rather than corrected. Two names across the two 2.x ledgers, so two rows each, and one of the two NAMES had a correctable shape — count rows or names deliberately, since the provenance note that holds the measurements speaks of the name. Deleted because neither name pinned a CONTEST, which is what the roster is for: at the shape each really produces exactly one rule admits it, so `classify()` was never adjudicating anything. `'Nguyen, Van'` diffs at no baseline at all, so no run ever asks `classify()` about it; `'Jane née and Jones Smith'` is a malformed harvest from a radar corpus whose right parse nobody can state, so a pin on it defends no boundary anyone would argue for — which is why the correctable one went too. compare.py's provenance note holds the per-name, per-baseline measurements behind both. The deletion emptied both 2.x sections, which stay present as explicit empty mappings — the completeness guards require it, and it is what makes emptiness sayable as a POSITION: a row is owed when someone argues a boundary, not before. What the emptiness is NOT — that these ledgers hold no contested diff — is measured and stated beside those sections, and whether the position should change now that those contests are measured rather than merely unexamined is [#501](https://github.com/derek73/python-nameparser/issues/501), open. Recorded with the deletion because it is the half a later sweep will get wrong: "only one rule admits it" is NOT on its own grounds to delete a row. A substantial minority of the surviving 1.4.0 rows are in that position and stay — the count and the names are beside them in test_ledger_guards.py — because the shapes they pin are shapes runs actually make, so a widened `fields` or a moved rule hands the name over and the test says so. The four deleted rows could not do that work at any edit, which is the difference. +- 2026-09-03 #497 — the prose sweep is sorted by this file's own criterion (#451, "past-tense counts stay in a ledger comment; live ones do not") into three treatments, and the treatment follows from what the count is DOING rather than from how wrong it is. A count that IS the finding and reads past tense stays. A count standing beside the enumeration it counts loses the digit — the list is self-checking and the digit is the only part that can rot. A live exhaustiveness claim with nothing recomputing it is REGROUNDED on the property it was reaching for, so that the sentence stops depending on a number — saying a widening claims all but a handful of the whole population the assertion is over says what a fraction said and cannot go stale. One judgement goes with it and is the arc's own: a figure that could NOT be reproduced was not replaced with a fresh one. `_reaches_non_vocabulary`'s docstring now carries the shape of its finding, the instruction for measuring it, and no LIVE figure for the magnitude the old figures asserted — a replacement digit nobody can re-derive is the defect rather than the repair, and this file has already been burned by that shape (the #414 roster comment under the fields-only arc's `explains = N` bullet). The past-tense digits explaining why the live ones are gone stay, which is the first treatment applied inside the third, and a later sweep should not read them as survivors of the pass. + +Found rather than decided, and worth as much: + +- **The cost that decided a decline had never been measured, and the decline survived losing it.** "The multi-minute worker pass" entered 2026-08-05 in `7767ba2`, at ONE site, with no measurement beside it. The census of where it reached, the timing figures that replace it, their recompute and the `_run_worker` timing trap all live ONCE, in the rule-order arc's `Declined:` bullet on precise per-name contest detection above — the argument they falsify — and none of them is restated here; that census is the figure in this arc most likely to move, so pointing at its owner rather than copying it is the #451 rule applied to the very claim that demonstrates why. Two notes on reading it that belong here rather than there. Not every site spells the phrase — one says the pass "costs minutes" — so count the CLAIM and not the wording (`git grep -n -i minute 514a2a1`, this branch's base). And measured, the worker pass is a fraction of a second and a whole gate run is seconds at most, where the phrase said minutes: read that class and not the digits. Three things transfer past them, and a coda about removing it. The claim was never a measurement anyone got wrong — it was a plausible sentence about a subprocess that installs a wheel, and it propagated by COPYING, the mechanism the fields-only arc recorded for ledger comments and which reaches code comments identically. It was load-bearing and its conclusion still stands, rewritten on COVERAGE: a run-time check sees one ledger, for the baseline that run was invoked with, and only when somebody invokes the tool, where a static predicate covers a rule from the moment it is written. And that is how a false number stays invisible for a month — an argument with two supports loses one and nothing downstream of it changes, so no reader ever meets a contradiction. Removing it was itself hard to do cleanly: review of the commit that removed the phrase found four instances of the very defect that commit exists to remove, fixed in `40c82a7`. +- **A row pinning a race with one runner is never exercised as a contest, so its shape only ever had to agree with itself.** That is why nothing caught the four, and it is the general shape rather than a fact about two names. The four arrived with #452's work (`ee6aaa6`, PR #455) under two claims of measurement — the roster header's "The diff shapes are measured against the 1.4.0 wheel, not guessed" and the 2.1.0 block's own "measured against this baseline's own run rather than copied from the 2.0.0 rows" — and every one of the 31 rows at 1.4.0 agrees with the run today, at all four baselines. Do NOT read that as the 1.4.0 half having been sound: one of those 31 was guessed too, `田中さん II`, wrong on a genuinely CONTESTED row, found by hand a day before this branch opened (`e5e4955`, the finding the arc above filed as #497) and already corrected when the 31 were re-measured. So the one-runner mechanism explains why THESE four survived a check that would have caught a contested row, not why every wrong shape survived — the contested one needed somebody to sit down with the wheel, which is the cost the run now removes. The transferable test is what to ask of a recorded INPUT: not "is it checked" but "would its CONSUMER answer differently if it were wrong". mechanisms.md#RECORDED-ROSTERS forbids re-deriving the EXPECTATION from the inputs the check reads; this roster obeyed that to the letter and still went wrong, on the other side of the assertion, which is why that entry now carries the limit. +- **The scan that scoped the sweep was wrong three ways, and it is the scan rather than the three fixed lines that the next sweep will reuse.** Two of its 19 machine-readable hits were false positives — a version number, and a hypothesis rather than a measurement — so the real population was 17. Its file list omitted `expected_since_2.2.0.toml`, which held a further copy of a claim it was already treating in two other ledgers, so the sweep would have shipped two of three siblings corrected. And its predicate matched only lines BEGINNING with `#`, which made every docstring invisible to it: the stalest figures in the whole sweep survived the first pass for that reason alone, and a docstring pass over the two guard modules then found two more. Read the sweep as BOUNDED and not exhaustive — that docstring pass covered `tests/v2/test_ledger_guards.py` and `tests/v2/test_differential.py`, and compare.py's own docstrings were never swept. And a fourth thing, which is the finding about the finding: the scan's PREDICATE was not preserved. Its 19 hits and their 17 survivors are recorded in `8593f3a`'s commit message and cannot be re-derived from the tree, because nothing says what it matched beyond the leading `#`. That is the reusable artifact of a sweep — not the lines it fixed — so the next one has to write its own and has no earlier hit count to check itself against. Write the predicate down where the results go. +- **What the stale claims were, because the classes recur and the digits do not.** (i) A count of a population that had grown by half underneath it: a guard comment gave a Latin-half widening as claiming "644 of the 654 unclassified corpus names", where the same expression claims 994 of 1008 today (`len(_UNCLASSIFIED_NAMES)`, intersected with `_claimed("[A-Za-z]")`, in tests/v2/test_ledger_guards.py) — and since the argument was that the widening reaches essentially the whole population the assertion is over, it is written that way now and needs no recompute. (ii) A count of FILES: "all five corpora", in three places, where six `corpus*.jsonl` sit on disk and a run globs every one of them — the corpus count was never the argument, and the gate's own `corpora:` line is the check. (iii) A live count inside the very paragraph that names stale counts as the file's standing hazard — `fix(#274)`'s ledger comment walked "all 31 the regex claims" where that regex claims 32 today (`len(_claimed(rule["name_regex"]))`). (iv) A count over a self-checking enumeration: "Six such widenings" above a list of four, where nobody can now recover which two are missing and the list was always the evidence. The fifth is the one worth the space, and the docstring pass found it: `_carries` said the `isascii()` split covers "12 of the 16 entries", where `MAIDEN_MARKERS` ships 17 with 12 of them non-ASCII — the NUMERATOR was right and stayed right while the denominator rotted. A count can go stale in one of its two halves, which is the shape a re-reader is least likely to catch, and it is a reason to prefer naming the predicate over counting its output. +- **The tier split inverts the ranking that #452's still-open half rests on.** Beside the fields-equals-union check that landed, #452 carries a second proposal: a ceiling on how much of the corpus one rule may REACH, measured there as a share of the whole corpus. #468's tiering makes that the wrong denominator, because a radar diff cannot fail the run — a rule reaching many radar names is not the hazard a rule reaching many contract names is. Measured over the corpus population and tier split the rule-order arc's RECOMPUTE above already owns and this bullet does not restate, the two comma rules reach 288 names, 25.8% of all of them, but 64 contract names, 19.6% of that tier; `fix(#271/#272/#298)` reaches 108, 9.7%, and 80 contract, 24.5%. So by total reach the comma rules are the outlier and by contract reach the CJK rule is, and a ceiling on total reach would rank the wrong rule as the biggest hazard. Read the INVERSION rather than the four percentages: it is a claim about which tier the denominator counts, and it survives any drift that does not reverse the two tiers' composition. Recorded here because the proposal is open and the measurements in it predate the tiering. RECOMPUTE: tier every corpus name off `compare._CORPUS_TIERS`, contract winning where a string sits in files of both tiers, then take each rule's `name_regex` reach split by tier. + +Declined: + +- `explains = N` per rule as ledger data (2026-09-03, again) — #452's own first sketch, declined there on churn, re-examined here only because this arc pins a different recorded value and owed an account of why the two differ, and declined again on the same grounds. The third decision above is the argument in full: a shape cannot move when a corpus does and a count can. The landed fields-equals-union check already catches the drift that motivated the sketch — a rule's explained set changing SHAPE underneath it — and needs no number to do it. The question belongs to [#452](https://github.com/derek73/python-nameparser/issues/452), which is OPEN and is exactly it; nothing here forecloses it, and its remaining reach-ceiling half should be re-measured on the tier finding above. +- The differential in CI (2026-09-03) — considered and dropped rather than deferred. It is a release gate that already runs locally on demand, and no sibling check — dormancy, over-declaration, the undeclared-contest refusal, this one — needed a CI job to be worth having. Worth recording as a DECLINE and not an omission precisely because the timing finding removes the only reason nobody proposed it: a job costing seconds is cheap, and cheapness was never the criterion. +- A general sweep of every number-carrying comment line (2026-09-03) — the sweep is scoped to the claim-shaped ones, which the scan put at 19 and which were 17. The population it declines is larger by nearly two orders of magnitude: measured 2026-09-03, 1430 comment lines carry a digit across the differential tooling, its four ledgers and the two guard modules (RECOMPUTE: count lines matching `^\s*#` that also contain a digit, over `tools/differential/*.py`, `tools/differential/*.toml`, `tests/v2/test_ledger_guards.py` and `tests/v2/test_differential.py`). Most are issue references, version numbers and codepoints, none of which rot. Read the RATIO and not either digit: a hand sweep two orders of magnitude past the claims is how wrong claims get INTRODUCED rather than removed, and this arc demonstrated that at the smaller scale — the commit that removed the false cost claim needed a follow-up for four instances of the same defect inside itself, and the first prose sweep shipped having skipped three instances it had already identified (`8593f3a`). + ### comma-suffix-arc — #291/#296/#316 (2026-07-26 → 2026-08-01) #291 was filed 2026-07-26 out of the 2.0 vocabulary cleanup, with diff --git a/docs/design/mechanisms.md b/docs/design/mechanisms.md index 716cc55b..ba47bb00 100644 --- a/docs/design/mechanisms.md +++ b/docs/design/mechanisms.md @@ -73,7 +73,7 @@ than inventing one. ## RECORDED-ROSTERS — record the answer, don't re-derive it -Problem shape. A guard needs to know what the answer WAS, so it can detect the answer changing. Contract statement. Store the measured answer as literal data (a roster) and compare against it; never re-derive the expectation from the same inputs the check reads, because a derivation from the same data always agrees with itself. Lives in. tests/v2/test_ledger_guards.py (_CORPUS_CLAIMS, _EXCLUSION_EFFECT, _CROSS_RULE_WINNERS, _SPAN_BEARING_RULES, _HONORIFIC_SOURCES, _LATIN_ALTERNATION_SOURCES), tools/differential/compare.py (_CORPUS_FLOORS), tests/v2/test_facade_cases.py (_CORE_ONLY_IDS). Reach for it when. Writing a check whose expected value is computed by the code under test, or a comment that enumerates ids/counts — make it data the suite asserts. +Problem shape. A guard needs to know what the answer WAS, so it can detect the answer changing. Contract statement. Store the measured answer as literal data (a roster) and compare against it; never re-derive the expectation from the same inputs the check reads, because a derivation from the same data always agrees with itself. Known limit, and it is what the contract statement does NOT cover: a roster row can hold a recorded INPUT as well as a recorded expectation, and only the expectation half is asserted. `_CROSS_RULE_WINNERS` obeyed this entry to the letter and still carried wrong rows — it recorded a diff shape per contested name AND the rule that should win, fed the shape to `classify()` as an argument, and asserted only the winner, so a wrong shape that still routed to the same rule agreed with itself for as long as nobody re-measured it by hand. The test to apply to a recorded input is not "is it checked" but "would the consumer answer DIFFERENTLY if it were wrong": where one rule alone admits a name's diff, the consumer answers the same either way and the row is inert whatever it says. The repair is to move the input to where something measures it — the shapes now live in tools/differential/compare.py as `_RECORDED_DIFFS` and a differential run verifies them against the diffs it has already computed, the roster reading them from there — so a recorded roster's recorded half is checked by a run rather than by the guard that consumes it (#497). Lives in. tests/v2/test_ledger_guards.py (_CORPUS_CLAIMS, _EXCLUSION_EFFECT, _CROSS_RULE_WINNERS, _ORDER_EXEMPTION_EFFECT, _SPAN_BEARING_RULES, _HONORIFIC_SOURCES, _LATIN_ALTERNATION_SOURCES), tools/differential/compare.py (_CORPUS_FLOORS, _RECORDED_DIFFS), tests/v2/test_facade_cases.py (_CORE_ONLY_IDS). Reach for it when. Writing a check whose expected value is computed by the code under test, or a comment that enumerates ids/counts — make it data the suite asserts. And when a roster row carries anything the check CONSUMES rather than compares, ask what measures that half. ## LEDGER-RULE-SEPARATION — file order decides, fields narrow by subset From f70a0addf2fae5d04b9947c87e2e4e8c775c47e7 Mon Sep 17 00:00:00 2001 From: Derek Gulbranson Date: Thu, 3 Sep 2026 04:00:12 -0700 Subject: [PATCH 18/23] tooling(differential): two claims the sweep did not reach, in its own file Both are #497's class, in the file that implements the check for it. 'every test in test_differential.py monkeypatches _run_worker' is false -- 20 of 168 mention it at all, and the hermeticity the sentence is defending rests on the module header plus faked Popen, not on a universal. The claim is sound; the paraphrase of it was not. 'its five corpus files ... the other three are still bare strings' is wrong twice: six files, four bare-string. It is the same 'all five corpora' claim swept out of three other files, surviving because that sweep was scoped to the ledgers and the guard modules and never reached compare.py. Both now name the predicate rather than counting its output. Refs #497 Co-Authored-By: Claude Opus 5 --- tools/differential/compare.py | 20 ++++++++++++++------ 1 file changed, 14 insertions(+), 6 deletions(-) diff --git a/tools/differential/compare.py b/tools/differential/compare.py index 020168e3..0d4f37e0 100644 --- a/tools/differential/compare.py +++ b/tools/differential/compare.py @@ -1644,8 +1644,12 @@ class _ShapeMismatch(NamedTuple): #: HERE rather than beside the roster because this is where the #: measurement happens. main() computes every name's real diff, so a #: run can check these; the unit suite cannot, and deliberately -- it -#: spawns no uv and no network, and every test in -#: tests/v2/test_differential.py monkeypatches _run_worker. Same +#: spawns no uv and no network -- tests/v2/test_differential.py says +#: so in its own header, and the ones that need a baseline fake +#: _run_worker or fake Popen beneath it rather than installing a +#: wheel. Not "every test monkeypatches _run_worker": most never +#: mention it, and that loose paraphrase is #497's own subject +#: arriving inside #497's evidence. Same #: direction as _CORPUS_FLOORS above, which lives here and is read from #: there. #: @@ -1817,10 +1821,14 @@ def _load_entries(path: Path) -> list[dict[str, object]]: string (the original format) or an object with a "name" plus optional metadata -- "tests" labels from build_corpus.py, and a "shape" id from build_shapes_corpus.py (#469). Tolerating both - means compare.py itself never needs a flag day across its five - corpus files: corpus.jsonl and corpus_shapes.jsonl carry object - lines, the other three are still bare strings, and both shapes - stay legal everywhere a corpus line is read. + means compare.py itself never needs a flag day as corpora are + added or converted: corpus.jsonl and corpus_shapes.jsonl carry + object lines, the rest are still bare strings, and both shapes + stay legal everywhere a corpus line is read. Counting the files + here said five over six, and the object/string split three over + four, which is the "all five corpora" claim #497 swept out of + three other files -- it survived because that sweep never + reached compare.py. The glob is the count. A "tests" label is read only when the radar block prints, at the very end of the run and well past the worker pass, so a malformed From a9bb77adcf73599507536e0bcf547619c1575c51 Mon Sep 17 00:00:00 2001 From: Derek Gulbranson Date: Thu, 3 Sep 2026 04:13:40 -0700 Subject: [PATCH 19/23] docs(design): the sweep-decline figure drifted before the PR opened The bullet declining a general sweep gave an exact count of number-carrying comment lines. It was right at d9c3bd8 and wrong two commits later, because f70a0ad added two such lines -- a live count going stale inside the entry that declines a sweep for live counts, within one branch. Regrounded on the argument the sentence was always making: the ratio is four figures against 17, and the ratio is what carries it. The recompute stays for anyone who wants the digit. Refs #497 Co-Authored-By: Claude Opus 5 --- docs/design/decisions.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/design/decisions.md b/docs/design/decisions.md index 8e224e9a..78493c3e 100644 --- a/docs/design/decisions.md +++ b/docs/design/decisions.md @@ -813,7 +813,7 @@ Declined: - `explains = N` per rule as ledger data (2026-09-03, again) — #452's own first sketch, declined there on churn, re-examined here only because this arc pins a different recorded value and owed an account of why the two differ, and declined again on the same grounds. The third decision above is the argument in full: a shape cannot move when a corpus does and a count can. The landed fields-equals-union check already catches the drift that motivated the sketch — a rule's explained set changing SHAPE underneath it — and needs no number to do it. The question belongs to [#452](https://github.com/derek73/python-nameparser/issues/452), which is OPEN and is exactly it; nothing here forecloses it, and its remaining reach-ceiling half should be re-measured on the tier finding above. - The differential in CI (2026-09-03) — considered and dropped rather than deferred. It is a release gate that already runs locally on demand, and no sibling check — dormancy, over-declaration, the undeclared-contest refusal, this one — needed a CI job to be worth having. Worth recording as a DECLINE and not an omission precisely because the timing finding removes the only reason nobody proposed it: a job costing seconds is cheap, and cheapness was never the criterion. -- A general sweep of every number-carrying comment line (2026-09-03) — the sweep is scoped to the claim-shaped ones, which the scan put at 19 and which were 17. The population it declines is larger by nearly two orders of magnitude: measured 2026-09-03, 1430 comment lines carry a digit across the differential tooling, its four ledgers and the two guard modules (RECOMPUTE: count lines matching `^\s*#` that also contain a digit, over `tools/differential/*.py`, `tools/differential/*.toml`, `tests/v2/test_ledger_guards.py` and `tests/v2/test_differential.py`). Most are issue references, version numbers and codepoints, none of which rot. Read the RATIO and not either digit: a hand sweep two orders of magnitude past the claims is how wrong claims get INTRODUCED rather than removed, and this arc demonstrated that at the smaller scale — the commit that removed the false cost claim needed a follow-up for four instances of the same defect inside itself, and the first prose sweep shipped having skipped three instances it had already identified (`8593f3a`). +- A general sweep of every number-carrying comment line (2026-09-03) — the sweep is scoped to the claim-shaped ones, which the scan put at 19 and which were 17. The population it declines is larger by nearly two orders of magnitude: measured 2026-09-03, comment lines carrying a digit run to four figures across the differential tooling, its four ledgers and the two guard modules, against 17 claim-shaped ones (RECOMPUTE: count lines matching `^\s*#` that also contain a digit, over `tools/differential/*.py`, `tools/differential/*.toml`, `tests/v2/test_ledger_guards.py` and `tests/v2/test_differential.py`). The exact figure is deliberately not given: an earlier draft of this bullet carried one, and the NEXT commit on this branch added two comment lines and falsified it — inside the entry declining a sweep for exactly that.) Most are issue references, version numbers and codepoints, none of which rot. Read the RATIO and not either digit: a hand sweep two orders of magnitude past the claims is how wrong claims get INTRODUCED rather than removed, and this arc demonstrated that at the smaller scale — the commit that removed the false cost claim needed a follow-up for four instances of the same defect inside itself, and the first prose sweep shipped having skipped three instances it had already identified (`8593f3a`). ### comma-suffix-arc — #291/#296/#316 (2026-07-26 → 2026-08-01) From 9360919f82e11cb0a4cd0376ef05ffd4716b424a Mon Sep 17 00:00:00 2001 From: Derek Gulbranson Date: Thu, 3 Sep 2026 12:20:36 -0700 Subject: [PATCH 20/23] tooling(differential): the window between the two recorded-shape halves MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A name can sit in a corpus file AND outside the compared set: the baseline-minimum skip drops an order-bearing entry the baseline cannot honor. A recorded row on such a name is checked by neither half -- recorded_diff_mismatches skips it as uncompared, the `gone` refusal passes it as still-in-a-file. Measured: a wrong shape on 'de la Cruz née Vega' over the full corpus at 1.4.0 exited 0 in 375 lines naming it in none of them, and the window is co-located with the only populated section (1.4.0 carries the 31 rows and is the only baseline where the skip fires). Not a refusal -- that is the thing recorded_diff_mismatches' docstring argues against, and the argument stands. "Do not refuse" and "say nothing" are different decisions and only the first was made, so main() now prints a NOT CHECKED note over the window, naming the rows and feeding no exit code. Also pins three surviving mutations in the moved-shape report, which every fixture missed by carrying exactly one row (assignment for the append, a literal 1 for the count, a truncated row loop); pins the per-ledger dispatch with a 2.0.0 run and a 1.4.0 decoy; and validates _RECORDED_DIFFS row values in test_ledger_guards.py against the same illegal states validate_rules refuses for a rule's `fields`, where a repeated role and an empty shape passed the whole suite silently. Co-Authored-By: Claude Opus 5 --- tests/v2/test_differential.py | 115 ++++++++++++++++++++++++++++++++- tests/v2/test_ledger_guards.py | 87 ++++++++++++++++++++++++- tools/differential/compare.py | 60 ++++++++++++++++- 3 files changed, 259 insertions(+), 3 deletions(-) diff --git a/tests/v2/test_differential.py b/tests/v2/test_differential.py index 1d4cbc0d..b609ff2f 100644 --- a/tests/v2/test_differential.py +++ b/tests/v2/test_differential.py @@ -2096,6 +2096,45 @@ def test_a_moved_shape_does_not_truncate_the_report( assert code == 1, out +def test_the_shape_report_names_every_contradicted_row( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None: + """The multi-row case the block's own layout is argued from. + + The header leads the block once, rather than the instruction riding + every line, BECAUSE a real parser move lands on many of these rows + at once -- and every other fixture here, unit and main(), carries + exactly one row, so that case was the one nothing exercised. Three + mutations lived in it: `out[:] = [...]` for the append in + recorded_diff_mismatches, a literal 1 for `len(shape_bad)` in the + header, and `shape_bad[:1]` for the row loop. Same harm as + test_a_moved_shape_does_not_truncate_the_report, one level in: a + truncated block hides a finding while reporting one. + + Two names, one rule explaining both diffs, and both recorded at a + shape neither produces. + """ + monkeypatch.setitem(compare._RECORDED_DIFFS, + "expected_since_1.4.0.toml", + {"John Smith": ("nickname",), + "Alice Jones": ("nickname",)}) + code, out = _run_main( + tmp_path, monkeypatch, + '[[change]]\nissue = "claimed"\nname_regex = "Smith|Jones"\n' + 'fields = ["family"]\n', _DIFFERS, + extra=[("Alice Jones", + {"title": "", "first": "Alice", "middle": "", + "last": "JONESY", "suffix": "", "nickname": "", + "maiden": ""})]) + # the COUNT, which a literal 1 would still print for two rows + assert "MOVED SHAPE expected_since_1.4.0.toml: 2 recorded" in out, out + # ... and both rows, which a truncated loop would not + assert "John Smith" in out and "Alice Jones" in out, out + assert out.count("_RECORDED_DIFFS records") == 2, out + # the header still leads ONCE, which is what the count is for + assert out.count("FINDING") == 1, out + assert code == 1, out + + def test_the_shape_report_over_an_unmeasured_name_claims_no_cause( tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None: """`measured` None has two reachable causes and the report must @@ -2128,6 +2167,39 @@ def test_the_shape_report_over_an_unmeasured_name_claims_no_cause( assert code == 1, out +def test_the_shape_check_reads_the_section_for_the_LEDGER_it_ran( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None: + """_RECORDED_DIFFS is keyed per ledger, and nothing pinned the key. + + Replacing the lookup with a hardcoded + `_RECORDED_DIFFS['expected_since_1.4.0.toml']` survived the whole + suite: every other main() test here runs at the default 1.4.0 + baseline, so the right answer and the hardcoded one are the same + dict. Harmless only while the 2.x sections stay empty, which is + open (#501) -- and the shapes are baseline-relative by + construction, so reading the wrong section is reading measurements + of a different comparison. + + A correct row in the 2.0.0 section and a DECOY in the 1.4.0 one: + dispatched right, the run is quiet; dispatched to 1.4.0, it reports + a moved shape on the same name. Set wholesale rather than per key, + so the shipped 31 rows cannot supply the answer either way. + """ + monkeypatch.setattr( + compare, "_RECORDED_DIFFS", + {"expected_since_2.0.0.toml": {"John Smith": ("family",)}, + "expected_since_1.4.0.toml": {"John Smith": ("nickname",)}}) + code, out = _run_main( + tmp_path, monkeypatch, + '[[change]]\nissue = "claimed"\nname_regex = "Smith"\n' + 'fields = ["family"]\n', + {**_SAME_FACADE, "last": "SMYTHE"}, baseline="2.0.0", + baseline_v2={**_SAME_V2, "family": "SMYTHE"}) + assert "MOVED SHAPE" not in out, out + assert "nickname" not in out, out + assert code == 0, out + + def test_a_full_run_refuses_a_recorded_name_no_corpus_holds( tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None: """The caller's half of the skip (#497 addendum). @@ -2209,6 +2281,19 @@ def test_a_name_this_baseline_skipped_is_not_a_name_the_corpus_lost( the shipped rosters cannot show the swap, because none of those three carries a row yet. + IT PINS "DOES NOT REFUSE", which is what it was written for, and it + used to pin "says nothing" as a side effect of asserting the name + was absent from stdout. Those are different decisions and only the + first was ever argued: a row on a skipped name is checked by + NEITHER half -- recorded_diff_mismatches drops it as uncompared and + the `gone` refusal passes it as still-in-a-file -- so silence made + a wrong shape on one of these three names a silent pass at the one + baseline whose section carries rows. Measured before the NOT + CHECKED note: a wrong shape on 'de la Cruz née Vega' over the full + corpus at 1.4.0 exited 0 in 375 lines naming it in none of them. So + the name is now REPORTED and the run still does not refuse, and + both halves are asserted below. + Hand-rolled rather than via _run_main, which writes bare-string corpus lines and so cannot produce a skipped entry at all; same construction as @@ -2248,8 +2333,16 @@ def _fake(v: str, w: bool, n: list[dict]) -> tuple[dict, list[dict]]: code = compare.main() out = buf.getvalue() assert _WORKER_CALL["names"] == ["John Smith"], out - assert name not in out, out + # does not REFUSE: the `gone` refusal's own sentence, which the + # sibling above asserts IS raised for a name no corpus holds + assert "no corpus holds any more" not in out, out assert code == 0, out + # ... and does not stay silent either. The note names the row, says + # it went unchecked, and says the row is not the thing at fault -- + # the three things a reader needs to not delete it. + assert "NOT CHECKED" in out, out + assert name in out, out + assert "do not delete" in out, out def test_radar_diff_with_no_rule_exits_0_and_is_reported( @@ -3503,6 +3596,26 @@ def test_a_recorded_shape_the_run_contradicts_is_reported() -> None: ("田中さん II", ("given", "suffix"), ("family", "given", "suffix"))] +def test_every_contradicted_row_is_returned_not_just_one() -> None: + """`out.append`, not `out[:] = [...]`. + + Every other fixture for this function carries one recorded row, and + over a one-row roster an assignment and an append are the same + function. The case that separates them is the one the report's + layout is argued from -- a parser move landing on many rows at once + -- so it is pinned at both levels; + test_the_shape_report_names_every_contradicted_row is main()'s. + """ + diffing = [("Smith, Jr.", {"family"}, None), + ("Kim, Jr.", {"family"}, None)] + got = compare.recorded_diff_mismatches( + {"Smith, Jr.": ("family", "suffix"), "Kim, Jr.": ("given",)}, + diffing, {"Smith, Jr.", "Kim, Jr."}) + assert [(m.name, m.recorded, m.measured) for m in got] == [ + ("Smith, Jr.", ("family", "suffix"), ("family",)), + ("Kim, Jr.", ("given",), ("family",))] + + def test_a_recorded_name_that_stops_diffing_is_reported() -> None: """Recorded means it diffed. If it no longer does, that is a finding about the parser, not a row to delete.""" diff --git a/tests/v2/test_ledger_guards.py b/tests/v2/test_ledger_guards.py index 9ac46ad0..19175c5f 100644 --- a/tests/v2/test_ledger_guards.py +++ b/tests/v2/test_ledger_guards.py @@ -2703,6 +2703,18 @@ def test_the_recorded_rule_still_wins_each_contested_name() -> None: never = _exclusions(ledger) shapes = compare._RECORDED_DIFFS[ledger_name] for name, expected in winners.items(): + # explicit, because a bare KeyError here names a dict and a + # string and nothing else. The readable half-recorded-pin + # message lives in test_every_pinned_winner_has_a_recorded_shape, + # which a `-k` selection or a first failure may not have run. + assert name in shapes, ( + f"{ledger_name}: {name!r} is pinned a winner in " + f"_CROSS_RULE_WINNERS with no shape in " + f"compare._RECORDED_DIFFS, so there is nothing to ask " + f"classify() about. Record the shape a run measures for " + f"it, or drop the pin -- " + f"test_every_pinned_winner_has_a_recorded_shape is the " + f"guard that states this as its own subject") fields = shapes[name] got = compare.classify(name, set(fields), rules, never) assert got is not None and got.startswith(expected), ( @@ -2730,7 +2742,34 @@ def test_every_pinned_winner_has_a_recorded_shape() -> None: """The roster names a contested name; _RECORDED_DIFFS says what it diffs. A name in one and not the other is a half-recorded pin -- the winner cannot be checked without the shape, and a shape nothing - pins a winner for is unverified by the run for no purpose.""" + pins a winner for is unverified by the run for no purpose. + + The row VALUES are checked here too, against the three illegal + states compare.validate_rules already refuses for a rule's `fields` + (a name outside the role vocabulary, '_initials' beside a role, a + repeated role) plus the empty one. A recorded shape is fed to + classify() as the same kind of input a rule's `fields` is, so the + wording below mirrors those refusals deliberately -- the two should + read as one check written in two places. + + WHY HERE. What each illegal row does WITHOUT these assertions, + measured 2026-09-03 by injecting one into the shipped 1.4.0 section + and running this file: a repeated role and an empty shape pass the + whole suite silently, and surface only as a MOVED SHAPE finding + from a gate run -- whose message says the winner beside the shape + "was recorded for the OLD shape", telling a contributor with a + copy-paste slip that the parser moved. A misspelled role and an + '_initials' beside a role happened to fail the sibling test above + as well, because on the rows they were injected into the corrupted + shape changed which rule classify() picks -- but that is a property + of those rows, not of the defect: a shape a second rule admits at + both spellings changes no winner and says nothing. Either way the + message names a rule handover rather than the typo, and the loop + that does catch these needs a baseline wheel and a full corpus + pass, so the row ships green through CI. At pytest speed the answer + is a sentence naming the typo. The `departed` assertion asks + compare.py's `gone` question the same way and for the same reason. + """ compare = load_tool("compare") for ledger_name, winners in _CROSS_RULE_WINNERS.items(): shapes = compare._RECORDED_DIFFS.get(ledger_name, {}) @@ -2738,6 +2777,52 @@ def test_every_pinned_winner_has_a_recorded_shape() -> None: f"{ledger_name}: winners without a recorded shape " f"{sorted(set(winners) - set(shapes))}; shapes with no " f"pinned winner {sorted(set(shapes) - set(winners))}") + # `gone`'s question, at pytest speed and against every ledger + # rather than only the one a run was pointed at. The gate asks + # it over the corpora a full run loaded; _CORPUS_NAMES is every + # name in every corpus*.jsonl, which is the same population. + departed = sorted(set(shapes) - set(_CORPUS_NAMES)) + assert not departed, ( + f"{ledger_name}: {departed} carry a recorded shape and sit " + f"in no corpus. Nothing measures such a row, so it agrees " + f"with itself forever. Settle whether the name left " + f"DELIBERATELY before editing either roster -- " + f"compare.py's own refusal over the full corpus carries the " + f"question and both repairs") + for name, shape in shapes.items(): + where = f"{ledger_name}: the recorded shape for {name!r}" + bad = sorted(set(shape) - compare._RULE_FIELDS) + assert not bad, ( + f"{where} names {bad}, which are not roles; expected " + f"from {sorted(compare._RULE_FIELDS)}. classify() is " + f"asked about the shape as a SET, so a misspelled role " + f"is simply a role the diff does not carry: it never " + f"reports as a typo, only as a rule handover or a " + f"MOVED SHAPE finding blaming the parser. Checked " + f"against _RULE_FIELDS -- the set validate_rules checks " + f"a rule's 'fields' against, since a shape is the same " + f"kind of input -- and not against V2_FIELDS, because " + f"'_ambiguities' is legal in a 2.x row (it cannot " + f"appear below 2.0) even though no row carries it today") + dups = sorted({f for f in shape if shape.count(f) > 1}) + assert not dups, ( + f"{where} repeats {dups}. It is read as a set, so the " + f"repeat changes nothing classify() matches -- it is a " + f"copy-paste slip that would otherwise pass every check " + f"here silently, and the '_initials' check below would " + f"read ('_initials', '_initials') as '_initials' alone") + others = sorted(set(shape) - {"_initials"}) + assert not ("_initials" in shape and others), ( + f"{where} lists '_initials' beside {others}. " + f"'_initials' enters a diff only when every role and " + f"the ambiguity kinds agree (#484), so no diff can " + f"carry it with another field: this shape is one no run " + f"can measure, and the row is unfalsifiable rather than " + f"merely wrong") + assert shape, ( + f"{where} is empty. main() appends to `diffing` only " + f"where a comparison DIFFED, so no run produces an " + f"empty shape and the row can never be contradicted") # Per-ledger equality above iterates the ROSTER's keys, so a # _RECORDED_DIFFS section for a ledger the roster does not name is # invisible to it -- and to the run, which reads shapes per ledger diff --git a/tools/differential/compare.py b/tools/differential/compare.py index 0d4f37e0..56144178 100644 --- a/tools/differential/compare.py +++ b/tools/differential/compare.py @@ -1617,7 +1617,13 @@ class _ShapeMismatch(NamedTuple): which side is which where `m[1]`/`m[2]` would not. """ name: str - #: the shape the roster records, sorted + #: the shape the roster records. Sorted HERE, at construction, like + #: `measured` beside it -- a property of the instance and NOT a + #: requirement on the literal row: a row spelled in any other order + #: sorts to the same tuple and compares equal, so a sortedness + #: guard over _RECORDED_DIFFS would refuse nothing and pin nothing. + #: test_a_recorded_shape_matching_the_run_is_no_mismatch pins both + #: sides of that. recorded: tuple[str, ...] #: what this run measured, sorted; None when the run produced no #: default-order diff of the name at all -- TWO states reach that, @@ -1784,6 +1790,13 @@ def recorded_diff_mismatches( that: it is _CORPUS_FLOORS against the files on disk and knows nothing about the skip. It is the earlier-draft `vacant` bug recorded below, one baseline over. + Those three names are the WINDOW between the two halves, and not + refusing is not the same decision as saying nothing: a row on one + of them is checked by neither side, so main() prints a NOT CHECKED + note over `set(recorded) & set(corpus_names) - compared` -- naming + them, claiming nothing about them, and feeding no exit code. That + note is the caller's too, for the reason this whole paragraph is: + only the caller holds both lists. A name that IS compared and produces no default-order diff is reported with `measured` None, and two states reach that: the @@ -2450,6 +2463,51 @@ def _tree_parse(name: str, order: str | None) -> object: # there, and `recorded` is the per-ledger row dict both halves read, # bound up beside it. compared = {str(n) for n in corpus} + # THE WINDOW BETWEEN THE TWO HALVES, and it is not empty. A name can + # sit in a corpus file this run READ and still be outside `compared`, + # because the baseline-minimum skip above drops an order-bearing + # entry the baseline cannot honor. Such a row falls between both + # checks: recorded_diff_mismatches skips a name outside `compared`, + # and the `gone` refusal passes it because the skip takes a name out + # of the RUN and out of no file. Measured 2026-09-03: a deliberately + # wrong shape on 'de la Cruz née Vega' over the FULL corpus at 1.4.0 + # left exit 0 and 375 stdout lines with no mention of the name -- + # and the window is co-located with the only populated section, + # since 1.4.0 is both where the 31 rows live and the only baseline + # where the skip fires (every order-bearing shape's min_baseline is + # 2.0.0). + # + # A NOTE. Not a refusal, not in the exit code, and the distinction + # is the whole of it: recorded_diff_mismatches' docstring argues + # that refusing off the POST-skip list would tell a contributor to + # delete a row for a name sitting in corpus_shapes.jsonl at the + # compat baseline, and that argument stands. It is an argument + # against REFUSING, not against SAYING, and only the first of those + # was ever made. + # + # Intersected with `corpus_names` rather than gated on + # `full_corpus`, which is where this parts from the two checks that + # do read that flag. Under `--corpus` the pre-skip list is already + # narrowed to what this run loaded, so the intersection names only + # rows this run READ and then dropped -- a fact about the run in + # both modes, and never the departed-name question the `gone` half + # owns. Nothing else stands between the two lists: `entries` is + # rebuilt exactly once between them, by that skip. + unchecked = sorted((set(recorded) & set(corpus_names)) - compared) + if unchecked: + print(f"NOT CHECKED {ledger.name}: {len(unchecked)} recorded " + f"diff shape(s) name an entry this baseline skipped, so " + f"this run measured no diff to check them against. " + f"Informational, outside the exit code, and NOT a stale " + f"row: each name is in a corpus this run read -- the " + f"skip takes an order-bearing entry out of the RUN and " + f"out of no file -- so do not delete one over this. The " + f"shape report below speaks for every other row and for " + f"none of these; re-run at a baseline that can honor " + f"their order to check them:") + for n in unchecked: + print(f" {n!r}") + print() shape_bad = recorded_diff_mismatches(recorded, diffing, compared) if shape_bad: # The instruction and the disclaimer are properties of the From 02ecdd5061aab85f01cba87c0d66faab94a084bb Mon Sep 17 00:00:00 2001 From: Derek Gulbranson Date: Thu, 3 Sep 2026 12:42:02 -0700 Subject: [PATCH 21/23] docs+tooling(differential): the prose this PR made stale about itself A five-agent review of #502 found the arc reproducing its own subject: claims measured on one commit and falsified by the next two. The criterion in mechanisms.md#RECORDED-ROSTERS was the worst of them. "Where one rule alone admits a name's diff, the consumer answers the same either way" is false 11 times out of 13 over the 1.4.0 roster -- a wrong shape routes to a DIFFERENT rule, so the assertion catches it ('Andrews, M.D.' wins fix(comma-family) at its recorded shape and fix(comma-precomma-family) at {family}). It also contradicted decisions.md, which says on the same day that single-admittance is not grounds to delete a row. The restatement the review proposed -- "the same rule admits both shapes" -- is false too: file order still decides, and 152 of the 246 such pairs route differently. What is true is per-pair, and is what the entry now says. Four sites carried "disowning a comparison it has just published" after 40c82a7 retired that framing at two others. Measured print order is `baseline:`, then a silent comparison loop, then `corpus:` -- so at the position all four describe, nothing of the comparison has printed. The fourth (tests/v2/test_differential.py) was outside the review's list. Also: _RECORDED_DIFFS' keying argument cited two names in two ledgers that b793ebc deleted; the MOVED SHAPE provenance was the last site still naming one cause where the README, the printed block and the roster header name two; decisions.md pointed at a compare.py note f70a0ad had rewritten to say the opposite, and quoted a 168 that is 171; the PYTHONHASHSEED recipe built its seven-role set alphabetically where the test builds it from V2_FIELDS, which is 1 of 200 against 0; a ledger comment kept the retired tuple key; the OVER-DECLARED parenthetical called a 47-word three-sentence block one sentence; and the checklist's "digit here is checked" covered one of two digits. Two things recorded that no site said: MOVED SHAPE feeds the exit code without asking the tier, and 21 of the 31 rows sit on radar names -- so radar parser drift can fail the run; and the release checklist runs three baselines against four ledgers with rules, leaving 2.1.0 without a dormancy, over-declaration or shape check. Noted, not silently fixed. Gate 0 at all four baselines (352/247/155/14), 0 unexplained, 0 MOVED SHAPE, 0 NOT CHECKED. Co-Authored-By: Claude Opus 5 --- AGENTS.md | 6 ++ docs/design/decisions.md | 12 ++-- docs/design/mechanisms.md | 2 +- tests/v2/test_differential.py | 29 ++++++--- tests/v2/test_ledger_guards.py | 21 +++++-- tools/differential/README.md | 25 +++++++- tools/differential/compare.py | 65 +++++++++++++++----- tools/differential/expected_since_1.4.0.toml | 6 +- tools/differential/expected_since_2.0.0.toml | 6 +- tools/differential/expected_since_2.1.0.toml | 6 +- tools/differential/expected_since_2.2.0.toml | 6 +- 11 files changed, 137 insertions(+), 47 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 44985c9a..af2eaadc 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -106,6 +106,12 @@ uv run sphinx-build -b html docs dist/docs # uv run python tools/differential/compare.py --baseline 1.4.0 # the v1 compat contract # uv run python tools/differential/compare.py # the previous minor # uv run python tools/differential/compare.py --baseline 2.0.0 # 2.0.0's ledger has rules too +# NOTE (2026-09-03): that is THREE commands against FOUR ledgers with +# rules -- `--baseline 2.1.0` is the one no line above reaches, so the +# "EVERY baseline" instruction is not what this list does. Since #497 +# a third check is per-ledger as well: the recorded-shape one reads +# _RECORDED_DIFFS[ledger.name], so the ledger nobody runs gets no +# dormancy check, no over-declaration check, and no shape check. # Redirect to a file rather than piping — under zsh a pipe replaces the exit # code with the pipe's, so a failing run reads as a passing one. The # classified summary it prints is the source for the release notes' behavior diff --git a/docs/design/decisions.md b/docs/design/decisions.md index 78493c3e..808ea961 100644 --- a/docs/design/decisions.md +++ b/docs/design/decisions.md @@ -783,7 +783,7 @@ Declined: - Reordering the eleven (2026-09-02) — the obvious way to make the check unnecessary, and wrong twice. Reordering moves which rule classifies a name, which is a behavior change to the ledger and breaks `_CROSS_RULE_WINNERS`; and no pair is even a reorder CANDIDATE, because in all eleven the narrower rule's own prose would be FALSE or MERELY PARTIAL of the co-matched names if it won, and either disqualifies it as classifier-of-record. Partiality is the commoner half and the arc's load-bearing distinction — compound versus component, which the `fix(#400/#274)`/`fix(#400)` exemption calls "the canonical compound-versus-component shape". Read the eleven `why` texts for it: `fix(#400)` "says nothing about a maiden marker"; a widened `fix(comma-family) lone post-comma piece routes to suffix/title` "would take the union and report only half of it"; `fix(cjk-glued-honorific-peel)` says "nothing about a division"; `feat(#273)`'s prose "has nothing to say about" the remainder rejoining as a name; and two of the three regex-accident pairs are partial rather than false, the `fix(suffix-routing)` jr and numeral rules genuinely naming the `{family, suffix}` half of `김민준씨 Jr.` and of `田中さん II` and describing neither the peel nor the segmentation that moves `given`. Falsity is the minority, and one of the eleven says so of itself: the exemption `fix(comma-family) a comma followed only by titles keeps the given/family split` carries over `fix(comma-precomma-family)` calls its own pair "the one where the narrower rule would be actively WRONG rather than merely partial", and adds that the claim "is about THIS name and is deliberately not generalised". `马丁·路德·金씨` is the clearest compound, and `王先生, V.` the reverse case, where the winner is already the least wrong reading available — and stays so, since the family-side twin that would describe it was weighed as #496 and declined (the next bullet). - A family-side twin of `fix(cjk-comma-honorific-peel)` for a PRE-comma glued honorific (2026-09-02, [#496](https://github.com/derek73/python-nameparser/issues/496), closed as not planned) — the rule that would honestly describe `王先生, V.`, `田中さん, V.` and `김민준씨, V.`, whose diff is the glued peel alone while the label they carry says comma compound (the finding bullet above). Declined on two grounds. FIRST, no gate can demand it: all 17 names the compound/peel pair is contested across measure radar — but by two different warrants, and an audit that assumes one will find the count short. Sixteen sit in `corpus_cjk_tolerated.jsonl`, the file #488 created by demoting the composed comma/Latin-wrapper CJK forms to tolerated input, `김, 민준씨` — the name #382 was filed over — among them. The seventeenth, `Dr 田中さん, V.`, is in `corpus_issues.jsonl` alone: harvested and append-only, radar since #468, and untouched by #488. (`Dr 김민준씨, Jr.` is in both files, and reads radar from either.) Radar means an unmatched diff is reported rather than fatal for want of a RULE; it is not an unconditional never-fatal, since a `[[never]]` exclusion outranks the tier and stays fatal on a radar name — `_CORPUS_TIERS`' own note, and `main()`'s two-reasons comment, which routes an excluded name to `unexplained` rather than to `radar`. No exclusion refuses these, so nothing here can demand the twin. SECOND, [#495](https://github.com/derek73/python-nameparser/issues/495) points the other way over the same corner: it asks whether the radar-only rules already in the ledger still earn their place and records `fix(cjk-comma-compound)` as having zero contract-tier reach, so #496 proposed a SECOND rule for the very names #495 is weighing a first one away from. Fewer rules for this corner is the coherent direction, and that is what makes the decline decisive rather than merely permissive. REOPEN it if a pre-comma glued-honorific name ever reads contract — and watch BOTH routes, because the two files promote by different mechanisms. A `corpus_cjk_tolerated.jsonl` name is promoted by clearing `tolerated` on its case rows, which moves the text into `corpus_cjk.jsonl` (`_CORPUS_TIERS`, and build_cjk_corpus.py's split). `Dr 田中さん, V.` has no case row at all, so there is no flag to clear: it changes tier only by being CHOSEN — a new unmarked row, or a rules.md example — which puts the text in a contract corpus, and the (name, order) dedup loads contract files first and keeps that reading. Read the tier off `_CORPUS_TIERS` either way, never off a hand-built file→tier map (the roster caution above). -- Precise per-name contest detection at differential-run time (2026-09-02; its stated reason replaced 2026-09-03, [#497](https://github.com/derek73/python-nameparser/issues/497)) — it would replace the static predicate's over-reporting with the measured nine-of-eleven split, and it needs the pinned-wheel worker pass to do it. The decline stands, on COVERAGE alone: a run-time check sees one ledger — the one for the baseline that run was invoked with — and only once somebody invokes the tool, whereas the static predicate covers a rule from the moment it is written, including a rule nobody has ever run the wheel against, which is the whole point of #382. Nor can the unit suite simply adopt the precise check: `tests/v2/test_differential.py` is deliberately hermetic — its own header says "nothing here spawns `uv` or the network", and the tests that need a baseline fake `_run_worker` or `Popen` outright — so moving it to pytest speed means spawning a subprocess and, on a cold cache, reaching PyPI. WITHDRAWN, and named here so it is not re-argued: this bullet used to say the check would sit "behind a multi-minute run" and gave that as "the whole of the reason". No measurement was recorded beside it: the phrase entered on 2026-08-05 in `7767ba2` at a single site in `tools/differential/compare.py`, and was copied outward until eight sites carried it — four there, three in `tests/v2/test_differential.py`, and this bullet. Measured 2026-09-03 over the corpus population named above, a whole run is 0.33s wall at baseline 1.4.0 and 0.57s at 2.2.0, the default; the worker pass is 0.10s and 0.29s of those, timed inside `main()`. All four baselines back to back total 1.97s, and a run with `uv`'s cache emptied first, so the pinned wheel is downloaded rather than reused, is still under a second. Read the class and not the digits: the worker pass is a fraction of a second and the whole gate is seconds at most, where the withdrawn phrase said minutes — so no argument here may be built on what the worker pass costs. RECOMPUTE: `time (uv run python tools/differential/compare.py --baseline 1.4.0 >/dev/null)`. For the worker's share, wrap `compare._run_worker` in a timer and then call `main()` — calling `_run_worker` directly on every loaded entry aborts at a 1.4.0 baseline, since the seven order-bearing shape-4/5 entries have to be dropped by the baseline-minimum skip first. Do NOT restate the decline as an error-direction argument either: within the NESTED pairs computing real diffs can only remove them, but the predicate is not a superset of the contests, since nesting is sufficient for one and not necessary (the bullet above, and #498). What the static check buys is coverage of a rule nobody has run the wheel against, not a guarantee of refusing everything it should. +- Precise per-name contest detection at differential-run time (2026-09-02; its stated reason replaced 2026-09-03, [#497](https://github.com/derek73/python-nameparser/issues/497)) — it would replace the static predicate's over-reporting with the measured nine-of-eleven split, and it needs the pinned-wheel worker pass to do it. The decline stands, on COVERAGE alone: a run-time check sees one ledger — the one for the baseline that run was invoked with — and only once somebody invokes the tool, whereas the static predicate covers a rule from the moment it is written, including a rule nobody has ever run the wheel against, which is the whole point of #382. Nor can the unit suite simply adopt the precise check: `tests/v2/test_differential.py` is deliberately hermetic — its own header says "nothing here spawns `uv` or the network", and the tests that need a baseline fake `_run_worker` or `Popen` outright — so moving it to pytest speed means spawning a subprocess and, on a cold cache, reaching PyPI. WITHDRAWN, and named here so it is not re-argued: this bullet used to say the check would sit "behind a multi-minute run" and gave that as "the whole of the reason". No measurement was recorded beside it: the phrase entered on 2026-08-05 in `7767ba2` at a single site in `tools/differential/compare.py`, and was copied outward until eight sites carried it — four there, three in `tests/v2/test_differential.py`, and this bullet. Measured 2026-09-03 over the corpus population named above, a whole run is 0.33s wall at baseline 1.4.0 and 0.57s at 2.2.0, the default; the worker pass is 0.10s and 0.29s of those, timed inside `main()`. All four baselines back to back total 1.97s, and a run with `uv`'s cache emptied first, so the pinned wheel is downloaded rather than reused, was still under a second — that last one is NETWORK- and environment-dependent and the RECOMPUTE below does not reach it, covering the warm run alone. Reproducing it means pointing `UV_CACHE_DIR` at an empty directory so the wheel is fetched rather than reused, and the number will differ on a different link. Read it as "the download is not the story either", not as a figure to check against. Read the class and not the digits: the worker pass is a fraction of a second and the whole gate is seconds at most, where the withdrawn phrase said minutes — so no argument here may be built on what the worker pass costs. RECOMPUTE: `time (uv run python tools/differential/compare.py --baseline 1.4.0 >/dev/null)`. For the worker's share, wrap `compare._run_worker` in a timer and then call `main()` — calling `_run_worker` directly on every loaded entry aborts at a 1.4.0 baseline, since the seven order-bearing shape-4/5 entries have to be dropped by the baseline-minimum skip first. Do NOT restate the decline as an error-direction argument either: within the NESTED pairs computing real diffs can only remove them, but the predicate is not a superset of the contests, since nesting is sufficient for one and not necessary (the bullet above, and #498). What the static check buys is coverage of a rule nobody has run the wheel against, not a guarantee of refusing everything it should. - Giving the peel rule a predicate the compound rule fails (2026-09-02, #382 option 1) — narrowing `fix(cjk-comma-honorific-peel)`'s `name_regex` to the honorific-bearing shapes, whose stated effect in #382 was that the pair "becomes order-independent and the original contract holds again". Declined because that effect is UNREACHABLE, and measured rather than argued. The two rules ship a BYTE-IDENTICAL `name_regex` (verified 2026-09-02 by comparing the two strings), and `_rule_reach` computes each rule's names from its OWN pattern, so narrowing the peel rule's regex narrows the peel rule's reach and nothing else: the compound rule goes on reaching every one of the names. The general form, which no regex edit escapes — where the earlier rule's `fields` are a SUBSET of the later one's, order-independence requires the LATER rule to stop reaching the name, and an edit to the earlier rule's own regex cannot cause that. Here the peel rule's `{given, suffix}` is a strict subset of the compound rule's `{family, given, suffix, title}`, so every diff the peel rule admits the compound rule admits too. Implemented to check it: narrowing the peel regex to a `GLUED_HONORIFICS` alternation keeps all seven of the co-matched names whose 1.4.0 diff is exactly `{given, suffix}`, and swapping the two rules in the NARROWED ledger still reattributes all seven. So the pair keeps the arrangement the arc settled on, and that is the answer rather than a cost trade: it is narrow-first, the declaration-free default, so it appears in none of the eleven contests `order_contests` reports and owes no `precedes_narrower` block, and all seven names are pinned by name in `_CROSS_RULE_WINNERS` (23 corpus names reach both regexes; the seven are the ones #375's reorder mutation moves). The cost stands as a second reason and not the first: the narrowing would hand-copy more honorific vocabulary into the pattern and grow the `_HONORIFIC_SOURCES` sync-roster surface (mechanisms.md#CURATED-VOCABULARY-ALTERNATION's second half). Option 2's convention plus a check on the wide-first exceptions to it is what the arc took, and option 3 it REFUSED on a false premise (the finding above). One thing this does not settle, and the ledger comment on the peel rule overstates it: `fields` separates the two only for the UNION rows whose diff includes `family`; for those seven it is file order that decides. - Scoping the check to contract-tier contests only (2026-09-02) — five of the eleven are contested only over radar names, so this would have cut the file's exemptions by nearly half. Declined because `_CROSS_RULE_WINNERS` already pins radar names — measured, most of the names it pins are radar-tier (22 of 33 today) — so the repo would be inconsistent with itself about whether a radar contest matters. Whether the radar-only CJK comma rules still earn their place after #488's demotion is a real question and is filed as #495; it is a question about those rules, not about the check. - A shared name-population function for both callers (2026-09-02, the spec's own first sketch) — see the two-populations decision above. `main()` and the unit guard must read different populations, so one function would break `--corpus`; they agree by a docstring that names the function it mirrors instead. @@ -795,10 +795,10 @@ The seventh ledger arc, and the one that makes a recorded INPUT measurable. The Decisions that landed: -- 2026-09-03 [#497](https://github.com/derek73/python-nameparser/issues/497) — the recorded shapes live in the TOOL, as `compare._RECORDED_DIFFS`, and the RUN verifies them; `_CROSS_RULE_WINNERS` keeps only which rule wins a contested name and reads the shape from there. There because it is the one place both halves exist: `main()` has measured every name's real diff by the time it asks, where `validate_rules` runs before any corpus is read and the unit suite installs no wheel at all — `tests/v2/test_differential.py`'s own header says "nothing here spawns `uv` or the network", and the tests that need a baseline fake `_run_worker` or fake `Popen` under it. State it that way and not as "every test monkeypatches `_run_worker`", which is the short form the same claim takes in compare.py's `_RECORDED_DIFFS` provenance note and is false: measured 2026-09-03, of 168 test functions in that module 20 mention `_run_worker` at all and six exercise the real one with `Popen` faked beneath it. RECOMPUTE by walking the module's `def test_*` bodies for the name. The hermeticity claim is unharmed and the loose paraphrase of it is what fails, which is this arc's subject arriving inside its own evidence. Read HERMETICITY as the reason and not speed. The rival placement is the unit suite, and the reason it is refused is that a check there would have to install a wheel — not that installing one is slow. The timing finding below retires every cost argument in this file and moves this decision not at all, which is the test such a placement argument has to pass and the reason to say out loud which ground carries it. `_CORPUS_FLOORS` is the precedent — tool-owned, corpus-shaped data that the guards read and monkeypatch — and this is the same direction. -- 2026-09-03 #497 — ONE roster, TWO checks, TWO PLACEMENTS, and the asymmetry is the decision rather than where the code happened to fit. The ABSENT-NAME half — a recorded row naming a string no corpus holds any more, which nothing else would ever report — refuses PRE-worker, beside `vacant`, because it reads the ledger and the loaded names and nothing the worker produces. It takes the PLACEMENT #382 chose for its own pre-worker refusal and NOT #382's reason for it, which was the cost claim below — `compare.py`'s comment at that placement is one of the sites the timing finding retires. The ground it stands on was rewritten on this branch: a refusal raised after the wheel is installed and the corpus compared prints below the run's own `baseline:` header, disowning a comparison it has just published. Recorded because it is the second decision the false number was under, and the two came apart differently — the declined check below kept its conclusion on a new argument, and here the ORDER was right all along while the reason for it was not. Only re-deriving a claim says which of its supports was doing the work; a placement that reads unchanged is not thereby unaffected. The MOVED-SHAPE half is the opposite case — it needs the diffs, so it stays after the comparison — and there it may not RAISE. It PRINTS and feeds the exit code, like `over_declared_rules`, its structural sibling and the other post-worker check on recorded data: a raise at that point lands MID-report and takes dormancy, OVER-DECLARED, UNEXPLAINED and the radar block with it, which compare.py measures on a narrowed 1.4.0 run beside the check. A stale roster row must never hide an unexplained diff, that being the gate's primary output. The two pre-worker refusals may raise because nothing has printed yet; this one cannot, and the difference is PRINT ORDER rather than severity. The two halves also read opposite name lists — pre-skip for the files question, post-skip for the run question — which is `recorded_diff_mismatches`' docstring's to state and is not restated here. +- 2026-09-03 [#497](https://github.com/derek73/python-nameparser/issues/497) — the recorded shapes live in the TOOL, as `compare._RECORDED_DIFFS`, and the RUN verifies them; `_CROSS_RULE_WINNERS` keeps only which rule wins a contested name and reads the shape from there. There because it is the one place both halves exist: `main()` has measured every name's real diff by the time it asks, where `validate_rules` runs before any corpus is read and the unit suite installs no wheel at all — `tests/v2/test_differential.py`'s own header says "nothing here spawns `uv` or the network", and the tests that need a baseline fake `_run_worker` or fake `Popen` under it. State it that way and not as "every test monkeypatches `_run_worker`", which is false: measured 2026-09-03 on this branch, of 171 `def test_*` functions in that module 20 mention `_run_worker` at all and six reach the real one with `Popen` faked beneath them by `_fake_popen`. RECOMPUTE by walking the module's `def test_*` bodies for the name. That short form is what compare.py's `_RECORDED_DIFFS` provenance note SAID when this entry was written; `f70a0ad`, two commits later, withdrew it there, and the note now rejects the paraphrase rather than carrying it — so follow that pointer to the history and not to the text. The denominator has moved once already inside this arc: `f70a0ad`'s own message says 168, which is why the argument here is "most never mention it" and the digits carry a date. The hermeticity claim is unharmed and the loose paraphrase of it is what fails, which is this arc's subject arriving inside its own evidence. Read HERMETICITY as the reason and not speed. The rival placement is the unit suite, and the reason it is refused is that a check there would have to install a wheel — not that installing one is slow. The timing finding below retires every cost argument in this file and moves this decision not at all, which is the test such a placement argument has to pass and the reason to say out loud which ground carries it. `_CORPUS_FLOORS` is the precedent — tool-owned, corpus-shaped data that the guards read and monkeypatch — and this is the same direction. +- 2026-09-03 #497 — ONE roster, TWO checks, TWO PLACEMENTS — and, since `9360919`, a NOTE over the gap between them, recorded at the end of this entry. The asymmetry of the two checks is the decision rather than where the code happened to fit. The ABSENT-NAME half — a recorded row naming a string no corpus holds any more, which nothing else would ever report — refuses PRE-worker, beside `vacant`, because it reads the ledger and the loaded names and nothing the worker produces. It takes the PLACEMENT #382 chose for its own pre-worker refusal and NOT #382's reason for it, which was the cost claim below — `compare.py`'s comment at that placement is one of the sites the timing finding retires. The ground it stands on was rewritten on this branch: a refusal raised after the wheel is installed and the corpus compared prints below the run's own published `baseline:` header, for a comparison it will never report. Not "disowning a comparison it has just published" — measured print order is `baseline:` at the tell, then the comparison loop, then `corpus: … intentional diffs:`, so at that position the header has printed and no line of the comparison has. The ORDER is the argument, and that corrected wording is what `compare.py` now carries at all three of its placement comments. Recorded because it is the second decision the false number was under, and the two came apart differently — the declined check below kept its conclusion on a new argument, and here the ORDER was right all along while the reason for it was not. Only re-deriving a claim says which of its supports was doing the work; a placement that reads unchanged is not thereby unaffected. The MOVED-SHAPE half is the opposite case — it needs the diffs, so it stays after the comparison — and there it may not RAISE. It PRINTS and feeds the exit code, like `over_declared_rules`, its structural sibling and the other post-worker check on recorded data: a raise at that point lands MID-report and takes dormancy, OVER-DECLARED, UNEXPLAINED and the radar block with it, which compare.py measures on a narrowed 1.4.0 run beside the check. A stale roster row must never hide an unexplained diff, that being the gate's primary output. The two pre-worker refusals may raise because nothing has printed yet; this one cannot, and the difference is PRINT ORDER rather than severity. The two halves also read opposite name lists, which is `recorded_diff_mismatches`' docstring's to state and is not restated here. What IS decided here is the THIRD thing this roster produces, added by `9360919` once the gap between those two lists was measured and found not empty: an order-bearing entry the baseline cannot honor sits in a corpus file and outside the compared set at once, so a row on such a name is checked by neither half and reported by neither. `main()` prints a `NOT CHECKED` note over it. A NOTE and not a refusal — the docstring's argument against refusing off the post-skip list stands, and it is an argument against REFUSING and not against SAYING, which are two decisions of which only the first was ever made. Informational, outside the exit code, and gated on neither `full_corpus` nor `args.corpus`, the intersection narrowing itself. Measured 2026-09-03 at 1.4.0 — both the only baseline where the skip fires and the only ledger with rows — the window is three names, none of which carries a roster row today, so the note prints on no run yet. The measurement that motivated it put a deliberately wrong shape on `'de la Cruz née Vega'` over the full corpus at 1.4.0: before `9360919` that exited 0 in 375 stdout lines naming the name in none of them, and it now exits 0 in 378 with the `NOT CHECKED` note naming it — re-measured 2026-09-03 by corrupting the row in memory around `main()`, which leaves the worktree alone. - 2026-09-03 #497 — a SHAPE may be pinned where a COUNT may not, and that line is the whole of what keeps this roster affordable. A diff shape is one name under two parser versions, so adding a name to a corpus cannot move it; an explained-name count moves on every corpus edit. Shapes drift only when the PARSER changes those names — so the drift and the reason to re-read the roster are the same event, which is exactly what a recorded number usually fails to be. That is why #452's churn objection to `explains = N` (the fields-only arc above, 2026-08-28) does not reach this roster, and why pinning shapes here does not weaken it: the four ledgers carry 201 rules today, one `_CORPUS_CLAIMS` entry each, every one liable to move whenever a corpus does, against 31 recorded shapes that move only with the parser. Read the two CAUSES and not the two totals — the argument is that one set is coupled to corpus edits and the other is not, and it survives either number moving. RECOMPUTE: `sum(len(v) for v in _CORPUS_CLAIMS.values())` in tests/v2/test_ledger_guards.py, and `sum(len(v) for v in _RECORDED_DIFFS.values())` in compare.py. -- 2026-09-03 #497 — four of the 35 rows the roster carried were DELETED rather than corrected. Two names across the two 2.x ledgers, so two rows each, and one of the two NAMES had a correctable shape — count rows or names deliberately, since the provenance note that holds the measurements speaks of the name. Deleted because neither name pinned a CONTEST, which is what the roster is for: at the shape each really produces exactly one rule admits it, so `classify()` was never adjudicating anything. `'Nguyen, Van'` diffs at no baseline at all, so no run ever asks `classify()` about it; `'Jane née and Jones Smith'` is a malformed harvest from a radar corpus whose right parse nobody can state, so a pin on it defends no boundary anyone would argue for — which is why the correctable one went too. compare.py's provenance note holds the per-name, per-baseline measurements behind both. The deletion emptied both 2.x sections, which stay present as explicit empty mappings — the completeness guards require it, and it is what makes emptiness sayable as a POSITION: a row is owed when someone argues a boundary, not before. What the emptiness is NOT — that these ledgers hold no contested diff — is measured and stated beside those sections, and whether the position should change now that those contests are measured rather than merely unexamined is [#501](https://github.com/derek73/python-nameparser/issues/501), open. Recorded with the deletion because it is the half a later sweep will get wrong: "only one rule admits it" is NOT on its own grounds to delete a row. A substantial minority of the surviving 1.4.0 rows are in that position and stay — the count and the names are beside them in test_ledger_guards.py — because the shapes they pin are shapes runs actually make, so a widened `fields` or a moved rule hands the name over and the test says so. The four deleted rows could not do that work at any edit, which is the difference. +- 2026-09-03 #497 — four of the 35 rows the roster carried were DELETED rather than corrected. Two names across the two 2.x ledgers, so two rows each, and one of the two NAMES had a correctable shape — count rows or names deliberately, since the provenance note that holds the measurements speaks of the name. Deleted because neither name pinned a CONTEST, which is what the roster is for: at the shape each really produces exactly one rule admits it, so `classify()` was never adjudicating anything. `'Nguyen, Van'` diffs at no baseline at all, so no run ever asks `classify()` about it; `'Jane née and Jones Smith'` is a malformed harvest from a radar corpus whose right parse nobody can state, so a pin on it defends no boundary anyone would argue for — which is why the correctable one went too. compare.py's provenance note holds the per-name, per-baseline measurements behind both. The deletion emptied both 2.x sections, which stay present as explicit empty mappings — the completeness guards require it, and it is what makes emptiness sayable as a POSITION: a row is owed when someone argues a boundary, not before. What the emptiness is NOT — that these ledgers hold no contested diff — is measured and stated beside those sections, and whether the position should change now that those contests are measured rather than merely unexamined is [#501](https://github.com/derek73/python-nameparser/issues/501), open. Recorded with the deletion because it is the half a later sweep will get wrong: "only one rule admits it" is NOT on its own grounds to delete a row. A substantial minority of the surviving 1.4.0 rows are in that position and stay — the count and its recompute are beside them in test_ledger_guards.py, where six of the thirteen say so in their own comments and the other seven do not, which is why the count is recomputed rather than read off the roster — because the shapes they pin are shapes runs actually make, so a widened `fields` or a moved rule hands the name over and the test says so. Counting admitters is not the test at all: 11 of those 13 route to a DIFFERENT rule under some other shape, and mechanisms.md#RECORDED-ROSTERS now carries that measurement in place of the criterion it stated first. The four deleted rows could not do that work at any edit, which is the difference. - 2026-09-03 #497 — the prose sweep is sorted by this file's own criterion (#451, "past-tense counts stay in a ledger comment; live ones do not") into three treatments, and the treatment follows from what the count is DOING rather than from how wrong it is. A count that IS the finding and reads past tense stays. A count standing beside the enumeration it counts loses the digit — the list is self-checking and the digit is the only part that can rot. A live exhaustiveness claim with nothing recomputing it is REGROUNDED on the property it was reaching for, so that the sentence stops depending on a number — saying a widening claims all but a handful of the whole population the assertion is over says what a fraction said and cannot go stale. One judgement goes with it and is the arc's own: a figure that could NOT be reproduced was not replaced with a fresh one. `_reaches_non_vocabulary`'s docstring now carries the shape of its finding, the instruction for measuring it, and no LIVE figure for the magnitude the old figures asserted — a replacement digit nobody can re-derive is the defect rather than the repair, and this file has already been burned by that shape (the #414 roster comment under the fields-only arc's `explains = N` bullet). The past-tense digits explaining why the live ones are gone stay, which is the first treatment applied inside the third, and a later sweep should not read them as survivors of the pass. Found rather than decided, and worth as much: @@ -806,14 +806,14 @@ Found rather than decided, and worth as much: - **The cost that decided a decline had never been measured, and the decline survived losing it.** "The multi-minute worker pass" entered 2026-08-05 in `7767ba2`, at ONE site, with no measurement beside it. The census of where it reached, the timing figures that replace it, their recompute and the `_run_worker` timing trap all live ONCE, in the rule-order arc's `Declined:` bullet on precise per-name contest detection above — the argument they falsify — and none of them is restated here; that census is the figure in this arc most likely to move, so pointing at its owner rather than copying it is the #451 rule applied to the very claim that demonstrates why. Two notes on reading it that belong here rather than there. Not every site spells the phrase — one says the pass "costs minutes" — so count the CLAIM and not the wording (`git grep -n -i minute 514a2a1`, this branch's base). And measured, the worker pass is a fraction of a second and a whole gate run is seconds at most, where the phrase said minutes: read that class and not the digits. Three things transfer past them, and a coda about removing it. The claim was never a measurement anyone got wrong — it was a plausible sentence about a subprocess that installs a wheel, and it propagated by COPYING, the mechanism the fields-only arc recorded for ledger comments and which reaches code comments identically. It was load-bearing and its conclusion still stands, rewritten on COVERAGE: a run-time check sees one ledger, for the baseline that run was invoked with, and only when somebody invokes the tool, where a static predicate covers a rule from the moment it is written. And that is how a false number stays invisible for a month — an argument with two supports loses one and nothing downstream of it changes, so no reader ever meets a contradiction. Removing it was itself hard to do cleanly: review of the commit that removed the phrase found four instances of the very defect that commit exists to remove, fixed in `40c82a7`. - **A row pinning a race with one runner is never exercised as a contest, so its shape only ever had to agree with itself.** That is why nothing caught the four, and it is the general shape rather than a fact about two names. The four arrived with #452's work (`ee6aaa6`, PR #455) under two claims of measurement — the roster header's "The diff shapes are measured against the 1.4.0 wheel, not guessed" and the 2.1.0 block's own "measured against this baseline's own run rather than copied from the 2.0.0 rows" — and every one of the 31 rows at 1.4.0 agrees with the run today, at all four baselines. Do NOT read that as the 1.4.0 half having been sound: one of those 31 was guessed too, `田中さん II`, wrong on a genuinely CONTESTED row, found by hand a day before this branch opened (`e5e4955`, the finding the arc above filed as #497) and already corrected when the 31 were re-measured. So the one-runner mechanism explains why THESE four survived a check that would have caught a contested row, not why every wrong shape survived — the contested one needed somebody to sit down with the wheel, which is the cost the run now removes. The transferable test is what to ask of a recorded INPUT: not "is it checked" but "would its CONSUMER answer differently if it were wrong". mechanisms.md#RECORDED-ROSTERS forbids re-deriving the EXPECTATION from the inputs the check reads; this roster obeyed that to the letter and still went wrong, on the other side of the assertion, which is why that entry now carries the limit. - **The scan that scoped the sweep was wrong three ways, and it is the scan rather than the three fixed lines that the next sweep will reuse.** Two of its 19 machine-readable hits were false positives — a version number, and a hypothesis rather than a measurement — so the real population was 17. Its file list omitted `expected_since_2.2.0.toml`, which held a further copy of a claim it was already treating in two other ledgers, so the sweep would have shipped two of three siblings corrected. And its predicate matched only lines BEGINNING with `#`, which made every docstring invisible to it: the stalest figures in the whole sweep survived the first pass for that reason alone, and a docstring pass over the two guard modules then found two more. Read the sweep as BOUNDED and not exhaustive — that docstring pass covered `tests/v2/test_ledger_guards.py` and `tests/v2/test_differential.py`, and compare.py's own docstrings were never swept. And a fourth thing, which is the finding about the finding: the scan's PREDICATE was not preserved. Its 19 hits and their 17 survivors are recorded in `8593f3a`'s commit message and cannot be re-derived from the tree, because nothing says what it matched beyond the leading `#`. That is the reusable artifact of a sweep — not the lines it fixed — so the next one has to write its own and has no earlier hit count to check itself against. Write the predicate down where the results go. -- **What the stale claims were, because the classes recur and the digits do not.** (i) A count of a population that had grown by half underneath it: a guard comment gave a Latin-half widening as claiming "644 of the 654 unclassified corpus names", where the same expression claims 994 of 1008 today (`len(_UNCLASSIFIED_NAMES)`, intersected with `_claimed("[A-Za-z]")`, in tests/v2/test_ledger_guards.py) — and since the argument was that the widening reaches essentially the whole population the assertion is over, it is written that way now and needs no recompute. (ii) A count of FILES: "all five corpora", in three places, where six `corpus*.jsonl` sit on disk and a run globs every one of them — the corpus count was never the argument, and the gate's own `corpora:` line is the check. (iii) A live count inside the very paragraph that names stale counts as the file's standing hazard — `fix(#274)`'s ledger comment walked "all 31 the regex claims" where that regex claims 32 today (`len(_claimed(rule["name_regex"]))`). (iv) A count over a self-checking enumeration: "Six such widenings" above a list of four, where nobody can now recover which two are missing and the list was always the evidence. The fifth is the one worth the space, and the docstring pass found it: `_carries` said the `isascii()` split covers "12 of the 16 entries", where `MAIDEN_MARKERS` ships 17 with 12 of them non-ASCII — the NUMERATOR was right and stayed right while the denominator rotted. A count can go stale in one of its two halves, which is the shape a re-reader is least likely to catch, and it is a reason to prefer naming the predicate over counting its output. +- **What the stale claims were, because the classes recur and the digits do not.** (i) A count of a population that had grown by half underneath it: a guard comment gave a Latin-half widening as claiming "644 of the 654 unclassified corpus names", where the same expression claims 994 of 1008 today (`len(_UNCLASSIFIED_NAMES)`, intersected with `_claimed("[A-Za-z]")`, in tests/v2/test_ledger_guards.py) — and since the argument was that the widening reaches essentially the whole population the assertion is over, it is written that way now and needs no recompute. (ii) A count of FILES: "all five corpora", in three places, where six `corpus*.jsonl` sit on disk and a run globs every one of them — the corpus count was never the argument, and the gate's own `corpora:` line is the check. (iii) A live count inside the very paragraph that names stale counts as the file's standing hazard — `fix(#274)`'s ledger comment walked "all 31 the regex claims" where that regex claims 32 today (`len(_claimed(rule["name_regex"]))`). (iv) A count over a self-checking enumeration: "Six such widenings" above a list of four, where nobody can now recover which two are missing and the list was always the evidence. The fifth is the one worth the space, and the docstring pass found it: `_carries` said the `isascii()` split covers "12 of the 16 entries", where at sweep time `MAIDEN_MARKERS` shipped 17 with 12 of them non-ASCII (measured 2026-09-03; RECOMPUTE with `len(MAIDEN_MARKERS)` and `sum(not m.isascii() for m in MAIDEN_MARKERS)` from `nameparser.config.maiden_markers`) — the NUMERATOR was right and stayed right while the denominator rotted. The repair names the predicate rather than counting its output, which is why the guard's own note past-tenses both figures and points the reader at the `isascii()` split itself. A count can go stale in one of its two halves, which is the shape a re-reader is least likely to catch, and it is a reason to prefer naming the predicate over counting its output. - **The tier split inverts the ranking that #452's still-open half rests on.** Beside the fields-equals-union check that landed, #452 carries a second proposal: a ceiling on how much of the corpus one rule may REACH, measured there as a share of the whole corpus. #468's tiering makes that the wrong denominator, because a radar diff cannot fail the run — a rule reaching many radar names is not the hazard a rule reaching many contract names is. Measured over the corpus population and tier split the rule-order arc's RECOMPUTE above already owns and this bullet does not restate, the two comma rules reach 288 names, 25.8% of all of them, but 64 contract names, 19.6% of that tier; `fix(#271/#272/#298)` reaches 108, 9.7%, and 80 contract, 24.5%. So by total reach the comma rules are the outlier and by contract reach the CJK rule is, and a ceiling on total reach would rank the wrong rule as the biggest hazard. Read the INVERSION rather than the four percentages: it is a claim about which tier the denominator counts, and it survives any drift that does not reverse the two tiers' composition. Recorded here because the proposal is open and the measurements in it predate the tiering. RECOMPUTE: tier every corpus name off `compare._CORPUS_TIERS`, contract winning where a string sits in files of both tiers, then take each rule's `name_regex` reach split by tier. Declined: - `explains = N` per rule as ledger data (2026-09-03, again) — #452's own first sketch, declined there on churn, re-examined here only because this arc pins a different recorded value and owed an account of why the two differ, and declined again on the same grounds. The third decision above is the argument in full: a shape cannot move when a corpus does and a count can. The landed fields-equals-union check already catches the drift that motivated the sketch — a rule's explained set changing SHAPE underneath it — and needs no number to do it. The question belongs to [#452](https://github.com/derek73/python-nameparser/issues/452), which is OPEN and is exactly it; nothing here forecloses it, and its remaining reach-ceiling half should be re-measured on the tier finding above. - The differential in CI (2026-09-03) — considered and dropped rather than deferred. It is a release gate that already runs locally on demand, and no sibling check — dormancy, over-declaration, the undeclared-contest refusal, this one — needed a CI job to be worth having. Worth recording as a DECLINE and not an omission precisely because the timing finding removes the only reason nobody proposed it: a job costing seconds is cheap, and cheapness was never the criterion. -- A general sweep of every number-carrying comment line (2026-09-03) — the sweep is scoped to the claim-shaped ones, which the scan put at 19 and which were 17. The population it declines is larger by nearly two orders of magnitude: measured 2026-09-03, comment lines carrying a digit run to four figures across the differential tooling, its four ledgers and the two guard modules, against 17 claim-shaped ones (RECOMPUTE: count lines matching `^\s*#` that also contain a digit, over `tools/differential/*.py`, `tools/differential/*.toml`, `tests/v2/test_ledger_guards.py` and `tests/v2/test_differential.py`). The exact figure is deliberately not given: an earlier draft of this bullet carried one, and the NEXT commit on this branch added two comment lines and falsified it — inside the entry declining a sweep for exactly that.) Most are issue references, version numbers and codepoints, none of which rot. Read the RATIO and not either digit: a hand sweep two orders of magnitude past the claims is how wrong claims get INTRODUCED rather than removed, and this arc demonstrated that at the smaller scale — the commit that removed the false cost claim needed a follow-up for four instances of the same defect inside itself, and the first prose sweep shipped having skipped three instances it had already identified (`8593f3a`). +- A general sweep of every number-carrying comment line (2026-09-03) — the sweep is scoped to the claim-shaped ones, which the scan put at 19 and which were 17 — the pair of figures that cannot be re-derived from the tree, since the scan's predicate was never preserved; they live in `8593f3a`'s commit message and the bullet above says so. The population it declines is larger by nearly two orders of magnitude: measured 2026-09-03, comment lines carrying a digit run to four figures across the differential tooling, its four ledgers and the two guard modules, against 17 claim-shaped ones (RECOMPUTE: count lines matching `^\s*#` that also contain a digit, over `tools/differential/*.py`, `tools/differential/*.toml`, `tests/v2/test_ledger_guards.py` and `tests/v2/test_differential.py`). The exact figure is deliberately not given: an earlier draft of this bullet carried one, and the NEXT commit on this branch added two comment lines and falsified it — inside the entry declining a sweep for exactly that. Most are issue references, version numbers and codepoints, none of which rot. Read the RATIO and not either digit: a hand sweep two orders of magnitude past the claims is how wrong claims get INTRODUCED rather than removed, and this arc demonstrated that at the smaller scale — the commit that removed the false cost claim needed a follow-up for four instances of the same defect inside itself, and the first prose sweep shipped having skipped three instances it had already identified (`8593f3a`). ### comma-suffix-arc — #291/#296/#316 (2026-07-26 → 2026-08-01) diff --git a/docs/design/mechanisms.md b/docs/design/mechanisms.md index ba47bb00..b90ce0a5 100644 --- a/docs/design/mechanisms.md +++ b/docs/design/mechanisms.md @@ -73,7 +73,7 @@ than inventing one. ## RECORDED-ROSTERS — record the answer, don't re-derive it -Problem shape. A guard needs to know what the answer WAS, so it can detect the answer changing. Contract statement. Store the measured answer as literal data (a roster) and compare against it; never re-derive the expectation from the same inputs the check reads, because a derivation from the same data always agrees with itself. Known limit, and it is what the contract statement does NOT cover: a roster row can hold a recorded INPUT as well as a recorded expectation, and only the expectation half is asserted. `_CROSS_RULE_WINNERS` obeyed this entry to the letter and still carried wrong rows — it recorded a diff shape per contested name AND the rule that should win, fed the shape to `classify()` as an argument, and asserted only the winner, so a wrong shape that still routed to the same rule agreed with itself for as long as nobody re-measured it by hand. The test to apply to a recorded input is not "is it checked" but "would the consumer answer DIFFERENTLY if it were wrong": where one rule alone admits a name's diff, the consumer answers the same either way and the row is inert whatever it says. The repair is to move the input to where something measures it — the shapes now live in tools/differential/compare.py as `_RECORDED_DIFFS` and a differential run verifies them against the diffs it has already computed, the roster reading them from there — so a recorded roster's recorded half is checked by a run rather than by the guard that consumes it (#497). Lives in. tests/v2/test_ledger_guards.py (_CORPUS_CLAIMS, _EXCLUSION_EFFECT, _CROSS_RULE_WINNERS, _ORDER_EXEMPTION_EFFECT, _SPAN_BEARING_RULES, _HONORIFIC_SOURCES, _LATIN_ALTERNATION_SOURCES), tools/differential/compare.py (_CORPUS_FLOORS, _RECORDED_DIFFS), tests/v2/test_facade_cases.py (_CORE_ONLY_IDS). Reach for it when. Writing a check whose expected value is computed by the code under test, or a comment that enumerates ids/counts — make it data the suite asserts. And when a roster row carries anything the check CONSUMES rather than compares, ask what measures that half. +Problem shape. A guard needs to know what the answer WAS, so it can detect the answer changing. Contract statement. Store the measured answer as literal data (a roster) and compare against it; never re-derive the expectation from the same inputs the check reads, because a derivation from the same data always agrees with itself. Known limit, and it is what the contract statement does NOT cover: a roster row can hold a recorded INPUT as well as a recorded expectation, and only the expectation half is asserted. `_CROSS_RULE_WINNERS` obeyed this entry to the letter and still carried wrong rows — it recorded a diff shape per contested name AND the rule that should win, fed the shape to `classify()` as an argument, and asserted only the winner, so a wrong shape that still routed to the same rule agreed with itself for as long as nobody re-measured it by hand. The test to apply to a recorded input is not "is it checked" but "would the consumer answer DIFFERENTLY if it were wrong" — which is a question about the PAIR of shapes and not about the row: a row is inert exactly where the recorded shape and the real one route to the same rule. Counting admitters does not answer it, and is the wrong instrument this entry itself reached for first. Measured 2026-09-03 over `_RECORDED_DIFFS['expected_since_1.4.0.toml']`: 13 of the 31 rows have exactly one rule admitting their recorded shape, and 11 of those 13 route to a DIFFERENT rule under some other shape, so the assertion would have caught a wrong one — `'Andrews, M.D.'` wins `fix(comma-family)` at its recorded `('given','suffix')` and `fix(comma-precomma-family)` at `('family',)`. Nor does "the same rule admits both shapes" make a row inert, since file order still decides: over that ledger 152 of the 246 shape pairs sharing an admitter route to different rules. RECOMPUTE: load the ledger, sort with `_sorted_rules`, and call `classify(name, shape, rules, exclusions, None)` at each row's recorded shape and at every other subset of the roles the roster uses. The four rows #497 deleted were inert for the PARTICULAR wrong shape they happened to carry, and not for any wrong shape — `'Jane née and Jones Smith'` routes to `fix(#445)` at both its recorded and its real shape in either 2.x ledger, and to `fix(#412)` at `('family',)`. The repair is to move the input to where something measures it — the shapes now live in tools/differential/compare.py as `_RECORDED_DIFFS` and a differential run verifies them against the diffs it has already computed, the roster reading them from there — so a recorded roster's recorded half is checked by a run rather than by the guard that consumes it (#497). Lives in. tests/v2/test_ledger_guards.py (_CORPUS_CLAIMS, _EXCLUSION_EFFECT, _CROSS_RULE_WINNERS, _ORDER_EXEMPTION_EFFECT, _SPAN_BEARING_RULES, _HONORIFIC_SOURCES, _LATIN_ALTERNATION_SOURCES), tools/differential/compare.py (_CORPUS_FLOORS, _RECORDED_DIFFS), tests/v2/test_facade_cases.py (_CORE_ONLY_IDS). Reach for it when. Writing a check whose expected value is computed by the code under test, or a comment that enumerates ids/counts — make it data the suite asserts. And when a roster row carries anything the check CONSUMES rather than compares, ask what measures that half. ## LEDGER-RULE-SEPARATION — file order decides, fields narrow by subset diff --git a/tests/v2/test_differential.py b/tests/v2/test_differential.py index b609ff2f..f123558f 100644 --- a/tests/v2/test_differential.py +++ b/tests/v2/test_differential.py @@ -2214,9 +2214,14 @@ def test_a_full_run_refuses_a_recorded_name_no_corpus_holds( Pre-worker, like both refusals it sits beside: it reads the ledger and the loaded names and nothing the comparison produces, and a - refusal raised afterwards prints below the run's own `baseline:` - header for a comparison it is about to disown. Its measured half - cannot do that, and prints instead. + refusal raised afterwards prints below the run's own published + `baseline:` header, for a comparison it will never report -- not + "a comparison it is about to disown", since nothing of the + comparison prints until the `corpus: ... intentional diffs:` line + below it. It is the ORDER that earns the placement, which is the + wording compare.py carries at all three of its own placement + comments. Its measured half cannot refuse at all, and prints + instead. """ monkeypatch.setitem(compare._RECORDED_DIFFS, "expected_since_1.4.0.toml", @@ -3568,13 +3573,21 @@ def test_a_recorded_shape_matching_the_run_is_no_mismatch() -> None: # mismatch on others. All seven roles rather than the two above # because an unsorted reading cannot be refuted outright -- a set # CAN iterate alphabetically, and the two-role set above does under - # 85 of PYTHONHASHSEED 0..199. The seven-role set did under none of - # them, so the size is what makes this row an instrument rather - # than a coin flip. Recompute either count with + # 85 of PYTHONHASHSEED 0..199. The seven-role set THE ASSERTION + # BELOW BUILDS does under none of them, so the size is what makes + # this row an instrument rather than a coin flip. Measured + # 2026-09-03; recompute either count with # for s in $(seq 0 199); do PYTHONHASHSEED=$s python3 -c \ - # 'print(list({"family","suffix"})==sorted({"family","suffix"}))' + # 'S={"family","suffix"};print(list(S)==sorted(S))' # done | grep -c True - # substituting compare.V2_FIELDS for the two-role set. + # substituting, for the seven-role count, + # S=set(("title","given","middle","family","suffix", + # "nickname","maiden")) + # -- compare.V2_FIELDS in ITS OWN order, which is the set the + # assertion passes. INSERTION ORDER is part of the measurement and + # not decoration: a bare set literal spelled alphabetically instead + # iterates sorted under seed 56, so a recipe retyped that way reads + # 1 of 200 where the assertion's own set reads 0. assert compare.recorded_diff_mismatches( {"Smith, Jr.": tuple(sorted(compare.V2_FIELDS))}, [("Smith, Jr.", set(compare.V2_FIELDS), None)], diff --git a/tests/v2/test_ledger_guards.py b/tests/v2/test_ledger_guards.py index 19175c5f..93b2966e 100644 --- a/tests/v2/test_ledger_guards.py +++ b/tests/v2/test_ledger_guards.py @@ -2666,11 +2666,22 @@ def test_every_rule_claims_the_recorded_share_of_the_corpus() -> None: # in this file presumes it. # # Nor is "only one rule admits it" grounds on its own to delete a - # row: 13 of the 31 above are in that position too, and say so (the - # jr rule's surplus, the bound-given trio). They stay because the - # shapes they pin are shapes runs actually make, so widening a - # `fields` or moving a rule hands the name over and this test says - # so. The four deleted rows could not do that work at any edit. + # row: 13 of the 31 above are in that position too. SIX of the 13 + # say so where they sit -- the jr rule's surplus and the + # bound-given trio, three rows each -- and the other seven do not, + # so take the count from the recompute rather than from the + # comments. RECOMPUTE (measured 2026-09-03): count the rows of + # compare._RECORDED_DIFFS['expected_since_1.4.0.toml'] for which + # exactly one rule of compare._sorted_rules over this ledger + # satisfies compare._entry_matches at the recorded shape. They + # stay because the shapes they pin are shapes runs actually make, + # so widening a `fields` or moving a rule hands the name over and + # this test says so -- and 11 of the 13 route to a DIFFERENT rule + # under some other shape, so the assertion is doing work on them. + # mechanisms.md#RECORDED-ROSTERS carries that measurement and the + # reason counting admitters is the wrong instrument for the + # question. The four deleted rows could not do that work at any + # edit. "expected_since_2.0.0.toml": {}, "expected_since_2.1.0.toml": {}, } diff --git a/tools/differential/README.md b/tools/differential/README.md index beedd6f6..380b6f23 100644 --- a/tools/differential/README.md +++ b/tools/differential/README.md @@ -242,6 +242,19 @@ than a note about those five. A stays UNEXPLAINED and fails the run even when the name itself sits in a radar file. +A recorded diff shape (`MOVED SHAPE`, below) is the second thing that +outranks the tier, and the larger of the two: it feeds the exit code +without asking what tier the name sits in. Measured 2026-09-03, 21 of +the 31 rows in `_RECORDED_DIFFS['expected_since_1.4.0.toml']` sit on +radar-tier names, so radar PARSER DRIFT can fail the run wherever +someone has pinned a shape (RECOMPUTE: for each row, take the tier of +the first corpus file holding the name with contract files sorted +first, as `main()` loads them). That is a pin doing what pins do -- +someone wrote the row by hand and it says what the name does -- but it +is not what "a radar diff can never fail the run" leads a reader to +expect. Read the tier rule as being about UNMATCHED diffs, which is +the only thing it was ever measured over. + A corpus line is a bare JSON string or an object carrying `name` and, optionally, `tests` or `shape` -- the input-shape id from `tools/differential/shapes.py`, which is where each shape's notation, @@ -588,6 +601,13 @@ dynamic check, so the release checklist runs `compare.py` at every baseline that has a ledger with rules -- a ledger left out of the checklist gets no dynamic dormancy check at all, and a `dormant` declaration in it goes unaudited. Adding a new ledger means adding its baseline to the checklist. +Measured 2026-09-03, that has already slipped: the root `AGENTS.md` checklist +lists THREE commands -- 1.4.0, the bare default, and 2.0.0 -- against four +ledgers carrying rules, so `expected_since_2.1.0.toml` is the one no listed +command reaches. The over-declaration check (#452) and the recorded-shape +check (#497) are per-ledger for the same reason the dormancy one is -- the +latter reads `_RECORDED_DIFFS[ledger.name]` -- so a ledger left out of the +checklist gets none of the three. The static tier still exempts a rule on the mere PRESENCE of `dormant`; it never asks whether the reason is still true. Only the dynamic check can @@ -732,12 +752,13 @@ Four checks now read the flag differently, and the differences are deliberate ra `tests/v2/test_ledger_guards.py` carries `_CROSS_RULE_WINNERS`, a roster pinning which rule should win a contested name. To ask that question it needs the name's diff SHAPE, which it feeds to `classify()` as an input -- so the shape is never itself checked, and a guessed one agrees with itself forever. That is how `田中さん II` sat recorded as `{given, suffix}` under a docstring promising the shapes were measured; the real diff is `{family, given, suffix}` (#497). The unit suite cannot catch it: it spawns no worker, deliberately, so it has no measured diff to compare against. -So the shapes live in `_RECORDED_DIFFS` in `compare.py`, keyed per ledger (a string moves a different set of roles against different baselines), and every run checks them. Two findings come out of it, and they behave differently on purpose. +So the shapes live in `_RECORDED_DIFFS` in `compare.py`, keyed per ledger (a string moves a different set of roles against different baselines), and every run checks them. THREE things come out of it -- two findings and a note -- and they behave differently on purpose. - **`MOVED SHAPE`** -- a recorded shape the run contradicts. Printed after the comparison, alongside `EXPLAINED NOTHING` and `OVER-DECLARED`, and it feeds the exit code the same way. It does not raise: a refusal there would land mid-report and take the `UNEXPLAINED` block down with it, and a stale roster row must never hide an unexplained diff. A moved shape is a FINDING, not a number to update, and it names no cause because it cannot: the parser may have changed what the name does, or the row may have been wrong when it was recorded -- which is what #497 found, four rows recording a shape no run makes under a roster promising the shapes were measured. Either way the winner pinned beside it in `_CROSS_RULE_WINNERS` was recorded against the OLD shape, so read both before editing either. Where the run measured no default-order diff at all, the report says so and names no cause: two states reach it (the parser stopped moving the name, or the name is compared only under a declared order) and the check cannot separate them. - **A row naming a name no corpus holds** -- refused, before the worker runs, on a FULL run only. Nothing measures such a row, so it agrees with itself forever, which is the defect above in a second form. The repair is not mechanical: ask first whether the name left deliberately (`git log -S'' -- tools/differential/corpus*.jsonl`), then either delete the row and its `_CROSS_RULE_WINNERS` partner, or restore the name to the corpus that lost it. Under `--corpus` this prints nothing, since a narrowed run legitimately holds almost none of the roster. +- **`NOT CHECKED`** -- a NOTE rather than a finding, printed after the comparison and deliberately outside the exit code (#497, `9360919`). It names recorded rows whose corpus entry THIS BASELINE SKIPPED: an order-bearing entry a baseline with no `Policy` cannot honor is dropped before the comparison runs, so there is no measured diff to check the row against. Such a row falls between the two checks above -- `MOVED SHAPE` skips a name it did not compare, and the departed-name refusal passes it because the skip takes a name out of the RUN and out of no file -- and without the note it is checked by neither and reported by neither. So the note says which rows the shape report is silent about and claims nothing else. Do NOT delete a row over it: the name is in a corpus this run read. Re-run at a baseline that can honor its order to check it. Measured at 1.4.0, which is both the only baseline where the skip fires and the only ledger with rows, the window is `de Mesnil Jean, Dr.`, `de la Cruz Juan Carlos, Dr.` and `de la Cruz née Vega` -- none carrying a roster row today, so the note prints on no run yet. "Do not refuse" and "say nothing" are two decisions, and only the first was ever argued. -The two read opposite name lists, and swapping them is the live hazard. "Was this name compared?" is about the RUN, so the shape check reads the list AFTER the baseline-minimum shape skip. "Does any corpus still hold this name?" is about the FILES, and the skip empties no file, so the departed-name check reads the list BEFORE it. `recorded_diff_mismatches`' docstring in `compare.py` carries the measurement and the recompute. +The two CHECKS read opposite name lists, and swapping them is the live hazard. "Was this name compared?" is about the RUN, so the shape check reads the list AFTER the baseline-minimum shape skip. "Does any corpus still hold this name?" is about the FILES, and the skip empties no file, so the departed-name check reads the list BEFORE it. The note reads both, being exactly the difference: the roster intersected with the pre-skip list, minus the post-skip one. It is also the one of the three that does not consult `full_corpus` -- that intersection already narrows itself under `--corpus`. `recorded_diff_mismatches`' docstring in `compare.py` carries the measurement and the recompute. ### Shapes that must never be explained (`[[never]]`) diff --git a/tools/differential/compare.py b/tools/differential/compare.py index 56144178..49cf2de3 100644 --- a/tools/differential/compare.py +++ b/tools/differential/compare.py @@ -1636,16 +1636,20 @@ class _ShapeMismatch(NamedTuple): #: (tests/v2/test_ledger_guards.py), per ledger (#497). #: #: Baseline-relative by construction, which is why it is keyed per -#: ledger rather than once by name even though the two names appearing -#: in two ledgers carry the same shape in both: a string moves a -#: different set of roles against different baselines, and decisions.md -#: (2026-08-28 #452) records `fix(#296) a lone post-comma credential is -#: a suffix` moving four roles at 1.4.0 and two at both 2.x baselines. +#: ledger rather than once by name: a string moves a different set of +#: roles against different baselines, and decisions.md (2026-08-28 +#: #452) records `fix(#296) a lone post-comma credential is a suffix` +#: moving four roles at 1.4.0 and two at both 2.x baselines. No name +#: sits in two sections today -- the three non-1.4.0 ones are empty -- +#: so the keying is held by that argument alone. #: -#: A recorded shape that MOVES is a FINDING, not a number to update: -#: the parser changed what that name does, and the winner pinned beside -#: it in _CROSS_RULE_WINNERS was recorded for the OLD shape, so both -#: want reading before either is edited. +#: A recorded shape that MOVES is a FINDING, not a number to update, +#: and it names no cause because it cannot: the parser may have changed +#: what that name does, or the row may have been wrong when it was +#: recorded, which is what the provenance below found four times. +#: Either way the winner pinned beside it in _CROSS_RULE_WINNERS was +#: recorded for the OLD shape, so both want reading before either is +#: edited. #: #: HERE rather than beside the roster because this is where the #: measurement happens. main() computes every name's real diff, so a @@ -1759,8 +1763,17 @@ def recorded_diff_mismatches( skip runs. The two differ by exactly the entries an old baseline cannot honor -- 7 of them at 1.4.0, where 1120 load and 1113 compare; re-measure by running this file with `--baseline 1.4.0` - and reading its `skipped` and `corpus:` lines. decisions.md, "the - rule-order arc", records the same trap for the contest checks, + and reading its `skipped` and `corpus:` lines. Those are ENTRY + counts. The paragraph further down counts NAMES, which is why the + same skip reads as 7 here and as 3 there: an entry is a (name, + order) pair, so the 1120 deduped entries carry 1116 distinct + strings -- three strings are compared under more than one order + ('de la Cruz Juan Carlos' under three, 'John Smith, Dr.' and 'de + la Cruz née Vega' under two) -- and four of the seven skipped + entries are the whole of the three names named below, while the + other three have a same-name twin that survives the skip. + decisions.md, "the rule-order arc", records the same trap for the + contest checks, which read the pre-skip list on purpose. Pass that list here and a roster row whose only entry was skipped reports as a name that stopped diffing, when it was never compared at all. @@ -2162,8 +2175,15 @@ def main() -> int: # reads the ledger and the loaded names and nothing the worker # produces, and a refusal raised after the worker has installed the # pinned wheel and compared the whole corpus prints below the run's - # own `baseline:` header, disowning a comparison it just published - # (#382). The measured half of this check is the opposite case and + # own published `baseline:` header, for a comparison that will + # never be reported (#382). Not "disowning a comparison it just + # published": measured print order is `baseline:` at the tell, + # then the comparison loop, which prints nothing, then `corpus: + # ... intentional diffs:`, so at that later point the header + # would have printed and no line of the comparison would have. It + # is the ORDER of the two that earns the placement, as it does at + # the twin comment above. HERE, nothing has printed at all. + # The measured half of this check is the opposite case and # sits after the comparison, where the diff it reads exists. # # PRE-skip `corpus_names`, where the measured half takes the @@ -2269,8 +2289,11 @@ def main() -> int: # The tree is checked BEFORE the worker runs. It depends on nothing # the worker produces, and validate_rules' own reasoning applies: a # misconfiguration that aborts after the whole install-and-compare - # pass is one the run reports below its own `baseline:` line, - # having already published a comparison it is about to disown. It + # pass is one the run reports below its own published `baseline:` + # line, having installed the wheel and compared the whole corpus + # for a report it will never make -- not "disowning a comparison it + # published", since nothing of the comparison prints until the + # `corpus: ... intentional diffs:` line further down. It # is the ORDER that earns the placement, not the clock. This # comment used to say that pass # "costs minutes" -- a magnitude nobody had measured, and wrong: @@ -2516,8 +2539,16 @@ def _tree_parse(name: str, order: str | None) -> object: # below uses for the two blocks that share it. A real parser # move lands on many of these rows at once, and per-row # repetition would bury the names under its own advice. - # (OVER-DECLARED repeats per row, but its shared text is one - # sentence.) No cause is named for a row with no measured + # (OVER-DECLARED repeats its shared text per row and is not + # the precedent for doing so here: measured 2026-09-03 its + # row-invariant text is three sentences and 47 words, against + # this lead's two and 39, so it is the LONGER of the two -- it + # gets away with the repetition because its rows come one per + # over-declared RULE, where these come one per name and a real + # parser move lands on many at once. An earlier draft of this + # parenthetical called that text one sentence and cited it as + # the model.) No cause is + # named for a row with no measured # shape: the check cannot tell the two apart -- # recorded_diff_mismatches' docstring says why they are # collapsed -- and naming one would send half the readers to diff --git a/tools/differential/expected_since_1.4.0.toml b/tools/differential/expected_since_1.4.0.toml index 63e1ae67..d316da79 100644 --- a/tools/differential/expected_since_1.4.0.toml +++ b/tools/differential/expected_since_1.4.0.toml @@ -429,8 +429,10 @@ definitionally family") while the family move is the whole of the credential reading here: 'MD' stops being a first name and becomes the one-word name. This rule is the routing rule PLUS the pre-comma merge on a single string, so a widened routing rule would take the union and -report only half of it. _CROSS_RULE_WINNERS pins ('MD, PHD', -('family','given','suffix','title')) here. Both names sit in +report only half of it. _CROSS_RULE_WINNERS pins 'MD, PHD' here, +keyed by the name alone since #497, and the shape it is adjudicated +at -- ('family','given','suffix','title') -- is the one +compare._RECORDED_DIFFS records for the name. Both names sit in corpus_issues.jsonl alone -- radar since #468's tier split, and untouched by #488, which created corpus_cjk_tolerated.jsonl and holds neither name -- so neither can demand a rule any more and nothing diff --git a/tools/differential/expected_since_2.0.0.toml b/tools/differential/expected_since_2.0.0.toml index 7d0f4915..5c361f6d 100644 --- a/tools/differential/expected_since_2.0.0.toml +++ b/tools/differential/expected_since_2.0.0.toml @@ -1422,8 +1422,10 @@ issue = "fix(#462) the facade keeps an initial-shaped conjunction letter" # bare e/y is the connective and does not move. The regex reaches 18 # corpus names, of which 14 diffed when the rule was written # (2026-09-02); _CORPUS_CLAIMS in tests/v2/test_ledger_guards.py pins -# that reach and fails if it moves, so the digit here is checked -# rather than remembered. The names it reaches and does not move are +# that reach as _Claim(18, ...) and fails if it moves, so the REACH +# digit is checked rather than remembered. The 14 is not: it is a +# dated snapshot of what diffed that day and nothing re-runs it. +# The names it reaches and does not move are # what the gap IS, and the invariant behind it is a GROUP rather than # a count: # they carry the E/Y in the GIVEN group, which has always initialed diff --git a/tools/differential/expected_since_2.1.0.toml b/tools/differential/expected_since_2.1.0.toml index 6d2840e7..e84c74d6 100644 --- a/tools/differential/expected_since_2.1.0.toml +++ b/tools/differential/expected_since_2.1.0.toml @@ -1371,8 +1371,10 @@ issue = "fix(#462) the facade keeps an initial-shaped conjunction letter" # bare e/y is the connective and does not move. The regex reaches 18 # corpus names, of which 14 diffed when the rule was written # (2026-09-02); _CORPUS_CLAIMS in tests/v2/test_ledger_guards.py pins -# that reach and fails if it moves, so the digit here is checked -# rather than remembered. The names it reaches and does not move are +# that reach as _Claim(18, ...) and fails if it moves, so the REACH +# digit is checked rather than remembered. The 14 is not: it is a +# dated snapshot of what diffed that day and nothing re-runs it. +# The names it reaches and does not move are # what the gap IS, and the invariant behind it is a GROUP rather than # a count: # they carry the E/Y in the GIVEN group, which has always initialed diff --git a/tools/differential/expected_since_2.2.0.toml b/tools/differential/expected_since_2.2.0.toml index ffa11422..46cbabf0 100644 --- a/tools/differential/expected_since_2.2.0.toml +++ b/tools/differential/expected_since_2.2.0.toml @@ -50,8 +50,10 @@ issue = "fix(#462) the facade keeps an initial-shaped conjunction letter" # bare e/y is the connective and does not move. The regex reaches 18 # corpus names, of which 14 diffed when the rule was written # (2026-09-02); _CORPUS_CLAIMS in tests/v2/test_ledger_guards.py pins -# that reach and fails if it moves, so the digit here is checked -# rather than remembered. The names it reaches and does not move are +# that reach as _Claim(18, ...) and fails if it moves, so the REACH +# digit is checked rather than remembered. The 14 is not: it is a +# dated snapshot of what diffed that day and nothing re-runs it. +# The names it reaches and does not move are # what the gap IS, and the invariant behind it is a GROUP rather than # a count: # they carry the E/Y in the GIVEN group, which has always initialed From 17a9e662a99d675e1db96bd28cfb37b49830a284 Mon Sep 17 00:00:00 2001 From: Derek Gulbranson Date: Thu, 3 Sep 2026 12:59:40 -0700 Subject: [PATCH 22/23] docs+tooling(differential): five claims this branch made stale or miscounted Four of the five were true when written and false by the time they shipped, all inside this branch. decisions.md's "22 of 33 today" for _CROSS_RULE_WINNERS was measured at 960c6d0 and then falsified by b793ebc, the same branch's own deletion of four rows -- fourteen lines below, the same file says so. Measured now: 31 rows, 31 distinct names, 21 radar / 10 contract. The digits go entirely; the argument was always the MAJORITY, which survives them, and a recompute replaces the count. compare.py's "375 stdout lines with no mention of the name" is the pre-9360919 side of a measurement whose post side sits four lines below it, in the code implementing the note. decisions.md was corrected for exactly this in 02ecdd5 and compare.py was not. Re-measured 2026-09-03: 378 lines, exit 0, the NOT CHECKED note naming the name; both sides now stated with the commit between them. README's "the one of the three that does not consult full_corpus" counts one where two do not: only the departed-name refusal reads the flag. MOVED SHAPE reads no flag and its docstring says so -- measured, a --corpus corpus_issues.jsonl run at 1.4.0 with a corrupted shape on 'Carod i' prints MOVED SHAPE and exits 1. A reader was being told a narrowed failure is suppressed like the departed-name one. "f70a0ad, two commits later" is one commit later; d9c3bd8 is its parent. The sentence sends the reader into history, so the miscount defeats it. And README's "the larger of the two" over the [[never]] exclusions named no metric and inverts under two of them: 31 rows against 2 exclusions one way, and 21 radar names pinned against 60 corpus names those two patterns reach (37 radar) the other. The comparative goes; both measurements stay, dated. Co-Authored-By: Claude Opus 5 --- docs/design/decisions.md | 4 ++-- tools/differential/README.md | 23 +++++++++++++++-------- tools/differential/compare.py | 19 ++++++++++++------- 3 files changed, 29 insertions(+), 17 deletions(-) diff --git a/docs/design/decisions.md b/docs/design/decisions.md index 808ea961..5fb4f3bc 100644 --- a/docs/design/decisions.md +++ b/docs/design/decisions.md @@ -785,7 +785,7 @@ Declined: - A family-side twin of `fix(cjk-comma-honorific-peel)` for a PRE-comma glued honorific (2026-09-02, [#496](https://github.com/derek73/python-nameparser/issues/496), closed as not planned) — the rule that would honestly describe `王先生, V.`, `田中さん, V.` and `김민준씨, V.`, whose diff is the glued peel alone while the label they carry says comma compound (the finding bullet above). Declined on two grounds. FIRST, no gate can demand it: all 17 names the compound/peel pair is contested across measure radar — but by two different warrants, and an audit that assumes one will find the count short. Sixteen sit in `corpus_cjk_tolerated.jsonl`, the file #488 created by demoting the composed comma/Latin-wrapper CJK forms to tolerated input, `김, 민준씨` — the name #382 was filed over — among them. The seventeenth, `Dr 田中さん, V.`, is in `corpus_issues.jsonl` alone: harvested and append-only, radar since #468, and untouched by #488. (`Dr 김민준씨, Jr.` is in both files, and reads radar from either.) Radar means an unmatched diff is reported rather than fatal for want of a RULE; it is not an unconditional never-fatal, since a `[[never]]` exclusion outranks the tier and stays fatal on a radar name — `_CORPUS_TIERS`' own note, and `main()`'s two-reasons comment, which routes an excluded name to `unexplained` rather than to `radar`. No exclusion refuses these, so nothing here can demand the twin. SECOND, [#495](https://github.com/derek73/python-nameparser/issues/495) points the other way over the same corner: it asks whether the radar-only rules already in the ledger still earn their place and records `fix(cjk-comma-compound)` as having zero contract-tier reach, so #496 proposed a SECOND rule for the very names #495 is weighing a first one away from. Fewer rules for this corner is the coherent direction, and that is what makes the decline decisive rather than merely permissive. REOPEN it if a pre-comma glued-honorific name ever reads contract — and watch BOTH routes, because the two files promote by different mechanisms. A `corpus_cjk_tolerated.jsonl` name is promoted by clearing `tolerated` on its case rows, which moves the text into `corpus_cjk.jsonl` (`_CORPUS_TIERS`, and build_cjk_corpus.py's split). `Dr 田中さん, V.` has no case row at all, so there is no flag to clear: it changes tier only by being CHOSEN — a new unmarked row, or a rules.md example — which puts the text in a contract corpus, and the (name, order) dedup loads contract files first and keeps that reading. Read the tier off `_CORPUS_TIERS` either way, never off a hand-built file→tier map (the roster caution above). - Precise per-name contest detection at differential-run time (2026-09-02; its stated reason replaced 2026-09-03, [#497](https://github.com/derek73/python-nameparser/issues/497)) — it would replace the static predicate's over-reporting with the measured nine-of-eleven split, and it needs the pinned-wheel worker pass to do it. The decline stands, on COVERAGE alone: a run-time check sees one ledger — the one for the baseline that run was invoked with — and only once somebody invokes the tool, whereas the static predicate covers a rule from the moment it is written, including a rule nobody has ever run the wheel against, which is the whole point of #382. Nor can the unit suite simply adopt the precise check: `tests/v2/test_differential.py` is deliberately hermetic — its own header says "nothing here spawns `uv` or the network", and the tests that need a baseline fake `_run_worker` or `Popen` outright — so moving it to pytest speed means spawning a subprocess and, on a cold cache, reaching PyPI. WITHDRAWN, and named here so it is not re-argued: this bullet used to say the check would sit "behind a multi-minute run" and gave that as "the whole of the reason". No measurement was recorded beside it: the phrase entered on 2026-08-05 in `7767ba2` at a single site in `tools/differential/compare.py`, and was copied outward until eight sites carried it — four there, three in `tests/v2/test_differential.py`, and this bullet. Measured 2026-09-03 over the corpus population named above, a whole run is 0.33s wall at baseline 1.4.0 and 0.57s at 2.2.0, the default; the worker pass is 0.10s and 0.29s of those, timed inside `main()`. All four baselines back to back total 1.97s, and a run with `uv`'s cache emptied first, so the pinned wheel is downloaded rather than reused, was still under a second — that last one is NETWORK- and environment-dependent and the RECOMPUTE below does not reach it, covering the warm run alone. Reproducing it means pointing `UV_CACHE_DIR` at an empty directory so the wheel is fetched rather than reused, and the number will differ on a different link. Read it as "the download is not the story either", not as a figure to check against. Read the class and not the digits: the worker pass is a fraction of a second and the whole gate is seconds at most, where the withdrawn phrase said minutes — so no argument here may be built on what the worker pass costs. RECOMPUTE: `time (uv run python tools/differential/compare.py --baseline 1.4.0 >/dev/null)`. For the worker's share, wrap `compare._run_worker` in a timer and then call `main()` — calling `_run_worker` directly on every loaded entry aborts at a 1.4.0 baseline, since the seven order-bearing shape-4/5 entries have to be dropped by the baseline-minimum skip first. Do NOT restate the decline as an error-direction argument either: within the NESTED pairs computing real diffs can only remove them, but the predicate is not a superset of the contests, since nesting is sufficient for one and not necessary (the bullet above, and #498). What the static check buys is coverage of a rule nobody has run the wheel against, not a guarantee of refusing everything it should. - Giving the peel rule a predicate the compound rule fails (2026-09-02, #382 option 1) — narrowing `fix(cjk-comma-honorific-peel)`'s `name_regex` to the honorific-bearing shapes, whose stated effect in #382 was that the pair "becomes order-independent and the original contract holds again". Declined because that effect is UNREACHABLE, and measured rather than argued. The two rules ship a BYTE-IDENTICAL `name_regex` (verified 2026-09-02 by comparing the two strings), and `_rule_reach` computes each rule's names from its OWN pattern, so narrowing the peel rule's regex narrows the peel rule's reach and nothing else: the compound rule goes on reaching every one of the names. The general form, which no regex edit escapes — where the earlier rule's `fields` are a SUBSET of the later one's, order-independence requires the LATER rule to stop reaching the name, and an edit to the earlier rule's own regex cannot cause that. Here the peel rule's `{given, suffix}` is a strict subset of the compound rule's `{family, given, suffix, title}`, so every diff the peel rule admits the compound rule admits too. Implemented to check it: narrowing the peel regex to a `GLUED_HONORIFICS` alternation keeps all seven of the co-matched names whose 1.4.0 diff is exactly `{given, suffix}`, and swapping the two rules in the NARROWED ledger still reattributes all seven. So the pair keeps the arrangement the arc settled on, and that is the answer rather than a cost trade: it is narrow-first, the declaration-free default, so it appears in none of the eleven contests `order_contests` reports and owes no `precedes_narrower` block, and all seven names are pinned by name in `_CROSS_RULE_WINNERS` (23 corpus names reach both regexes; the seven are the ones #375's reorder mutation moves). The cost stands as a second reason and not the first: the narrowing would hand-copy more honorific vocabulary into the pattern and grow the `_HONORIFIC_SOURCES` sync-roster surface (mechanisms.md#CURATED-VOCABULARY-ALTERNATION's second half). Option 2's convention plus a check on the wide-first exceptions to it is what the arc took, and option 3 it REFUSED on a false premise (the finding above). One thing this does not settle, and the ledger comment on the peel rule overstates it: `fields` separates the two only for the UNION rows whose diff includes `family`; for those seven it is file order that decides. -- Scoping the check to contract-tier contests only (2026-09-02) — five of the eleven are contested only over radar names, so this would have cut the file's exemptions by nearly half. Declined because `_CROSS_RULE_WINNERS` already pins radar names — measured, most of the names it pins are radar-tier (22 of 33 today) — so the repo would be inconsistent with itself about whether a radar contest matters. Whether the radar-only CJK comma rules still earn their place after #488's demotion is a real question and is filed as #495; it is a question about those rules, not about the check. +- Scoping the check to contract-tier contests only (2026-09-02) — five of the eleven are contested only over radar names, so this would have cut the file's exemptions by nearly half. Declined because `_CROSS_RULE_WINNERS` already pins radar names — measured, MOST of the names it pins are radar-tier — so the repo would be inconsistent with itself about whether a radar contest matters. The majority is the whole of the argument and no digit belongs here: a count written at this spot was stale before the branch that wrote it merged, since the 2026-09-03 #497 entry below then deleted four of the roster's rows. RECOMPUTE: union the names in `_CROSS_RULE_WINNERS` (tests/v2/test_ledger_guards.py) and read each one's tier off `compare._CORPUS_TIERS`, via the first `corpus*.jsonl` file holding the name with contract files sorted first, as `main()` loads them. Whether the radar-only CJK comma rules still earn their place after #488's demotion is a real question and is filed as #495; it is a question about those rules, not about the check. - A shared name-population function for both callers (2026-09-02, the spec's own first sketch) — see the two-populations decision above. `main()` and the unit guard must read different populations, so one function would break `--corpus`; they agree by a docstring that names the function it mirrors instead. - Deleting the rules whose explained names are all radar (2026-09-02, [#495](https://github.com/derek73/python-nameparser/issues/495), closed as not planned) — filed off #488's demotion on the reasoning that a rule nothing can demand is a rule to remove, and refused on what deleting one actually does. Measured at baseline 1.4.0, 19 of the 72 explaining rules explain only radar names, between them 48 of the 352 intentional diffs — and that set is a SUPERSET of what the issue proposed deleting, not a different set. Three of the five rules #495 tabulated are in it — `fix(#296) a credential-only comma string reads a name and its postnominal`, `fix(cjk-comma-compound)`, and the roman-numeral `fix(suffix-routing)` — and they are exactly the three its option 2 named as having zero contract reach. The other two are outside it for the reason that disqualified them as candidates there too: they explain contract names, fourteen of them for `fix(cjk-glued-honorific-peel)` and `'Smith Jr.'` (corpus_rules.jsonl) for the jr `fix(suffix-routing)`. So the criterion reaches everything the issue wanted gone and more, and what refuses it is the fate of those 48 diffs. Measured by driving `classify()` over each one with its own rule removed: 25 report UNCLASSIFIED and 23 are claimed by ANOTHER rule (26 and 22 with all 19 deleted at once). For nearly half the population, deleting does not leave the diff watched-but-unexplained — it leaves it explained, by a rule whose comment is about something else. Nine of `fix(cjk-comma-compound)`'s eleven land on `fix(cjk-glued-honorific-peel)`; all seven of `fix(cjk-comma-honorific-peel)`'s land on `fix(cjk-comma-compound)`; `'Carod i'` lands on the roman-numeral `fix(suffix-routing)`, and `'Bob Jones, author'` on `fix(comma-precomma-family)`. That is the rule-broader-than-the-diff absorption this file already names twice — `fix(#424)` at decisions.md#H1, going on classifying a diff that shrank underneath it while the fields-only catch-all's heading stepped 14 -> 17 with nothing announcing either, and the fields-only arc's "a fields-only rule silently falsifies OTHER comments" — reached this time on purpose rather than by accident, which is not an improvement. Applied, it deletes `fix(#342)` and `fix(#397)`, the two NOT WANTED rules whose whole job is classifying a reading nobody wants so the gate stays green while the bug is open, and which carry the delete-when-fixed instruction the dormancy check enforces once the fix lands (the fields-only arc above). The cost there is NOT a red gate, and a first draft of this bullet said it was: both names sit in corpus_issues.jsonl, which is radar, and on a radar corpus an unmatched diff prints UNCLASSIFIED (radar) while the run keeps exiting 0 — UNEXPLAINED is the contract tier's word (tools/differential/README.md). Measured with both rules gone, the run reports `unexplained: 0; radar unclassified: 1` and exits 0. The one is `'Aishwarya Rai'`; `'Carod i'` is not reported at all, the numeral rule having taken it — so one of the two open bugs loses its marker not to silence but to a wrong label. Verified end to end against a ledger with `fix(#397)` removed: exit 0, 352 intentional diffs still, no UNCLASSIFIED, no EXPLAINED NOTHING, and `'Carod i'` printed under `fix(suffix-routing) a two-token name ending in a roman numeral`. It also deletes `feat(#273)`, `feat(#269) Arabic بن prefix chains onto family (non-Latin new-recognition)` and `fix(#360) ste moved into the never-given particles with mc` — records of new recognition behavior and of a vocabulary move — and the last two have to be named to the RULE rather than to the issue, as the ledger holds sibling rules under both strings. The `feat(#269)` sibling is the sharp one: `feat(#269) non-Latin titles/conjunctions recognized` does not qualify, explaining the contract-tier `'Хосе И Мария Сантос'` (corpus_rules.jsonl), so a delete by issue prefix takes both and sends that name UNEXPLAINED — measured, and a red gate. The criterion that does justify deleting a rule is that it explains NOTHING, which `dormant` already names and the dormancy check already enforces; a rule explaining eleven radar names is not dormant. What the radar tier is FOR answers the rest: decisions.md#cjk-comma-demotion puts it as giving the differential "somewhere to WATCH a name without promising it", and a rule explaining a radar name IS the watching. The five exemptions citing #495 are rewritten in the same diff rather than left standing: four of them put it as a LIVE question — the two under `fix(#296) a credential-only comma string`, the one on the `fix(cjk-comma-compound)` block, and the one on the `'田中さん II'` numeral contest — and the fifth, on the jr contest, called neither rule a candidate in the present tense. Each records the answer now and keeps every measurement it carried. One of those measurements was wrong and is corrected with them: `fix(comma-family) lone post-comma piece routes to suffix/title, not first` explains eight names of which FIVE are contract, not six — the six having shipped in #499. It is not a mis-map: six is what tiering by FILE MEMBERSHIP gives from a CORRECT `_CORPUS_TIERS`, and the single name it gets wrong is `'John Smith, Dr.'`, the third RECOMPUTE trap below. #496's decline was argued inside the `fix(cjk-comma-compound)` one and stands without it: the twin rule was refused because nothing can demand it, which is untouched by what #495 answered. RECOMPUTE: do not join names to files by hand. `_CORPUS_TIERS` maps FILES, the report prints names with no file, and 24 of the 352 explained names sit in files of BOTH tiers, so a hand join has to reproduce `main()`'s contract-first dedup AND its baseline-minimum skip to get them right. Let the run do it: call `compare.main()` unchanged under two wrappers — one around `_run_worker`, whose `entries` argument is the final list with `tier` and `file` already stamped on every entry, and one around `classify()`, recording (name, order, diff, issue) for each of its 352 calls — then reclassify offline from those records against `rules` minus whichever rule is under test, having first confirmed the wrapped run reproduced the shipped one (352 intentional, 0 unexplained, 0 radar unclassified). Parse no printed repr by stripping quotes: three of the 352 names do not survive it — `"'Smitty' Dr. Jones"`, `"'Smitty' Jones Jr."`, and the spelling of محمد بن سلمان wrapped in RIGHT-TO-LEFT MARKs — and dropping them silently is what produced the wrong count of 20 on the first pass, the first of the three being `fix(#410)`'s only contract name. `ast.literal_eval` reads the printout correctly. The ten-name truncation is NOT the hazard it looks: contract files load first, so any contract name a rule explains sits in the printed prefix, and tiering from the first ten alone gives the same 19. THIRD, and the only one of the three that changes an answer on this tree: a name's tier, in any claim scoped to a BASELINE, is a property of the ENTRY that run compared and not of the files the string appears in. The two part where a string sits in files of both tiers AND its contract copy carries a shape tag, because the dedup key is `(name, order)` — a shape declaring an order leaves TWO entries rather than one contract winner, and the baseline-minimum skip can then drop the contract one. Measured, 2 of those 24 names carry a shape tag and exactly 1 bites at 1.4.0: `'John Smith, Dr.'` is in corpus_issues.jsonl (radar, no shape) and corpus_shapes.jsonl (contract, shape 4, FAMILY_FIRST, min_baseline 2.0.0); both entries survive the dedup, which the 2.0.0 run shows by comparing both, and at 1.4.0 the shape-4 one is among the seven skipped, leaving the radar entry as the only comparison. `'Beethoven, Ludwig van'`'s shape 2 is order-None with a 1.4.0 minimum, so the dedup collapses it and contract wins at every baseline. File-membership tiering gets exactly 1 of the 352 wrong, and that one is what `fix(comma-family)`'s six-versus-five turned on; `entry["tier"]` gets it right for free. The radar-only count of 19 is the same under either tiering. To exercise a ledger edit, point `_allowlist_for` at a scratch file rather than copying the tree — the corpus glob resolves from `HERE` and `_check_tree` refuses a nameparser outside `REPO_ROOT`, so a copy elsewhere cannot run at all. @@ -795,7 +795,7 @@ The seventh ledger arc, and the one that makes a recorded INPUT measurable. The Decisions that landed: -- 2026-09-03 [#497](https://github.com/derek73/python-nameparser/issues/497) — the recorded shapes live in the TOOL, as `compare._RECORDED_DIFFS`, and the RUN verifies them; `_CROSS_RULE_WINNERS` keeps only which rule wins a contested name and reads the shape from there. There because it is the one place both halves exist: `main()` has measured every name's real diff by the time it asks, where `validate_rules` runs before any corpus is read and the unit suite installs no wheel at all — `tests/v2/test_differential.py`'s own header says "nothing here spawns `uv` or the network", and the tests that need a baseline fake `_run_worker` or fake `Popen` under it. State it that way and not as "every test monkeypatches `_run_worker`", which is false: measured 2026-09-03 on this branch, of 171 `def test_*` functions in that module 20 mention `_run_worker` at all and six reach the real one with `Popen` faked beneath them by `_fake_popen`. RECOMPUTE by walking the module's `def test_*` bodies for the name. That short form is what compare.py's `_RECORDED_DIFFS` provenance note SAID when this entry was written; `f70a0ad`, two commits later, withdrew it there, and the note now rejects the paraphrase rather than carrying it — so follow that pointer to the history and not to the text. The denominator has moved once already inside this arc: `f70a0ad`'s own message says 168, which is why the argument here is "most never mention it" and the digits carry a date. The hermeticity claim is unharmed and the loose paraphrase of it is what fails, which is this arc's subject arriving inside its own evidence. Read HERMETICITY as the reason and not speed. The rival placement is the unit suite, and the reason it is refused is that a check there would have to install a wheel — not that installing one is slow. The timing finding below retires every cost argument in this file and moves this decision not at all, which is the test such a placement argument has to pass and the reason to say out loud which ground carries it. `_CORPUS_FLOORS` is the precedent — tool-owned, corpus-shaped data that the guards read and monkeypatch — and this is the same direction. +- 2026-09-03 [#497](https://github.com/derek73/python-nameparser/issues/497) — the recorded shapes live in the TOOL, as `compare._RECORDED_DIFFS`, and the RUN verifies them; `_CROSS_RULE_WINNERS` keeps only which rule wins a contested name and reads the shape from there. There because it is the one place both halves exist: `main()` has measured every name's real diff by the time it asks, where `validate_rules` runs before any corpus is read and the unit suite installs no wheel at all — `tests/v2/test_differential.py`'s own header says "nothing here spawns `uv` or the network", and the tests that need a baseline fake `_run_worker` or fake `Popen` under it. State it that way and not as "every test monkeypatches `_run_worker`", which is false: measured 2026-09-03 on this branch, of 171 `def test_*` functions in that module 20 mention `_run_worker` at all and six reach the real one with `Popen` faked beneath them by `_fake_popen`. RECOMPUTE by walking the module's `def test_*` bodies for the name. That short form is what compare.py's `_RECORDED_DIFFS` provenance note SAID when this entry was written; `f70a0ad`, the very next commit, withdrew it there, and the note now rejects the paraphrase rather than carrying it — so follow that pointer to the history and not to the text. The denominator has moved once already inside this arc: `f70a0ad`'s own message says 168, which is why the argument here is "most never mention it" and the digits carry a date. The hermeticity claim is unharmed and the loose paraphrase of it is what fails, which is this arc's subject arriving inside its own evidence. Read HERMETICITY as the reason and not speed. The rival placement is the unit suite, and the reason it is refused is that a check there would have to install a wheel — not that installing one is slow. The timing finding below retires every cost argument in this file and moves this decision not at all, which is the test such a placement argument has to pass and the reason to say out loud which ground carries it. `_CORPUS_FLOORS` is the precedent — tool-owned, corpus-shaped data that the guards read and monkeypatch — and this is the same direction. - 2026-09-03 #497 — ONE roster, TWO checks, TWO PLACEMENTS — and, since `9360919`, a NOTE over the gap between them, recorded at the end of this entry. The asymmetry of the two checks is the decision rather than where the code happened to fit. The ABSENT-NAME half — a recorded row naming a string no corpus holds any more, which nothing else would ever report — refuses PRE-worker, beside `vacant`, because it reads the ledger and the loaded names and nothing the worker produces. It takes the PLACEMENT #382 chose for its own pre-worker refusal and NOT #382's reason for it, which was the cost claim below — `compare.py`'s comment at that placement is one of the sites the timing finding retires. The ground it stands on was rewritten on this branch: a refusal raised after the wheel is installed and the corpus compared prints below the run's own published `baseline:` header, for a comparison it will never report. Not "disowning a comparison it has just published" — measured print order is `baseline:` at the tell, then the comparison loop, then `corpus: … intentional diffs:`, so at that position the header has printed and no line of the comparison has. The ORDER is the argument, and that corrected wording is what `compare.py` now carries at all three of its placement comments. Recorded because it is the second decision the false number was under, and the two came apart differently — the declined check below kept its conclusion on a new argument, and here the ORDER was right all along while the reason for it was not. Only re-deriving a claim says which of its supports was doing the work; a placement that reads unchanged is not thereby unaffected. The MOVED-SHAPE half is the opposite case — it needs the diffs, so it stays after the comparison — and there it may not RAISE. It PRINTS and feeds the exit code, like `over_declared_rules`, its structural sibling and the other post-worker check on recorded data: a raise at that point lands MID-report and takes dormancy, OVER-DECLARED, UNEXPLAINED and the radar block with it, which compare.py measures on a narrowed 1.4.0 run beside the check. A stale roster row must never hide an unexplained diff, that being the gate's primary output. The two pre-worker refusals may raise because nothing has printed yet; this one cannot, and the difference is PRINT ORDER rather than severity. The two halves also read opposite name lists, which is `recorded_diff_mismatches`' docstring's to state and is not restated here. What IS decided here is the THIRD thing this roster produces, added by `9360919` once the gap between those two lists was measured and found not empty: an order-bearing entry the baseline cannot honor sits in a corpus file and outside the compared set at once, so a row on such a name is checked by neither half and reported by neither. `main()` prints a `NOT CHECKED` note over it. A NOTE and not a refusal — the docstring's argument against refusing off the post-skip list stands, and it is an argument against REFUSING and not against SAYING, which are two decisions of which only the first was ever made. Informational, outside the exit code, and gated on neither `full_corpus` nor `args.corpus`, the intersection narrowing itself. Measured 2026-09-03 at 1.4.0 — both the only baseline where the skip fires and the only ledger with rows — the window is three names, none of which carries a roster row today, so the note prints on no run yet. The measurement that motivated it put a deliberately wrong shape on `'de la Cruz née Vega'` over the full corpus at 1.4.0: before `9360919` that exited 0 in 375 stdout lines naming the name in none of them, and it now exits 0 in 378 with the `NOT CHECKED` note naming it — re-measured 2026-09-03 by corrupting the row in memory around `main()`, which leaves the worktree alone. - 2026-09-03 #497 — a SHAPE may be pinned where a COUNT may not, and that line is the whole of what keeps this roster affordable. A diff shape is one name under two parser versions, so adding a name to a corpus cannot move it; an explained-name count moves on every corpus edit. Shapes drift only when the PARSER changes those names — so the drift and the reason to re-read the roster are the same event, which is exactly what a recorded number usually fails to be. That is why #452's churn objection to `explains = N` (the fields-only arc above, 2026-08-28) does not reach this roster, and why pinning shapes here does not weaken it: the four ledgers carry 201 rules today, one `_CORPUS_CLAIMS` entry each, every one liable to move whenever a corpus does, against 31 recorded shapes that move only with the parser. Read the two CAUSES and not the two totals — the argument is that one set is coupled to corpus edits and the other is not, and it survives either number moving. RECOMPUTE: `sum(len(v) for v in _CORPUS_CLAIMS.values())` in tests/v2/test_ledger_guards.py, and `sum(len(v) for v in _RECORDED_DIFFS.values())` in compare.py. - 2026-09-03 #497 — four of the 35 rows the roster carried were DELETED rather than corrected. Two names across the two 2.x ledgers, so two rows each, and one of the two NAMES had a correctable shape — count rows or names deliberately, since the provenance note that holds the measurements speaks of the name. Deleted because neither name pinned a CONTEST, which is what the roster is for: at the shape each really produces exactly one rule admits it, so `classify()` was never adjudicating anything. `'Nguyen, Van'` diffs at no baseline at all, so no run ever asks `classify()` about it; `'Jane née and Jones Smith'` is a malformed harvest from a radar corpus whose right parse nobody can state, so a pin on it defends no boundary anyone would argue for — which is why the correctable one went too. compare.py's provenance note holds the per-name, per-baseline measurements behind both. The deletion emptied both 2.x sections, which stay present as explicit empty mappings — the completeness guards require it, and it is what makes emptiness sayable as a POSITION: a row is owed when someone argues a boundary, not before. What the emptiness is NOT — that these ledgers hold no contested diff — is measured and stated beside those sections, and whether the position should change now that those contests are measured rather than merely unexamined is [#501](https://github.com/derek73/python-nameparser/issues/501), open. Recorded with the deletion because it is the half a later sweep will get wrong: "only one rule admits it" is NOT on its own grounds to delete a row. A substantial minority of the surviving 1.4.0 rows are in that position and stay — the count and its recompute are beside them in test_ledger_guards.py, where six of the thirteen say so in their own comments and the other seven do not, which is why the count is recomputed rather than read off the roster — because the shapes they pin are shapes runs actually make, so a widened `fields` or a moved rule hands the name over and the test says so. Counting admitters is not the test at all: 11 of those 13 route to a DIFFERENT rule under some other shape, and mechanisms.md#RECORDED-ROSTERS now carries that measurement in place of the criterion it stated first. The four deleted rows could not do that work at any edit, which is the difference. diff --git a/tools/differential/README.md b/tools/differential/README.md index 380b6f23..23c8eb11 100644 --- a/tools/differential/README.md +++ b/tools/differential/README.md @@ -243,13 +243,20 @@ stays UNEXPLAINED and fails the run even when the name itself sits in a radar file. A recorded diff shape (`MOVED SHAPE`, below) is the second thing that -outranks the tier, and the larger of the two: it feeds the exit code -without asking what tier the name sits in. Measured 2026-09-03, 21 of -the 31 rows in `_RECORDED_DIFFS['expected_since_1.4.0.toml']` sit on -radar-tier names, so radar PARSER DRIFT can fail the run wherever -someone has pinned a shape (RECOMPUTE: for each row, take the tier of -the first corpus file holding the name with contract files sorted -first, as `main()` loads them). That is a pin doing what pins do -- +outranks the tier: it feeds the exit code without asking what tier the +name sits in. Measured 2026-09-03, MOST of the rows in +`_RECORDED_DIFFS['expected_since_1.4.0.toml']` sit on radar-tier names +-- 21 of 31 -- so radar PARSER DRIFT can fail the run wherever someone +has pinned a shape (RECOMPUTE: for each row, take the tier of the first +corpus file holding the name with contract files sorted first, as +`main()` loads them). Which of the two outranks the tier more WIDELY is +deliberately not claimed here, because the answer inverts with the unit +and neither unit is the point: the exclusions number two against 31 +rows, while the two `[[never]]` patterns reach 60 corpus names between +them, 37 of those radar-tier -- more radar names than the roster pins +(measured 2026-09-03 by matching each `name_regex` against every name +in the `corpus*.jsonl` glob, tiers read the same way). That is a pin +doing what pins do -- someone wrote the row by hand and it says what the name does -- but it is not what "a radar diff can never fail the run" leads a reader to expect. Read the tier rule as being about UNMATCHED diffs, which is @@ -758,7 +765,7 @@ So the shapes live in `_RECORDED_DIFFS` in `compare.py`, keyed per ledger (a str - **A row naming a name no corpus holds** -- refused, before the worker runs, on a FULL run only. Nothing measures such a row, so it agrees with itself forever, which is the defect above in a second form. The repair is not mechanical: ask first whether the name left deliberately (`git log -S'' -- tools/differential/corpus*.jsonl`), then either delete the row and its `_CROSS_RULE_WINNERS` partner, or restore the name to the corpus that lost it. Under `--corpus` this prints nothing, since a narrowed run legitimately holds almost none of the roster. - **`NOT CHECKED`** -- a NOTE rather than a finding, printed after the comparison and deliberately outside the exit code (#497, `9360919`). It names recorded rows whose corpus entry THIS BASELINE SKIPPED: an order-bearing entry a baseline with no `Policy` cannot honor is dropped before the comparison runs, so there is no measured diff to check the row against. Such a row falls between the two checks above -- `MOVED SHAPE` skips a name it did not compare, and the departed-name refusal passes it because the skip takes a name out of the RUN and out of no file -- and without the note it is checked by neither and reported by neither. So the note says which rows the shape report is silent about and claims nothing else. Do NOT delete a row over it: the name is in a corpus this run read. Re-run at a baseline that can honor its order to check it. Measured at 1.4.0, which is both the only baseline where the skip fires and the only ledger with rows, the window is `de Mesnil Jean, Dr.`, `de la Cruz Juan Carlos, Dr.` and `de la Cruz née Vega` -- none carrying a roster row today, so the note prints on no run yet. "Do not refuse" and "say nothing" are two decisions, and only the first was ever argued. -The two CHECKS read opposite name lists, and swapping them is the live hazard. "Was this name compared?" is about the RUN, so the shape check reads the list AFTER the baseline-minimum shape skip. "Does any corpus still hold this name?" is about the FILES, and the skip empties no file, so the departed-name check reads the list BEFORE it. The note reads both, being exactly the difference: the roster intersected with the pre-skip list, minus the post-skip one. It is also the one of the three that does not consult `full_corpus` -- that intersection already narrows itself under `--corpus`. `recorded_diff_mismatches`' docstring in `compare.py` carries the measurement and the recompute. +The two CHECKS read opposite name lists, and swapping them is the live hazard. "Was this name compared?" is about the RUN, so the shape check reads the list AFTER the baseline-minimum shape skip. "Does any corpus still hold this name?" is about the FILES, and the skip empties no file, so the departed-name check reads the list BEFORE it. The note reads both, being exactly the difference: the roster intersected with the pre-skip list, minus the post-skip one. TWO of the three consult `full_corpus` not at all, and the departed-name refusal is the only one that reads it: the note's intersection already narrows itself under `--corpus`, and `MOVED SHAPE` never asks the flag either -- it is silent under `--corpus` only about rows whose name this run did not compare, so a narrowed run holding a pinned name still reports and still FAILS. Measured 2026-09-03: `--corpus corpus_issues.jsonl` at 1.4.0 with a corrupted shape on `Carod i` prints `MOVED SHAPE` and exits 1 (RECOMPUTE by corrupting that row in `compare._RECORDED_DIFFS` in memory around `main()`). Do not read the departed-name suppression across to it -- a `MOVED SHAPE` from a narrowed run is a real finding, and a `full_corpus` gate added here would be a behavior change and not a restoration. `compare.py`'s comment at the note says the note parts "from the two checks that do read that flag", counting `vacant` and `gone` -- its own frame, and a different pair from the three listed here. `recorded_diff_mismatches`' docstring in `compare.py` carries the measurement and the recompute for the two lists. ### Shapes that must never be explained (`[[never]]`) diff --git a/tools/differential/compare.py b/tools/differential/compare.py index 49cf2de3..050fa061 100644 --- a/tools/differential/compare.py +++ b/tools/differential/compare.py @@ -2492,13 +2492,18 @@ def _tree_parse(name: str, order: str | None) -> object: # entry the baseline cannot honor. Such a row falls between both # checks: recorded_diff_mismatches skips a name outside `compared`, # and the `gone` refusal passes it because the skip takes a name out - # of the RUN and out of no file. Measured 2026-09-03: a deliberately - # wrong shape on 'de la Cruz née Vega' over the FULL corpus at 1.4.0 - # left exit 0 and 375 stdout lines with no mention of the name -- - # and the window is co-located with the only populated section, - # since 1.4.0 is both where the 31 rows live and the only baseline - # where the skip fires (every order-bearing shape's min_baseline is - # 2.0.0). + # of the RUN and out of no file. Measured 2026-09-03 by putting a + # deliberately wrong shape on 'de la Cruz née Vega' over the FULL + # corpus at 1.4.0, and BOTH sides of the note are stated because + # only the pair says what the note bought: before `9360919` that + # run exited 0 in 375 stdout lines naming the name in none of them, + # and it now exits 0 in 378, the three added lines being the NOT + # CHECKED note below naming it. Re-measure by corrupting the row in + # _RECORDED_DIFFS in memory around main(), which leaves the + # worktree alone. The window is co-located with the only populated + # section, since 1.4.0 is both where the roster's rows live and the + # only baseline where the skip fires (every order-bearing shape's + # min_baseline is 2.0.0). # # A NOTE. Not a refusal, not in the exit code, and the distinction # is the whole of it: recorded_diff_mismatches' docstring argues From b3f03cf148a7a3b570e0a6fd86b762a255685ee1 Mon Sep 17 00:00:00 2001 From: Derek Gulbranson Date: Thu, 3 Sep 2026 13:03:12 -0700 Subject: [PATCH 23/23] docs(agents): the unreachable ledger gets an owner (#503) The note recorded that the checklist runs three commands against four rule-bearing ledgers and changed nothing a releaser executes, so the gap had no owner and each sweep re-found it. #503 is where the fix gets decided; the note now cites it and says what it weighs. Two things the note did not carry. Which checks 2.1.0 actually misses: dormancy and over-declaration are live gaps, while the shape check is latent only because that ledger's _RECORDED_DIFFS section is empty -- and #501, open, measured six contested diffs across the two 2.x ledgers with no pin, one of them at 2.1.0, so the first row to land there is checked by nothing a releaser runs. And that cost is not the obstacle, measured rather than assumed: 0.43s at 1.4.0, 0.56s at 2.2.0, 1.97s for all four back to back, with the recompute in decisions.md. The three options stay #503's to choose between. Option 2's local precedent is named because it is already settled elsewhere in the tool: compare.py's --corpus default is deliberately a glob, on the ground that "one that has to be asked for by name is one that stops being run", which is this note's failure one argument up. Co-Authored-By: Claude Opus 5 --- AGENTS.md | 30 +++++++++++++++++++++++++----- 1 file changed, 25 insertions(+), 5 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index af2eaadc..c672e1eb 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -106,12 +106,32 @@ uv run sphinx-build -b html docs dist/docs # uv run python tools/differential/compare.py --baseline 1.4.0 # the v1 compat contract # uv run python tools/differential/compare.py # the previous minor # uv run python tools/differential/compare.py --baseline 2.0.0 # 2.0.0's ledger has rules too -# NOTE (2026-09-03): that is THREE commands against FOUR ledgers with -# rules -- `--baseline 2.1.0` is the one no line above reaches, so the -# "EVERY baseline" instruction is not what this list does. Since #497 -# a third check is per-ledger as well: the recorded-shape one reads -# _RECORDED_DIFFS[ledger.name], so the ledger nobody runs gets no +# NOTE (2026-09-03, #503): that is THREE commands against FOUR ledgers +# with rules -- `--baseline 2.1.0` is the one no line above reaches, so +# the "EVERY baseline" instruction is not what this list does. Since +# #497 a third check is per-ledger as well: the recorded-shape one +# reads _RECORDED_DIFFS[ledger.name], so the ledger nobody runs gets no # dormancy check, no over-declaration check, and no shape check. +# Two of those three are gaps TODAY; the shape check is latent only +# because expected_since_2.1.0.toml's _RECORDED_DIFFS section is empty +# -- and whether the 2.x sections stay empty is #501, which measured +# six contested diffs across the two 2.x ledgers with no pin, one of +# them at 2.1.0 (`MD, PHD`). The first 2.1.0 row to land would be +# checked by nothing a releaser runs. +# Cost is not the obstacle and no fix may be argued from it: measured +# 2026-09-03, a whole run is under a second at every baseline and all +# four back to back are about two seconds (0.43s at 1.4.0, 0.56s at +# 2.2.0, 1.97s for the four; decisions.md's 2026-09-03 #497 timing +# finding carries the recompute and retires every cost argument in +# this repo's differential prose). +# #503 weighs three fixes and none of them is this note's to make: +# add the fourth command; have the checklist DERIVE the list from the +# expected_since_*.toml glob rather than enumerate it; or say that +# 2.1.0 is deliberately outside the audit and fix the "EVERY baseline" +# line to match. The local precedent for the second is compare.py's +# own `--corpus` default, a glob and deliberately so -- "one that has +# to be asked for by name is one that stops being run" -- which is the +# same failure this note records, one argument up. # Redirect to a file rather than piping — under zsh a pipe replaces the exit # code with the pipe's, so a failing run reads as a passing one. The # classified summary it prints is the source for the release notes' behavior