You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
docs: the review round's prose corrections, and every tagged row names its slot
Five prose inaccuracies from the #487 review, two of them mine and
three pre-existing but falsified or moved by this PR, plus the slot
annotations that make the matrix readable from the rows themselves.
No shape tag, corpus entry, floor, claim or ledger rule changes here --
notes are not in build_shapes_corpus.py's projection, so regenerating
after this commit rewrites corpus_shapes.jsonl byte for byte.
decisions.md, the #486 bullet: "None of the 30 needed a ledger rule"
was false on a plain reading -- the seven shape 4/5 entries ARE
classified by ledger rules today, and scoping exactly those rules is
what the `orders` bullet three lines above is about. It now says none
of the 17 names #486 tagged or authored needed one, and says out loud
that this is not a claim about all 30 entries.
Two same-dated counts this PR falsified rather than swept -- the
dated-snapshot defence fails when the supersession lands the same day,
so accuracy was the only option. decisions.md's #468 bullet said "a
comparison of 1113 names today" (now 1120, with the earlier figure kept
as history); mechanisms.md's LEDGER-RULE-SEPARATION entry said `[a-z]`
"reaches 963 of 1113 comparisons". Re-measured here with the same
(name, order)-dedup the gate uses, independently of the review's
recount and agreeing with it: 970 of 1120 (` ` reaches 1057). Both keep
their 2026-09-01 date.
expected_since_1.4.0.toml, two of mine from bebb9d1: the capture-count
history read "(46, then 51, then 55)" and git archaeology says the
comment's copy went 34 -> 46 -> 51 and never carried 55 -- 55 was the
value the pin already held while the copy still said 51, and that lag
is now stated as the argument for deleting the copy rather than
correcting it again. And "gave that string a case row and a shape tag"
overstated the change: `nickname_quotes` predates this work by a long
way and only the TAG is new. While there, the replacement illustration
is named as the hazard it is -- 'John (Jack) Kennedy' is itself
`nickname_parens`'s text, one tag from falsifying the sentence the same
way, and no example is left that is neither a corpus name nor a case
row.
Slot auditability: every shape 1-3 row's notes now name the variation
slot it witnesses, in the row's own voice, matching what the three
authored rows and the two annotated rows already did. 18 rows gained a
clause; all 23 shape 1-3 rows have one now, so the matrix reconstructs
from the rows around `grep -n 'shape=' tests/v2/cases.py` without
consulting a commit message.
Two inaccuracies in f203017's own message cannot be fixed without
rebasing a pushed branch, so they are recorded here instead. Its
promotion arithmetic reads "Ten of the fourteen names were already in a
corpus ... or is a no-op for the three the rules corpus already held",
which double-counts: the correct split of the 14 tagged names is 8
promoted from radar-only to contract, 2 already contract through the
rules corpus, and 4 in no corpus at all. And the exclusion-example slip
that message repeats ('John "Jack" Kennedy' as the entry's second
examples string, promoted out of the radar corpora) was corrected in
bebb9d1: it is the first, and it was in no corpus before.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Copy file name to clipboardExpand all lines: docs/design/decisions.md
+2-2Lines changed: 2 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -694,12 +694,12 @@ The fourth ledger arc, and the first to change what the gate is FOR rather than
694
694
Decisions that landed:
695
695
696
696
- 2026-09-01 #468 — every corpus carries a TIER, and only the CONTRACT tier can fail the run. `corpus.jsonl` (scraped from v1's test banks) and `corpus_issues.jsonl` (harvested from the tracker) became RADAR: their diffs still classify against the ledger, so release notes keep their grouping, but an unmatched one prints under `UNCLASSIFIED (radar)` and the run still exits 0. `corpus_cjk.jsonl`, `corpus_rules.jsonl` and `corpus_shapes.jsonl` keep the full discipline. The reason is a VOCABULARY gap and not a volume one: the gate demanded a rule-precision explanation for names nobody chose, and the honest explanation for many of them was "this string exercises nothing" — which no ledger key can say: a rule means "this diff is intended and here is why", an exclusion means "this shape must never be explained", and neither is the sentence wanted. A rule invented to say it would be the #372 shape by construction, since the only thing it could narrow on is which file the name happens to sit in. `_CORPUS_TIERS` in compare.py is the roster and is fail-closed like `_CORPUS_FLOORS`: a corpus added without a tier is a hard error, so the split forces a decision when a corpus arrives rather than defaulting one.
697
-
- 2026-09-01 #468 — nothing is DELETED to quiet the gate; promotion is the answer to "then how does a radar name ever bind again". A name enters the contract by being CHOSEN — a tests/v2/cases.py row plus a shape tag, which puts it in `corpus_shapes.jsonl` at the next regeneration. Deleting the string was the alternative and is worse in both directions: it loses the radar signal, and it makes the gate's own population editable by whoever finds the gate inconvenient, which is the one edit a differential gate must not make cheap. A meaningless radar string costs one parse and one report line against a comparison of 1113 names today.
697
+
- 2026-09-01 #468 — nothing is DELETED to quiet the gate; promotion is the answer to "then how does a radar name ever bind again". A name enters the contract by being CHOSEN — a tests/v2/cases.py row plus a shape tag, which puts it in `corpus_shapes.jsonl` at the next regeneration. Deleting the string was the alternative and is worse in both directions: it loses the radar signal, and it makes the gate's own population editable by whoever finds the gate inconvenient, which is the one edit a differential gate must not make cheap. A meaningless radar string costs one parse and one report line against a comparison of 1120 names today (1113 when this bullet was written, hours earlier the same day; #486's bullet below carries the arithmetic).
698
698
- 2026-09-01 (decided in review) — a `[[never]]` exclusion OUTRANKS the tier: a diff it refuses reports UNEXPLAINED and exits 1 on either tier. Not a carve-out but the tier's own principle applied — the split governs names nobody looked at, and an exclusion is chosen exactly as a rule is, someone having written its `why` and its `examples`. It is also what the shipped ledgers already promise, and the measurement is what settled it: the 1.4.0 ledger's trailing-`Ph. D.` exclusion, whose `why` says "a diff here is a regression", reaches exactly three corpus names (`John Smith Ph. D.`, `John Smith, Ph. D.`, `John Smith, Ph.D.`) and every one of them lives in `corpus.jsonl` or `corpus_issues.jsonl` alone. Reading the tier as outranking the exclusion would have retired that entry outright, silently, in the commit that split the corpora. Recompute by matching each `[[never]]` entry's `name_regex` over the corpus files and reading which files the matches come from.
699
699
- 2026-09-01 (decided in review) — a ledger RULE may narrow by comparison order, through a new `orders` key; it is optional, and its absence is the order-blind behavior every rule written before shape-tagged entries has. Forced by the first order-scoped rule anyone wrote: `feat(#395)`'s fold on `de la Cruz Juan Carlos` moves `{family, given, middle}` under the two family-first orders, and the most plausible regression of that very change — the fold leaking into the DEFAULT order — moves those same three roles on that same string, which `corpus_rules.jsonl` carries as a bare string and therefore compares. Order-blind, the rule describing the feature would have absorbed the regression of the feature and called it intentional: #372's failure mode reached from a new direction, and on the one name best placed to hide it. The legal set is BORROWED from shapes.py rather than hand-copied (`_legal_orders`), the same call build_cjk_corpus.py makes for the script table — an order no shape declares is an order no comparison runs under, so a rule scoped to a typo could only ever be dormant. Amended 2026-09-01 (decided in review): the legal set gains a `DEFAULT` SENTINEL, and the two legacy rules that needed it were scoped. Found when the reviewer traced the absorption running the OTHER way from the one above — the order-blind `fix(#399) a maiden marker bounds the particle chain that swallowed it` rule reaches `de la Cruz née Vega`, which `corpus_shapes.jsonl` compares under both family-first orders, and it sorts AHEAD of the two rules written for that name, so a family-first-only `{family, maiden}` regression there would have classified as intentional. The fix a rule wants is `orders`, and no rule could say "default order only": the members are constant NAMES, the default order is the absence of one, and TOML has no null inside an array. So `"DEFAULT"` names it. `fix(#399) …that swallowed it` took `["DEFAULT"]` in both 2.x ledgers and `fix(#296) … 'John Smith, Dr.' keeps its split and its title` took `["DEFAULT", "FAMILY_FIRST"]`, those being the comparisons each one's diffs actually come from. Order-blind rules stay legal — every ledger written before shape tags is full of them — so the absorption is made VISIBLE rather than illegal: a run prints an `ORDER-BLIND` block for each (issue, name, order) where a rule with no `orders` key explained an order-bearing diff, informational and outside the exit code. Recompute by running the gate at each of the four baselines and reading that block; it is empty at all four today.
700
700
- 2026-09-01 (decided in review) — EXCLUSIONS stay order-blind, and `validate_exclusions` rejects an `orders` key as unknown. The dormancy arc's monotone-refusal reasoning is the whole argument: an exclusion only ever removes a name from classification, so the widest thing an over-wide one can do is make a name report UNEXPLAINED and fail the run. There is no silent direction to protect against, and the narrowing a rule needs is the one an exclusion must not have — "never explain this, but only under FAMILY_FIRST" is a promise with a hole in it.
701
701
- 2026-09-01 #469 — "some names are only supported under one order" is answered by the SHAPE rather than by a ledger exception. shapes.py records, per input shape, the `name_order` the shape is an input FOR and the oldest baseline whose worker can honor that order; `compare.py` parses a tagged entry under that order on both sides, compares it on the v2 surface alone (the facade is the v1-compat surface, and a family-first name is not a v1 contract), and leaves out an entry whose run predates its `min_baseline` — reported as `skipped N names tagged shape(s) [...]` and counted in the `corpora:` line, so a shrunken comparison is never silent. Structural, and that is the point: nothing has to be written into any ledger to say a family-first name is not a default-order promise. Today's arithmetic, all of it recomputable from the checked-in files and one gate run: 13 shape entries, of which 4 dedupe into names already compared under the default order, for 1113 comparisons; shapes 4 and 5 declare `min_baseline` 2.0.0, so 7 entries are skipped at `--baseline 1.4.0` and 1106 names compare there.
702
-
- 2026-09-01 #486 (later the same day, so read this bullet as superseding the arithmetic in the one above rather than the two figures disagreeing) — the shape 1-3 variation matrix filled the given-first half of the inventory, which the feature work that authored shapes 4 and 5 had left at whatever the pipeline PRs happened to tag. 14 existing case rows gained a tag and 3 rows were authored for slots no row instantiated, taking `corpus_shapes.jsonl` from 13 entries to 30 and shapes 1/2/3 from 1/4/1 names to 9/11/3. The arithmetic moves with it and the recipe is unchanged — run `uv run python tools/differential/compare.py` and read its `corpora:` and `corpus:` lines, adding `--baseline 1.4.0` for the skip figures: 30 shape entries, of which 14 dedupe into names already compared under the default order, for 1120 comparisons; shapes 4 and 5 still declare `min_baseline` 2.0.0, so 7 entries are skipped at `--baseline 1.4.0` and 1113 names compare there. The dedupe count is the half worth reading, because it is what the tier promotion looks like from this side: 8 of the newly tagged names were in a RADAR corpus only, so the tag moved them into the contract tier without adding a comparison, and 2 more were already contract through the rules corpus. None of the 30 needed a ledger rule — the seven names that were in no corpus at all diff at no baseline, and intentional stayed 226/205/113/0 across the change.
702
+
- 2026-09-01 #486 (later the same day, so read this bullet as superseding the arithmetic in the one above rather than the two figures disagreeing) — the shape 1-3 variation matrix filled the given-first half of the inventory, which the feature work that authored shapes 4 and 5 had left at whatever the pipeline PRs happened to tag. 14 existing case rows gained a tag and 3 rows were authored for slots no row instantiated, taking `corpus_shapes.jsonl` from 13 entries to 30 and shapes 1/2/3 from 1/4/1 names to 9/11/3. The arithmetic moves with it and the recipe is unchanged — run `uv run python tools/differential/compare.py` and read its `corpora:` and `corpus:` lines, adding `--baseline 1.4.0` for the skip figures: 30 shape entries, of which 14 dedupe into names already compared under the default order, for 1120 comparisons; shapes 4 and 5 still declare `min_baseline` 2.0.0, so 7 entries are skipped at `--baseline 1.4.0` and 1113 names compare there. The dedupe count is the half worth reading, because it is what the tier promotion looks like from this side: 8 of the newly tagged names were in a RADAR corpus only, so the tag moved them into the contract tier without adding a comparison, and 2 more were already contract through the rules corpus. None of the 17 names #486 tagged or authored needed a ledger rule — the seven of them that were in no corpus at all diff at no baseline, and intentional stayed 226/205/113/0 across the change. Not a claim about all 30 entries: the seven shape 4/5 entries are classified by ledger rules today, and scoping exactly those rules is what the `orders` bullet three above is about.
703
703
- 2026-09-01 #469 — the CJK arrangement is deliberately OUTSIDE the shape inventory. Whether an unspaced CJK name is a third family-first shape is the open question on #469, and `corpus_cjk.jsonl` — contract-tier already, generated from the case table already — covers that ground meanwhile. Tagging it in would have to ANSWER the question first, since a shape row cannot be written without a `name_order` and a `min_baseline` for it, and the inventory is a table of arrangements rather than where that argument belongs.
Copy file name to clipboardExpand all lines: docs/design/mechanisms.md
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -77,7 +77,7 @@ Problem shape. A guard needs to know what the answer WAS, so it can detect the a
77
77
78
78
## LEDGER-RULE-SEPARATION — file order decides, fields narrow by subset
79
79
80
-
Problem shape. Two differential-ledger rules claim overlapping names. Contract statement. Every ledger rule must carry a `name_regex` — since #451 `validate_rules` REJECTS a rule with `fields` and no `name_regex`, and one with neither was already rejected — so every rule sits in ONE tier, the sort is stable, and FILE ORDER decides every contest: the narrower rule must be written first. `fields` narrows a rule by subset; it does not separate rules by sorting. Narrowing by subset is not the whole contract: since #452 a rule's `fields` must EQUAL the union of the diffs it explains, and `compare.py` reports OVER-DECLARED and exits non-zero otherwise — a declared role no diff moves is not inert, it lets the rule keep claiming a name whose diff SHRINKS into the excess (decisions.md#differential-ledger). Since #468 there is a THIRD narrowing key: `orders` admits only the comparison orders it lists, the key being optional and its absence the order-blind reading every earlier rule has — a name compared under two orders can move the same roles for opposite reasons, so a rule describing an order-scoped fold would otherwise absorb that fold leaking into the default order (decisions.md#differential-ledger carries the worked case, and the legal set is borrowed from tools/differential/shapes.py rather than copied — plus one member no shape can declare, the `DEFAULT` sentinel naming the comparison run under no declared order, TOML having no null to put in an array). Exclusions take no `orders` and stay order-blind, deliberately. The ban ends the SHAPE and not the property it enabled: a required `name_regex` bounds nothing by itself, since the only width check is the sentinel probe — measured, `[a-z]` validates and reaches 963 of 1113 comparisons (2026-09-01). What changed is that such a rule now carries a `_CORPUS_CLAIMS` reach and digest, so its breadth is visible once at recording time rather than never (#452). The two-tier sort in `_sorted_rules` is KEPT although the ban makes it the identity on every ledger that loads (four ledgers load today, measured 2026-09-01; the open cycle's carries no rules, so the identity holds trivially there): it is the defence for a reader that does not call `validate_rules` first — a future tool, a REPL, a test fixture — and its docstring in tools/differential/compare.py says so. How it works. Detail is owned by tools/differential/README.md. The file-order clause is measured, not theoretical: in the 1.4 ledger the comma-honorific-peel rule's fields are a strict subset of the comma-compound rule's, both carry a name_regex, and a pure reorder reattributes seven names — caught by _CROSS_RULE_WINNERS and by nothing else in the suite (#375's mutation). Whether that pair should be separated by a predicate instead of by order is
80
+
Problem shape. Two differential-ledger rules claim overlapping names. Contract statement. Every ledger rule must carry a `name_regex` — since #451 `validate_rules` REJECTS a rule with `fields` and no `name_regex`, and one with neither was already rejected — so every rule sits in ONE tier, the sort is stable, and FILE ORDER decides every contest: the narrower rule must be written first. `fields` narrows a rule by subset; it does not separate rules by sorting. Narrowing by subset is not the whole contract: since #452 a rule's `fields` must EQUAL the union of the diffs it explains, and `compare.py` reports OVER-DECLARED and exits non-zero otherwise — a declared role no diff moves is not inert, it lets the rule keep claiming a name whose diff SHRINKS into the excess (decisions.md#differential-ledger). Since #468 there is a THIRD narrowing key: `orders` admits only the comparison orders it lists, the key being optional and its absence the order-blind reading every earlier rule has — a name compared under two orders can move the same roles for opposite reasons, so a rule describing an order-scoped fold would otherwise absorb that fold leaking into the default order (decisions.md#differential-ledger carries the worked case, and the legal set is borrowed from tools/differential/shapes.py rather than copied — plus one member no shape can declare, the `DEFAULT` sentinel naming the comparison run under no declared order, TOML having no null to put in an array). Exclusions take no `orders` and stay order-blind, deliberately. The ban ends the SHAPE and not the property it enabled: a required `name_regex` bounds nothing by itself, since the only width check is the sentinel probe — measured, `[a-z]` validates and reaches 970 of 1120 comparisons (2026-09-01, re-measured the same day after #486 widened the shapes corpus; it read 963 of 1113 before that). What changed is that such a rule now carries a `_CORPUS_CLAIMS` reach and digest, so its breadth is visible once at recording time rather than never (#452). The two-tier sort in `_sorted_rules` is KEPT although the ban makes it the identity on every ledger that loads (four ledgers load today, measured 2026-09-01; the open cycle's carries no rules, so the identity holds trivially there): it is the defence for a reader that does not call `validate_rules` first — a future tool, a REPL, a test fixture — and its docstring in tools/differential/compare.py says so. How it works. Detail is owned by tools/differential/README.md. The file-order clause is measured, not theoretical: in the 1.4 ledger the comma-honorific-peel rule's fields are a strict subset of the comma-compound rule's, both carry a name_regex, and a pure reorder reattributes seven names — caught by _CROSS_RULE_WINNERS and by nothing else in the suite (#375's mutation). Whether that pair should be separated by a predicate instead of by order is
81
81
[#382](https://github.com/derek73/python-nameparser/issues/382). The old #271/#272
82
82
slug taboo is RETIRED (#333): the canonical-rule selector that keyed on those substrings is deliberately deleted — rule authors are free to use them in compound slugs — and the surviving rosters select on their own explicit keys (_HONORIFIC_SOURCES and _LATIN_ALTERNATION_SOURCES by named issue strings, _SPAN_BEARING_RULES by exact leading fix(...) tag). Lives in. tools/differential/compare.py, the expected_since_*.toml ledgers. Reach for it when. A ledger rule's behavior seems to depend on where it sits in the file — it does, and the reorder mutation is the test (run twice in #375; it fails _CROSS_RULE_WINNERS). History: #372 (closed) measured the then-existing fields-only rule owning 1639 of 5257 name×field pairs as filed (2026-08-10); #375/#376 then cut its classifier-of-record share sharply, and the residual pair ownership was read as the last-resort tier working as designed rather than a defect — until #451 retired the shape outright (decisions.md#differential-ledger). #372's two proposed mechanical checks were DECLINED with measurements (see decisions.md#differential-ledger), not left open.
0 commit comments