diff --git a/AGENTS.md b/AGENTS.md index 9a1ddbbc..44985c9a 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -26,7 +26,7 @@ Three committed contributor docs carry the parser's normative rules and their re **Counting claims.** A bare count in prose is either an assertion or a liability, keyed by who observes its staleness: asserted counts (a test holds the number) fail CI at change time — the useful kind; dated snapshots ("51 sites at spec time") cannot go stale; standing present-tense prose counts are the forbidden class — promote to an assertion, add a date, or state the invariant and let a test count. After changing how many times something runs, sweep for counts, not for the thing's name. -**Release-log claims.** Quantified or universal behavior claims in release bullets must come from the differential gate's classified summary, be verified against rules.md examples, or -- for a view the gate cannot see -- carry a recompute recipe stored with the design entry the bullet cites; never write one from memory. The classified summary covers the CONTRACT tier plus whatever radar diffs a rule classifies; a radar corpus's unmatched diffs are listed under UNCLASSIFIED (radar) and are not in it, so a claim quantified from the summary alone is silent about them. The gate compares the seven role fields and `_ambiguities`, so `capitalized()`, `initials()` and any future render view are invisible to it (decisions.md#R4, #R3) and the first two sources cannot reach a claim about one: a gate run is byte-identical across the change, and an example line witnesses an output without counting anything. A recipe names the corpus files, the policy sweep, and -- the part that is easy to omit and fatal -- THE COMPARATOR, which must be something the shipped tree is not: #408's first recipe said to compare `initials()` against a folded-first partition, which is what `initials()` now IS, so it reproduced 0 where the bullet claimed 660 and was the only stated provenance for the number. Run the recipe as written before shipping the bullet. Cross-version numbers (a released wheel, the pre-change tree) are dated snapshots under Counting claims, since nothing in the repository re-runs them. Per-rule ledger toml comments asserting PARSER behavior cite rule IDs under the excerpt discipline; free prose is for ledger mechanics only (owned by tools/differential/README.md). +**Release-log claims.** Quantified or universal behavior claims in release bullets must come from the differential gate's classified summary, be verified against rules.md examples, or -- for a view the gate cannot see -- carry a recompute recipe stored with the design entry the bullet cites; never write one from memory. The classified summary covers the CONTRACT tier plus whatever radar diffs a rule classifies; a radar corpus's unmatched diffs are listed under UNCLASSIFIED (radar) and are not in it, so a claim quantified from the summary alone is silent about them. The gate compares the seven role fields, `_ambiguities` and -- since #484 -- `initials()`, under the `_initials` pseudo-field; so an initials-only change DOES show in the classified summary. But that pseudo-field sees only the names whose initials moved WHILE EVERY ROLE STAYED PUT -- main()'s roles-identical guard keeps it out of any diff a role is already in -- so a count taken from it is a FLOOR on initials movement, not the population: measured 2026-09-02 at 2.1.0 → tree on the v2 surface, 83 of the 1120 compared ENTRIES (1116 distinct names; seven entries carry a declared order rather than the default, and three strings are compared under more than one) changed their `initials()` string and only 28 were visible under `_initials`, the other 55 having moved a role as well. A bullet about how many names' initials changed still needs the recompute recipe. `capitalized()` and any other render view stay invisible to the gate (decisions.md#R4, #R3), and for those the first two sources still cannot reach a claim: a gate run is byte-identical across the change, and an example line witnesses an output without counting anything. A recipe names the corpus files, the policy sweep, and -- the part that is easy to omit and fatal -- THE COMPARATOR, which must be something the shipped tree is not: #408's first recipe said to compare `initials()` against a folded-first partition, which is what `initials()` now IS, so it reproduced 0 where the bullet claimed 660 and was the only stated provenance for the number. Run the recipe as written before shipping the bullet. Cross-version numbers (a released wheel, the pre-change tree) are dated snapshots under Counting claims, since nothing in the repository re-runs them. Per-rule ledger toml comments asserting PARSER behavior cite rule IDs under the excerpt discipline; free prose is for ledger mechanics only (owned by tools/differential/README.md). **Working on docs/design/ has its own AGENTS.md.** `docs/design/AGENTS.md` carries the landing-a-design distillation checklist, the primary-source review rule, the dated-count convention, and the review axes. Claude Code loads it automatically when a session reads or edits anything under docs/design/; if your tool does not do nested discovery, read it yourself before touching those files or reviewing a change to them. diff --git a/docs/design/decisions.md b/docs/design/decisions.md index ea625bb6..5d763d69 100644 --- a/docs/design/decisions.md +++ b/docs/design/decisions.md @@ -747,6 +747,14 @@ Decisions that landed: - 2026-09-01 #469 (later the same day, so read this bullet as SUPERSEDING the one above rather than the two disagreeing) — the CJK arrangement is IN the inventory after all, as shapes 6 and 7, and the bullet above is right about why it could not be until the question was answered: a shape row cannot be written without settling what it admits. What settled it was splitting the question the bullet treats as one. There is no single "unspaced CJK name" shape — there is a pure family-first arrangement (6) and a source-order transcription listing (7), and the composed forms are not arrangements at all. Both take `order=None`, which is the part that had looked impossible: the family-first reading is SCRIPT-carried rather than declared, so a pure shape 6/7 string already parses correctly under the DEFAULT policy and there is nothing for `order` to name. `min_baseline` is then documentary rather than a skip trigger, an asymmetry with shapes 4/5 that tools/differential/shapes.py's docstring states so it reads as designed. The doctrine half is its own entry (cjk-comma-demotion above); what belongs here is that `corpus_cjk.jsonl` no longer "covers that ground meanwhile" — the ground is split between it and a radar file now. - 2026-09-01 #469 — the `tolerated` flag on a case row is the demotion's VEHICLE, and it is recorded here rather than as a mechanisms.md entry. It is an explicit declaration, reviewed one row at a time exactly as a `shape=` tag is, mutually exclusive with one, and restricted to rows whose text bears a classified codepoint; `build_cjk_corpus.py` partitions on it, and a text marked on one row and not another is a hard error, the flag being per-text in effect. Why not a mechanisms entry: that catalog is keyed by RECURRING problem shape and its contract statements are citable verbatim from code comments, and this is one arc's device with no second instance and no comment needing to cite it — the tier mechanics it rides on are already owned by tools/differential/README.md, so an entry would restate a source rather than say something the source does not (the #473 lesson under decisions.md#review-agent-single-source). If a second arc ever needs "record the behavior, promise nothing", that is the point to promote it and this bullet is the first instance to cite. +### differential-ledger, the initials view (2026-09-01, #484/#462) + +The fifth ledger arc: the gate gains a column it never had. `initials()` is a derived render view, so a change in it with every role identical was invisible to the seven-field comparison by construction — the `middle_as_family_fold_order` row's `notes` in `tests/v2/cases.py` say as much, and #408 moved initials for a whole minor with the gate green. Mechanics (which rows carry `_initials`, the roles-identical guard, the one-field rule constraint) are owned by tools/differential/README.md; these are the decisions. + +- 2026-09-01 #484 — `_initials` is a pseudo-field on BOTH surfaces, the facade's at every baseline (1.4.0 has `initials()`) and the core's from 2.0 on, and it enters a name's diff ONLY when the seven roles and `_ambiguities` agree on every compared surface. The alternative — strict subset semantics, as `_ambiguities` has — was measured and rejected: names whose roles AND initials both move are claimed by 53 of the 72 explaining rules at 1.4.0, 43 of 66 at 2.0.0 and 36 of 59 at 2.1.0 (recompute below), every one of which would have had to grow `_initials`, and a rule declaring it absorbs any initials change on its names anyway, so the churn bought no discrimination. A role move drags its initials with it; the pseudo-field is a render-layer instrument, not a second copy of the role comparison. Consequences a rule author must know: `fields = ["_initials"]` is a rule of its own kind and lists nothing else — validate_rules refuses the mix as dead; and a count taken from the `_initials` summary is a FLOOR on initials movement, not the population (AGENTS.md carries the 83 / 28 / 55 measurement). `family_base` was considered as a second derived view and NOT added (YAGNI; adding one is the same two rows per surface, recorded here rather than on the issue). +- WHAT IT FOUND, measured 2026-09-01 by patching compare.py's own worker to emit `initials()` and comparing against the tree — and measured on THIS BRANCH'S BASE, before `31623d9` fixed #462, which is the tree the parenthetical describes: there the tree matched 2.2.0 exactly, 0 diffs of any kind. Three clusters of initials-only names, 140 at 1.4.0 and 28 at each of 2.0.0 and 2.1.0. On the SHIPPED tree the same recipe gives 126 at 1.4.0 and 42 / 42 / 14 at 2.0.0 / 2.1.0 / 2.2.0 (re-measured 2026-09-02; the gate itself is the recipe now, see RECOMPUTE). The two readings reconcile through cluster (ii) alone — 126 + 14 = 140, 28 + 14 = 42, 0 + 14 = 14 — the 14 #462 names having moved from the 1.4.0 side to the 2.x side when the fix landed. (i) The R2 readmission, 27 names at 1.4.0, 2.0.0 and 2.1.0 — `Anh Do` → `A. D.`, `Juan van der` → `J. v. d.` — shipped in 2.2.0 with a release note and the R2 entry above but no ledger rule, because none could see it; now `fix(#385/#402)`, a literal list of the 27 (the shape "a part of nothing but particles" is no property of the raw string), plus `de los Santos` under its own `fix(#360)` rule since ITS initials moved when `los` became a particle, not with R2. (ii) The facade dropping a middle `E`/`Y` initial, 14 names at 1.4.0 only, all but one radar tier — the population beside the count, per docs/design/AGENTS.md's population rule, is the names carrying such a letter in the MIDDLE or FAMILY group, which is the only position the bug reached, and there are 14 of them: 14 of 14 in that position moved. The `fix(#462)` rule's regex (`(?:^|[\s,])(?:[EY]|[EeYy]\.)(?=[\s,]|$)`) reaches 18 corpus names, four more than the population, and the other four (`E Anne D`, `E Jones`, `E Maria`, `Y. L.`) carry the letter in the GIVEN group, which has always initialed every word it holds, so nothing moved. The discriminator is the GROUP and not the position: `E Anne D,Leonardo` leads with the E too, but its comma re-roles the run into the family group, and it is one of the 14 that move: #462, a 1.4 parity break shipped in 2.0.0 and recorded nowhere. Fixed in the same PR (R3's bullet below); the 14 then move against the three 2.x baselines instead and carry a `fix(#462)` rule in each. (iii) Per-word grouping, 98 names at 1.4.0 only: 1.4.0 initialed one group per `*_list` element and rendered a multi-word element's letters with the separator and no delimiter — `Juan Velasquez y Garcia` → `J. V G.` at 1.4.0 and `J. V. G.` here, `Abdul Salam Hassan` → `A S. H.` and `A. S. H.` — and the 2.0 facade initials each word. Measure with the FULL name, not with the run alone: `Velasquez y Garcia` and `Abdul Salam` as whole inputs give `V. G.` and `A. S.` at 1.4.0 and here alike, the run being the family and initialed per word either way, so the bare forms this bullet illustrated with until 2026-09-02 reproduced nothing. Strip periods and spaces and all 98 agree with 1.4.0 letter for letter. Changed in 2.0.0, mentioned once in passing (the #408 bullet under R3: "v1's initials GRANULARITY"), never classified. Classified LATE, on purpose: mechanisms.md#FACADE-CONTRACT promises 1.4-warning-free code keeps working except for release-log-classified fixes, and a classification three minors overdue is still owed, so the 2.3.0 log carries a bullet that says the behavior has held since 2.0.0. Four rules rather than one, because the Latin-alternation guard keys a rule to a single copied vocabulary and the 98 copy three (CONJUNCTIONS 66, BOUND_GIVEN_NAMES 19, PARTICLES 11) plus the `Ph. D.` merge (2). The connective rule cannot claim cluster (ii), and it takes BOTH of its exclusions — the case-sensitivity and the trailing whitespace lookahead — to keep it out. Of the 14, 7 carry a bare capital `E` and 7 a dotted `E.`/`e.`: the case-sensitivity excludes the bare capitals, and the alternation's trailing whitespace lookahead excludes the dotted ones whatever their case, a period not being whitespace. Since the fix in `31623d9` those names agree with 1.4.0 and diff only against the 2.x baselines, so neither half is keeping two live clusters apart today. What they do is make any FUTURE facade initials change on these shapes surface as UNEXPLAINED at 1.4.0 rather than be absorbed here as per-word grouping. Gate totals moved from 226 / 205 / 113 / 0 intentional to 352 / 247 / 155 / 14, zero unexplained, exactly the new rules' explained counts. +- RECOMPUTE: load `tools/differential/compare.py` by path; build the entries as main() does (`_load_entries`, `_load_shapes`, `_CORPUS_TIERS`, dedup by (name, order), drop order-bearing entries below their shape's minimum); patch `_worker_source` to append `_initials` to both rows; `_run_worker`; compare against the tree's facade and core. TWO RECIPES, and only the second of them needs that patch. (a) The CLASSIFIED TOTAL per baseline is read off the gate itself, no patch and no in-memory run: `compare.py --baseline X` prints a `## issue (N)` heading per rule, and the total is the sum of the headings of the rules whose `fields = ["_initials"]` — at 1.4.0 `fix(#385/#402)` 27, `fix(#360) los` 1 and the four `fix(initials-per-word)` rules 19 / 66 / 11 / 2, summing to 126; at 2.0.0 and at 2.1.0 `fix(#385/#402)` 27, `fix(#360) los` 1 and `fix(#462)` 14, summing to 42; at 2.2.0 `fix(#462)` alone, 14. What the gate does NOT print is the per-name detail — a heading lists at most ten of its names and no diff fields at all, and the UNEXPLAINED and radar blocks are empty on a green tree — so it answers how many and under which rule, never which names or which surface moved. (b) The PER-NAME SPLIT, and the pre-fix 140 / 28, need the in-memory run described above: load compare.py by path, run main()'s comparison loop or hook `diffing`, and keep the names whose diff is exactly `{"_initials"}`. READ THE PATCH STEP AS HISTORY, and it belongs to (b) alone: since `a748862` the shipped worker template emits `_initials` itself, so patching `_worker_source` is what a measurement BELOW that commit needs and re-applying it on this tree changes nothing. The pre-fix 140 / 28 must be taken on the branch's base, before `31623d9`, rather than on this tree. Split "initials only" from "roles and initials"; within the former, lowercase and strip `[.\s]` to separate per-word grouping (equal after stripping) from content changes. The guard's liveness is proved by mutation, not by the green run: with `not diff and` removed from the guard, a roles-and-initials name has its role diff REPLACED by `{_initials}`, which no existing rule declares, and the 1.4.0 run reports 117 of them as no longer classified — 79 unexplained plus 38 radar, from 0 and 0. The same mutation is where the first bullet's churn figures come from: the rules that lose at least one name are the rules strict subset semantics would have had to widen, counted against the rules that explain anything at that baseline (53 of 72, 43 of 66, 36 of 59 at 1.4.0 / 2.0.0 / 2.1.0; re-derived 2026-09-02, the 1.4.0 numerator having been recorded as 43 until then). + ### comma-suffix-arc — #291/#296/#316 (2026-07-26 → 2026-08-01) #291 was filed 2026-07-26 out of the 2.0 vocabulary cleanup, with @@ -792,7 +800,7 @@ Declined: - 2026-08-18 #404/#385 — R2's DISCRIMINATOR REVERSED. It said a family written wholly out of particle vocabulary has a base where one of those words "is itself borne as an ordinary surname", and that only the words that are never anyone's name stay particles. That is a vocabulary test, and it is the wrong one: a particle earns its name by joining forward to the word it modifies, so a particle with nothing to join is not doing a particle's work whatever it is borne as. POSITION decides, and the borne-as-surname question does not arise. The invariant that replaces it, and the reason this is worth a rule rather than three bug fixes: a non-empty family always has a non-empty base. A particle needs a base to attach to. Measured, 43 of 751 corpus names x 3 orders violated it — "Del Toro" under FAMILY_FIRST reported family 'Del' with no base, losing the surname from the base and from the initials. Marked rather than untagged (mechanisms.md#MARK-DONT-STRIP): `particle` is stable API and means "from the particle vocabulary wherever it lands", which stays true; UNJOINED_TAG records what was decided. That also leaves #405's ambiguity emitter a predicate to key on, which stripping would have destroyed for exactly the never-given half where the fork is most interesting. - Blast radius: 44 of 2,253 parses move and every one is VIEWS-only — the seven role fields are byte-identical everywhere, so the differential harness cannot see this change at all (it compares roles only). Tests are the whole verification, and the invariant is one of them. + Blast radius: 44 of 2,253 parses move and every one is VIEWS-only — the seven role fields are byte-identical everywhere, so the differential harness cannot see this change at all (it compares roles only). Tests are the whole verification, and the invariant is one of them. Amended 2026-09-02 (#484): the harness now compares `initials()` under the `_initials` pseudo-field for names whose roles agree, so it CAN see this change — the 27 names are classified by the `fix(#385/#402)` rule at 1.4.0, 2.0.0 and 2.1.0. Tests are no longer the whole verification. - Recorded 2026-08-16, intent UNVERIFIED: the v1-era design held that a family name cannot be only particles ("Anh Do" — Do is a surname AND a particle), so last_base was guarded non-empty. The facade still guards (HumanName("Anh Do").last_base == "Do"); the v2 core does not (parse("Anh Do").family_base == "", family_particles == "Do"), and the surname vanishes from initials (parse("Anh Do").initials() == "A."). - 2026-08-16 #385 RESOLVED by the collision criterion, not on its own terms: the issue's option 3 ("guard the view only when a word is vocabulary-ambiguous") is what decisions.md#vocabulary-collisions produces when applied here. @@ -817,16 +825,17 @@ R3's earlier history is under `decisions.md#R2`, which this entry does not repea WHAT THE MEASUREMENT ADDED, and it reframes the issue from a consistency fix to a parity restoration. v1 had this feature as `CONSTANTS.middle_name_as_last` (`_config_shim` maps it to `Policy.middle_as_family`) and had NO general `name_order`, so only the two default-order policies have a v1 reference at all. Measured on the released 1.4.0 wheel over the 1094-name deduped corpus, from a script run outside the worktree that asserts `nameparser.__version__` first: at the default policy 1 name moves and agrees with 1.4.0 neither before nor after; under `middle_as_family` 71 move, of which **54 come into exact 1.4.0 agreement and NOT ONE moves away from it**. Worked pair: `Doe, Dr. John A.` gives `J. A. D.` at 1.4.0, gave `J. D. A.` here, and gives `J. A. D.` again; `Brundridge, Contessa A` likewise `C. A. B.` / `C. B. A.` / `C. A. B.`. The facade had been giving the folded answer all along — `HumanName.initials()` reads `first_list`/`middle_list`/`last_list`, and `_facade._list_for` prepends the carriers exactly as `_text_for` does — so the core view was simultaneously out of step with the field beside it, with the facade above it, and with 1.4.0 behind it. A view that stopped honoring a record had been reproducing a v1 bug v1 did not have. THE 17 THAT AGREE WITH 1.4.0 NEITHER WAY, recorded because the obvious explanation is wrong — and the FIRST REPLACEMENT for it was wrong the same way, corrected by this branch's docs review before merge. It is not that their parse already differs: only 2 of the 17 parse differently from 1.4.0 under this policy, `Smith, John V.` (#432's dotted numeral) and `남궁민수 지훈` (the unspaced-hangul surname split and the script order, rules.md#W1). The other 15 have SEVEN BYTE-IDENTICAL FIELDS at 1.4.0 and here — `der, y van` INCLUDED, which an earlier draft of this bullet filed with the two above by carrying over a measurement taken at the OTHER policy, where its parse does differ. Of those 15, 14 differ from 1.4.0 only in v1's initials GRANULARITY — v1 initials one group per `*_list` element, so a run its parse joined into one element (`Velasquez y Garcia`, `abdul salam`) yields `V G.` where 2.x, one token per word, yields `V. G.` — and strip the periods and spaces and all 14 come into 1.4.0's letter ORDER after this change, none having been in it before (`JVGQ` → `JQVG`). The 15th is `der, y van`, and it is worth naming rather than lumping: its fields match 1.4.0 exactly and its initials do not, because 1.4.0 contributes NOTHING for a family that is all particles (`van der` → `y.`) where 2.x contributes its words. That is R3's own adjudicated clause and decisions.md#R2's territory — R3 carries `"Juan van der" → initials="J. v. d."` against 1.4.0's `J.` — so this one name is a divergence this document has already decided in favor of, not an unexplained residue. So the parity claim is stronger than "54 of 71 and the rest unexplained": 68 of the 71 come into 1.4.0's ordering, 54 of them exactly, one more is R2's decided divergence, and 2 have a parse difference that predates this change. BLAST RADIUS by policy, over the same corpus × three `name_order` values × `middle_as_family` off/on (6564 parses, 660 move). Default order: 1 / 71. FAMILY_FIRST: 1 / 294. FAMILY_FIRST_GIVEN_LAST: 1 / 292. STATE THE POPULATION BESIDE THE COUNT, per this file's own rule: the shape an order change can be observed in is a contributing folded token standing AFTER a contributing unfolded one of the same role, and at the default order the corpus holds exactly 1 such name without `middle_as_family` and 72 with it. So the movers are 1 of 1 and 71 of 72 — a near-total hit rate over a corpus that carries one instance of P6's half, not a blast radius near zero. (The one non-mover is `Chang, Andy C I`, where the folded `C` and the family `Chang` share a letter and the reorder is invisible in initials.) Nine corpus names carry the tag at all at the default order; the other eight have a single contributing family token, which makes the partition the identity. The `middle_as_family` half is where the reach is, which follows from the producers: O3 folds every middle word, while P6's attachment moves initials only where the attached run is READMITTED by the unjoined mark — a family that is all particles — so it reaches one corpus name in each order and the same one. RECOMPUTE, and the COMPARATOR is the half that has to be stated, because the obvious phrasing of it measures nothing: parse the `tools/differential/corpus*.jsonl` files deduped under each of the six policies and compare `initials()` against THE PRE-CHANGE RENDERING — the same contributing tokens left in WRITTEN order. What this sentence said until 2026-08-30 was "against a folded-first partition of `tokens_for(role)`", which is what `initials()` now IS: that recipe compares the shipped implementation with itself and yields 0 differing parses, the inert-measurement shape (mechanisms.md's Verification shapes section) written into the very sentence meant to prevent it, and the only recompute path the repository offered for docs/release_log.rst's counts as well. CONTRIBUTING is the other half of the recipe and it comes FIRST: apply this view's own skip filter — outside the given group a token tagged `particle` or `conjunction` contributes nothing unless it also carries UNJOINED_TAG — and compare the two orders only over what survives it. Leave the filter out and every skipped word reads as a difference instead: 1546 rather than 660 against written order, and 1018 rather than 0 against the folded partition. Re-measured this way on 2026-08-30 and reproducing the counts above: 660 of 6564 parses, 1 / 71 at the default order, 1 / 294 and 1 / 292 under the two family-first orders. - THE VERIFICATION LIMIT, and it is decisions.md#R4's, restated because a plausible reading of "one name moves at the default policy" is that the ledger owes it a rule. It does not, and could not: `tools/differential/compare.py` compares the seven role fields plus `_ambiguities`, and `initials()` is not among them — the same blindness recorded for `capitalized()` under R4. Measured rather than reasoned: the gate was run at all three baselines on the pre-change tree (exported with `git archive HEAD` into a scratch directory, so nothing in the working tree moved) and on the changed one, and the two outputs are byte-identical apart from the tree path and the baseline-worker temp path — 1094 corpus names, 229 / 194 / 102 intentional diffs, `unexplained: 0`, and 67 / 58 / 51 headings with identical per-heading counts, at 1.4.0 / 2.0.0 / 2.1.0. NO LEDGER RULE WAS ADDED, and none was needed. What stands in its place is `rules.md#R3`'s example line and `tests/v2/test_render.py`, both mutation-checked. + THE VERIFICATION LIMIT, and it is decisions.md#R4's, restated because a plausible reading of "one name moves at the default policy" is that the ledger owes it a rule. It does not, and could not: `tools/differential/compare.py` compares the seven role fields plus `_ambiguities`, and `initials()` is not among them — the same blindness recorded for `capitalized()` under R4. Measured rather than reasoned: the gate was run at all three baselines on the pre-change tree (exported with `git archive HEAD` into a scratch directory, so nothing in the working tree moved) and on the changed one, and the two outputs are byte-identical apart from the tree path and the baseline-worker temp path — 1094 corpus names, 229 / 194 / 102 intentional diffs, `unexplained: 0`, and 67 / 58 / 51 headings with identical per-heading counts, at 1.4.0 / 2.0.0 / 2.1.0. NO LEDGER RULE WAS ADDED, and none was needed. What stands in its place is `rules.md#R3`'s example line and `tests/v2/test_render.py`, both mutation-checked. Amended 2026-09-02 (#484): the blindness this paragraph rests on is retired — the gate compares `initials()` for roles-identical names now, so an initials-only change DOES owe a ledger rule. #408 still owes none, and for a different reason than the one recorded here: its movement is 1.4.0 parity, and the one corpus name it moves at the default order (`der, y van`) diffs on ROLES at all three baselines, under `fix(#379)`, so the pseudo-field never enters. The semantics are in `decisions.md#differential-ledger, the initials view`. WHY R3 GAINED A SENTENCE rather than leaning on R1. R1 already says folded family words "render before the rest of the family wherever they stood in the string" — but its subject is "Every FIELD", and `initials()` is not a field: R3 calls it "this view", and it takes a format spec and two separators rather than being read as an attribute. So R1 does not reach it by its own words, and #408 is precisely what leaving that to inference costs. R3's new clause states the order in R3's own voice, carries the O3 example line (`"Hassan, Mohamad Ahmad Ali"` under `middle_as_family` → `initials="M. A. A. H."`, reusing R1's own input so `corpus_rules.jsonl` is unchanged at 241 names), and R1 and R3 now name each other in `interacts:` — the field says which order, the view says it follows the field. P6's half gets no example line, for the same reason R1 carries none: the only input that witnesses it is `der, y van`, whose PARSE is itself contested, so pinning its initials normatively would pin a value another open question can move. It is pinned in the unit test instead, which is where today's-behavior pins belong. THE PARTITION IS PER ROLE, not scoped to FAMILY, and this is a deliberate choice rather than a copied line. Both producers re-role to FAMILY, so FAMILY is the only role a parse can put the tag on today and the GIVEN and MIDDLE arms are unreachable. They are there because `_text_for` partitions for every role it renders and the two views must not diverge again — the same uniformity `_post_rules`' own UNJOINED_TAG loop takes for its three roles, "for uniformity with the rule, not because all three are observable". A producer that ever folded into another part would otherwise reopen #408 there with nothing to fail. `test_initials_folds_in_every_role_it_renders` pins it from a hand-built name, since no input string can. Mutation-checked four ways — scoping the partition to FAMILY, and skipping it for GIVEN, for MIDDLE, for FAMILY — and each fails that test. It took THREE drafts of the fixture to make that true, and the two misses are the same miss one role apart. The first carried two GIVEN tokens and ONE MIDDLE: a one-element partition is the identity, so the MIDDLE arm was asserted and unpinned, and skipping the partition for MIDDLE passed the entire suite. The second fixed MIDDLE and carried NO FAMILY token at all, in a test named for every role it renders — so skipping the partition for FAMILY passed this test, and was caught only by its siblings and by R3's example line, neither of which is about per-role application. A zero-element group is the identity too, and reads even less like a gap than a one-element one, which is why the second miss survived a review that had just named the first. Two tokens per role is what closes it, and the general form is carried as mechanisms.md#TWO-ELEMENT-GROUPS rather than left here: a test written to pin a partition, a sort or a dedup needs at least two elements in every group it claims to cover, or the claim rides on a no-op. +- 2026-09-01 #462 — DONE, the facade's twin: `HumanName.initials()` dropped a dotted or bare-capital `E`/`Y` from the middle and family groups because `_process_initial` tested conjunction membership alone, where 1.4.0's `is_conjunction` was "in the set AND NOT `is_an_initial`". `Scott E. Werner` gave `S. W.` from 2.0.0 through 2.2.0 and gives `S. E. W.` again; `John E Smith` (bare capital) and `Juan Y. Garcia` likewise; `parse().initials()` never had the bug, since `_classify` tags `E.` an initial and the render reads the tag (mechanisms.md#RENDER-HONORS-THE-PARSE). The issue said the bare form was "still correctly dropped", which is true of lowercase `e` and false of bare capital `E` — 7 of the 14 corpus names that move are bare capitals. Restored with `_render._INITIAL`, v1's own `initial` shape, because the facade may import `_render` and not `_pipeline`; scoped to `_process_initial`, its only caller, so a future reader of `_is_conjunction` does not inherit a decision made for initials. Found by #484's pseudo-field at the 1.4.0 baseline, where it had sat as 14 unreported diffs; the gate could not see it before because the seven fields never moved. ### R4 — case repair reads the unjoined mark - 2026-08-29 #407 — DECIDED: case repair consults the mark #404 introduced, and consults it for the WHOLE PART rather than for a particle standing alone. Until now three views read it and the fourth did not, which is how `parse("ANH DO")` came to report family_base 'DO' and initials 'A. D.' while `capitalized()` returned 'Anh do' — the same word treated as an ordinary name word by three readers and as a particle by the fourth. The scope question is the only real choice here, and standing-alone loses on the first Vietnamese name that tests it: `anh van do` has family `van do`, two particle-vocabulary words and neither of them alone, so a rule keyed on aloneness would capitalize `Anh Do` and leave `Anh van do`, making the same surname read differently for having a given-name word in front of it. rules.md#R2 states the test as "Every word", not "standing alone", and this is the case that shows why it is stated that way. One predicate, four views. - 2026-08-29 — this is a DELIBERATE DEVIATION from 1.4.0, not a parity restoration, said plainly because the surrounding branch work is mostly restorations and a reader will otherwise assume this is one. Measured on the released 1.4.0 wheel: `HumanName('ANH DO').capitalize()` leaves `'Anh do'` and `HumanName('anh van do').capitalize()` leaves `'Anh van do'`; both are `'Anh Do'` and `'Anh Van Do'` on this branch. There was nothing to restore — v1 lowercased on vocabulary membership alone and had no notion of a particle with nothing to join, the mark being a 2.x invention. - 2026-08-29 — the ACCEPTED COST, agreed before the change rather than discovered after it: a degenerate family that is nothing but particles capitalizes too, so `juan van der` renders `Juan Van Der` where 1.4.0 gave `Juan van der`. This is the same trade rules.md#R2 already took for `family_base` — those words ARE the base, so they are name words, and a parser that called them name words for sorting and particles for display would be the incoherent option. Blast radius, measured by running both predicates in one process, and stated over BOTH populations because the neighbouring bullets in this entry measure over 1094 and a bare "4 and 23" would read as measured there too. Over the 1090 names the corpora carried before this change: 4 names move under a plain `capitalized()` — `ANH DO`, `vai la`, `van ma van`, and `der, y van`, which becomes `y Van Der` and keeps its lowercase conjunction — and 23 under `force=True`, the extra 19 being mixed-case inputs the gate holds back (rules.md#R5). Over today's 1094 it is 5 and 24: this change's own R4 example rows added `anh van do`, and that name is itself a mover. -- 2026-08-29 — THE VERIFICATION LIMIT, which matters more than the change. `tools/differential/compare.py` compares the seven role fields plus `_ambiguities` and nothing else, so `capitalized()` is not a compared field and NO ledger run at any baseline can see this deviation: the corpus counts and the three intentional-diff totals hold across it precisely because the harness is blind to it, and a green run is evidence about roles, not about case. mechanisms.md#FACADE-CONTRACT promises that v1-visible changes are "release-log-classified fixes, each verified in the differential ledger". The first half is kept — docs/release_log.rst carries the 2.2.0 bullet — and the SECOND HALF CANNOT BE, for this change or for any other that moves only case. Standing in its place: rules.md#R4's own example rows (`ANH DO`, `anh van do`, and the `juan de la vega` boundary), rules.md#R5's, and the v1-facade tests in tests/test_capitalization.py. The ones that pin the fix are mutation-checked against deletion of the tag consult; the `juan de la vega` control passes that mutation BY DESIGN, which is what makes it a control rather than a witness. Read the silence as a SCOPE and not as a gap: the harness compares parses, case repair is a render view, and nothing is owed on the tracker for it — the next bullet is why that is a decision rather than a deferral. +- 2026-08-29 — THE VERIFICATION LIMIT, which matters more than the change. `tools/differential/compare.py` compares the seven role fields plus `_ambiguities` and nothing else, so `capitalized()` is not a compared field and NO ledger run at any baseline can see this deviation: the corpus counts and the three intentional-diff totals hold across it precisely because the harness is blind to it, and a green run is evidence about roles, not about case. mechanisms.md#FACADE-CONTRACT promises that v1-visible changes are "release-log-classified fixes, each verified in the differential ledger". The first half is kept — docs/release_log.rst carries the 2.2.0 bullet — and the SECOND HALF CANNOT BE, for this change or for any other that moves only case. Standing in its place: rules.md#R4's own example rows (`ANH DO`, `anh van do`, and the `juan de la vega` boundary), rules.md#R5's, and the v1-facade tests in tests/test_capitalization.py. The ones that pin the fix are mutation-checked against deletion of the tag consult; the `juan de la vega` control passes that mutation BY DESIGN, which is what makes it a control rather than a witness. Read the silence as a SCOPE and not as a gap: the harness compares parses, case repair is a render view, and nothing is owed on the tracker for it — the next bullet is why that is a decision rather than a deferral. Amended 2026-09-02 (#484): the premise moved and the conclusion did not. `compare.py` compares the seven role fields plus `_ambiguities` AND `_initials`, so "nothing else" is now one pseudo-field short and a render view is no longer invisible by construction — but `initials()` is the only one added, deliberately (`decisions.md#differential-ledger, the initials view`), so `capitalized()` is still not a compared field, and the next bullet's "the compared surfaces stay the seven roles plus `_ambiguities`" should be read as naming the surfaces case repair is absent from. Do NOT read #484 as the precedent that answers the next bullet: it MEETS that bullet's two objections rather than waiving them. `_initials` is a diff-of-diffs gated on the fields agreeing — precisely the instrument the next bullet declines for case — and the column did arrive pre-failed, by 140 entries at 1.4.0 as measured on this branch's base — 126 of them classified by six new 1.4.0 rules on the shipped tree, and the other 14 by `fix(#462)` at the three 2.x baselines once the facade fix moved them there. What earned it its place is a demonstrated ESCAPE: #408 moved initials for a whole minor with the gate green, and the 98 per-word names had been moving since 2.0.0, unseen and unclassified. So the next bullet's case for no `_capitalized` column stands until a case change escapes the same way, and that is the test a future proposal has to meet. - 2026-08-29 — WHY CASE IS NOT A COMPARED SURFACE, settled here rather than left open, because "add a column" is the obvious answer to the bullet above and it is the wrong instrument. What the harness is FOR is parsing behavior — tools/differential/README.md says to run it "when touching parsing behavior, and before cutting a release" — and what earns it its two-process cost is that a role is EMERGENT: the eight stages fold over one name and each can move what the next sees, so the reach of a vocabulary or rule edit is not derivable by reading the diff and has to be measured over a corpus instead. Case repair is the opposite kind of thing. `_render._cap_word` is a pure function of `(word, role, tags, lexicon)` with four branches — the particle/conjunction lowercasing, the exceptions map, the Mac/Mc convention, and plain `capitalize` — sitting behind R5's single gate, so an edit to it is legible: you can see what you touched. The measurement agrees with the reasoning. Over the 1094 corpus names the 1.4.0 and 2.1.0 wheels ALREADY disagree on `capitalize(force=True)` for 139 of them (recompute by running `HumanName(n).capitalize(force=True)` under each wheel over the corpus files deduped, and diffing the two maps; the count was the same over the 1090 names the corpora carried before this change). Nearly every one is an echo of a parse diff the field comparison already classifies — a token that changed role capitalizes differently — so a column would re-report the ledger's own explained diffs in a vocabulary its rules cannot match, and would arrive pre-failed by 139 entries. Isolating the residue means comparing case only where the fields agree, a diff-of-diffs, to reach a class that a unit test reaches directly. So the compared surfaces stay the seven roles plus `_ambiguities`, by design and not by omission, and case is pinned where case belongs: the `rules.md#R4` and `#R5` example rows, and tests/test_capitalization.py. - 2026-08-29 — DEBT DISCHARGED, the obligation the last bullet of decisions.md#R5 left this commit; recorded here so the pointer resolves rather than dangling. The FALSEHOOD: R4 promised repair "vocabulary exceptions (McDonald) included" flatly, which `str(parse('Juan Mcdonald').capitalized())` — `'Juan Mcdonald'` — falsifies, the gate refusing before any vocabulary is consulted. The statement now scopes that promise to where repair acts at all and hands the where to R5. A SECOND error in the same eight words turned up while rewriting them, and is worth recording because it had survived unread for as long as the first: McDonald is not a vocabulary exception. The shipped exceptions are five entries — II, III, IV, M.D., Ph.D. (recompute with `Lexicon.default().capitalization_exceptions_map`) — and `Mc`/`Mac` is a separate convention applied to any name shaped like one, so the old parenthetical illustrated the promise with the one mechanism it was not about. The statement now names an exception the vocabulary actually records (Ph.D.) and the Mac/Mc convention separately. The AMBIGUITY: "an already-correct name comes back unchanged" is true read as idempotence and reads as false under bearer-correctness, and the repair was to say WHICH is meant — a name already written the way repair would write it comes back unchanged — plus one clause refusing the inference a reader might otherwise draw, that a spelling its bearer chose is not spared for having been chosen. That clause is deliberately a REFUSAL and not a carve-out: sparing deliberately single-cased names is behavior the parser does not have and that R5's rationale declines on the record, so `str(parse('bell hooks').capitalized())` is still `'Bell Hooks'`. R4's boundary moved too: the old `"Juan McDonald" → capitalized="Juan McDonald"` row was inert — it passes with R5's gate deleted — so it became a `capitalized_forced=` row, which asserts what repair APPLIES rather than what the gate keeps and fails when the Mac/Mc convention is removed (measured by neutering it in one process: `'Juan Mcdonald'`). The boundary is now `juan de la vega`, a working particle run R4's new clause deliberately does not reach. diff --git a/docs/release_log.rst b/docs/release_log.rst index 18625eb6..8cc77c8d 100644 --- a/docs/release_log.rst +++ b/docs/release_log.rst @@ -2,6 +2,14 @@ Release Log =========== * 2.3.0 - Unreleased + nameparser 2.3 is under development. + + **Behavior Changes** + + - **Fix HumanName.initials() dropping a middle- or family-group initial that is also a one-letter conjunction.** ``HumanName("Scott E. Werner").initials()`` gives ``S. E. W.`` again where 2.0.0 through 2.2.0 gave ``S. W.``; ``Juan Y. Garcia`` and a bare ASCII capital ``John E Smith`` likewise. v1 excluded initial-shaped words from its conjunction test and the 2.0 facade had not; ``parse(...).initials()`` was already right and is unchanged. A bare lowercase ``john e smith`` still reads the ``e`` as the connective. See the ``R3`` entry of ``docs/design/decisions.md`` (closes #462) + + - **Record a 2.0.0 change to HumanName.initials() that no release note had classified:** since 2.0.0 the facade initials each WORD of a name part, where 1.4.0 initialed a joined run as one group -- ``HumanName("Juan Velasquez y Garcia").initials()`` is ``J. V. G.`` and was ``J. V G.``; ``Abdul Salam Hassan`` is ``A. S. H.`` and was ``A S. H.``. Nothing changes in 2.3.0; the differential gate now compares ``initials()`` (#484) and this is what it found. See the ``differential-ledger, the initials view`` entry of ``docs/design/decisions.md`` + * 2.2.0 - August 31, 2026 nameparser 2.2 is a rename plus about thirty parsing fixes. diff --git a/nameparser/_facade.py b/nameparser/_facade.py index 051d44e9..3ad9bac4 100644 --- a/nameparser/_facade.py +++ b/nameparser/_facade.py @@ -512,16 +512,32 @@ def last_base(self) -> str: # -- initials ------------------------------------------------------------- def _process_initial(self, name_part: str, firstname: bool = False) -> str: - # v1 parser.py:427 verbatim: particles/conjunctions are filtered - # from initials unless the part is a first name. split() rather - # than split(" "): *_list attributes assigned directly bypass - # whitespace normalization, and split(" ") yields empty strings - # for repeated spaces (#232). + # after v1 parser.py:427, not verbatim: particles and + # conjunctions are filtered from initials unless the part is a + # first name. split() rather than split(" ") because split(" ") + # yields '' between repeated spaces and `part[0]` below would + # raise IndexError on it (#232). v1 stated the reason as + # `*_list` attributes bypassing whitespace normalization, which + # no longer holds -- the `*_list` properties are read-only in + # 2.x, and assignment through `hn.middle = ...` normalizes -- + # but a doubled space anywhere in a part still reaches here. parts = name_part.split() initials = [] for part in parts: - if not (self._is_particle(part) - or self._is_conjunction(part)) or firstname: + # v1 parser.py:771 (1.4.0): is_conjunction was "in the + # conjunctions set AND NOT is_an_initial", so a dotted or + # bare-capital E/Y is the initial it looks like rather + # than the connective. The 2.0 facade dropped that half + # and lost the middle initial of 'Scott E. Werner' (#462). + # _render._INITIAL is v1's `initial` shape, kept in step + # with the pipeline's copy by tests/v2/test_regex_sync.py; + # the facade may import _render but not _pipeline + # (tests/v2/test_layering.py). Scoped here rather than in + # _is_conjunction: this is the only caller, and a future + # one should not inherit a decision made for initials. + conjunction = (self._is_conjunction(part) + and not _render._INITIAL.fullmatch(part)) + if not (self._is_particle(part) or conjunction) or firstname: initials.append(part[0]) if len(initials) > 0: return self.initials_separator.join(initials) @@ -554,11 +570,14 @@ def group_initials(names: list[str], # -- nothing survived # the filter, so the whole group is particles. The # facade's twin of the core's - # UNJOINED_TAG. NOT pinned against it: both case runners - # compare the seven role fields only, and Case carries no - # initials column, so the one covering test is - # tests/test_initials.py::test_initials_middle_name_all_prefixes. _split_last already applies the same guard to - # the base, which is why last_base was never empty here. + # UNJOINED_TAG. NOT pinned against it by the case runners, + # which compare the seven role fields only (Case carries + # no initials column); the covering test is + # tests/test_initials.py::test_initials_middle_name_all_prefixes, + # and since #484 the differential compares initials() on + # both surfaces for names whose roles agree. _split_last + # already applies the same guard to the base, which is why + # last_base was never empty here. return [w[0] for w in words] return (group_initials(self.first_list, True), group_initials(self.middle_list), diff --git a/tests/test_initials.py b/tests/test_initials.py index 39e9b261..dcb80b2b 100644 --- a/tests/test_initials.py +++ b/tests/test_initials.py @@ -231,3 +231,39 @@ def test_initials_separator_default_on_constants(self) -> None: # fixture restores). from nameparser.config import CONSTANTS self.assertEqual(CONSTANTS.initials_separator, " ") + + def test_initials_keep_an_initial_shaped_conjunction_letter(self) -> None: + # #462: v1's is_conjunction was "in the set AND NOT + # is_an_initial", so a dotted or bare-capital E/Y is the + # initial it looks like, not the Italian/Spanish connective. + # The 2.0 facade lost that exclusion and gave 'S. W.'; + # parse().initials() never had the bug. 1.4.0 parity, all four. + for name, want in (("Scott E. Werner", "S. E. W."), + ("John E Smith", "J. E. S."), + ("Juan Y. Garcia", "J. Y. G."), + # the letter is the WHOLE middle group here, + # so before the fix the group yielded nothing + # and was dropped outright: 'S. V.', a name + # short of its middle rather than a letter + ("Vega, Santa de Y", "S. Y. V."), + ("Хосе И. Мария Сантос", "Х. И. М. С.")): + hn = HumanName(name) + self.m(hn.initials(), want, hn) + + def test_initials_drop_a_bare_non_ascii_conjunction_letter(self) -> None: + # v1's bound, kept: _render._INITIAL is `^(\w\.|[A-Z])$`, so + # its DOTTED half is Unicode-aware (the 'И.' above survives) + # and its bare-capital half is ASCII-only. A bare Cyrillic 'И' + # is therefore not initial-shaped, stays the connective, and + # drops -- 'Х. М. С.' for a four-word name. #462 restored v1's + # exclusion; it did not widen the shape v1 excluded on. + hn = HumanName("Хосе Мария И Сантос") + self.m(hn.initials(), "Х. М. С.", hn) + + def test_initials_still_drop_a_lowercase_conjunction(self) -> None: + # the boundary #462 leaves alone: a bare lowercase e/y IS the + # connective, and 1.4.0 and 2.x agree + hn = HumanName("john e smith") + self.m(hn.initials(), "j. s.", hn) + hn = HumanName("maria y lopez") + self.m(hn.initials(), "m. l.", hn) diff --git a/tests/v2/cases.py b/tests/v2/cases.py index 89ddc68d..29e3f00e 100644 --- a/tests/v2/cases.py +++ b/tests/v2/cases.py @@ -347,7 +347,10 @@ def _check_cjk_shape_purity(self) -> None: "initials column, so the half #408 moved is pinned in " "rules.md#R3 and tests/v2/test_render.py: initials were " "'M. H. A. A.' here and are 'M. A. A. H.' now, which is " - "also 1.4.0's answer"), + "also 1.4.0's answer. Since #484 the differential compares " + "initials() too, for names whose roles are identical -- " + "but only under the policies it runs, and middle_as_family " + "is not one, so this row stays pinned here"), Case("ambiguous_surname_acronyms", "Jack MA", {"given": "Jack", "family": "MA"}, ambiguities=("suffix-or-name",), diff --git a/tests/v2/test_differential.py b/tests/v2/test_differential.py index 091c2bc1..1ee07d1f 100644 --- a/tests/v2/test_differential.py +++ b/tests/v2/test_differential.py @@ -117,6 +117,27 @@ def test_worker_source_gates_the_v2_import_on_the_baseline() -> None: assert "WANT_V2 = True" in compare._worker_source("2.0.0", want_v2=True) +def test_worker_source_emits_initials_on_both_surfaces() -> None: + """#484: initials() is a derived view the seven-field diff cannot + see. The facade has had initials() since 1.x, so the facade row + carries it at EVERY baseline; the v2 row carries it wherever the + v2 surface is compared at all. + + A text check, and the two asserts do not buy the same thing. + `_v2_row` is defined unconditionally in the template body, which + is version-independent -- so its assert at want_v2=False pins + template TEXT, not behavior. The facade line is the one whose + BEHAVIOR depends on the installed wheel -- 1.4.0's + `HumanName.initials()` has to exist and answer -- which no text + check can see, and is why the sibling below RUNS the template + instead of reading it: + test_the_worker_reads_a_default_order_line_as_the_tree_does.""" + for version, want_v2 in (("1.4.0", False), ("2.0.0", True)): + src = compare._worker_source(version, want_v2=want_v2) + assert 'row["facade"]["_initials"] = hn.initials() or ""' in src + assert 'row["_initials"] = p.initials() or ""' in src + + def test_every_shape_orders_resolve_and_bound_sanely() -> None: """The inventory's two contracts: an `order` is a public constant name on the installed tree that Policy actually accepts as a @@ -175,7 +196,12 @@ def _fake(v: str, w: bool, "__file__": "/wheel/nameparser/__init__.py"}, [{"facade": {"title": "", "first": "John", "middle": "", "last": "Smith", "suffix": "", "nickname": "", - "maiden": ""}}]) + "maiden": "", + # the tree's own initials for this name. + # This test is about the skip decision, so + # the fake baseline agrees by construction + # on the #484 pseudo-field too. + "_initials": "J. S."}}]) monkeypatch.setattr(compare, "_run_worker", _fake) monkeypatch.setattr(sys, "argv", ["compare.py", "--baseline", "1.4.0", @@ -222,6 +248,7 @@ def test_an_order_none_shapes_later_minimum_does_not_skip_the_entry( # produces, so an old/new facade that agree by construction keeps # a real diff from muddying the assertion. old_facade = {k: (v or "") for k, v in HumanName(name).as_dict().items()} + old_facade["_initials"] = HumanName(name).initials() or "" sent: dict = {} def _fake(v: str, w: bool, @@ -244,18 +271,31 @@ def _fake(v: str, w: bool, assert "corpus_x.jsonl (1)" in out -def _tree_v2_row(name: str, order: str) -> dict: +def _tree_v2_row(name: str, order: str | None) -> dict: """The tree's own v2 reading of `name` under `order`, built the same way main()'s tree side and the worker template's _v2_row both build it. Used to fabricate a baseline row that agrees (or, with a field mutated, disagrees) with the tree, without needing a real - baseline wheel.""" - from nameparser import Parser, Policy - import nameparser as _np - p = Parser(policy=Policy(name_order=getattr(_np, order))).parse(name) + baseline wheel. + + `order` is None for the DEFAULT order -- the comparison that + declares no name_order at all, so it reads through `parse()` + rather than a Policy-bearing Parser. One builder covering both + branches rather than two hand copies, for the reason the three + row-building copies in compare.py carry a comment about: a + duplicate drifts the moment a field is added, and #484 added one. + """ + if order is None: + from nameparser import parse + p = parse(name) + else: + from nameparser import Parser, Policy + import nameparser as _np + p = Parser(policy=Policy(name_order=getattr(_np, order))).parse(name) row = {f: (getattr(p, f, "") or "") for f in compare.V2_FIELDS} row["_ambiguities"] = sorted( {a.kind.name for a in getattr(p, "ambiguities", ())}) + row["_initials"] = p.initials() or "" return row @@ -436,6 +476,74 @@ def _fake(v: str, w: bool, return code, buf.getvalue() +def _order_bearing_initials_run( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch, + ledger_body: str) -> tuple[int, str]: + """_order_bearing_run's `_initials` twin: the same shape-4 entry, + with the CORE's initials moved and every role left alone. + + A second helper rather than a parameter on the first, because the + two differ in what they are about: that one builds a role diff, and + this one builds the diff main() only ever forms when no role moved + at all.""" + import contextlib + import io + import json as _json + import sys + name = "Ménil Christophe du" + corpus = tmp_path / "corpus_x.jsonl" + corpus.write_text( + _json.dumps({"name": name, "shape": 4}, ensure_ascii=False) + "\n", + encoding="utf-8") + (tmp_path / "expected_since_2.0.0.toml").write_text( + ledger_body, encoding="utf-8") + monkeypatch.setitem(compare._CORPUS_FLOORS, corpus.name, 1) + monkeypatch.setitem(compare._CORPUS_TIERS, corpus.name, "contract") + monkeypatch.setattr(compare, "HERE", tmp_path) + v2_row = _tree_v2_row(name, "FAMILY_FIRST") + v2_row["_initials"] = "M. X." # the view moved; the roles did not + + def _fake(v: str, w: bool, + entries: list[dict[str, object]]) -> tuple[dict, list[dict]]: + return ({"__version__": v, + "__file__": "/wheel/nameparser/__init__.py"}, + [{"v2": v2_row}]) + + monkeypatch.setattr(compare, "_run_worker", _fake) + monkeypatch.setattr(sys, "argv", ["compare.py", "--baseline", "2.0.0", + "--corpus", str(corpus)]) + buf = io.StringIO() + with contextlib.redirect_stdout(buf): + code = compare.main() + return code, buf.getvalue() + + +def test_an_order_bearing_initials_only_diff_reports_without_a_surface_tag( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None: + """An order-bearing entry never consults the facade -- main() + leaves `new` empty for it -- so the facade pair is ('', ''), + `facade_moved` is False, and the v2 `_initials` line falls to the + ORDER-AWARE branch, which suppresses the tag. A "[v2 surface only]" + here would be a lie: no facade reading was compared to be `only` + different from. + + This is also the one path on which an `_initials` diff can reach + the report at all under a declared order, which is why every + `_initials` ledger rule carries orders = ["DEFAULT"]: such a diff + is the CORE's, and no rule whose prose says "facade" may absorb + it.""" + code, out = _order_bearing_initials_run( + tmp_path, monkeypatch, + '[[change]]\nissue = "unrelated"\nname_regex = "ZZZ"\n' + 'fields = ["family"]\n') + assert code == 1 + assert "UNEXPLAINED 'Ménil Christophe du' [order: FAMILY_FIRST]" in out + # 'C. M.' and not 'M. C. d.': the tree side is read under + # FAMILY_FIRST, where 'Ménil' is the family and 'du' its particle + assert "_initials: 'M. X.' -> 'C. M.'" in out + assert "[v2 surface" not in out + + def test_an_order_blind_rule_absorbing_an_order_bearing_diff_is_reported( tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None: """The leak this notice makes visible runs the OTHER way from the @@ -577,6 +685,56 @@ def test_the_worker_reads_an_order_bearing_line_as_the_tree_does() -> None: assert "facade" not in row +def test_the_worker_reads_a_default_order_line_as_the_tree_does() -> None: + """The worker's FACADE row, RUN rather than compiled -- the third + copy of the row-building code, pinned behaviorally. + + The other two copies (main()'s tree side, the template's _v2_row) + are exercised by every main() test; this one crosses a process + boundary in a real run and so is reachable only by executing the + template. test_worker_source_emits_initials_on_both_surfaces + pins its TEXT, which cannot see a row that builds the right keys + from the wrong parse -- and #484 added a key to exactly this row. + + The name is chosen to tell the two SURFACES apart. `Ph. D., John` + is the one corpus name whose facade and core initials differ today + -- 'J. P D.' against 'J. P. D.', the phd-merge element grouped one + way by HumanName and another by parse() -- so the facade assertion + below can no longer pass by reading the core's value into the + facade row. Under a name the two surfaces agree on (this test used + `Ménil Christophe du`, 'M. C. d.' both ways) that swap is + invisible. + + exec'd in-process for the same reason as its order-bearing + sibling: `import nameparser` then resolves to this checkout, so + the row is the tree's own reading and can be compared against the + tree's own HumanName and parse().""" + import contextlib + import io + import json as _json + import sys + import nameparser + from nameparser import HumanName + name = "Ph. D., John" + source = compare._worker_source(nameparser.__version__, want_v2=True) + stdin = io.StringIO(_json.dumps( + {"name": name, "order": None}, ensure_ascii=False) + "\n") + real_stdin, buf = sys.stdin, io.StringIO() + try: + sys.stdin = stdin + with contextlib.redirect_stdout(buf): + exec(compile(source, "baseline_worker.py", "exec"), {}) + finally: + sys.stdin = real_stdin + tell, row = (_json.loads(line) for line in buf.getvalue().splitlines()) + assert tell["__version__"] == nameparser.__version__ + assert row["facade"] == { + **{k: v or "" for k, v in HumanName(name).as_dict().items() + if k in compare.FIELDS}, + "_initials": HumanName(name).initials() or ""} + assert row["v2"] == _tree_v2_row(name, None) + + def test_dormancy_diagnoses_a_reverted_scoped_rule_as_reverted() -> None: """An order-scoped rule that stops explaining anything has had its behavior reverted, and must say so even when another rule explains @@ -764,8 +922,9 @@ def test_a_rule_with_a_regex_and_no_fields_is_rejected() -> None: A rule with no `fields` narrows by name and by nothing else, so on any name its regex reaches it claims every diff shape there is -- - measured, every one of the 255 shapes - eight roles allow at a 2.x baseline. #452 made that worse than it looks by giving + measured, every one of the 256 shapes the seven roles, + `_ambiguities` and the standalone `_initials` allow at a 2.x + baseline. #452 made that worse than it looks by giving the shape a second job: over_declared_rules skips a rule with no `fields`, correctly, since one declaring no roles cannot over-declare them. So deleting the `fields` line is the response to @@ -961,9 +1120,12 @@ def test_v2_fields_matches_the_Role_enum() -> None: ({"issue": "x", "name_regex": ".+"}, "matches every one of"), ({"issue": "x", "name_regex": r"\b"}, "matches every one of"), ({"issue": "x", "name_regex": r"[\s\S]"}, "matches every one of"), - # seven roles without _ambiguities: below baseline 2.0 that IS the - # whole vocabulary, so it claims every diff. name_regex is along - # for the ride so this pins the roles check, not the #451 one. + # all seven roles: the roles are the only names that ever co-occur + # in one diff -- _ambiguities cannot appear below baseline 2.0 and + # _initials only ever appears alone -- so listing all seven is + # already the widest a rule can be, and it claims every role diff. + # name_regex is along for the ride so this pins the roles check, + # not the #451 one. ({"issue": "x", "name_regex": "Smith", "fields": ["title", "given", "middle", "family", "suffix", "nickname", "maiden"]}, @@ -971,6 +1133,12 @@ def test_v2_fields_matches_the_Role_enum() -> None: # uncompilable: without this it raises mid-run, after the worker ({"issue": "x", "name_regex": "Smith("}, "invalid 'name_regex'"), ({"issue": "x", "name_regex": "Smith", "fields": []}, "empty 'fields'"), + # a repeated name is a copy-paste slip the set-based subset test + # would swallow; refused so it cannot masquerade as a narrowing + ({"issue": "x", "name_regex": "Smith", "fields": ["family", "family"]}, + "repeats"), + ({"issue": "x", "name_regex": "Smith", + "fields": ["_initials", "_initials"]}, "repeats"), ({"issue": "x", "name_regex": "Smith", "fields": ["famly"]}, "not roles"), # facade vocabulary is not role vocabulary; it would never match @@ -1058,6 +1226,37 @@ def test_ambiguities_is_a_legal_field_name() -> None: "ledger.toml") +def test_initials_is_a_legal_field_name_alone() -> None: + """#484: the derived-view pseudo-field, legal by itself.""" + compare.validate_rules( + [{"issue": "x", "name_regex": "Smith", "fields": ["_initials"]}], + "ledger.toml") + compare.validate_exclusions( + [{"why": "x", "name_regex": "Smith", "examples": ["John Smith"], + "fields": ["_initials"]}], "ledger.toml") + + +@pytest.mark.parametrize("fields", [ + ["_initials", "family"], + ["family", "_initials"], + ["_initials", "_ambiguities"], +]) +def test_initials_mixed_with_another_field_is_rejected( + fields: list[str]) -> None: + """main() adds `_initials` to a diff only when nothing else moved, + so a rule declaring it beside a role can never match on it: the + half is dead, and dead rule text is the shape every other check in + validate_rules exists to refuse.""" + with pytest.raises(SystemExit, match="silently dead"): + compare.validate_rules( + [{"issue": "x", "name_regex": "Smith", "fields": fields}], + "ledger.toml") + with pytest.raises(SystemExit, match="silently dead"): + compare.validate_exclusions( + [{"why": "x", "name_regex": "Smith", "examples": ["John Smith"], + "fields": fields}], "ledger.toml") + + #: What _run_worker was asked for, so a test can prove main forwarded #: the baseline and the corpus rather than defaults of its own. _WORKER_CALL: dict = {} @@ -1093,11 +1292,50 @@ def _run_main(tmp_path: Path, monkeypatch: pytest.MonkeyPatch, ledger_body: str, "\n".join(json.dumps(n) for n in names) + "\n", encoding="utf-8") (tmp_path / f"expected_since_{baseline}.toml").write_text( ledger_body, encoding="utf-8") - rows: list[dict] = [{"facade": baseline_facade}] + # Copies, not the caller's dicts: the `_initials` default below + # writes into every row, and the fixtures are module-level + # constants (_SAME_FACADE, _DIFFERS, _SAME_V2). Mutating them in + # place would leak one test's baseline into the next. + rows: list[dict] = [{"facade": dict(baseline_facade)}] if baseline_v2 is not None: - rows[0]["v2"] = baseline_v2 + rows[0]["v2"] = dict(baseline_v2) for _, facade in (extra or ()): - rows.append({"facade": facade}) + rows.append({"facade": dict(facade)}) + # #484: the tree emits `_initials` on every row and compares it + # whenever the roles agree. A fixture row that omitted the key + # would read as an initials diff against the tree's own initials, + # so an omitted key means "same as the tree"; a test that wants an + # initials diff writes the key explicitly. + # + # Which makes a MISSPELLED key the dangerous one, and the reason + # for the refusal below: `{**_SAME_FACADE, "_initals": "J. X."}` + # is a row main() never reads the stray key from, so the real + # `_initials` defaults to the tree's own answer, the surfaces + # agree, and the test passes while pinning nothing. Refuse the row + # instead. Checked BEFORE the setdefault so the message can name + # the key the caller wrote rather than the one the helper added. + _facade_keys = set(compare.FIELDS) | {"_initials"} + _v2_keys = set(compare.V2_FIELDS) | {"_ambiguities", "_initials"} + for n, row in zip(names, rows): + for which, legal in (("facade", _facade_keys), ("v2", _v2_keys)): + if which not in row: + continue + stray = sorted(set(row[which]) - legal) + if stray: + raise AssertionError( + f"fixture row for {n!r} ({which}) carries " + f"{stray}, which main() never reads. The row would " + f"silently agree with the tree on every field it " + f"does read -- a misspelled '_initials' defaults to " + f"the tree's own initials -- so the test would pass " + f"having pinned nothing. Expected keys: " + f"{sorted(legal)}") + from nameparser import HumanName as _HN + for n, row in zip(names, rows): + row["facade"].setdefault("_initials", _HN(n).initials() or "") + if "v2" in row: + from nameparser import parse as _parse + row["v2"].setdefault("_initials", _parse(n).initials() or "") _WORKER_CALL.clear() def _fake(v: str, w: bool, n: list[dict]) -> tuple[dict, list[dict]]: @@ -1187,6 +1425,25 @@ def test_radar_diff_with_no_rule_exits_0_and_is_reported( assert "UNEXPLAINED" not in out +def test_radar_initials_only_diff_prints_its_pseudo_field_line( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None: + """The radar block forwards `initials_only` too. Every other + `_initials` print test goes through the UNEXPLAINED block, and the + two blocks call _print_field_diffs from separate call sites -- so + passing `initials_only=False` at the radar one leaves a + `UNCLASSIFIED (radar)` header with NO field lines under it, which + is a report nobody can act on and which no other test here sees. + + An empty ledger for the same reason as the sibling above: a + ZZZ decoy rule would be dormant in a one-name corpus and exit 1 + for a reason that has nothing to do with the pseudo-field.""" + code, out = _run_main( + tmp_path, monkeypatch, "", _INITIALS_MOVED, tier="radar") + assert code == 0 + assert "UNCLASSIFIED (radar) 'John Smith'" in out + assert "_initials: 'J. X.' -> 'J. S.'" in out + + def test_radar_diff_matching_a_rule_still_classifies( tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None: """Radar names keep feeding the release-note grouping, and a rule @@ -1732,6 +1989,182 @@ def test_check_tree_resolves_before_comparing() -> None: _SAME_V2 = {"title": "", "given": "John", "middle": "", "family": "Smith", "suffix": "", "nickname": "", "maiden": "", "_ambiguities": []} +#: 'John Smith' with every role identical and only the facade's +#: initials moved: the render-layer drift #484 exists to see. +_INITIALS_MOVED = {**_SAME_FACADE, "_initials": "J. X."} + + +def test_main_reports_an_initials_only_diff_under_the_pseudo_field( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None: + """#484: a name whose seven roles agree on both surfaces and whose + initials do not is a diff, reported under `_initials` so the block + can be pasted into a rule.""" + code, out = _run_main( + tmp_path, monkeypatch, + '[[change]]\nissue = "unrelated"\nname_regex = "ZZZ"\n' + 'fields = ["family"]\n', _INITIALS_MOVED) + assert code == 1 + assert "UNEXPLAINED 'John Smith'" in out + assert "_initials: 'J. X.' -> 'J. S.'" in out + + +def test_main_classifies_an_initials_only_diff_by_an_initials_rule( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None: + code, out = _run_main( + tmp_path, monkeypatch, + '[[change]]\nissue = "render-drift"\nname_regex = "Smith"\n' + 'fields = ["_initials"]\n', _INITIALS_MOVED) + assert code == 0 + assert "## render-drift (1)" in out + + +def test_main_drops_initials_from_a_diff_where_a_role_moved( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None: + """The roles-identical guard, pinned as the exact condition. The + rule below declares `family` only; if `_initials` entered the diff + beside the moved role, the subset test would decline it and the + run would exit 1. Delete `not diff and` from the guard and this + is the test that fails.""" + code, out = _run_main( + tmp_path, monkeypatch, + '[[change]]\nissue = "role-only"\nname_regex = "Smith"\n' + 'fields = ["family"]\n', {**_DIFFERS, "_initials": "J. X."}) + assert code == 0 + assert "## role-only (1)" in out + assert "_initials" not in out + + +def test_main_does_not_print_initials_beside_a_moved_role( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None: + """The print half of the guard. An UNEXPLAINED block exists to be + pasted into a rule, and a rule listing `_initials` beside a role + is refused by validate_rules -- so the block must never show the + pair. Pass `initials_only=True` unconditionally (or delete the + `if initials_only:`) in _print_field_diffs and this fails while + the classification test above still passes.""" + code, out = _run_main( + tmp_path, monkeypatch, + '[[change]]\nissue = "unrelated"\nname_regex = "ZZZ"\n' + 'fields = ["family"]\n', {**_DIFFERS, "_initials": "J. X."}) + assert code == 1 + assert "family: 'SMYTHE' -> 'Smith'" in out + assert "_initials" not in out + + +def test_main_reports_a_v2_only_initials_diff_with_the_surface_tag( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None: + """The v2 half of the pseudo-field: the facade agrees and the core + view moved, which is the shape #408 had for a whole minor.""" + code, out = _run_main( + tmp_path, monkeypatch, + '[[change]]\nissue = "unrelated"\nname_regex = "ZZZ"\n' + 'fields = ["family"]\n', + _SAME_FACADE, baseline="2.0.0", + baseline_v2={**_SAME_V2, "_initials": "J. X."}) + assert code == 1 + assert "_initials: 'J. X.' -> 'J. S.' [v2 surface only]" in out + + +def test_main_prints_both_initials_lines_when_the_surfaces_moved_differently( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None: + """The facade's and the core's initials() are independent + implementations, so unlike a role they can move to different + strings; a block that showed only the facade's movement would + have a rule written for it in the belief the core agreed.""" + code, out = _run_main( + tmp_path, monkeypatch, + '[[change]]\nissue = "unrelated"\nname_regex = "ZZZ"\n' + 'fields = ["family"]\n', + {**_SAME_FACADE, "_initials": "J. X."}, baseline="2.0.0", + baseline_v2={**_SAME_V2, "_initials": "J. Y."}) + assert code == 1 + assert "_initials: 'J. X.' -> 'J. S.'" in out + assert "_initials: 'J. Y.' -> 'J. S.' [v2 surface]" in out + assert out.count("_initials:") == 2 + + +def test_main_prints_one_initials_line_when_both_surfaces_moved_alike( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None: + """The print-once convention, kept for the case it was written + for: two surfaces moving to the SAME pair are one movement and one + rule, so the facade line stands for both and carries no surface + tag. + + The mutant this kills is replacing the WHOLE + `(not facade_moved or v2_pair != facade_pair)` guard with a bare + `if v2_moved:` -- then the v2 line prints here too, while the + sibling above still sees its two lines and passes. Deleting only + the pair comparison is the SIBLING's mutant, not this one: it + leaves `v2_moved and not facade_moved`, which is False here, so + this case still prints once and this test cannot see it.""" + code, out = _run_main( + tmp_path, monkeypatch, + '[[change]]\nissue = "unrelated"\nname_regex = "ZZZ"\n' + 'fields = ["family"]\n', + {**_SAME_FACADE, "_initials": "J. X."}, baseline="2.0.0", + baseline_v2={**_SAME_V2, "_initials": "J. X."}) + assert code == 1 + assert "_initials: 'J. X.' -> 'J. S.'" in out + assert out.count("_initials:") == 1 + assert "[v2 surface" not in out + + +def test_main_keeps_initials_out_of_a_diff_a_V2_role_moved( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None: + """The v2 half of the roles-identical guard. Its facade half is + pinned above (test_main_drops_initials_from_a_diff_where_a_role_ + moved); this one moves the role on the CORE surface only, where the + guard reads a different dict. The rule declares `family` alone, so + an `_initials` that leaked into the diff beside it would fail the + subset test and exit 1.""" + code, out = _run_main( + tmp_path, monkeypatch, + '[[change]]\nissue = "role-only"\nname_regex = "Smith"\n' + 'fields = ["family"]\n', + _SAME_FACADE, baseline="2.0.0", + baseline_v2={**_SAME_V2, "family": "SMYTHE", "_initials": "J. X."}) + assert code == 0 + assert "## role-only (1)" in out + assert "_initials" not in out + + +def test_main_prints_one_initials_line_when_only_the_facade_moved( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None: + """The facade moved and the core, compared, AGREED -- the reading + the docstring of _print_field_diffs warns a bare facade line does + NOT license, and the case that makes the warning necessary. Both + surfaces were consulted here; the v2 line is suppressed because + the core did not move, not because it was never asked. + + Distinct from test_main_reports_an_initials_only_diff_under_the_ + pseudo_field, which runs at 1.4.0 where there is no core surface + to agree.""" + code, out = _run_main( + tmp_path, monkeypatch, + '[[change]]\nissue = "unrelated"\nname_regex = "ZZZ"\n' + 'fields = ["family"]\n', + {**_SAME_FACADE, "_initials": "J. X."}, baseline="2.0.0", + baseline_v2=dict(_SAME_V2)) + assert code == 1 + assert "_initials: 'J. X.' -> 'J. S.'" in out + assert out.count("_initials:") == 1 + assert "[v2 surface" not in out + + +def test_run_main_refuses_a_fixture_row_with_an_unknown_key( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None: + """The fixture's own guard, and the reason it exists: main() reads + `_initials` off the row and the helper defaults a MISSING one to + the tree's own answer, so a misspelled key is not a loud failure + but a silent agreement -- the test passes having compared the tree + against itself. Refuse the row instead of letting it pass.""" + with pytest.raises(AssertionError, match="_initals"): + _run_main( + tmp_path, monkeypatch, + '[[change]]\nissue = "unrelated"\nname_regex = "ZZZ"\n' + 'fields = ["family"]\n', + {**_SAME_FACADE, "_initals": "J. X."}) + def test_main_compares_the_v2_surface_from_baseline_2_0( tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None: @@ -1926,6 +2359,8 @@ def test_classify_refuses_an_excluded_shape() -> None: "not a list of strings"), ({"why": "x", "name_regex": "a", "examples": ["a"], "fields": []}, "empty 'fields'"), + ({"why": "x", "name_regex": "a", "examples": ["a"], + "fields": ["_initials", "_initials"]}, "repeats"), ({"why": "x", "name_regex": "a", "examples": ["a"], "fields": ["nope"]}, "not roles"), # the facade's vocabulary is not the role vocabulary diff --git a/tests/v2/test_ledger_guards.py b/tests/v2/test_ledger_guards.py index 127e1467..3f611bfb 100644 --- a/tests/v2/test_ledger_guards.py +++ b/tests/v2/test_ledger_guards.py @@ -44,6 +44,7 @@ class declares, which members an alternation offers. Those are exact # hand copy of a constant with a source of truth, inside the module # written to forbid exactly that. from nameparser._lexicon import _PHRASE_FIELDS, _normalize +from nameparser.config.bound_given_names import BOUND_GIVEN_NAMES from nameparser.config.conjunctions import CONJUNCTIONS from nameparser.config.maiden_markers import MAIDEN_MARKERS from nameparser.config.particles import PARTICLES @@ -353,7 +354,9 @@ def test_script_ranges_membership_is_decided() -> None: "fix(#298)", # the 间隔号 lookahead }), "expected_since_2.1.0.toml": frozenset(), # 2.2 cycle: no span-bearing rule - "expected_since_2.2.0.toml": frozenset(), # open cycle, no rules yet + # open cycle: its one rule, fix(#462), is a Latin letter shape and + # copies no script range + "expected_since_2.2.0.toml": frozenset(), } #: The leading `fix(...)`/`feat(...)` tag of a rule's `issue`, which is @@ -835,6 +838,24 @@ def test_case_shape_ids_exist_in_the_inventory() -> None: # acronym rule above as well, since 'ma' is acronym vocabulary too. "fix(suffix-routing) the dotted M.A. spelling reads as a credential (ma-do)": ("Jack Ma", "Jack MA", "John Smith M.A."), + # #484: the connective rule is case-sensitive on the single letters + # so that the #462 shapes -- a capital or dotted E that is an + # INITIAL the facade drops -- are never claimed as the per-word + # grouping change. Since the facade fix these names AGREE with + # 1.4.0 and diff only against the 2.x baselines, where fix(#462) + # claims them, so a #462 REGRESSION is the only way they can diff + # at this baseline again -- and it must surface as UNEXPLAINED + # rather than be absorbed as per-word grouping. That is what the + # case-sensitivity buys, and it is why this roster keeps probing + # for it after the bug is gone. + "a connective run initials": ("Jose E Maria Santos", + "JOSE E MARIA SANTOS", + "Scott E. Werner", "Amy E Maid"), + # fix(#462)'s boundary: lowercase bare e/y is the connective; + # 'E.T.' is a run of initials the rule has no view on; a bare I + # is not conjunction vocabulary at all. + "fix(#462)": ("john e smith", "maria y lopez", "E.T. Smith", + "Maier, Amy I, Jr."), } @@ -1351,6 +1372,27 @@ class _LatinCopy(NamedTuple): "fix(suffix-routing) a two-token name ending in a credential acronym keeps it in `suffix`": _LatinCopy(vocabulary=SUFFIX_ACRONYMS, covers=frozenset({"mc", "mp"})), + # #484's per-word rules copy one vocabulary EACH, which is why the + # 98 names take three rules: the roster keys a rule to a single + # source. B1 is case-sensitive on purpose (its comment says why), + # so both spellings of `and` are members and cover one entry. + "a connective run initials": _LatinCopy( + vocabulary=CONJUNCTIONS, + covers=frozenset({"y", "e", "&", "and", "of", "the"})), + "a bound-given run initials": _LatinCopy( + vocabulary=BOUND_GIVEN_NAMES, + covers=frozenset({"abdul", "abu"})), + # Three entries, not the five the chain names: this roster keys ONE + # vocabulary per rule, and the rule's first draft carried a second + # alternation (`(?:\s+(?:der|la))?`) that this test would have had + # to pin against the same entry. It could not: `covers` is one set, + # and the two alternations cover disjoint halves of it. The draft + # group was inert -- same 107 names, same digest, with and without + # it -- so the ledger dropped it rather than the roster growing a + # second shape. The rule's comment records the measurement. + "a particle chain inside": _LatinCopy( + vocabulary=PARTICLES, + covers=frozenset({"van", "von", "de"})), } #: Alternations that copy no vocabulary, so discovery must not demand a @@ -1416,6 +1458,30 @@ class _LatinCopy(NamedTuple): # here stands behind. This roster records only that a wordlist is # not what is copied. frozenset({"X", "IX", "IV", "V?I{1,3}", "V"}), + # #484: the anchor groups of the `_initials` rules -- start-of-name + # or whitespace, start-of-name or whitespace-or-comma. Anchors, not + # words; the same reason as fix(#400)'s pair above. + frozenset({"^", "\\s"}), + frozenset({"^", "[\\s,]"}), + # fix(#385/#402)'s 24 spellings: the 27 corpus names whose + # all-particle part moved, listed because "a part of nothing but + # particles" is not a property a regex over the raw string can + # state. A list of names, the fix(#445) precedent -- that rule and + # fix(#410) and fix(#335) are the 1.4.0 ledger's literal lists. + frozenset({"anh do", "smith van der", "yin le", "yin a le", "vai la", + "jong van der", "jong, van der", "juan van der", + "mesnil garcia de", "mesnil garcia van", "mesnil de", + "sander van", "van ma van", "anh van do", + "beethoven ludwig van", "berg jan de jr\\.", "john van mc", + "jong anke de", "juan de", "ménil christophe de", + "ménil de", "nguyen thi van", "nguyen, van le", + "van berg jan de"}), + # fix(#462)'s letter shape: a bare capital E/Y or a dotted E./Y. + # It is the initial SHAPE (v1's `initial` regex, _render._INITIAL) + # intersected with the single-letter conjunctions, not a copy of + # CONJUNCTIONS -- "e." is no entry, and matching it against the + # vocabulary would be a false claim of correspondence. + frozenset({"[EY]", "[EeYy]\\."}), }) def _unjustified_reach(name_regex: str, members: set[str]) -> list[str]: @@ -1979,6 +2045,33 @@ def _claim(rule: dict) -> _Claim: _Claim(2, ('family', 'suffix'), "ed72c9672214", None), "fix(suffix-routing) the dotted M.A. spelling reads as a credential (ma-do)": _Claim(1, ('family', 'suffix'), "17379620526b", None), + # #484's six `_initials` rules. Four of them reach far more + # than they explain, and the gap is the pseudo-field's own + # doing rather than a widening: `_initials` enters a diff ONLY + # when the seven roles and the ambiguity kinds all agree, so a + # name whose regex the rule matches contributes nothing unless + # its parse is otherwise identical across the two surfaces. + # Reach against explained, measured 2026-09-02 at baseline + # 1.4.0 -- a snapshot of that run, not a standing count: + # 27/27, 1/1, 96/66, 41/19, 107/11, 18/2. The reach half is + # what this roster holds; the explained half moves with the + # corpus and is re-read from the gate. The phd rule's 18 is the widest + # gap and the most literal regex -- `\bph\. d\.` matches every + # spelling of the fragment the corpora carry, and the trailing + # ones are protected by the [[never]] entry above, which is + # what _EXCLUSION_EFFECT's grown `absorbed_by` records. + "fix(#385/#402) an all-particle name part initials its words (R2)": + _Claim(27, ('_initials',), "6b242c287db8", ('DEFAULT',)), + "fix(#360) los joined the particles, so it no longer initials": + _Claim(1, ('_initials',), "cd721215f463", ('DEFAULT',)), + "fix(initials-per-word) a connective run initials each word (facade, since 2.0.0)": + _Claim(96, ('_initials',), "fa69850d2cd4", ('DEFAULT',)), + "fix(initials-per-word) a bound-given run initials each word (facade, since 2.0.0)": + _Claim(41, ('_initials',), "e99f56c955d5", ('DEFAULT',)), + "fix(initials-per-word) a particle chain inside a name part initials each word (facade, since 2.0.0)": + _Claim(107, ('_initials',), "bdc4da864f59", ('DEFAULT',)), + "fix(initials-per-word) the Ph. D. merge initials each word (facade, since 2.0.0)": + _Claim(18, ('_initials',), "f67d8ebddd56", ('DEFAULT',)), }, "expected_since_2.0.0.toml": { "fix(#335) a marker-led clause leaves the one name word its bare reading": @@ -2107,8 +2200,35 @@ def _claim(rule: dict) -> _Claim: _Claim(1, ('family', 'given', 'middle', 'suffix', 'title'), "fc6bc9e605e1", ('FAMILY_FIRST',)), "feat(#395)/fix(#296) a comma followed only by a title leaves the pre-comma name to the declared order's fold, the given-last spelling": _Claim(1, ('family', 'given', 'middle', 'suffix', 'title'), "3e43a2be022e", ('FAMILY_FIRST_GIVEN_LAST',)), + # #484's two `_initials` rules. Both are literal name lists, so + # reach equals what they explain here -- 27 and 1 -- and both + # digests match the 1.4.0 ledger's, which is the point of + # copying the list verbatim rather than restating it. + "fix(#385/#402) an all-particle name part initials its words (R2)": + _Claim(27, ('_initials',), "6b242c287db8", ('DEFAULT',)), + "fix(#360) los joined the particles, so it no longer initials": + _Claim(1, ('_initials',), "cd721215f463", ('DEFAULT',)), + # fix(#462)'s reach is 18 where it explains 14. Its regex is a + # letter SHAPE rather than a name list, and the four extra -- + # 'E Anne D', 'E Jones', 'E Maria', 'Y. L.' -- carry the E/Y in + # the GIVEN group, which has always initialed every word it + # holds whatever the vocabulary says, so the fix does not move + # them. The discriminator is the GROUP and not the position: + # 'E Anne D,Leonardo' also leads with the E, but its comma + # re-roles the whole run into the FAMILY group, and it is one + # of the 14 that move. + # The digest is the same in all three 2.x ledgers because the + # regex is the same string in each. + "fix(#462) the facade keeps an initial-shaped conjunction letter": + _Claim(18, ('_initials',), "3dd0e0276be6", ('DEFAULT',)), + }, + # The 2.3 cycle's first rule, and a facade-only render fix: every + # role is identical, so `_initials` alone. Reach and digest as in + # the 2.0.0 mapping above, the same regex classifying the same 14. + "expected_since_2.2.0.toml": { + "fix(#462) the facade keeps an initial-shaped conjunction letter": + _Claim(18, ('_initials',), "3dd0e0276be6", ('DEFAULT',)), }, - "expected_since_2.2.0.toml": {}, # open cycle, no rules yet "expected_since_2.1.0.toml": { "fix(#371) a suffix never begins a name: the Ph./D. merge declines at the head": _Claim(4, ('family', 'given', 'middle', 'suffix', 'title'), "1425d85a2d86", None), @@ -2222,6 +2342,19 @@ def _claim(rule: dict) -> _Claim: _Claim(1, ('family', 'given', 'middle', 'suffix', 'title'), "fc6bc9e605e1", ('FAMILY_FIRST',)), "feat(#395)/fix(#296) a comma followed only by a title leaves the pre-comma name to the declared order's fold, the given-last spelling": _Claim(1, ('family', 'given', 'middle', 'suffix', 'title'), "3e43a2be022e", ('FAMILY_FIRST_GIVEN_LAST',)), + # #484's two, the same pair as the 2.0.0 ledger's: the change + # shipped in 2.2.0, so it is equally visible from either 2.x + # baseline, and the reaches and digests agree because the two + # files carry the same literal list. + "fix(#385/#402) an all-particle name part initials its words (R2)": + _Claim(27, ('_initials',), "6b242c287db8", ('DEFAULT',)), + "fix(#360) los joined the particles, so it no longer initials": + _Claim(1, ('_initials',), "cd721215f463", ('DEFAULT',)), + # fix(#462), reach and digest as in the 2.0.0 mapping: the same + # regex over the same corpora, and the facade bug it fixes is + # in every 2.x wheel, so the baseline makes no difference. + "fix(#462) the facade keeps an initial-shaped conjunction letter": + _Claim(18, ('_initials',), "3dd0e0276be6", ('DEFAULT',)), }, } @@ -2320,7 +2453,8 @@ def test_every_rule_claims_the_recorded_share_of_the_corpus() -> None: #: recorded winner is order-scoped, and it stops being true the day one #: is. _CROSS_RULE_WINNERS: dict[str, dict[tuple[str, tuple[str, ...]], str]] = { - "expected_since_2.2.0.toml": {}, # open cycle, no rules and so no contest + # open cycle: one rule, so nothing for a second one to contest + "expected_since_2.2.0.toml": {}, "expected_since_1.4.0.toml": { ("Andrews, M.D.", ("given", "suffix")): "fix(comma-family)", ("田中, 太郎さん", ("given", "suffix")): "fix(cjk-comma-honorific-peel)", @@ -2376,6 +2510,21 @@ def test_every_rule_claims_the_recorded_share_of_the_corpus() -> None: ("田中さん II", ("given", "suffix")): "fix(cjk-glued-honorific-peel) glued honorific peels into " "suffix", + # #484's three `_initials` contests with a literal rule on one + # side. fix(initials-per-word)'s particle-chain rule reaches all + # three, declares the same lone `_initials` field, and loses + # only because both literal rules are written ahead of it -- + # file order, which _sorted_rules leaves untouched now that + # every rule carries a name_regex. Each of the three really does + # diff on initials, so narrowing or deleting a literal rule + # would hand its name to the per-word rule rather than surface + # it: the handover this row exists to catch. + ("de los Santos", ("_initials",)): + "fix(#360) los joined the particles, so it no longer initials", + ("van Berg Jan de", ("_initials",)): + "fix(#385/#402) an all-particle name part initials its words (R2)", + ("van ma van", ("_initials",)): + "fix(#385/#402) an all-particle name part initials its words (R2)", # the glued/spaced boundary. 'Andersonさん' and '김민준씨' left # suffix-routing for a rule that names them; '김민준 씨.' is # spaced and stays on the spaced rule, which #372 taught to @@ -2437,6 +2586,34 @@ def test_every_rule_claims_the_recorded_share_of_the_corpus() -> None: "fix(#296) a lone post-comma credential is a suffix", ("김민준씨 Jr.", ("family", "given", "suffix")): "fix(cjk-glued-honorific-peel) glued honorific peels into suffix", + # #484's per-word rules. Measured over the corpus, exactly ONE + # name is reached by two of them AND diffs on `_initials`: + # 'de la Vega y Santos Juan' carries both a connective and a + # particle chain, and the connective rule wins it on file + # order alone -- the two declare the same single field, so + # `fields` cannot separate them and only position does. Five + # more names are reached by two ('Jane van der Berg née y + # Jones', 'Maria Luisa y de la Cruz', 'Sir abdul van der + # Berg', 'abdul Ph. D. Smith Berg', 'van der Berg, abdul née + # Jones') and none of them has an `_initials` diff to pin: + # three move ROLES, so the pseudo-field never enters, and two + # do not diff at all. + ("de la Vega y Santos Juan", ("_initials",)): + "fix(initials-per-word) a connective run", + # Not contested today -- the bound-given rule is the only one + # of the four whose regex reaches these three -- and recorded + # anyway, because the bound rule sits BEHIND the connective + # one in file order and the two vocabularies are one widened + # alternation apart. A connective rule grown to reach 'Abu' + # or a particle rule grown to reach a trailing 'van' takes + # these names silently: reach is per-rule and the gate's total + # is per-corpus, so nothing else here would say so. + ("Abu Bakr Al Baghdadi, MD", ("_initials",)): + "fix(initials-per-word) a bound-given run", + ("abu bakr al baghdadi", ("_initials",)): + "fix(initials-per-word) a bound-given run", + ("Berg, abdul van", ("_initials",)): + "fix(initials-per-word) a bound-given run", }, # The two 2.x ledgers had NO section here until #452, and the # coverage assertion below was `<=`, so their absence read as "no @@ -2655,10 +2832,37 @@ class _Excluded(NamedTuple): # -- any of the siblings could widen onto a protected # reading and this roster would stay green. The same # identity-free weakness _SPAN_BEARING_RULES records. + # + # #484's phd-merge initials rule JOINED the tuple, + # and it is the first entrant that reaches a + # protected shape through the `_initials` + # pseudo-field rather than through a role: its + # `fields` is exactly ["_initials"], so the only + # subset it can claim is the singleton, and + # _protectable_fields builds that subset because it + # reads compare._RULE_FIELDS rather than Role. + # + # Decided rather than absorbed. The rule's regex is + # the bare fragment `\bph\. d\.`, which reaches all + # 18 corpus spellings while explaining the two + # LEADING ones it is named for; a trailing 'Ph. D.' + # is what this entry protects, and the exclusion + # refuses those names before any rule is consulted, + # so the rule claims nothing there in a real run. + # What the growth records is that it now STANDS + # READY to -- delete this entry and the initials + # reading of a trailing 'Ph. D.' would classify + # under a rule whose prose is about the merge at + # the front of a name. Narrowing the regex to the + # leading spelling would empty this tuple again and + # is the alternative on the table if the exclusion + # is ever retired. ("fix(comma-family) lone post-comma piece routes to " "suffix/title, not first", "fix(comma-precomma-family) pre-comma run reads as " - "family, not given")), + "family, not given", + "fix(initials-per-word) the Ph. D. merge initials " + "each word (facade, since 2.0.0)")), '(^|[\\w.]\\s+)[("\'][^)"\']+[)"\'](\\s+\\w|\\s*$)': # 51 -> 54 as rules.md gained the bracketed Polish examples # (#434): 'Maria Kowalska (z domu Nowak)', 'Maria Kowalska diff --git a/tools/differential/README.md b/tools/differential/README.md index 7c4e76a6..ef56168b 100644 --- a/tools/differential/README.md +++ b/tools/differential/README.md @@ -69,13 +69,21 @@ prints it on a `tree:` line — see the third trap below for why a bare import was not enough. A rule's `fields` names roles the way `Role` does, whichever surface -the diff came from, plus the pseudo-field `_ambiguities` for a change -in reported `AmbiguityKind`s. The roster is not restated here: it is -`Role`'s members, `validate_rules` rejects anything outside them, and -a copy in prose is a copy that goes stale when a role is added. The -facade reports `first`/`last`; those are canonicalized on the way in, -and the `UNEXPLAINED` block prints the canonical name so what you read -is what you write. +the diff came from, plus two pseudo-fields. `_ambiguities` carries a +change in reported `AmbiguityKind`s. `_initials` (#484) carries a +change in `initials()` -- the facade's at every baseline, the core's +from 2.0 on -- and enters a name's diff ONLY when every role and the +ambiguity kinds agree on every compared surface: a role move drags +its initials with it, so that movement is the role diff's consequence +and is neither compared nor printed, while an initials change with +the fields identical is render-layer drift the field comparison +cannot see. A rule that lists `_initials` therefore lists nothing +else; `validate_rules` refuses the mix as silently dead. The roster is +not restated here: it is `Role`'s members, `validate_rules` rejects +anything outside them, and a copy in prose is a copy that goes stale +when a role is added. The facade reports `first`/`last`; those are +canonicalized on the way in, and the `UNEXPLAINED` block prints the +canonical name so what you read is what you write. ## The three invocation traps @@ -436,12 +444,20 @@ neither (it would match every diff), one carrying `fields` and no `name_regex` (#451 -- no name narrowing, so it claims every name whose diff fits its roles), and one carrying `name_regex` and no `fields` (#456 -- no role narrowing, so on any name its regex reaches it claims -every diff shape there is, measured, 255 of them from baseline 2.0 on and 127 below it). The three are one +every diff shape there is, measured, 256 of them from baseline 2.0 on and 128 below it). The three are one rule with one reason: a rule narrows by name AND by role, or it is not a rule. Note the two bans are each other's obvious wrong answer -- deleting `fields` to silence an over-declaration failure lands on #456's, and adding `fields` while dropping the regex lands on #451's. +**`fields = ["_initials"]` is a rule of its own kind** (#484). It +classifies a change in the derived `initials()` view on a name whose +seven roles did not move, and nothing else -- `main()` never puts +`_initials` into a diff beside a role, so a rule mixing the two is +refused at startup. Such rules close the 1.4.0 ledger, and each 2.x +ledger carries the subset visible from its baseline; their comments +say which view change each one names. + **`orders` is the optional third narrowing** (#468). A rule may carry `orders = ["FAMILY_FIRST", ...]` -- public order-constant names, taken from the ones `shapes.py` declares, so `validate_rules` rejects a name diff --git a/tools/differential/compare.py b/tools/differential/compare.py index 4e86872e..e04c9159 100644 --- a/tools/differential/compare.py +++ b/tools/differential/compare.py @@ -44,13 +44,19 @@ _V1_TO_ROLE = {"first": "given", "last": "family"} #: An unclassified diff, carrying BOTH surfaces' before/after: -#: (name, old_facade, new_facade, old_v2, new_v2, order). Both halves -#: are kept because a diff can exist on the v2 surface alone, and a -#: report that named such a diff without showing it would be -#: unactionable. `order` rides along so the report can say which -#: order produced the diff -- None for the default order. +#: (name, old_facade, new_facade, old_v2, new_v2, order, +#: initials_only). Both halves are kept because a diff can exist on +#: the v2 surface alone, and a report that named such a diff without +#: showing it would be unactionable. `order` rides along so the report +#: can say which order produced the diff -- None for the default +#: order. `initials_only` is main()'s own verdict on the diff set +#: (#484): it rides along so _print_field_diffs can be TOLD whether +#: the derived view belongs in the block rather than re-deriving it +#: from the rows, which is the shape two copies of one rule take just +#: before they drift. _Unexplained = tuple[str, dict[str, str], dict[str, str], - dict[str, object], dict[str, object], str | None] + dict[str, object], dict[str, object], str | None, + bool] def _parse_version(text: str) -> tuple[int, int, int]: @@ -209,6 +215,8 @@ def _v2_row(p): row = {f: (getattr(p, f, "") or "") for f in V2_FIELDS} row["_ambiguities"] = sorted( {a.kind.name for a in getattr(p, "ambiguities", ())}) + # #484: a derived view, compared only where the roles agree + row["_initials"] = p.initials() or "" return row @@ -225,9 +233,12 @@ def _v2_row(p): {"v2": _v2_row(_parser_for(order).parse(name))}, ensure_ascii=False), flush=True) continue + hn = HumanName(name) row = {"facade": {k: v or "" - for k, v in HumanName(name).as_dict().items() + for k, v in hn.as_dict().items() if k in V1_FIELDS}} + # #484: HumanName.initials() exists at every baseline, 1.4.0 included + row["facade"]["_initials"] = hn.initials() or "" if WANT_V2: row["v2"] = _v2_row(parse(name)) print(json.dumps(row, ensure_ascii=False), flush=True) @@ -396,7 +407,8 @@ def _order_tag(order: str | None) -> str: def _print_field_diffs(old_facade: dict[str, str], new: dict[str, str], old_v2: dict[str, object], new_v2: dict[str, object], - order: str | None = None) -> None: + order: str | None = None, *, + initials_only: bool) -> None: """Print each moved field under one name, Role's, whichever surface(s) moved it. Shared by the UNEXPLAINED and UNCLASSIFIED (radar) blocks in main() -- one copy rather than two that can @@ -406,8 +418,9 @@ def _print_field_diffs(old_facade: dict[str, str], new: dict[str, str], Role's names, not the facade's: both report blocks exist to be turned into a ledger rule, and a rule naming the facade's `first` is rejected by validate_rules at startup. Both surfaces are - walked, and a field is reported once even when both moved, since - one rule covers it. + walked, and a ROLE is reported once even when both moved, since + one rule covers it. `_initials` is the exception, and the last + paragraph is why. `order` is None for a default-order entry, else the order both surfaces were read under. The "[v2 surface only]" tag means "the @@ -415,6 +428,35 @@ def _print_field_diffs(old_facade: dict[str, str], new: dict[str, str], entry, whose facade is never consulted at all (main() passes empty dicts for it), so the tag is suppressed there rather than printed with a meaning it does not have. + + `initials_only` is main()'s VERDICT on the diff set, not a hint: + the derived `_initials` view is printed only when it is what the + diff consists of. Taking the answer rather than re-deriving it + from the rows is the point -- main()'s roles-identical guard and + this print cannot drift apart, so a block always pastes into a + rule validate_rules accepts (`fields = ["family", "_initials"]` is + refused, and a block showing that pair would invite exactly it). + + The facade's initials() and the core's are INDEPENDENT + implementations -- the facade builds its string from + `_initials_lists`, not from `_render.initials` -- so unlike a role + the two can legitimately move to different strings, and the roles' + print-it-once convention would hide the core's movement behind the + facade's. So the v2 line is suppressed only when it would REPEAT + the facade's: it prints whenever the core moved to a different + (old, new) pair, tagged "[v2 surface]" to say both surfaces were + compared and moved differently. One rule still covers both, the + field name being the same -- but a block showing the facade's + movement alone would have that rule written in the belief the core + agreed. Read a bare facade line for what it says, then: the + absence of a v2 line means the core did not move to a DIFFERENT + pair -- it moved identically, or it did not move at all -- and + never that the core agreed. The tag needs no order check to stay + honest, because `facade_moved` can only be true where the facade + was CONSULTED: main() passes empty dicts for an order-bearing + entry, so the facade pair is ('', '') there, `facade_moved` is + False, and the v2 line falls to the order-aware branch, which + suppresses the tag. """ seen: set[str] = set() for f in FIELDS: @@ -429,6 +471,23 @@ def _print_field_diffs(old_facade: dict[str, str], new: dict[str, str], print(f" {_canonical_field(f)}: " f"{old_v2.get(f, '')!r} -> {new_v2.get(f, '')!r}" f"{tag}") + # #484: the derived view, printed exactly as main() compares it -- + # per surface, facade first, and twice when the two surfaces moved + # DIFFERENTLY, which a role cannot do (see the docstring). + if initials_only: + facade_pair = (old_facade.get("_initials", ""), + new.get("_initials", "")) + v2_pair = (old_v2.get("_initials", ""), new_v2.get("_initials", "")) + facade_moved = facade_pair[0] != facade_pair[1] + v2_moved = v2_pair[0] != v2_pair[1] + if facade_moved: + print(f" _initials: {facade_pair[0]!r} -> {facade_pair[1]!r}") + if v2_moved and (not facade_moved or v2_pair != facade_pair): + if facade_moved: + tag = " [v2 surface]" + else: + tag = "" if order is not None else " [v2 surface only]" + print(f" _initials: {v2_pair[0]!r} -> {v2_pair[1]!r}{tag}") def _is_latin_only(name: str) -> bool: @@ -444,10 +503,14 @@ def _is_latin_only(name: str) -> bool: #: Every legal entry in a rule's `fields`: the seven roles under Role's -#: names, plus the pseudo-field carrying reported AmbiguityKinds. The -#: ambiguity entry is legal and load-bearing -- a SEGMENTATION-only diff -#: is facade-identical, so this is the one name that can classify it. -_RULE_FIELDS = frozenset((*V2_FIELDS, "_ambiguities")) +#: names, plus two pseudo-fields. `_ambiguities` carries reported +#: AmbiguityKinds -- a SEGMENTATION-only diff is facade-identical, so +#: this is the one name that can classify it -- and cannot enter a diff +#: below baseline 2.0. `_initials` (#484) carries the derived +#: initials() view and CAN enter one at 1.4.0, through the facade; it +#: enters only when every role and the ambiguity kinds agree (main()), +#: so a rule listing it lists nothing else (validate_rules). +_RULE_FIELDS = frozenset((*V2_FIELDS, "_ambiguities", "_initials")) _RULE_KEYS = frozenset(("issue", "name_regex", "fields", "dormant", "orders")) @@ -760,16 +823,46 @@ def validate_rules(rules: list[dict[str, object]], ledger: str) -> None: f"roles; expected from {sorted(_RULE_FIELDS)}. A " f"name outside that set never matches, so the rule " f"is silently dead") + # `others` rather than len(fields) > 1: fields = + # ["_initials", "_initials"] is longer than one and yet + # mixes nothing, so the length test fired and printed an + # empty list -- a refusal naming no second field, which is + # the one thing the message exists to name. Nothing here + # rejects a repeated field name, and this check is not the + # place to start: it is about the MIX. + dups = sorted({f for f in fields if fields.count(f) > 1}) + if dups: + raise SystemExit( + f"{where} repeats {dups} in 'fields'. classify() " + f"reads 'fields' as a set, so the repeat changes " + f"nothing it matches -- it is a copy-paste slip that " + f"would otherwise pass every check below silently, " + f"and the '_initials' check in particular would read " + f"['_initials', '_initials'] as '_initials' alone") + others = sorted(set(fields) - {"_initials"}) + if "_initials" in fields and others: + raise SystemExit( + f"{where} lists '_initials' beside " + f"{others} in 'fields'. " + f"'_initials' enters a diff only when every role and " + f"the ambiguity kinds agree (#484), so no diff can " + f"carry it with another field: the '_initials' half " + f"of this rule is silently dead. Give the " + f"initials-only shape its own rule with " + f"fields = ['_initials']") if set(V2_FIELDS) <= set(fields): raise SystemExit( f"{where} lists all seven roles in 'fields', so the " - f"subset test admits every diff -- the narrowing is " - f"not narrowing anything. Checked against the seven " - f"roles rather than against every legal entry, " - f"because '_ambiguities' cannot enter a diff at all " - f"below baseline 2.0: there the seven ARE the whole " - f"vocabulary, and a rule listing them would have " - f"claimed every diff in the 1.4 ledger") + f"subset test admits every ROLE diff -- the " + f"narrowing is not narrowing anything. Checked " + f"against the seven roles rather than against every " + f"legal entry, because the roles are the only names " + f"that ever co-occur in one diff: '_ambiguities' " + f"cannot appear below baseline 2.0, and '_initials' " + f"only ever appears alone (#484's roles-identical " + f"guard in main()). So all seven is already the " + f"widest a rule can be, and that is the widening " + f"this check refuses") # LAST of the family, deliberately. This rejects a # WELL-FORMED `fields` that simply has no name beside it, so it # must not pre-empt the three checks above, each of which buys a @@ -796,8 +889,9 @@ def validate_rules(rules: list[dict[str, object]], ledger: str) -> None: f"{where} has 'name_regex' but no 'fields' (#456). It " f"narrows by name and by nothing else, so on any name " f"its regex reaches it claims EVERY diff shape there is -- " - f"255 of them from baseline 2.0 on, where `_ambiguities` " - f"joins the seven roles, and 127 below it. #452 makes " + f"256 of them from baseline 2.0 on, where `_ambiguities` " + f"joins the seven roles and `_initials` adds the one " + f"shape that stands alone, and 128 below it. #452 makes " f"that worse than it looks: " f"over_declared_rules skips a rule with no 'fields', " f"correctly, since one declaring no roles cannot " @@ -900,6 +994,29 @@ def validate_exclusions(entries: list[dict[str, object]], raise SystemExit( f"{where} names {bad} in 'fields', which are not " f"roles; expected from {sorted(_RULE_FIELDS)}") + # `others` rather than len(fields) > 1, as in + # validate_rules: a repeated '_initials' is longer than + # one and mixes nothing, and printed an empty list. + dups = sorted({f for f in fields if fields.count(f) > 1}) + if dups: + raise SystemExit( + f"{where} repeats {dups} in 'fields'. classify() " + f"reads 'fields' as a set, so the repeat changes " + f"nothing it matches -- it is a copy-paste slip that " + f"would otherwise pass every check below silently, " + f"and the '_initials' check in particular would read " + f"['_initials', '_initials'] as '_initials' alone") + others = sorted(set(fields) - {"_initials"}) + if "_initials" in fields and others: + raise SystemExit( + f"{where} lists '_initials' beside " + f"{others} in 'fields'. " + f"'_initials' enters a diff only when every role and " + f"the ambiguity kinds agree (#484), so no diff can " + f"carry it with another field: the '_initials' half " + f"of this entry is silently dead. Give the " + f"initials-only shape its own exclusion with " + f"fields = ['_initials']") if set(V2_FIELDS) <= set(fields): raise SystemExit( f"{where} lists all seven roles in 'fields', which " @@ -1489,8 +1606,11 @@ def _tree_parse(name: str, order: str | None) -> object: name = entry["name"] order = entry.get("order") if order is None: - new = {k: v or "" for k, v in HumanName(name).as_dict().items() + hn = HumanName(name) + new = {k: v or "" for k, v in hn.as_dict().items() if k in FIELDS} + # must stay identical to the worker template's facade row + new["_initials"] = hn.initials() or "" # canonicalized on the way in: the ledger speaks Role's # names, and the facade is the surface whose vocabulary # differs @@ -1511,16 +1631,33 @@ def _tree_parse(name: str, order: str | None) -> object: new_v2 = {f: (getattr(p, f, "") or "") for f in V2_FIELDS} new_v2["_ambiguities"] = sorted( {a.kind.name for a in getattr(p, "ambiguities", ())}) + new_v2["_initials"] = p.initials() or "" diff |= {_canonical_field(f) for f in (*V2_FIELDS, "_ambiguities") if old.get("v2", {}).get(f, "") != new_v2.get(f, "")} + # #484: initials() is a DERIVED view. It enters the diff only + # when every role and the ambiguity kinds agree on every + # compared surface -- render-layer drift, the one shape the + # field comparison cannot see. When a role moved, the initials + # movement is that move's consequence, not drift: it is neither + # compared nor printed, and the rule that explains the role + # diff explains it (decisions.md, "the initials view"). Strict + # subset semantics were measured and rejected there: they would + # have put `_initials` onto a long tail of existing rules for + # no added discrimination. + if not diff and ( + old.get("facade", {}).get("_initials", "") + != new.get("_initials", "") + or old.get("v2", {}).get("_initials", "") + != new_v2.get("_initials", "")): + diff = {"_initials"} if not diff: continue diffing.append((name, diff, order)) issue = classify(name, diff, rules, exclusions, order) if issue is None: row = (name, old.get("facade", {}), new, old.get("v2", {}), - new_v2, order) + new_v2, order, diff == {"_initials"}) # classify() returns None for two different reasons: no # rule matched, or a [[never]] entry refused the name -- # and only the first belongs to the tier split. An @@ -1608,22 +1745,26 @@ def _tree_parse(name: str, order: str | None) -> object: if unexplained or radar: print("Field names below are Role's, matching what a ledger " "`fields` rule must say.\n") - for name, old_facade, new, old_v2, new_v2, order in unexplained: + for (name, old_facade, new, old_v2, new_v2, order, + initials_only) in unexplained: # the order tag distinguishes a family-first regression from a # default-order one on the same name -- otherwise indistinguishable # in the report print(f"UNEXPLAINED {name!r}{_order_tag(order)}") - _print_field_diffs(old_facade, new, old_v2, new_v2, order) + _print_field_diffs(old_facade, new, old_v2, new_v2, order, + initials_only=initials_only) if radar: print("\nRadar tier (names the contract does not answer for, " "#468): shown, never blocking. Promote a name that " "matters via a cases.py row + shape tag, or -- for a " "demoted one -- by clearing `tolerated` on its rows.\n") - for entry, (name, old_facade, new, old_v2, new_v2, order) in radar: + for entry, (name, old_facade, new, old_v2, new_v2, order, + initials_only) in radar: labels = entry.get("tests") tag = f" [v1: {', '.join(labels)}]" if labels else "" print(f"UNCLASSIFIED (radar) {name!r}{tag}{_order_tag(order)}") - _print_field_diffs(old_facade, new, old_v2, new_v2, order) + _print_field_diffs(old_facade, new, old_v2, new_v2, order, + initials_only=initials_only) # A rule explaining nothing is as much a broken contract as an # unexplained diff: both mean the ledger no longer describes what the # code does. A rule explaining LESS than it declares is the third diff --git a/tools/differential/expected_since_1.4.0.toml b/tools/differential/expected_since_1.4.0.toml index e35d704b..03ae1e37 100644 --- a/tools/differential/expected_since_1.4.0.toml +++ b/tools/differential/expected_since_1.4.0.toml @@ -2247,3 +2247,165 @@ issue = "fix(suffix-routing) the dotted M.A. spelling reads as a credential (ma- # _MUST_NOT_MATCH probe for that reason. name_regex = "(?i)^jack\\s+m\\.a\\.$" fields = ["family", "suffix"] + +# --------------------------------------------------------------------- +# The `_initials` rules (#484). `_initials` is the differential's +# derived-view pseudo-field: it enters a name's diff ONLY when the +# seven roles and `_ambiguities` agree on every compared surface, so +# every rule below declares it alone (validate_rules refuses it +# beside a role). None of these classifies a role move; each +# classifies a change in what HumanName.initials() -- and from 2.0 +# on parse().initials() -- renders for a parse whose fields did not +# move. +# +# The order the file has them in: the two literal name LISTS first +# (fix(#385/#402), fix(#360)), then the three VOCABULARY rules -- +# connective, bound-given, particle chain -- then the Ph. D. merge. +# Only the bound-given/particle-chain pair is ordered by narrowness, +# and by measured reach rather than by reading: 41 corpus names +# against 107 (measured 2026-09-02 at this baseline, and recorded in +# _CORPUS_CLAIMS). The connective rule's position ahead of both is NOT +# a specificity claim -- its reach is 96, wider than the bound-given +# rule it precedes -- it is just where it was written. +# +# Every rule below carries orders = ["DEFAULT"]. Each describes a +# DEFAULT-order render, and an order-bearing entry never consults the +# facade at all -- main() passes empty dicts for it -- so an +# `_initials` diff under a declared order could only come from the +# CORE, and must not be absorbed by a rule whose prose says "facade". + +[[change]] +issue = "fix(#385/#402) an all-particle name part initials its words (R2)" +# 'Anh Do', 'Juan van der', 'Nguyen, Van Le': a name part whose every +# word is particle vocabulary is a part where none of them is doing a +# particle's work, so its words initial as ordinary name words -- +# rules.md#R2/#R3, shipped in 2.2.0 (release_log "Fix a family name +# made only of particle words reporting no base"), decisions.md#R2. +# 1.4.0 contributed nothing for such a part ('Anh Do' -> 'A.'; the +# tree gives 'A. D.'). The fields did not move, only the view, which +# is why no rule could classify this until #484 gave the gate an +# initials column. Reach exactly the 27 names that move (24 spellings, +# three of them case variants), listed literally rather than shaped: +# the shape is "a part of nothing but particles", which no regex over +# the raw string can state, and the fix(#445) rule above set the +# precedent for a name list in THIS ledger (fix(#410) and fix(#335) +# are two more). 'de los Santos' is NOT here: its +# initials moved for a different reason (the rule below). +name_regex = "(?i)^(?:anh do|smith van der|yin le|yin a le|vai la|jong van der|jong, van der|juan van der|mesnil garcia de|mesnil garcia van|mesnil de|sander van|van ma van|anh van do|beethoven ludwig van|berg jan de jr\\.|john van mc|jong anke de|juan de|ménil christophe de|ménil de|nguyen thi van|nguyen, van le|van berg jan de)$" +fields = ["_initials"] +orders = ["DEFAULT"] + +[[change]] +issue = "fix(#360) los joined the particles, so it no longer initials" +# 'de los Santos': #360 (2.2) added the Spanish/Portuguese articles to +# PARTICLES. The family was 'de los Santos' before and after; what +# changed is that 'los' is now a particle and contributes no initial +# ('l. S.' -> 'S.'). One name, literal. +name_regex = "(?i)^de los santos$" +fields = ["_initials"] +orders = ["DEFAULT"] + +[[change]] +issue = "fix(initials-per-word) a connective run initials each word (facade, since 2.0.0)" +# 'Juan y Eva Garcia', 'Dean of Chemistry Robert Johns', 'John & Jane': +# 1.4.0 built initials one group per *_list ELEMENT, and a run its +# parse joined into one element ('Velasquez y Garcia') gave one +# initial per word joined by the separator with NO delimiter -- +# 'V G.'. The 2.0 facade initials each WORD of an element, 'V. G.'. +# The parse is identical; this is the facade's rendering, changed in +# 2.0.0 and recorded first in 2.3.0's release log (decisions.md, +# "the initials view"). Strip periods and spaces and every name here +# agrees with 1.4.0 letter for letter. +# +# CASE-SENSITIVE, the whole alternation, deliberately -- and it is the +# SINGLE LETTERS the case decides, because a capital `E`/`Y` is an +# initial and so is a dotted `E.`/`Y.`/`e.`/`y.`. The dotted spellings +# are kept out by the PERIOD rather than by the case: the alternation +# wants whitespace after the letter, so 'john e. smith' and +# 'scott e. werner' cannot match whatever their case. The bare capital +# is kept out by the case alone, and that is the half worth keeping. +# The #462 names ('Amy E Maid', 'Jose E Maria Santos') AGREE with +# 1.4.0 since the facade fix and diff only against the 2.x baselines, +# where fix(#462) claims them. What the two exclusions buy is +# unconditional and outlives that fix: any future facade initials +# change touching these shapes must report UNEXPLAINED here rather +# than be absorbed as per-word grouping. +# `and`/`And` because both spellings are corpus names and 'Rob And +# Beth Edmunds' carries no lowercase alternation word to be reached +# by. `of`/`the` have no capital variant because no name in this +# cluster needs one -- a corpus fact and not a shape claim: a capital +# `Of` occurs nowhere in the corpora, and the two cluster names +# carrying a capital `The` ('part1 of The part2 ...' and its `And` +# twin) are already reached through their own lowercase `of`/`the`. +name_regex = "(?:^|\\s)(?:y|e|&|and|And|of|the)(?=\\s)" +fields = ["_initials"] +orders = ["DEFAULT"] + +[[change]] +issue = "fix(initials-per-word) a bound-given run initials each word (facade, since 2.0.0)" +# 'Abdul Salam Hassan, MD', 'Abu Bakar Salim', 'Berg, abdul van': the +# bound-given join makes one list element of the pair, and 1.4.0 +# initialed the element as one group ('A S. H.'); the 2.0 facade +# initials each word ('A. S. H.'). Same change as the rule above, on +# the BOUND_GIVEN_NAMES words the corpora carry. Written before the +# particle rule below DEFENSIVELY, not to win a live contest: nothing +# is contested yet. Two corpus names carry both a bound word and a +# particle chain -- 'Sir abdul van der Berg', which does not diff at +# all, and 'van der Berg, abdul née Jones', whose diff is ROLES and so +# never reaches the pseudo-field -- and the one name this rule shares +# with the Ph. D. rule ('abdul Ph. D. Smith Berg') moves roles too. +# _CROSS_RULE_WINNERS' comment lists these three among the FIVE names +# that two per-word rules both reach, and says why none of them is +# pinned there (its rows under this rule are other names). So the +# position decides nothing today; it is the narrower reading -- +# measured, 41 corpus names against the particle chain's 107 -- +# which is this file's +# write-the-narrower-rule-first convention (the header above, and +# #382's open question about separating such a pair by a predicate +# instead), and it is what would decide a future contest. +name_regex = "(?i)(?:^|[\\s,])(?:abdul|abu)\\s" +fields = ["_initials"] +orders = ["DEFAULT"] + +[[change]] +issue = "fix(initials-per-word) a particle chain inside a name part initials each word (facade, since 2.0.0)" +# 'John van der Berg Smith', 'pennie von bergen wessels', 'de la Cruz +# Juan Carlos': a particle chain that lands INSIDE a part -- a family +# of 'van der Berg Smith' or of 'von bergen wessels' -- is one +# list element, and 1.4.0 initialed it as one group ('J. B S.'); the +# 2.0 facade gives 'J. B. S.'. The shape is a chain-opening particle +# followed by at least two more words, so the particle cannot be the +# family's own tussenvoegsel standing before a single family word +# (those do not move: the particles contribute nothing either way). +# The chain lands in a MIDDLE name only where something later bounds +# the part: one of the eleven names this rule explains does, 'Jane van +# der Berg née Jr Jones', whose maiden marker ends the middle at 'Jr'. +# +# The chain's SECOND word is not spelled out. It was, as an optional +# `(?:\\s+(?:der|la))?` before the two-word tail, and that group was +# provably inert: X(?:A)?Y accepts XY either way, and requiring 'der' +# there only asks for a THIRD word after the particle, which is a +# subset of what the tail already demands. Measured, both spellings +# reach the same 107 corpus names with the same digest. It came out +# because it also put a second alternation in the rule, and +# _LATIN_ALTERNATION_SOURCES keys one vocabulary per rule -- so the +# roster could pin `van|von|de` or `der|la`, never both. +name_regex = "(?i)(?:^|\\s)(?:van|von|de)\\s+\\S+\\s+\\S+" +fields = ["_initials"] +orders = ["DEFAULT"] + +[[change]] +issue = "fix(initials-per-word) the Ph. D. merge initials each word (facade, since 2.0.0)" +# 'Ph. D., John' and 'Sir Ph. D. Van Johnson': the merged credential +# is one list element ('Ph. D.'), and the two facades group it +# differently -- 1.4.0 'J. P. D.', the tree 'J. P D.'; and in the +# titled name the element 'Van Johnson' went from 'V J.' to 'V. J.'. +# Same per-word change, on the phd-merge shape (decisions.md#phd-merge). +# Literal, no alternation. 'Ph. D., Jr.' also matches, and is harmless +# in both directions: its initials do not move ('D.' at 1.4.0 and +# 'D.' here), and its diff is ROLES anyway -- fix(comma-precomma-family) +# claims it on `first`/`last` -- so `_initials` never enters it and +# this rule can neither explain nor absorb it. +name_regex = "(?i)\\bph\\. d\\." +fields = ["_initials"] +orders = ["DEFAULT"] diff --git a/tools/differential/expected_since_2.0.0.toml b/tools/differential/expected_since_2.0.0.toml index 8dc167da..b1b55354 100644 --- a/tools/differential/expected_since_2.0.0.toml +++ b/tools/differential/expected_since_2.0.0.toml @@ -1380,3 +1380,51 @@ issue = "feat(#395)/fix(#296) a comma followed only by a title leaves the pre-co name_regex = "^de la Cruz Juan Carlos, Dr\\.$" fields = ["title", "given", "middle", "family", "suffix"] orders = ["FAMILY_FIRST_GIVEN_LAST"] + +# --------------------------------------------------------------------- +# The `_initials` rules (#484) -- see the block of the same name at the +# end of expected_since_1.4.0.toml for what the pseudo-field is. Only +# view changes AFTER this baseline are visible from it: the per-word +# grouping predates it (2.0.0) and is classified at 1.4.0 alone. +# fix(#462), below, is the 2.3 facade fix, visible from every 2.x +# baseline. + +[[change]] +issue = "fix(#385/#402) an all-particle name part initials its words (R2)" +# 'Anh Do' -> 'A. D.', 'Juan van der' -> 'J. v. d.': rules.md#R2/#R3, +# shipped in 2.2.0, decisions.md#R2. Fields identical at this +# baseline; only the view moved. The same 27-name literal list as the +# 1.4.0 rule, for the same reason. +name_regex = "(?i)^(?:anh do|smith van der|yin le|yin a le|vai la|jong van der|jong, van der|juan van der|mesnil garcia de|mesnil garcia van|mesnil de|sander van|van ma van|anh van do|beethoven ludwig van|berg jan de jr\\.|john van mc|jong anke de|juan de|ménil christophe de|ménil de|nguyen thi van|nguyen, van le|van berg jan de)$" +fields = ["_initials"] +orders = ["DEFAULT"] + +[[change]] +issue = "fix(#360) los joined the particles, so it no longer initials" +# 'de los Santos': 'l. S.' -> 'S.' once #360 (2.2) made 'los' a +# particle. One name, literal. +name_regex = "(?i)^de los santos$" +fields = ["_initials"] +orders = ["DEFAULT"] + +[[change]] +issue = "fix(#462) the facade keeps an initial-shaped conjunction letter" +# 'Scott E. Werner', 'Amy E Maid', 'Jose E. Maria Santos': +# HumanName.initials() dropped a dotted or bare-capital E/Y in the +# middle or family group because the 2.0 facade tested conjunction +# membership alone, where 1.4.0's is_conjunction also required "NOT +# is_an_initial". 'S. W.' -> 'S. E. W.', restoring 1.4.0; the core's +# initials() was already right. Fields identical -- the parse never +# read E as a conjunction, only the facade's render did -- so this is +# `_initials` alone (#484). The shape: a bare capital E or Y, or a +# dotted E./Y. of either case, standing as its own word. Lowercase +# bare e/y is the connective and does not move. The regex reaches 18 +# corpus names, of which 14 diffed when the rule was written +# (2026-09-02). The four it reaches and does not move are what the +# gap IS, and the invariant behind it is a GROUP rather than a count: +# they carry the E/Y in the GIVEN group, which has always initialed +# every word it holds whatever the vocabulary says, so there was +# nothing for the fix to restore. +name_regex = "(?:^|[\\s,])(?:[EY]|[EeYy]\\.)(?=[\\s,]|$)" +fields = ["_initials"] +orders = ["DEFAULT"] diff --git a/tools/differential/expected_since_2.1.0.toml b/tools/differential/expected_since_2.1.0.toml index e6d31875..e28ec3fb 100644 --- a/tools/differential/expected_since_2.1.0.toml +++ b/tools/differential/expected_since_2.1.0.toml @@ -1324,3 +1324,51 @@ issue = "feat(#395)/fix(#296) a comma followed only by a title leaves the pre-co name_regex = "^de la Cruz Juan Carlos, Dr\\.$" fields = ["title", "given", "middle", "family", "suffix"] orders = ["FAMILY_FIRST_GIVEN_LAST"] + +# --------------------------------------------------------------------- +# The `_initials` rules (#484) -- see the block of the same name at the +# end of expected_since_1.4.0.toml for what the pseudo-field is. Only +# view changes AFTER this baseline are visible from it: the per-word +# grouping predates it (2.0.0) and is classified at 1.4.0 alone. +# fix(#462), below, is the 2.3 facade fix, visible from every 2.x +# baseline. + +[[change]] +issue = "fix(#385/#402) an all-particle name part initials its words (R2)" +# 'Anh Do' -> 'A. D.', 'Juan van der' -> 'J. v. d.': rules.md#R2/#R3, +# shipped in 2.2.0, decisions.md#R2. Fields identical at this +# baseline; only the view moved. The same 27-name literal list as the +# 1.4.0 rule, for the same reason. +name_regex = "(?i)^(?:anh do|smith van der|yin le|yin a le|vai la|jong van der|jong, van der|juan van der|mesnil garcia de|mesnil garcia van|mesnil de|sander van|van ma van|anh van do|beethoven ludwig van|berg jan de jr\\.|john van mc|jong anke de|juan de|ménil christophe de|ménil de|nguyen thi van|nguyen, van le|van berg jan de)$" +fields = ["_initials"] +orders = ["DEFAULT"] + +[[change]] +issue = "fix(#360) los joined the particles, so it no longer initials" +# 'de los Santos': 'l. S.' -> 'S.' once #360 (2.2) made 'los' a +# particle. One name, literal. +name_regex = "(?i)^de los santos$" +fields = ["_initials"] +orders = ["DEFAULT"] + +[[change]] +issue = "fix(#462) the facade keeps an initial-shaped conjunction letter" +# 'Scott E. Werner', 'Amy E Maid', 'Jose E. Maria Santos': +# HumanName.initials() dropped a dotted or bare-capital E/Y in the +# middle or family group because the 2.0 facade tested conjunction +# membership alone, where 1.4.0's is_conjunction also required "NOT +# is_an_initial". 'S. W.' -> 'S. E. W.', restoring 1.4.0; the core's +# initials() was already right. Fields identical -- the parse never +# read E as a conjunction, only the facade's render did -- so this is +# `_initials` alone (#484). The shape: a bare capital E or Y, or a +# dotted E./Y. of either case, standing as its own word. Lowercase +# bare e/y is the connective and does not move. The regex reaches 18 +# corpus names, of which 14 diffed when the rule was written +# (2026-09-02). The four it reaches and does not move are what the +# gap IS, and the invariant behind it is a GROUP rather than a count: +# they carry the E/Y in the GIVEN group, which has always initialed +# every word it holds whatever the vocabulary says, so there was +# nothing for the fix to restore. +name_regex = "(?:^|[\\s,])(?:[EY]|[EeYy]\\.)(?=[\\s,]|$)" +fields = ["_initials"] +orders = ["DEFAULT"] diff --git a/tools/differential/expected_since_2.2.0.toml b/tools/differential/expected_since_2.2.0.toml index 0cade28b..8194499f 100644 --- a/tools/differential/expected_since_2.2.0.toml +++ b/tools/differential/expected_since_2.2.0.toml @@ -21,14 +21,39 @@ # assumed. # # tests/v2/test_differential.py permits exactly one empty ledger, the -# one DEFAULT_BASELINE names -- which is this file, for as long as the -# 2.3 cycle moves no corpus name at the default order. +# one DEFAULT_BASELINE names -- which is this file. That latitude went +# unused: the first entry arrived with fix(#462), the 2.3 cycle's first +# change to move a corpus name at the default order. # # tests/v2/test_ledger_guards.py records this file in -# _SPAN_BEARING_RULES with an empty set and in _CORPUS_CLAIMS with an -# empty mapping. The first rule that hand-copies _SCRIPT_RANGES has to -# be recorded in the former or the sweep fails. +# _SPAN_BEARING_RULES with an empty set and in _CORPUS_CLAIMS with the +# one rule below. The empty set still holds: fix(#462)'s regex is a +# letter shape, not a hand copy of _SCRIPT_RANGES. The first rule that +# does copy a script range has to be recorded in the former or the +# sweep fails. # # There is deliberately no `change = []` line. TOML forbids appending a # [[change]] table to a statically defined array, so that line would # block the first entry. + +[[change]] +issue = "fix(#462) the facade keeps an initial-shaped conjunction letter" +# 'Scott E. Werner', 'Amy E Maid', 'Jose E. Maria Santos': +# HumanName.initials() dropped a dotted or bare-capital E/Y in the +# middle or family group because the 2.0 facade tested conjunction +# membership alone, where 1.4.0's is_conjunction also required "NOT +# is_an_initial". 'S. W.' -> 'S. E. W.', restoring 1.4.0; the core's +# initials() was already right. Fields identical -- the parse never +# read E as a conjunction, only the facade's render did -- so this is +# `_initials` alone (#484). The shape: a bare capital E or Y, or a +# dotted E./Y. of either case, standing as its own word. Lowercase +# bare e/y is the connective and does not move. The regex reaches 18 +# corpus names, of which 14 diffed when the rule was written +# (2026-09-02). The four it reaches and does not move are what the +# gap IS, and the invariant behind it is a GROUP rather than a count: +# they carry the E/Y in the GIVEN group, which has always initialed +# every word it holds whatever the vocabulary says, so there was +# nothing for the fix to restore. +name_regex = "(?:^|[\\s,])(?:[EY]|[EeYy]\\.)(?=[\\s,]|$)" +fields = ["_initials"] +orders = ["DEFAULT"]