tests+docs(design): the period-honorific rows declare tolerated; W2 keeps its normative half

docs/design/rules.md

@@ -1063,14 +1063,16 @@ W2. Rationale: some East Asian honorifics glue directly onto the end
     never itself end a name, so the listed vocabulary carries its
     own license and needs no other gate.
     A listed honorific glued to the end of the name's last name
-    word splits off once and reads as a suffix. The split-off
-    ignores surrounding punctuation, but never treats a part that is
-    not name text as the name's end.
+    word splits off once and reads as a suffix. A part that is not
+    name text — a post-nominal word standing on its own — is never
+    the name's end: the split-off steps past it to the name word
+    behind, and never dissects it.
       "田中さん"                  →  suffix="さん"
       "김민준씨"                  →  suffix="씨"
-      "田中さん 様."              →  suffix="さん, 様."
+      "김민준 박사님"             →  suffix="박사님"
       "马丁·路德·金씨"            →  suffix="씨"
       "김지양"                    →  suffix=""  · boundary
+      "선생님"                    →  family="선생님"  · boundary
       "王君"                      →  family="王君"  · boundary
     history: decisions.md#W2 · interacts: W3 · implemented: nameparser/_pipeline/_script_segment.py
 
@@ -1079,8 +1081,10 @@ W3. Rationale: a family name declared by a comma is the writer's
     drew — but none of the East Asian writing systems declares a
     family name that way (the Background above), so every input this
     rule reads is a listing convention wrapped around a name whose
-    own script has already arranged it. What follows describes what
-    the parser does with such input; it does not promise it.
+    own script has already arranged it — the comma that declares the
+    family name, and the punctuation such a listing carries in with
+    it. What follows describes what the parser does with such input;
+    it does not promise it.
     Under a family comma the pre-comma text is the family by
     declaration and never divides, and the post-comma side is given
     text with no family to find; only the honorific split-off (W2)
@@ -1098,13 +1102,28 @@ W3. Rationale: a family name declared by a comma is the writer's
     name's end, and a glued honorific before the comma stays glued.
     The vocabulary question is C1's own, asked without C1's
     word-count condition: the two differ in what else they require,
-    not in what they ask of the words.
+    not in what they ask of the words. A period the listing leaves
+    behind is not what licenses the step past a post-nominal word,
+    and it is not ignored either. The step is W2's, taken on the
+    vocabulary alone and taken with no punctuation anywhere in the
+    input; a period on a SEPARATE post-nominal word rides along into
+    the suffix and moves no division (田中さん 様. divides where
+    田中さん 様 does). A period glued to the honorific's OWN word is
+    read instead, and decides against the split-off: the listed tail
+    no longer ends that word, so nothing peels and the text goes on
+    whole — 田中さん. and 김민준씨. each read as a title, measured
+    2026-09-05. Neither string is a case row or a corpus line, so no
+    row pins those two readings and they can move without the suite
+    or the differential saying so; the sentence reports them rather
+    than promising them, even by this rule's standard.
+    decisions.md#cjk-comma-demotion carries the parses.
       "남궁민수"                  →  family="남궁"
       "지훈, 남궁민수"            →  given="남궁민수"
       "남궁민수, 지훈"            →  family="남궁민수"  · boundary
       "田中さん, Dr."             →  family="田中さん"
       "田中さん, PhD"             →  suffix="さん, PhD"
-    tolerated: native CJK writing has no family-comma convention, so the four comma lines above illustrate current behavior — changeable without notice — rather than promise it; the comma-free line beside them is W1's claim, which is normative. All four comma names stay watched at every released baseline on the differential's radar tier (tools/differential/corpus_cjk_tolerated.jsonl, projected from the `tolerated` rows of tests/v2/cases.py) instead of its contract tier, and those rows pin them at HEAD.
+      "田中さん 様."              →  suffix="さん, 様."
+    tolerated: native CJK writing has neither a family-comma convention nor a period standing after an honorific, so the four comma lines above and the period line under them illustrate current behavior — changeable without notice — rather than promise it; the line carrying neither, beside them, is W1's claim, which is normative. All five stay watched at every released baseline on the differential's radar tier (tools/differential/corpus_cjk_tolerated.jsonl, projected from the `tolerated` rows of tests/v2/cases.py) instead of its contract tier, and those rows pin them at HEAD.
     history: decisions.md#W3 · interacts: W1, W2, C1 · implemented: nameparser/_pipeline/_script_segment.py
 
 W4. Rationale: Chinese, Japanese and Korean all write the family

nameparser/_pipeline/_script_segment.py

@@ -228,6 +228,19 @@ def _is_post_nominal(state: ParseState, i: int) -> bool:
     return is_suffix_strict(state.tokens[i].text, state.lexicon)
 
 
+# rules.md#W2: "A part that is not name text — a post-nominal word
+# standing on its own — is never the name's end: the split-off steps
+# past it to the name word behind, and never dissects it." (history:
+# decisions.md#W2)
+# The scan-back below is that clause, and it needs no punctuation to
+# fire. '김민준 박사님' steps past 박사님 to 김민준, finds no listed
+# tail there and returns None -- so 박사님 is left whole for suffix
+# classification rather than cut into 박사 + 님, which is what the
+# same input gives when the step is removed. '선생님' is post-nominal
+# entire with no name word behind it, so the scan yields no site at
+# all and the token stays whole. Both are contract-tier corpus names
+# and rules.md#W2 example lines; decisions.md#cjk-comma-demotion
+# carries the forced-predicate measurement behind them.
 def _peel_site(state: ParseState, flat: Sequence[int],
                tails: frozenset[str]) -> tuple[int, int] | None:
     """Where a peel would land in the token run `flat`: the index of the
@@ -272,14 +285,24 @@ def _peel_site(state: ParseState, flat: Sequence[int],
 
 
 # rules.md#W2: "a listed honorific glued to the end of the name's
-# last name word splits off once and reads as a suffix. The
-# split-off ignores surrounding punctuation, but never treats a
-# part that is not name text as the name's end." (history:
+# last name word splits off once and reads as a suffix." (history:
 # decisions.md#W2)
 # That the peel also reaches ACROSS a family comma is stated at
 # rules.md#W3 instead, which is a tolerated rule since the
 # 2026-09-01 comma demotion -- the crossing is what the parser does
-# today, not something W2 promises.
+# today, not something W2 promises. W3 also took, on 2026-09-05, the
+# reading of a period a listing leaves behind
+# (decisions.md#cjk-comma-demotion): a period on a SEPARATE
+# post-nominal word rides into the suffix and moves nothing ('様.'
+# is post-nominal-strict and the scan steps past it as it steps past
+# '様'), while a period glued to the honorific's OWN token stands
+# between the honorific and that token's end, so no listed tail
+# matches, the peel declines, and the whole text reads as a title
+# downstream ('田中さん.', '김민준씨.') -- measured 2026-09-05 and
+# pinned by nothing, since neither string is a case row or a corpus
+# line, so that reading can move with nothing reporting it. Neither
+# is a promise; the step past the post-nominal word itself is W2's,
+# above, and is.
 def _peel_honorific_tail(state: ParseState) -> ParseState:
     """#308: split a listed honorific off the END of the name's last
     NON-POST-NOMINAL token -- 田中さん -> 田中 + さん -- and let

tests/v2/cases.py

@@ -50,15 +50,28 @@
 
 def _has_ascii_letter(text: str) -> bool:
     """True when text contains an ASCII a-z/A-Z letter. Shapes 6/7's
-    purity check calls this ALONGSIDE a separate comma test -- this
-    function tests neither a comma nor a non-ASCII Latin letter on its
-    own. The ASCII restriction is deliberate: a diacritic or a letter
+    purity check calls this ALONGSIDE a comma test and a residue test
+    that takes every OTHER non-space ASCII character -- this function
+    tests neither a comma nor a non-ASCII Latin letter on its own. The
+    ASCII restriction is deliberate: a diacritic or a letter
     outside a-z/A-Z is not what a Latin WRAPPER around CJK text looks
     like in the corpus today (title/credential vocabulary is ASCII),
     and widening this is a call for whichever future row needs it."""
     return any(c.isascii() and c.isalpha() for c in text)
 
 
+def _stray_ascii(text: str) -> str:
+    """The distinct ASCII characters in text other than a space,
+    sorted, as one string ("" when there are none). The residue arm of
+    shapes 6/7's purity test: the comma and the ASCII letter keep
+    messages of their own because each names a composed form with its
+    own doctrine, and this takes everything else -- a period, a digit,
+    a parenthesis. A space is the one ASCII character a pure CJK
+    arrangement writes (rules.md#W4's '山田 太郎'), so it is the one
+    exemption."""
+    return "".join(sorted({c for c in text if c.isascii() and c != " "}))
+
+
 #: Shape 7's other admission besides an explicit divider: a
 #: transcription written wholly in katakana with no dividing
 #: punctuation at all (e.g. "マイケルジャクソン" or the spaced
@@ -113,7 +126,8 @@ class Case:
     #: tag -- mutually exclusive with `shape`, since a shape ADMITS a
     #: text to the contract and tolerated deliberately does not. Every
     #: composed/wrapped CJK form (a comma listing, a Latin title or
-    #: credential around a CJK name) is this table's ground for it,
+    #: credential around a CJK name, and since 2026-09-05 a trailing
+    #: ASCII period on an honorific) is this table's ground for it,
     #: not shapes 6/7's. Restricted to CJK-bearing text (`_has_cjk`):
     #: it exists to demote composed/wrapped CJK forms specifically, and
     #: a Latin row asking for it is a smell until some future arc
@@ -217,8 +231,13 @@ def _check_latin_shape_order(self) -> None:
 
     def _check_cjk_shape_purity(self) -> None:
         """Shapes 6/7 (2026-09-01): the CJK arrangements, admitted
-        wholly classified-script text only -- no comma, no Latin
-        letter. Every composed/wrapped form is tolerated=True's
+        wholly classified-script text only -- no ASCII character at
+        all except the space between two name words (WIDENED
+        2026-09-05: the check read 'no comma, no Latin letter', which
+        admitted the trailing-period honorifics '田中さん 様.' and
+        '김민준 씨.' -- a listing artifact no writing system produces,
+        and the same class as the forms it was already refusing).
+        Every composed/wrapped form is tolerated=True's
         ground, not a shape tag's, so this REFUSES rather than
         requires a particular arrangement beyond that purity test
         (plus shape 7's divider/katakana requirement, and shape 6's
@@ -255,6 +274,19 @@ def _check_cjk_shape_purity(self) -> None:
                 f"{self.id}: shape {self.shape} refuses a Latin "
                 f"letter; Latin-wrapped compositions belong under "
                 f"tolerated=True, not a shape tag")
+        # The residue, after the two forms with doctrine of their own:
+        # a space is the only ASCII character a pure CJK arrangement
+        # writes, so anything else ASCII came in with a convention
+        # from elsewhere -- a trailing period, a digit, a bracket --
+        # and is a composed form whatever it is called.
+        stray = _stray_ascii(self.text)
+        if stray:
+            raise ValueError(
+                f"{self.id}: shape {self.shape} refuses the ASCII "
+                f"{stray!r} ({self.text!r}); a space is the only "
+                f"ASCII character a pure arrangement carries, and "
+                f"composed forms belong under tolerated=True, not a "
+                f"shape tag")
         # U+00B7 (间隔号) marks a name transcription in SOURCE order
         # (W1 Accepted) -- shape 7's ground, not shape 6's family-
         # first one. The fullwidth nakaguro U+30FB is NOT a source-
@@ -3298,7 +3330,14 @@ def _check_cjk_shape_purity(self) -> None:
                "this has one -- which is no longer what separates "
                "them.) 1.4.0 read this first "
                "田中さん / last '様.', which is what 2.0 produced until "
-               "#320: parity before, a classified change after"),
+               "#320: parity before, a classified change after. "
+               "TOLERATED since 2026-09-05: a trailing ASCII period on "
+               "a CJK honorific is a listing artifact no writing "
+               "system produces -- the same class as a comma listing "
+               "or a Latin credential. The row still pins #320's "
+               "mechanism at HEAD; what moves is which corpus file "
+               "carries the text",
+         tolerated=True),
     Case("ja_honorific_period_does_not_stop_the_peel", "田中さん, 様.",
          {"family": "田中", "suffix": "さん, 様."},
          classification="fix(#320)",
@@ -3425,7 +3464,12 @@ def _check_cjk_shape_purity(self) -> None:
                "middle 민준 / family '씨.'; the fields above are #320's, "
                "not the segmenter's, so the row is classified to it -- "
                "as ko_honorific_ssi is classified to #307 without "
-               "naming the same segmentation it also depends on"),
+               "naming the same segmentation it also depends on. "
+               "TOLERATED since 2026-09-05 with its two period twins: "
+               "a trailing ASCII period on a CJK honorific is a "
+               "listing artifact no writing system produces. The row "
+               "still pins #320's mechanism at HEAD",
+         tolerated=True),
     Case("ko_honorific_glued_teacher", "김선생님",
          {"family": "김", "suffix": "선생님"},
          classification="fix(#307) + fix(#271)",
@@ -4019,7 +4063,13 @@ def _check_cjk_shape_purity(self) -> None:
                "pin nothing these two do not. Classified to #320 like "
                "its 씨 counterpart: 1.4.0 read this first 김민준 / last "
                "'양.', and the fields above are the ones this change "
-               "produced, not the segmenter's"),
+               "produced, not the segmenter's. TOLERATED since "
+               "2026-09-05 for the same reason as that counterpart -- "
+               "a trailing ASCII period on a CJK honorific is a "
+               "listing artifact no writing system produces -- and the "
+               "pair moves tiers together the way it moves fields "
+               "together. The row still pins #320's mechanism at HEAD",
+         tolerated=True),
     Case("ko_surname_yang_leads_a_segmentable_given", "양 지훈",
          {"family": "양", "given": "지훈"},
          classification="fix(#271)",

tests/v2/test_cases.py

@@ -74,10 +74,14 @@ def test_the_family_partitions_into_particles_and_base(
         f"particles={pn.family_particles!r} + base={pn.family_base!r}")
 
 
-#: Case.__post_init__'s shape checks, each probed for the one message it
-#: alone raises. A row here is a Case that must fail to construct, not
+#: Case.__post_init__'s shape checks, each probed for the message that
+#: identifies it. A row here is a Case that must fail to construct, not
 #: one that ever joins CASES -- unlike test_case above, this exercises
-#: the dataclass's own validation rather than the parser.
+#: the dataclass's own validation rather than the parser. One message
+#: is shared by three rows and deliberately: the residue arm of the
+#: purity check (2026-09-05) refuses every non-space ASCII character
+#: the comma and Latin-letter arms do not, so its probes differ in the
+#: character that trips it rather than in what they are told.
 @pytest.mark.parametrize("kwargs, match", [
     pytest.param(
         dict(text="Beethoven, Ludwig van", shape=2, locale="nl_NL"),
@@ -113,6 +117,30 @@ def test_the_family_partitions_into_particles_and_base(
         dict(text="김민준 V", shape=6),
         "refuses a Latin letter",
         id="shape-6-refuses-a-latin-letter"),
+    pytest.param(
+        # The 2026-09-05 widening, and the text that motivated it: the
+        # comma and Latin-letter arms above both said no of every
+        # composed form anyone had written down, and this one carries
+        # neither. It is a tolerated row today
+        # (ja_honorific_with_a_period_no_comma); the tag it must not be
+        # able to take back is what this probe holds.
+        dict(text="田中さん 様.", shape=6),
+        "refuses the ASCII",
+        id="shape-6-refuses-a-trailing-period"),
+    pytest.param(
+        dict(text="김민준 2", shape=6),
+        "refuses the ASCII",
+        id="shape-6-refuses-a-digit"),
+    pytest.param(
+        # Refused for its parentheses, BEFORE the transcription test
+        # this text would also fail -- the residue arm runs first, so
+        # the message names the ASCII rather than the divider. The
+        # nickname row this text belongs to (fix(#272)) stays contract
+        # and untagged: the purity gate is a property of a SHAPE tag,
+        # not of the corpus.
+        dict(text="山田 太郎 (マイケル・ジャクソン)", shape=7),
+        "refuses the ASCII",
+        id="shape-7-refuses-ascii-parentheses"),
     pytest.param(
         dict(text="김민준·지훈", shape=6),
         "belongs to shape 7",
@@ -176,14 +204,19 @@ def test_case_construction_rejects_a_bad_shape_tag(
 #: ja_nakaguro_han_takes_the_han_order), an interpunct-divided shape-7
 #: row, a SPACED wholly-katakana shape-7 row (the subtler admission --
 #: a transcription with no U+00B7 at all is still a shape, not a
-#: demotion, as long as every non-space character is katakana), and a
+#: demotion, as long as every non-space character is katakana), a
+#: SPACED HONORIFIC written without the period the residue arm refuses
+#: (the boundary the 2026-09-05 widening had to leave standing: what
+#: the demoted text loses is its period, not its arrangement), and a
 #: tolerated row built from the SAME text a shape probe above refuses
 #: as a comma -- the boundary reading the pair as intended: what a
 #: shape tag refuses, tolerated=True admits. Each must construct
 #: cleanly -- the purity rule is a REFUSAL rule, not a requirement
 #: that admits nothing.
 @pytest.mark.parametrize("kwargs", [
     pytest.param(dict(text="김민준", shape=6), id="pure-shape-6-constructs"),
+    pytest.param(dict(text="田中さん 様", shape=6),
+                 id="spaced-honorific-without-a-period-constructs"),
     pytest.param(dict(text="高橋・一郎", shape=6),
                  id="han-nakaguro-shape-6-constructs"),
     pytest.param(dict(text="威廉·莎士比亚", shape=7),