Skip to content

Commit 9932592

Browse files
derek73claude
andauthored
Retire the block rule format (#761) (#786)
* Retire the block rule format (#761) Every rule in docs/design/rules/ is in the skeleton format, so the block format's parser, measures, checks, tests and docs go. A rule written in the old format is now refused rather than skipped. Also fixes the four parser minors #768's review deferred: an assertion in a backtick span, a #-led bullet continuation, a second When or Tolerated paragraph, and the S10 comment naming an Except bullet by description. The README's Signal tag list now names every tag, held by a test. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Address the docs review of the block-format retirement (#761) Refuse a block-format rule line inside a field too, and a nested list in any marker under a bullet, so the docs' refusal lists hold; qualify the README's refusal list for backtick spans; complete the guard list in docs/design/AGENTS.md; and fix three descriptions of deleted behavior (parse_example_line, axis 12, test_corpora's floor). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Address the PR review of the block-format retirement (#761) Refuse text outside any rule other than a section heading and its one Background paragraph, and a numbered list at the start of a line, both of which were read as nothing or as prose; delete Rule.tolerated, which nothing read; give the bullet-lead and signal-tag checks synthetic rows that can fail; ratchet test_corpora's rule floor to 70; and correct the comments and docs the review found stale. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Close the gaps the fix-commit review found in the section check (#761) Refuse structure glued to a Background paragraph (a fence, heading, field label, list item or example; a quote-led wrapped line stays prose), a section heading with no Background and a section letter headed twice; make prose_blocks end a Background at a section heading as parse_rules_doc does; refuse a bare numbered item ending a line. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Ask the shared assertion predicate of a Background (#761) The Background check's own arrow pattern ran only on wrapped lines and ignored backtick spans; ask rules_skeleton.holds_assertion, the rule side's predicate less its quote-lead half, of every Background line, the first included. A list item interrupts a Background only numbered 1, as in CommonMark. The docs state the glued check's column-0 scope. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Say which numbered items interrupt a Background (#761) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
1 parent c7831d9 commit 9932592

13 files changed

Lines changed: 799 additions & 924 deletions

‎AGENTS.md‎

Lines changed: 4 additions & 4 deletions
Large diffs are not rendered by default.

‎docs/design/AGENTS.md‎

Lines changed: 10 additions & 11 deletions
Large diffs are not rendered by default.

‎docs/design/rules/README.md‎

Lines changed: 12 additions & 12 deletions
Large diffs are not rendered by default.

‎nameparser/_pipeline/_vocab.py‎

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -259,8 +259,9 @@ def is_initial_shaped(text: str) -> bool:
259259
# vocabulary lists, and those up to X it does not, by their shape
260260
# alone" -- its Then clause, the `_ROMAN` test; and rules/S10's Except
261261
# bullet "An initial in front makes it a name word too", the two
262-
# is_initial_shaped tests on `preceding`. Its one-letter Except, the
263-
# words-to-spare count, is the callers'. decisions/S10 (#718, #721)
262+
# is_initial_shaped tests on `preceding`. The words-to-spare count,
263+
# rules/S10's Except bullet "A numeral of one letter needs words to
264+
# spare", is the callers'. decisions/S10 (#718, #721)
264265
def is_trailing_numeral_suffix(text: str, preceding: str,
265266
one_case: bool | None) -> bool:
266267
"""The trailing walk's roman-numeral fork: a FINAL single-token

‎tests/v2/rules_budget.py‎

Lines changed: 17 additions & 69 deletions
Original file line numberDiff line numberDiff line change
@@ -14,57 +14,34 @@
1414
the new words read. A rule that shrinks fails too, so the record
1515
follows it down and the words cannot come back unseen.
1616
17-
Four measures for a block-format rule, all over the statement prose a citation may quote
18-
(rules_doc.prose_blocks, so the Signal:, example, pointer and marker
19-
lines are not counted):
20-
21-
- ``words``: the statement's words, the Rationale included and the
22-
Accepted: clauses not;
23-
- ``exceptions``: the statement's ``except`` and ``unless`` -- a proxy
24-
for the clauses that carve a case out of the rule, which is the way
25-
the split rules grew;
26-
- ``accepted``: the number of Accepted clauses (a line opening
27-
``Accepted:`` or ``Accepted,``, rules_doc.ACCEPTED_RE; the rules
28-
write the second as ``Accepted, and ...:``);
29-
- ``accepted_words``: their words, so a sentence added inside an
30-
existing Accepted clause moves the record too.
31-
32-
A skeleton-format rule (#761, rules_skeleton.py) has its own seven,
33-
read from the parsed fields rather than counted over prose, the word
34-
counts off rules_skeleton.skeleton_parts, the list a citation quotes:
35-
``words`` (every quotable field but the Accepted bullets: When,
36-
Tolerated, Then and its bullets, Because, Except and Precedence, so a
37-
word added anywhere in them moves the record), ``then_words`` (Then
38-
and its bullets), ``because_sentences``,
39-
``except``, ``precedence`` and ``accepted`` (bullet counts) and
40-
``accepted_words`` (the Accepted bullets' words). A rule that changes
41-
format shows as one ``format changed`` line, not a line per measure.
17+
Seven measures, read from the rule's parsed fields (rules_skeleton.py)
18+
rather than counted over prose, the word counts off
19+
rules_skeleton.skeleton_parts, the list a citation quotes: ``words``
20+
(every quotable field but the Accepted bullets: When, Tolerated, Then
21+
and its bullets, Because, Except and Precedence, so a word added
22+
anywhere in them moves the record), ``then_words`` (Then and its
23+
bullets), ``because_sentences``, ``except``, ``precedence`` and
24+
``accepted`` (bullet counts) and ``accepted_words`` (the Accepted
25+
bullets' words). The footer, the fences and the labels are not counted.
4226
"""
4327
from __future__ import annotations
4428

4529
import argparse
4630
import json
47-
import re
4831
from collections.abc import Callable
4932
from pathlib import Path
5033

5134
from tests.v2 import rules_skeleton
5235
from tests.v2.design_docs import read_rules_text
53-
from tests.v2.rules_doc import (
54-
Rule, parse_rules_doc, prose_blocks, split_accepted)
36+
from tests.v2.rules_doc import Rule, parse_rules_doc
5537
from tests.v2.rules_skeleton import count_sentences
5638

5739
RECORD = Path(__file__).with_name("rules_budget.json")
5840
RECORD_COMMAND = "uv run python -m tests.v2.rules_budget --record"
59-
BLOCK_MEASURES = ("words", "exceptions", "accepted", "accepted_words")
60-
SKELETON_MEASURES = ("words", "then_words", "because_sentences", "except",
61-
"precedence", "accepted", "accepted_words")
41+
MEASURES = ("words", "then_words", "because_sentences", "except",
42+
"precedence", "accepted", "accepted_words")
6243

63-
_RULE_ID_RE = re.compile(r"[A-Z]\d+")
64-
_EXCEPTION_RE = re.compile(r"\b(?:except|unless)\b", re.IGNORECASE)
65-
66-
67-
def _skeleton_row(rule: Rule) -> dict[str, int]:
44+
def _row(rule: Rule) -> dict[str, int]:
6845
sk = rule.skeleton
6946
assert sk is not None
7047
parts = rules_skeleton.skeleton_parts(sk)
@@ -86,30 +63,8 @@ def words(keep: Callable[[str], bool]) -> int:
8663
def measure(text: str) -> dict[str, dict[str, int]]:
8764
"""Each rule's measures, keyed by rule ID, in ID order. The text is
8865
parsed first, so a rules text the doc tests reject raises here
89-
rather than being measured, and the rules measured are exactly the
90-
rules parsed."""
91-
rules = parse_rules_doc(text)
92-
rule_ids = {r.rule_id for r in rules}
93-
skeletons = {r.rule_id: r for r in rules if r.skeleton is not None}
94-
out = {}
95-
for rid, lines in prose_blocks(text).items():
96-
if not _RULE_ID_RE.fullmatch(rid):
97-
continue # a section's Background
98-
if rid in skeletons:
99-
out[rid] = _skeleton_row(skeletons[rid])
100-
continue
101-
stmt_lines, clauses = split_accepted(lines)
102-
statement = " ".join(stmt_lines)
103-
out[rid] = {
104-
"words": len(statement.split()),
105-
"exceptions": len(_EXCEPTION_RE.findall(statement)),
106-
"accepted": len(clauses),
107-
"accepted_words": sum(len(" ".join(c).split()) for c in clauses),
108-
}
109-
if set(out) != rule_ids:
110-
raise ValueError(
111-
f"the rules measured and the rules parsed differ: "
112-
f"{sorted(set(out) ^ rule_ids)}")
66+
rather than being measured."""
67+
out = {r.rule_id: _row(r) for r in parse_rules_doc(text)}
11368
return dict(sorted(out.items(),
11469
key=lambda kv: (kv[0][0], int(kv[0][1:]))))
11570

@@ -155,19 +110,12 @@ def changes(record: dict[str, dict[str, int]],
155110
out.append(f"{rid}: recorded, but no rule has that ID")
156111
continue
157112
old_row, new_row = record[rid], now[rid]
158-
formats = (set(BLOCK_MEASURES), set(SKELETON_MEASURES))
159-
if (set(old_row) != set(new_row) and set(old_row) in formats
160-
and set(new_row) in formats):
161-
out.append(f"{rid}: format changed: {old_row} -> {new_row}")
162-
continue
163-
measures = (SKELETON_MEASURES if set(new_row) == set(SKELETON_MEASURES)
164-
else BLOCK_MEASURES)
165-
for key in measures:
113+
for key in MEASURES:
166114
old, new = old_row.get(key), new_row[key]
167115
if old != new:
168116
way = "grew" if old is None or new > old else "shrank"
169117
out.append(f"{rid}: {key} {way}, {old} -> {new}")
170-
for key in sorted(set(old_row) - set(measures)):
118+
for key in sorted(set(old_row) - set(MEASURES)):
171119
out.append(f"{rid}: recorded {key!r}, which is no longer "
172120
f"measured")
173121
return out

0 commit comments

Comments
 (0)