Skip to content

fix(mp2): aligned tables and ASCII boxes are layout, not context stuffing - #322

Open
Mark2Mac wants to merge 1 commit into
NVIDIA:mainfrom
Mark2Mac:fix/mp2-markdown-tables
Open

fix(mp2): aligned tables and ASCII boxes are layout, not context stuffing#322
Mark2Mac wants to merge 1 commit into
NVIDIA:mainfrom
Mark2Mac:fix/mp2-markdown-tables

Conversation

@Mark2Mac

Copy link
Copy Markdown
Contributor

The problem

MP2 (context window stuffing) matches (.{2,20}?)\1{20,} — a short unit repeated twenty times or more. There is already a guard for the obvious false positive:

non_ws_chars = set(captured) - {" ", "\t", "\n", "\r"}
if len(non_ws_chars) <= 1 and not any(c in captured for c in (" ", "\t")):
    continue

A run of one repeated character is skipped — but whitespace is explicitly exempted from that skip, so any unit containing a space still matches. In practice that is most formatting a real project contains:

| campo      | quando              | significato                       |
| ---------- | ------------------- | --------------------------------- |
│  └───────────────────────────────────────────────────────┘  │
lines.append(f"{Colors.DIM}{Colors.RESET}                                   ")

An aligned table, an ASCII box, a padded column. None of them displaces anything.

The change

The existing guard's reasoning, extended to the case it misses: a unit made only of whitespace and alignment characters — dashes, bars, colons, dots, box drawing U+2500–U+257F — repeats because something is being lined up.

It is deliberately not a blanket exemption. Above 400 characters layout stops being a plausible explanation, so a long run is still reported even when it is pure filler:

input before after
aligned markdown table MP2
ASCII box / horizontal rule MP2
padded column built in code MP2
"testo" + " " * 800 MP2 MP2
"lorem " * 60 (repeated content) MP2 MP2

Measured effect

25 real files taken from a corpus of 65 skill/plugin units, every one of which had MP2 findings:

MP2 findings   95  ->  3

The three survivors come from MP2's other patterns — prose that talks about exceeding the context window — not from the repetition pattern. They are a different question and this PR does not touch them.

Tests

Five new cases alongside the existing test_mp2_separator_not_flagged: aligned table, ASCII box and padded column produce nothing; a whitespace run past the layout width, and repeated content, still do.

Full suite: 1565 passed, 14 skipped, 6 xfailed.

…fing

MP2's repetition pattern `(.{2,20}?)\1{20,}` already has a guard for a single
character repeated, but it exempts whitespace explicitly, so any run built from
spaces still matches. That is most of what real projects contain: aligned
markdown tables, ASCII boxes, horizontal rules, and UI code that pads columns.

The guard is extended in the same spirit: a unit made only of whitespace and
alignment characters (dashes, bars, colons, box drawing U+2500-U+257F) repeats
because something is being *lined up*, not because context is being displaced.

It is not a blanket exemption. Above 400 characters layout stops being a
plausible explanation — nobody reads a table row or a box that wide — so a long
run is still reported even when it is pure filler. Repetition of *content* is
untouched.

Measured on 25 real files drawn from a corpus of 65 skill/plugin units, all of
which had MP2 findings:

  MP2 findings   95 -> 3

The three survivors come from MP2's *other* patterns (prose about exceeding the
context window), not from the repetition pattern, and are outside this change.

Tests: an aligned table, an ASCII box and a padded column produce nothing; a
whitespace run wider than the layout limit, and repeated content, still do.
Full suite: 1565 passed, 14 skipped, 6 xfailed.

Signed-off-by: Mark2Mac <Mark2Mac@users.noreply.github.com>

@rng1995 rng1995 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Automated SkillSpector Review]

Approved. The MP2 exemption is narrowly limited to short runs made only of whitespace/alignment characters; repeated content and long whitespace stuffing remain detected. The new table, box, padded-column, long-run, and repeated-content cases cover both sides of the boundary, and the focused pattern suite passes (244 tests).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants