test(semantic): close the decision gate, 69.4% -> 100% (and correct #490)

[kevincostner17]

What

PR #490 reported "19 mutants, 94.7%" for semantic/policy.py + guard.py. That figure was measured over 19 of the module pair's 62 mutation sites — an indexing defect in the mutation harness meant most (kind, index) pairs matched no node and were silently skipped. It described roughly a third of the module, not the module.

True figure: 69.4%, with 19 survivors. This adds 30 tests and closes it.

	Sites	Kill rate	Survivors
#490 reported	19	94.7%	1 "equivalent"
Actual, re-measured	62	69.4%	19
This PR	62	100.0%	0

not applied: 0 on every run. There are now no equivalent mutants in the gate pair.

Why the survivors mattered

This is the module pair the whole safety story rests on: policy.decide() is the gate that decides whether a repair is applied, suggested or skipped, and guard.py is the last of the five identifier-protection layers.

policy.py — identifier protection narrowed.

is_id = column in ctx.id_columns or (info is not None and info.identifier_like)

Flipped to and, a column that is detected as identifier-like but not explicitly listed in id_columns silently loses its protection. This is the brief's named "ID-protection removed" class, sitting in the gate itself.

policy.py — the gate's own review boundary. proposal.confidence < ctx.review_threshold → <= was movable. #491 closed the equivalent boundary in scoring.py; the gate's copy was never covered.

policy.py — issue_type == 'format_alignment' and _payload_preserving(proposal) → or would let a non-payload-preserving repair through the format-alignment path.

guard.py — the byte-identity comparison. The row-count check len(after) < len(before) was movable, and the NA comparison could be flipped so the guard misses a violation (see below).

This corrects a false claim shipped in #490

tests/test_policy_guard_mutants.py asserted in a docstring:

guard bool#5 (if left_na or right_na → and) is an equivalent mutant and is deliberately not chased. […] Verified across 289 input pairs […] zero behavioural differences. It cannot be killed because it does not change behaviour, so a test written to chase it would be asserting nothing.

It can be killed, and now is. The 289-pair pool held only ordinary scalars and contained no cell whose __eq__ returns True against anything — unittest.mock.ANY, and wildcard/matcher objects generally. For such a cell the fall-through bool(left == right) returns True, not False:

_cell_equal(None, ANY)  -> False   # violation detected
mutant(None, ANY)       -> True    # violation MISSED

So the mutant reports a missing value and a present one as equal, and the final identifier-protection layer passes a column whose NA cell was replaced by a real value. A 1296-pair differential finds 18 such disagreements, in both argument orders.

The claim was not merely imprecise: it asserted that an unkillable mutant sat on the last safety layer, when in fact an uncovered path let that layer miss a violation.

The note also cited mutant IDs, which are positional and did not survive the harness fix — its bool#5 is today's bool#2, and today's bool#5 is a different site entirely. The corrected docstring quotes the source line instead of an ID.

The methodological lesson is in the docstring: an equivalence claim is only as strong as the input pool it was checked over, and a pool of ordinary scalars cannot rule out exotic __eq__. Prefer a proof that the branch is unreachable over a differential that merely found no counterexample.

Scope

Tests only — no src/ change. No behaviour changes, so no changelog entry and no compatibility impact. The one non-new-file edit is the corrected docstring.

Verification

• New files + test_policy_guard_mutants.py + test_guard_protected.py — 63 passed
• Full suite, py3.12 — 7339 passed, 22 skipped, 0 failed, coverage 95.02%
• ruff check . clean (the repo's CI lint; ruff format is not enforced on tests/ and test_policy_guard_mutants.py is not format-clean on main either, so its formatting is left alone)
• Mutation: 62/62 killed

One latent defect found (reported, not fixed here)

decide() gates auto mode on the pre-override risk while _decision() rewrites the recorded risk afterwards:

risk = "high" if proposal.issue_type == "unsafe_ambiguous" else proposal.risk   # policy.py:66
...
if high_conf and proposal.risk != "high":                                        # policy.py:128

So a proposal with issue_type="unsafe_ambiguous" and risk="low" is auto-applied and recorded as high risk, contradicting the module docstring's "never auto-apply high risk".

Not reachable from the pipeline today — verified: the only in-tree producer (apply.py:131) calls make_proposal without risk_override, scoring.risk_for floors that issue type to "high" unconditionally, and every other risk_override in src/ is "high" or conditional-high. It becomes reachable the moment any expert emits unsafe_ambiguous with an explicit non-high override, and nothing prevents that. Minimal fix is to have the gate test the same value it records; that touches src/ and wants its own regression test.

[coderabbitai]

Important

• 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Advanced

Run ID: 53d838fa-c137-4145-800e-dc8cb1656686

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}