Skip to content

docs: add deletion probes for testing necessity empirically - #50

Open
cristim wants to merge 1 commit into
mainfrom
docs/deletion-probes
Open

docs: add deletion probes for testing necessity empirically#50
cristim wants to merge 1 commit into
mainfrom
docs/deletion-probes

Conversation

@cristim

@cristim cristim commented Aug 6, 2026

Copy link
Copy Markdown
Member

What

A "Deletion probes" subsection under "Verifying a Refactor" in coding-standards.md, wired into all three phases in CLAUDE.md.

Delete the candidate, run the verification, read the result. Necessity is an empirical question; stop answering it by reading the code.

Why

#44, #46 and #48 are all judgement-based — someone decides, by reading, whether a piece of machinery is needed. That judgement is precisely what fails: the author's local justification for a guard, a parameter or a fallback almost always holds up on its own terms, which is why over-engineering survives review even when reviewers are hunting for it.

Evidence, in both directions

From the six-PR cleanup that prompted this whole thread:

candidate reasoning said probe said
timeout => 1800 over Puppet's 300s default defensive padding catalog applied in 301s — load-bearing by one second
GOPRIVATE / GIT_CONFIG_GLOBAL beside a redirected HOME redundant private module fetch fails without them
~15 lines of idempotence machinery coherent local justification guards an unreachable state; removed, nothing changed

No amount of reading the manifest establishes the 301-second fact. A build does, in one run.

The part actually worth writing down

"Nothing broke" has two meanings, and conflating them is how a coverage gap gets mistaken for dead code:

  1. the code was genuinely unnecessary; or
  2. the verification is too weak to notice.

If deleting a guard that protects a real invariant breaks no test, the honest finding is "this guard is untested", not "this guard is unnecessary". Those cases point straight at the hole that let the risk in, and are usually worth more than the deletion would have been.

Where it lands

  • Plan — for any item whose necessity isn't self-evident, state what breaks without it. A task that can't answer is a task to cut, and the answer becomes the probe the reviewer runs later.
  • Implementation (§1c step 1) — probe your own additions before handing the diff to review. Cheaper here than in review, and what survives arrives with evidence attached.
  • Review (sixth dimension) — where necessity is genuinely arguable, don't argue it; probe it.

The through-line: a parameter proven necessary by a failing build is no longer a matter of opinion.

Scope

18 added lines across two files.

Closes #49

Summary by CodeRabbit

  • Documentation
    • Updated planning guidance to document the impact of omitting each task.
    • Added deletion-probe guidance for evaluating whether code or additions are necessary.
    • Clarified how to interpret verification results when testing removals.

The YAGNI standard, the sixth review dimension and the plan/implementation
rules are all judgement-based: someone reads the code and decides whether a
piece of machinery is needed. That judgement is exactly what fails, because
the author's local justification almost always holds up on its own terms.

Adds a "Deletion probes" subsection to coding-standards.md: delete the
candidate, run the verification, and read the result. Records the part that
matters most, that a clean run has two meanings and conflating them turns a
coverage gap into a mistaken deletion.

Wires it into all three phases in CLAUDE.md: plan states what breaks without
each non-obvious item, the implementer probes its own additions before
review, and the over-engineering dimension probes rather than argues when
necessity is contested.
@coderabbitai

coderabbitai Bot commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

📝 Walkthrough

Walkthrough

The documentation adds deletion probes for testing whether questionable code is necessary. Planning, implementation, and over-engineering review guidance now requires recording omission impact and interpreting verification results.

Changes

Deletion probe guidance

Layer / File(s) Summary
Define deletion probe procedure
coding-standards.md
Adds instructions to delete candidate code individually, run verification, record failures, identify verification gaps, and restore code between probes.
Apply probes across development phases
CLAUDE.md
Requires plans to state omission impact and requires deletion probes during implementation and over-engineering review.

Estimated code review effort: 1 (Trivial) | ~3 minutes

Possibly related issues

Possibly related PRs

  • LeanerCloud/dotclaude#44 — Adds related YAGNI and over-engineering guidance extended here with empirical deletion probes.
  • LeanerCloud/dotclaude#46 — Updates the same documentation files with related over-engineering and refactoring verification guidance.

Suggested labels: documentation, triaged, type/docs, impact/internal, effort/s, severity/low, urgency/eventually, priority/p3

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely describes the deletion-probe guidance added by the pull request.
Linked Issues check ✅ Passed The documentation covers the deletion-probe loop, interpretation, restoration, one-at-a-time testing, and use across planning, implementation, and review [#49].
Out of Scope Changes check ✅ Passed All changes update planning, implementation, review, and verification guidance directly required by the linked issue [#49].
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch docs/deletion-probes

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@coding-standards.md`:
- Around line 78-83: Update the deletion-probe guidance around “delete it, run
the full verification” to require a clean baseline verification on unchanged
code first, followed by a comparable verification after deleting only the
candidate. Count only new, reproducible failures as evidence that the candidate
is load-bearing; if either run is incomplete, non-comparable, pre-existing, or
flaky, restore the candidate and report the probe as inconclusive.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: d3dbfafe-6856-4193-ba83-5c2b8e4797da

📥 Commits

Reviewing files that changed from the base of the PR and between d6bb786 and 33e4dfa.

📒 Files selected for processing (2)
  • CLAUDE.md
  • coding-standards.md

Comment thread coding-standards.md
Comment on lines +78 to +83
Whether a piece of machinery is needed is an empirical question, so answer it empirically instead of by reasoning about the code. For each candidate: **delete it, run the full verification, and read the result.**

- **Something fails** → it is load-bearing. Restore it, and record *what* failed: that failure is the evidence it earns its place, and it belongs in the PR description or the protected list.
- **Nothing fails** → exactly one of two things is true, and you must say which:
1. the code was genuinely unnecessary — delete it for real; or
2. **the verification is too weak to notice** — restore the code and report a coverage gap.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟠 Major | ⚡ Quick win

Require a clean baseline before interpreting a deletion probe.

Line 80 treats any post-deletion failure as evidence that the candidate is load-bearing. A pre-existing or flaky failure can produce the same result. Run the same verification on the unchanged code first. Then run it after deleting only the candidate. Treat only a new, reproducible failure as evidence. If either run is incomplete or not comparable, restore the candidate and report the probe as inconclusive.

Proposed probe sequence
-For each candidate: **delete it, run the full verification, and read the result.**
+For each candidate, first run the full verification on the unchanged code and confirm that it completes cleanly. Then delete only that candidate and rerun the same verification.
 
-- **Something fails** → it is load-bearing. Restore it, and record *what* failed: that failure is the evidence it earns its place, and it belongs in the PR description or the protected list.
+- **A new, reproducible failure appears only after deletion** → it is load-bearing. Restore it, and record *what* failed: that failure is the evidence it earns its place, and it belongs in the PR description or the protected list.
 
+- If either verification run is incomplete or not comparable, restore the candidate and report the probe as inconclusive.

As per coding guidelines, verification must be explicit and must not silently accept invalid or incomplete results.

📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
Whether a piece of machinery is needed is an empirical question, so answer it empirically instead of by reasoning about the code. For each candidate: **delete it, run the full verification, and read the result.**
- **Something fails** → it is load-bearing. Restore it, and record *what* failed: that failure is the evidence it earns its place, and it belongs in the PR description or the protected list.
- **Nothing fails** → exactly one of two things is true, and you must say which:
1. the code was genuinely unnecessary — delete it for real; or
2. **the verification is too weak to notice** — restore the code and report a coverage gap.
Whether a piece of machinery is needed is an empirical question, so answer it empirically instead of by reasoning about the code. For each candidate, first run the full verification on the unchanged code and confirm that it completes cleanly. Then delete only that candidate and rerun the same verification.
- **A new, reproducible failure appears only after deletion** → it is load-bearing. Restore it, and record *what* failed: that failure is the evidence it earns its place, and it belongs in the PR description or the protected list.
- **Nothing fails** → exactly one of two things is true, and you must say which:
1. the code was genuinely unnecessary — delete it for real; or
2. **the verification is too weak to notice** — restore the code and report a coverage gap.
- If either verification run is incomplete or not comparable, restore the candidate and report the probe as inconclusive.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@coding-standards.md` around lines 78 - 83, Update the deletion-probe guidance
around “delete it, run the full verification” to require a clean baseline
verification on unchanged code first, followed by a comparable verification
after deleting only the candidate. Count only new, reproducible failures as
evidence that the candidate is load-bearing; if either run is incomplete,
non-comparable, pre-existing, or flaky, restore the candidate and report the
probe as inconclusive.

Source: Coding guidelines

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

docs: over-engineering rules rely on judgement; necessity should be tested empirically

1 participant