Skip to content

tooling(devx): report the files ESLint walks with ZERO rules resolved, behind a shrink-only ledger - #8339

Merged
baozhoutao merged 3 commits into
mainfrom
claude/issue-7908-lint-coverage-rules-predicate
Sep 7, 2026
Merged

tooling(devx): report the files ESLint walks with ZERO rules resolved, behind a shrink-only ledger#8339
baozhoutao merged 3 commits into
mainfrom
claude/issue-7908-lint-coverage-rules-predicate

Conversation

@claude

@claude claude Bot commented Sep 7, 2026

Copy link
Copy Markdown
Contributor

Fixes #7908

Two truths, and why the gap survived

Both of these are true on main at the same time, and that is the whole finding:

pnpm lint:coverage  ->  46/46 packages linted, 0 with outstanding errors
pnpm lint:root      ->  272 files walked, 95 of them resolve ZERO rules

scripts/check-lint-coverage.mjs asks whether a package RUNS ESLint. It reads package.json for a lint script and nothing else, so a package that runs ESLint over files ESLint has no rules for is, to that gate, fully covered. It is not wrong; it answers a different question, and its confident answer is the reason nobody looked further.

Every rule-bearing config object in eslint.config.js is scoped to TS-and-TSX (a brace expansion over those two extensions) or narrower — including the no-console exemption, whose scripts/ carve-out is spelled with the same TS-and-TSX suffix. So the entire .js / .mjs / .cjs family resolves an EMPTY rule set. ESLint parses those files, reports nothing, and exits 0; every downstream reading calls that clean.

This PR turns no rule on. Whether the JS family should get js.configs.recommended is a gate-strength decision with an UNMEASURED red set and its own owner.

The funnel, re-derived (not inherited)

Measured on fedfa3e4a with a real install, ESLint v10.8.1, eslint --print-config FILE cross-checked against the ESLint Node API:

file rules resolved
scripts/github-slug.mjs 0
scripts/check-lint-coverage.mjs 0
eslint.config.js 0
eslint-rules/index.js 0
playwright.config.ts 116 — control
apps/console/src/__tests__/bootSplash.test.ts 117 — control

Both controls were taken in the same run on the same install; without them the zeros are not a reading.

lint:root's own JSON population, this install: 272 files, .ts 174, .tsx 3 (116–117 rules each), .mjs 72, .js 22, .cjs 1 → 95 of 272 (35%) resolve zero rules.

Two differences from the card, which measured at fa7d66c4. The card read 252 files / 89 zero-rule; this branch reads 272 / 95. The percentage is the same (35%) and the mechanism is identical — the tree simply grew. The card's numbers are stale and these are the live ones.

The card also names a stale-looking eslint-disable at scripts/github-slug.mjs:44. Re-derived: there is none. Line 44 is the vendored NON_SLUG_CHARS regex, and a scan of all 124 zero-rule files finds zero real directives — the four textual hits are prose inside comments. That is consistent rather than surprising: reportUnusedDisableDirectives: 'error' is live over this population (it is the one config block with no files key), so a stale directive there would already be a lint:root error, and lint:root is at 0 errors. No card was filed for a defect that does not exist.

Repo-wide, which is the population this gate actually judges: 4440 files walked, 4316 resolve rules, 124 resolve zero.

The gate

New sibling scripts/check-lint-rule-coverage.mjs, wired as pnpm check:lint-rule-coverage. It enumerates the files ESLint walks (ESLint#isPathIgnored, not a re-implementation of the walk), resolves the rule count per file (ESLint#calculateConfigForFile, the API half of --print-config), and reports every file that resolves zero.

Cost, measured: 4440 walked out of 6700 on disk — directory walk 23ms, isPathIgnored 1.9s, calculateConfigForFile 29ms. About two seconds, one process, no lint pass. Per-file resolution is 3.65ms amortised, so there is no need to resolve per file-GROUP and the gate does not.

Enumeration validated against ground truth, not assumed: compared with the filePath set of a real eslint . --format json run at the lint:root scope — 272 files, 272 matches, zero difference either way. The pin test re-proves the equivalence against ESLint#lintFiles on a fixture tree.

Three return shapes from calculateConfigForFile, all load-bearing: undefined = not walked; an object with no rules = WALKED with zero rules (the finding); an object with rules = covered.

Home decision: a sibling, not an extension

  • Real need, measured. Different question (per FILE vs per PACKAGE), different input (an ESLint instance and a tree walk vs package.json reads), different cost class. Keeping them apart is what keeps BOTH numbers printed — 46/46 packages linted and 124 files resolve zero rules are the pair that makes the gap legible, and a merged gate prints one summary line again, which is how this survived.
  • Long-term. Extending would falsify check-lint-coverage.mjs's own header, which states its subject in its first line, and would give one gate a failure message that has to answer two unrelated questions.
  • Hard for an AI to get wrong. A distinct name and a distinct failure message; the remedy for each red is named in the output.
  • Startup focus. One check:* line in the root package.json. No workflow (see below), no new dependency, no rewrite of a working gate.

eslint.config.js and eslint-rules/ are COUNTED, not exempted

The objection triage raised is that "a predicate that reddens on its own config file is a bad first day." It does not redden, and the reason is the ledger: every vacuous file that exists today is a declared row, so the first run is GREEN with the config file counted. Once that is true, a blanket exemption is a SECOND mechanism doing what the ledger already does — and strictly worse, because an exemption is invisible while a row carries a reason and goes red when it stops applying.

Measured both ways before choosing:

  • counting them — 8 rows, 123 files on the base commit
  • exempting them — 6 rows, 103 files, and 20 files silently outside the instrument

The 20 are not incidental. eslint-rules/ is 19 hand-written files — ten rule implementations and their .test.js neighbours — and they implement the ratchets eslint.config.js itself sets to error. "The code that enforces our lint rules is unlinted" is a finding someone may want to act on; an exemption would delete it from the output permanently and unmeasured.

On the four axes: real need — the exemption serves no measured need, only an aesthetic one about day one, and the ledger already answers day one. Long-term — a second waiver mechanism is the kind of workaround this repo's contract-first rule bans. AI-error — a declared row is a strict contract that must be maintained; a blanket exemption is consumer-side tolerance, the shape that hides bulk mistakes. Startup focus — one concept instead of two.

The ledger, and the three directions it goes red

VACUOUS_GROUPS is an exemption list, never the population. Rows are globs, not paths: a row is a statement about a POPULATION, which is why a new scripts/check-foo.mjs does not redden the gate — the row already declares it, and nothing new is hidden.

That granularity is measured, not convenient: 38 new JS-family files landed under scripts/ and eslint-rules/ in the 14 days before this branch (git window 2026-07-30 to 2026-09-07, so the count is not truncated). An exact per-row COUNT would therefore have reddened on essentially every tooling PR, with "increment a number" as its remedy. A ratchet whose remedy is not the fix is a ratchet that gets switched off.

row files today why
scripts/ JS family 74 root repo tooling — where this repo keeps its gate scripts
eslint-rules/ JS 19 the custom rule implementations and their tests
eslint.config.js 1 the flat config itself
lazy-test.mjs 1 root test helper
e2e/ mjs 1 Playwright live-CI pin
packages/*/scripts/ mjs 21 per-package tooling, mostly plugin-gantt verify scripts
postcss.config js/mjs 6 build-tool configs at a package root
apps/site/next.config.mjs 1 same class

Three reds, so a row can only ever be narrowed or deleted:

  1. UNLEDGERED — a walked file resolving zero rules that no row matches. Names the file.
  2. OVER-BROAD — a row that also matches a walked file which DOES resolve rules. This is the "a file left the row" direction: narrow the glob. It makes partial progress visible without pinning a count.
  3. STALE — a row that matches no walked file at all, or only rule-bearing ones. Delete it.

Ablation, on the committed tree, with trap/restore

Every leg proved its mutation on disk (git hash-object against the HEAD blob, plus a grep -c on the injected text) before any result was read, and proved restore by blob hash and an empty git diff HEAD.

leg mutation exit what it printed
0 control none 0 GREEN 4440 walked / 4316 rule-bearing / 124 zero, all ledgered
1 new zz-ablation-7908.mjs at the repo root 1 RED 1 file(s) ESLint walks resolve ZERO rules and no ledger row declares them: zz-ablation-7908.mjs
2 widen a rule-bearing files to reach eslint-rules/ 1 RED the ledger row 'eslint-rules/...' also matches 19 file(s) that DO resolve rules
3 narrow every TS-and-TSX files entry to TSX only 0 GREEN see below — the leg's premise is false
4 add a files entry for .mts carrying no rules 1 RED 1 file(s) ... no ledger row declares them: vitest.config.mts
final none 0 GREEN git diff HEAD empty, eslint.config.js byte-identical to its HEAD blob

Leg 3 falsified its own premise, and that is a result worth having. The dispatch predicted that narrowing a files glob would drop a covered file to zero rules. Measured, it does not: packages/core/src/index.ts came back isPathIgnored: true, config undefined, and 776 files left the WALK while the zero-rule count stayed at 124. The mechanism is that .js/.cjs/.mjs are linted BY DEFAULT in flat config, while .ts is walked only because some object's files names it — so narrowing removes files from the walk, it cannot manufacture vacuity. Leg 4 is the corrected form and exercises the real-world shape: a files entry that carries no rules. Both directions are now in the gate's header and pinned by two cases.

One leg-4 honesty note: its first attempt used a regex that matched nothing (NO-MUTATION-DID-NOT-LAND, injected-text count 0). That reading was void and is not reported as a leg; the anchor was fixed and the leg re-run, which is the table row above.

Wiring, and what actually enforces this

package.json only. There is no ci.yml step, and .github/workflows/ was outside this session's declared file surface.

The dispatch's premise about #8301 did not hold and was re-derived. That card is no longer open: GitHub reports its state_reason as not_planned, at 2026-09-07T13:15Z, and its subject is check:unused-deps specifically, not a policy for this shift's gates. What it does establish is the enforcement route this gate inherits: the this repository is green case in the pin test runs the whole analysis inside pnpm test, so a new unledgered zero-rule file fails CI in the PR that adds it. The two costs #8301 named apply here too — the failure surfaces as one assertion inside a 120-file suite rather than a named step, and it lands in the heavy half of CI for a two-second install-only check. Recorded in the header rather than worked around. 15 of this repo's check:* scripts already run in no workflow.

Gates, each with its exit code

Exit codes captured by redirecting first and reading $? — never through a pipe.

command exit verdict line
pnpm exec vitest run scripts/__tests__/check-lint-rule-coverage.test.ts 0 17 tests passed
pnpm exec vitest run scripts/__tests__/ 0 120 files, 3554 tests passed
node scripts/check-lint-rule-coverage.mjs 0 OK 4440 file(s) walked ... 124 resolve ZERO rules -- every one declared by 8 ledger row(s)
pnpm lint:root 0 32 problems, 0 errors, 32 warnings
pnpm lint:coverage 0 46/46 packages linted, 0 with outstanding errors
pnpm check:entry-guard 0 74 scripts/ files, no guard outside the baseline
pnpm check:phantom-deps 0 every in-scope import declared
pnpm check:unused-deps 0 every gated declaration has a consumer
pnpm check:doc-examples 0 124 blocks, every one compiles or fails as ledgered
pnpm check:comment-mask-corpus 0 4442 files, 1 disagree — the residue #7882 holds open
pnpm type-check:scripts 0 clean
pnpm check:control-bytes 0 6623 tracked text files scanned
node scripts/check-changeset-presence.mjs 0 no changeset owed — 3 files changed, 0 published source
node scripts/check-governed-queue-guard.mjs --test (all 3 paths) 0 NOT GOVERNED for every path

Also scanned by hand for control bytes with a grep -naP over the changed files: no hits.

Filed out of scope

  • objectui#8337 — vitest.config.mts is walked by NO ESLint run in this repository. A neighbouring class this gate cannot see by construction: not vacuously linted, entirely unwalked, because .mts/.cts are in neither the rule-bearing globs nor ESLint's default lint set. One file today, zero .cts. Deduped against all 448 open issues.

Not addressed here, deliberately

  • Whether the JS family should get recommended rules. Its red set is UNMEASURED, and it is a gate-strength decision with its own owner. objectui#7908 is the card this PR answers, and the strength question is not part of it.
  • Whether eslint-rules/ should be linted at all.
  • linterOptions is untouched.

Files changed

  • scripts/check-lint-rule-coverage.mjs (new)
  • scripts/__tests__/check-lint-rule-coverage.test.ts (new)
  • package.json (one script line)

🤖 Generated with Claude Code

https://claude.ai/code/session_01FhBNJcLRZLe8M87VcUgpKr


Generated by Claude Code

`pnpm lint:coverage` says `46/46 packages linted, 0 with outstanding
errors` and cannot see this: its predicate is whether a package RUNS
ESLint, not whether ESLint HAS rules for what that package walks. Both
statements are true at once, which is why the gap survived.

Every rule-bearing config object in `eslint.config.js` is scoped
`files: ['**/*.{ts,tsx}']` or narrower, including the `no-console`
exemption whose `scripts/**` carve-out is spelled
`scripts/**/*.{ts,tsx}`. So the entire `.js`/`.mjs`/`.cjs` family
resolves an EMPTY rule set: ESLint parses those files, reports nothing,
and exits 0. Measured on fedfa3e with ESLint v10.8.1: `lint:root`
walks 272 files and 95 of them (35%) resolve zero rules, and that 95 is
where this repository keeps its gate scripts.

The new gate resolves the rule COUNT per walked file, repo-wide, and
reports every file that resolves zero. The 123 such files on the base
commit are declared as eight glob rows with reasons, so the first run is
green; the ledger goes red on an unledgered zero-rule file, on a row that
also claims files which now resolve rules, and on a row that declares
nothing any more. It can therefore only be narrowed or deleted.

This turns no rule on. Whether the JS family should get
`js.configs.recommended` is a gate-strength decision with an unmeasured
red set and its own card.

Refs: objectui#7908

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FhBNJcLRZLe8M87VcUgpKr
The fourth ablation leg was supposed to narrow a `files:` glob so a
covered file dropped to zero rules. Measured, it does not: narrowing
every `**/*.{ts,tsx}` to `**/*.tsx` moved 776 files OUT OF THE WALK
(`packages/core/src/index.ts` resolves `undefined`, `isPathIgnored`
true) and left the zero-rule count unchanged at 124.

The mechanism is that `.js`/`.cjs`/`.mjs` are linted BY DEFAULT in flat
config while `.ts` is walked only because some object's `files` names
it. So vacuity can only be produced by the default-lint set, or by a
`files:` entry that carries no rules -- which is the real-world way this
defect gets added to a config, and reds the gate naming the file.

Both directions are now in the gate's header and pinned by two cases.

Refs: objectui#7908

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FhBNJcLRZLe8M87VcUgpKr
…lanned

The dispatch for this gate cited objectui#8301 as the open question
owning whether this shift's gates get a workflow step. Re-derived: it is
CLOSED as `not planned` (2026-09-07T13:15Z) and its subject is
`check:unused-deps` specifically, not a policy for new gates.

What that card does establish, and what this gate inherits, is the
enforcement route: a `check:*` script with no `ci.yml` step is enforced
through its pin test's repository-green case inside `pnpm test`, at the
two costs #8301 named — attribution buried in a 120-file suite, and cost
placed in the heavy half of CI. Both apply here and are now written down
instead of being described as an open question.

Refs: objectui#7908

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FhBNJcLRZLe8M87VcUgpKr
@claude

claude Bot commented Sep 7, 2026

Copy link
Copy Markdown
Contributor Author

ACCEPT (#7908 — the rules-resolved predicate) — reviewed on the tree at head bf0210d13 against origin/main 614d85b19, read 2026-09-07T13:42Z; domain:devx @ objectui seat, PM session session_01FhBNJcLRZLe8M87VcUgpKr. Merges clean.

Verified from the diff and the tree, not the report:

  • Three files, additions only: the new gate, its 17-case test, one check:lint-rule-coverage line. eslint.config.js is not in the diff — the gate-strength question stayed out, as triage required.
  • VACUOUS_GROUPS is eight glob rows, each with a reason and the owning card. eslint-rules/**/*.js and eslint.config.js are deliberately COUNTED, not exempted, and the reasoning is the right one: the ledger already makes the first run green, so an exemption would buy nothing that exists while hiding 20 files — including the 19 hand-written implementations of the object-ui/* ratchets the config sets to error. "The code enforcing our lint rules is itself unlinted" is a finding, not a formality.
  • Three red directions in the code and pinned: UNLEDGERED, OVER-BROAD (a row that also matches a rule-bearing file — the "a file left the row" direction, whose remedy is narrowing the glob), and STALE. A row can therefore only be narrowed or deleted.
  • Census floors (1,000 / 1,000) sit far below the measured 4,438 / 4,315 and exist to catch a collapse rather than to pin a number — the non-vacuity shape the sibling gates use.
  • The enumeration was validated against ground truth: the gate's own walk reproduced the 272 file paths of a real lint:root JSON run exactly, zero difference either way.

Three results I want on the record because they contradict what was expected:

  1. The dispatch's fourth ablation leg was falsified. Narrowing a files: glob does NOT manufacture vacuity — 776 files simply left the walk and the zero-rule count held at 124, because JS-family files are linted by default in flat config while .ts is walked only when some object names it. The dev reported that as a result and ran a corrected leg ({ files: ['**/*.mts'] } with no rules → RED naming vitest.config.mts). Reporting a falsified premise beats reshaping the leg to fit.
  2. The card's own numbers were stale and one of its details does not exist. 272/95 here versus the card's 252/89 (same 35%, larger tree), and the card's stale eslint-disable at scripts/github-slug.mjs:44 is not there — line 44 is the vendored regex, and a scan of all 124 zero-rule files finds zero real directives. Consistent with reportUnusedDisableDirectives: 'error' being live over that population. ⛔ No card filed for a defect that does not exist.
  3. A third state exists between "walked with rules" and "walked with none": --print-config returns undefined for vitest.config.mts because .mts/.cts are in neither the rule-bearing globs nor ESLint's default set. Not vacuously linted — entirely unwalked, and invisible to this gate by construction. Filed as finding(devx): vitest.config.mts is walked by NO ESLint run in this repository -- .mts/.cts are in neither the rule-bearing globs nor ESLint's default lint set #8337.

And a correction this seat owes: the dev re-derived the dispatch's premise about #8301 and found it closed not_planned at 13:15Z. That was this seat's cross-reference, and it is now stale; corrected on PR #8322 where the claim was made. The gate's enforcement route is unchanged and honest: the pin test's this repository is green case runs inside pnpm test, the header states both costs triage would have named, and the test asserts the ABSENCE of a ci.yml step so the header's claim stays checkable.

No changeset owed; all three paths NOT GOVERNED. Fixes #7908.

Next: CI convergence → flip ready → post-flip guard → auto-merge SQUASH → queue confirmation → landing probe (the check:lint-rule-coverage line and the eight-row ledger on re-fetched origin/main; control: lint:coverage still printing 46/46 packages linted) → LANDED here and on #7908, which closes via Fixes.


Generated by Claude Code

@baozhoutao
baozhoutao marked this pull request as ready for review September 7, 2026 13:47
@baozhoutao
baozhoutao added this pull request to the merge queue Sep 7, 2026
@claude

claude Bot commented Sep 7, 2026

Copy link
Copy Markdown
Contributor Author

Armed: PR #8339 flipped ready at 2026-09-07T13:47:53Z on head bf0210d13 after CI converged all green (28 checks, read 13:47:41Z); post-flip Governed Surface Queue Guard completed/success at 13:48:09Z; auto-merge SQUASH enabled; queued (timestamp in the REST timeline). Landing stroke on merge: probe on re-fetched origin/main — the check:lint-rule-coverage script line and the eight-row VACUOUS_GROUPS ledger; control: lint:coverage still printing 46/46 packages linted — then LANDED here and on #7908, which closes via Fixes. — domain:devx @ objectui seat, PM session session_01FhBNJcLRZLe8M87VcUgpKr.


Generated by Claude Code

Merged via the queue into main with commit 9cefe85 Sep 7, 2026
31 checks passed
@baozhoutao
baozhoutao deleted the claude/issue-7908-lint-coverage-rules-predicate branch September 7, 2026 14:04
@claude

claude Bot commented Sep 7, 2026

Copy link
Copy Markdown
Contributor Author

LANDED: merged at 2026-09-07T14:04:28Z as 9cefe8549 (squash — one parent, via the merge queue). Content probe on re-fetched origin/main (read 14:04Z): the check:lint-rule-coverage script line present (1), scripts/check-lint-rule-coverage.mjs present, VACUOUS_GROUPS holding exactly 8 glob rows; control — eslint.config.js is byte-unchanged across this merge, so the gate-strength question stayed out as triage required. Merge commit is an ancestor of origin/main. #7908 closes via Fixes; its pm:dispatched is stripped in the same stroke. Follow-up filed by the dev: #8337 (vitest.config.mts is walked by no ESLint run at all — the third state, invisible to this gate by construction). — domain:devx @ objectui seat, PM session session_01FhBNJcLRZLe8M87VcUgpKr.


Generated by Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

finding(devx): eslint.config.js resolves ZERO rules for every .js/.mjs/.cjs file, so lint:root lints 35% of its own population vacuously

2 participants