From e1420f3f9153ef21f61c5ffb594dd9c5edee7569 Mon Sep 17 00:00:00 2001 From: os-justin Date: Wed, 9 Sep 2026 01:02:17 +0000 Subject: [PATCH 1/2] chore(tooling): census the workspace test-tsconfig divergence matrix (objectui#8714) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Three per-package `tsconfig` divergences had been found — each by a developer tripping over it, none by any instrument — and the card asked the question none of them could answer: how many packages differ, on how many axes, with how many distinct values each. Adds the instrument and the reading: - `scripts/tsconfig-test-parity-census.mjs` (`pnpm census:tsconfig-test-parity`) resolves every `tsconfig.test.json` through TypeScript's own config resolver, so `extends` chains flatten the way `tsc` flattens them, and groups the 38 projects by their effective value on every option. Report-only: it never fails on a divergence, because several of them are deliberate and load-bearing. The one thing it refuses is a silent zero — an empty population exits non-zero, since a census blind to its own corpus is indistinguishable from a repo whose test programs agree. - `docs/audits/2026-09-test-tsconfig-parity-census.md` records the matrix: 42 test type-programs, 38 with a dedicated project, 2,477 program files, 23 divergent axes, and 4 behavioural axes across the 35-package sibling cohort (`types` 6 values, `lib` 4, `include` 3, `exclude` 2). It also records the two routes a diff of the config text cannot see. A bare `import '@testing-library/jest-dom'` augments the matchers program-wide, so 83 files across four packages type-check only because a DIFFERENT file in the same package imports it; and one `/// ` gives `sdui-parser` the Node globals its ten `types`-unset siblings do not have. No `tsconfig` is edited and no gate is added. The audit proposes what a gate should assert and what it would have caught; ruling on that is a separate card. The one conditional fix the card authorised — naming jest-dom in `plugin-timeline` — is declined with evidence: 21 of 35 siblings do not name it, `permissions` documents its omission on purpose, and `fields` uses matchers in 86 files with `types` unset entirely, so there is no uniform practice to have deviated from. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01YBWFb5YgMU5dw8p2VKj16S --- .../8714-test-tsconfig-parity-census.md | 11 + .../2026-09-test-tsconfig-parity-census.md | 331 ++++++++++++++++++ package.json | 1 + .../tsconfig-test-parity-census.test.ts | 166 +++++++++ scripts/tsconfig-test-parity-census.mjs | 263 ++++++++++++++ 5 files changed, 772 insertions(+) create mode 100644 .changeset/8714-test-tsconfig-parity-census.md create mode 100644 docs/audits/2026-09-test-tsconfig-parity-census.md create mode 100644 scripts/__tests__/tsconfig-test-parity-census.test.ts create mode 100644 scripts/tsconfig-test-parity-census.mjs diff --git a/.changeset/8714-test-tsconfig-parity-census.md b/.changeset/8714-test-tsconfig-parity-census.md new file mode 100644 index 0000000000..320226b976 --- /dev/null +++ b/.changeset/8714-test-tsconfig-parity-census.md @@ -0,0 +1,11 @@ +--- +--- + +Adds `scripts/tsconfig-test-parity-census.mjs` (`pnpm census:tsconfig-test-parity`) and +`docs/audits/2026-09-test-tsconfig-parity-census.md`, the measurement objectui#8714 asked +for: every workspace `tsconfig.test.json` resolved through TypeScript's own config +resolver and compared axis by axis, plus the two ambient-type routes that live in the test +sources rather than in any config. + +Internal tooling and documentation only — no published package source, entry point or +publish-contract field is touched, so nothing releases from this change. diff --git a/docs/audits/2026-09-test-tsconfig-parity-census.md b/docs/audits/2026-09-test-tsconfig-parity-census.md new file mode 100644 index 0000000000..80a2cf0e3a --- /dev/null +++ b/docs/audits/2026-09-test-tsconfig-parity-census.md @@ -0,0 +1,331 @@ +# Audit: test-`tsconfig` parity census across the workspace (2026-09) + +**Question** (objectui#8714): the repo has found **three** per-package `tsconfig` +divergences — each by a developer tripping over it, none by any instrument. How many +packages differ, on how many axes, with how many distinct values each? + +**Why it exists**: the card that triggered this named one instance — +`plugin-timeline/tsconfig.test.json` omits `@testing-library/jest-dom` from +`compilerOptions.types` while `plugin-map`'s names it, so a jest-dom matcher there is +**green under vitest and `TS2339` under `tsc`**. Its own reading was that the tractable +card is not *"add jest-dom to plugin-timeline"* but *"nothing asserts that sibling +packages' test type-programs agree."* That question is unanswerable without this number, +and nobody had it. + +⛔ **This audit changes no `tsconfig`, adds no gate, and rules nothing.** Some +divergences are deliberate and load-bearing — one of them is measured below. Normalising +them would be a repo-wide behaviour change dressed as tidying. What lands with this audit +is the *instrument*: `scripts/tsconfig-test-parity-census.mjs` +(`pnpm census:tsconfig-test-parity`), report-only, re-runnable. + +**Instrument**: TypeScript's own config resolver +(`ts.getParsedCommandLineOfConfigFile`), so `extends` chains are flattened the way `tsc` +flattens them, and every reading below is taken from the *resolved* program rather than +from the config text. + +**Precondition**: every `tsc -p` reading here was taken with the dependency closure +**built** (`pnpm build`, 43/43 tasks green). An unbuilt tree gives `TS2307` for every +workspace import and makes all 38 type-programs look broken in the same way — a +precondition, never a result. + +--- + +## Summary + +| | | +| --- | --- | +| test type-programs in the workspace | **42** | +| …with a dedicated `tsconfig.test.json` (this census) | **38** | +| …that fuse tests into the package's ordinary `tsconfig.json` | **4** | +| program root files across the 38 | **2,477** | +| of the 38, compiling green today | **38 / 38** (measured, exit 0, 0 errors each) | +| divergent axes across all 38 | **23** | +| behavioural axes within the 35-package sibling cohort | **4** (`types`, `lib`, `include`, `exclude`) | +| distinct values on those four | 6 / 4 / 3 / 2 | +| **live cost already paid, in workarounds** | **2 sites** — see *Already paid* | +| **files typed only by leakage from another file** | **83** | + +⭐ **The headline**: it is not *"two packages differ on one key"* (three one-line fixes and +a note). It is **35 sibling packages, four behavioural axes, no axis with fewer than two +values and one with six** — plus two ambient-type routes that are **not in any `tsconfig` +at all**, which is why three separate cards each found "the" divergence and none of them +found the set. + +--- + +## Population, and proof it is not empty + +``` +$ pnpm census:tsconfig-test-parity | head -1 +# test-tsconfig parity census — 38 projects, 2477 program files +``` + +38 files, discovered from disk (`packages/*`, `apps/*`, `examples/*`), not from a list — +a list is a thing to forget, and a package outside the census reads exactly like a package +that agrees with its siblings. A census that reads zero because its glob is wrong is +indistinguishable from a uniform repo, so the script **exits non-zero on an empty +population** and `scripts/__tests__/tsconfig-test-parity-census.test.ts` runs that leg. + +**Two cohorts**, and the split matters because it explains 19 of the 23 axes: + +| cohort | n | `extends` | note | +| --- | --- | --- | --- | +| **A** | 35 | `../../tsconfig.json` (repo root) | the real sibling comparison | +| **B** | 3 | its own package `tsconfig.json` | `examples/console-starter`, `examples/schema-catalog`, `packages/vscode-extension` — a different base, so they diverge on 24 axes among themselves; not comparable to A | + +**The 4 fused programs** (no `tsconfig.test.json`; tests compiled by the package's build +config, under the build's options) are a structural axis in their own right: + +| package | test files | `types` | `lib` | +| --- | --- | --- | --- | +| `apps/console` | 93 | `["vitest/globals", "@testing-library/jest-dom"]` | `[ES2020, DOM, DOM.Iterable]` | +| `packages/data-objectstack` | 60 | `["node"]` | inherited `[ES2020, DOM, DOM.Iterable]` | +| `packages/cli` | 16 | *unset* | `["ES2020"]` — **no DOM** | +| `packages/test-support` | 5 | `["node", "vitest/globals"]` | inherited | + +`vitest/globals` appears in exactly **2 of the 42** programs. `apps/console` extends +nothing at all. + +--- + +## The matrix — cohort A (35 sibling packages) + +### 1. `compilerOptions.types` — 6 effective values (7 spellings) + +| value | n | packages | +| --- | --- | --- | +| *unset* | 11 | collaboration fields mobile plugin-designer plugin-editor plugin-markdown **plugin-timeline** plugin-tree providers react-runtime sdui-parser | +| `["node"]` | 9 | core create-plugin layout permissions plugin-dashboard plugin-detail plugin-gantt plugin-list types | +| `["node", "@testing-library/jest-dom"]` | 7 | auth components i18n plugin-chatbot plugin-form plugin-grid react | +| `["@testing-library/jest-dom", "node"]` | 6 | plugin-calendar plugin-charts plugin-kanban **plugin-map** plugin-report plugin-view | +| `["node", "vite/client", "@testing-library/jest-dom"]` | 1 | app-shell | +| `["node", "vite/client"]` | 1 | runner | + +The last two spellings are the same set in different order — the census normalises sets, +so those two rows are **one** effective value, and 6 is the honest count. + +**`types` is a switch, not a list**: naming it at all turns OFF automatic `@types/*` +inclusion. So *unset* and `["node"]` are not "one has jest-dom, one does not" — they are +two different mechanisms. + +### 2. `compilerOptions.lib` — 4 effective values + +| value | n | packages | +| --- | --- | --- | +| `[ES2020, DOM, DOM.Iterable]` | 27 | (the rest) | +| `[ES2022, DOM, DOM.Iterable]` | 4 | app-shell components mobile plugin-list | +| `[ES2020, DOM]` | 3 | i18n plugin-form types | +| `[ES2022, DOM]` | 1 | plugin-grid | + +**6 of the 38** programs are on ES2022. That number matters — see *Already paid*. + +### 3. `include` — 3 effective values + +| value | n | +| --- | --- | +| `["src/**/*.test.ts", "src/**/*.test.tsx"]` | 28 | +| `+ "src/**/*.d.ts"` | 6 — auth components plugin-charts plugin-dashboard plugin-grid plugin-map | +| `+ "src/**/*.bench.ts"` | 1 — core | + +### 4. `exclude` — 2 values + +Unset in 34; `["src/browser-process-shim.d.ts"]` in `components`. + +### Same program, different spelling (reported, not counted as divergence) + +`composite` (`false` ×32 / unset ×3) and `declaration` (`false` ×11 / unset ×24). Both +default to `false` in TypeScript, so the two spellings compile identically. The census +prints them marked `[same program, different spelling]` rather than hiding them, because +"why does my package say this and my neighbour's not" is a real question with a boring +answer. + +### Uniform across cohort A (the good news) + +`strict`, `module`, `moduleResolution` (`bundler`), `target` (ES2020), `jsx` (`react-jsx`), +`paths` (`{}` — every sibling drops the root source-tree paths so `@object-ui/*` resolves +through built `.d.ts`), `noEmit`, `skipLibCheck`, `isolatedModules`, `resolveJsonModule`, +`baseUrl`, `noUnusedLocals` (`false`), `noUnusedParameters` (`true`), +`useDefineForClassFields`. + +--- + +## The two axes that are not in any `tsconfig` + +This is the part a `diff` of the config files cannot produce, and it is why the card's +framing ("nothing compares them") was right about the *problem* and incomplete about the +*surface*. + +### Route 3 — jest-dom typed by a side-effect import in a **different file** + +`import '@testing-library/jest-dom'` is a global augmentation. Once **any** file in the +program has it, the matchers are typed for **every** file in that program. So a package +can be green with no `types` entry and no import of its own. + +| package | `types` names jest-dom | files using matchers | files importing it | **files typed only by leakage** | +| --- | --- | --- | --- | --- | +| `plugin-detail` | no (`["node"]`) | 55 | 24 | **40** | +| `plugin-dashboard` | no (`["node"]`) | 37 | 22 | **16** | +| `fields` | no (*unset*) | 86 | 93 | **15** | +| `plugin-list` | no (`["node"]`) | 22 | 14 | **12** | + +**83 files** across four packages compile today because a *different* file imports +jest-dom. Deleting the last importing file from `plugin-detail` turns 40 files `TS2339` in +one commit, and nothing in the package names the dependency. + +### Route 4 — Node globals from a `/// ` + +`sdui-parser` and `react-runtime` both leave `types` **unset**. Measured with a probe file +(`const _x: string = process.env.NODE_ENV ?? '';`) compiled by each project: + +``` +react-runtime TS2591: Cannot find name 'process'. +sdui-parser (clean) +``` + +`--explainFiles` names the cause: + +``` +node_modules/.../@types/node/index.d.ts + Type library referenced via 'node' from file + 'packages/sdui-parser/src/__tests__/dashboard-widget-options-census.test.ts' +``` + +One triple-slash directive on line 1 of one test file types Node globals for the whole +package. **`sdui-parser` is the only one of the 38 whose Node types arrive this way.** +Probed across all 11 `types`-unset projects plus two `["node"]` controls: 10 have no Node +globals, `sdui-parser` has them, both controls have them (control lit). + +--- + +## Already paid — LIVE vs LATENT + +A divergence matters only where code depends on it. Both known axes are **LATENT for +compilation** (all 38 programs are green) and **LIVE as workaround cost** — which is the +expensive kind, because the cost is invisible in CI. + +| axis | status | evidence | +| --- | --- | --- | +| jest-dom in `plugin-timeline` | **LATENT** — 0 of its 26 program files use a matcher | but **paid**: `ObjectTimeline.expandFls-7429.test.tsx:147` uses `expect(...).toBeTruthy()` with a comment, from PR #8713, *because* the matcher would not type-check | +| `lib` ES2020 | **LATENT** — of the 20 `.at(` matches inside ES2020 programs, 6 are comments and the other 14 sit on `any`-typed receivers (`makeAdapter()` returns `as Record`), so none is a real `TS2550` | but **paid**: **5 comment sites in `plugin-designer`** documenting index arithmetic written instead of `.at(-1)` | +| ambient `.d.ts` in `include` | **LATENT and deliberate** | see below | + +**Control leg for the `lib` reading** — a zero-hit sweep is evidence only if a lit control +fires. A probe file added to `plugin-timeline`'s program with a genuinely typed array: + +``` +packages/plugin-timeline/src/zzprobe.test.ts(5,15): error TS2550: Property 'at' does not + exist on type 'number[]'. … Try changing the 'lib' compiler option to 'es2022' or later. +packages/plugin-timeline/src/zzprobe.test.ts(9,16): error TS2339: Property + 'toBeInTheDocument' does not exist on type 'Assertion'. +``` + +Both fire. So the sweep's zeros are readings, not blindness — and the card's `TS2339` +claim reproduces exactly. + +⚠️ **One of the paid workarounds encodes a false belief.** +`plugin-designer/src/__tests__/DashboardEditor.i18nTitle.test.tsx` states: + +> `.at()` type-checks **nowhere in this repo** even though every runtime it ships on has it. + +**6 of the 38 programs are on ES2022**, where it does. That comment is what the absence of +a census costs: a developer generalised correctly from their own package and wrote the +generalisation into the tree as documentation. + +--- + +## Deliberate or accidental — and why you cannot tell by reading + +| divergence | verdict | reason | +| --- | --- | --- | +| `permissions` omits jest-dom | **deliberate** | its config says so: *"nothing here needs the jest-dom matchers, which these suites do not use"* | +| `app-shell` raises `lib` to ES2022 | **deliberate** | its config says so, and says why it was raised there and not in the package build config | +| `plugin-map` includes `src/**/*.d.ts` | **deliberate** | its config explains the `TS2882` on a CSS side-effect import | +| `plugin-view` / `plugin-report` **exclude** their `src/global.d.ts` | **deliberate — but undocumented, and only discoverable by measurement** | adding `src/**/*.d.ts` to `plugin-view`'s `include` (the shape 6 siblings carry) turns it **RED with 7 errors**: its `global.d.ts` declares `const process`, which collides with `@types/node`'s. The exclusion is load-bearing; nothing says so | +| `auth`, `plugin-charts`, `plugin-grid` include `src/**/*.d.ts` | **accidental (inert)** | those patterns match **zero files on disk**. Copied along with the config | +| `composite: false` / `declaration: false` present in some, absent in others | **accidental (inert)** | identical program either way | +| `plugin-timeline` omits jest-dom | **inconclusive — and that is the finding** | it does not name `types` at all, so it is not "the sibling that forgot a line"; it is one of 11 packages using a different mechanism | +| `sdui-parser` gets Node types from a triple-slash | **accidental** | no sibling does it; the directive is on line 1 of one file and nothing points at it | + +⭐ **The generalisable result**: three of these eight needed a `tsc` run to classify. **You +cannot read deliberate-vs-accidental off the config text.** That is the strongest argument +for an instrument and the strongest argument against normalising by hand. + +--- + +## The one-line fix, and why it was not made + +objectui#8714 authorised adding `@testing-library/jest-dom` to `plugin-timeline` **only +if** the census showed the omission accidental — *"its siblings uniformly name jest-dom +and nothing explains the exception."* + +**The condition fails.** 21 of 35 cohort-A siblings do not name it; `permissions` +documents its omission on purpose; `fields` uses matchers in 86 files with `types` unset +entirely. There is no uniform sibling practice for `plugin-timeline` to have deviated +from — it is a data point in the matrix, not a bug. **No `tsconfig` was edited.** + +Measured, for whoever rules on it (each leg run against a probe file using +`process.env`, restored after): + +| plugin-timeline `types` | result | +| --- | --- | +| *unset* (today) | jest-dom matchers `TS2339`; `process` `TS2591` | +| `["@testing-library/jest-dom"]` — the literal one line | matchers type-check; `process` still `TS2591` | +| `["node", "@testing-library/jest-dom"]` — the shape 7 siblings use | matchers type-check; `process` resolves; project green | + +If triage rules "fix the instance", the third row is the shape to use, not the first. + +--- + +## Proposed gate — ⛔ NOT implemented here + +If triage wants one, this is what it should assert. Implementing it is a separate card. + +**It should not assert uniformity.** Four of the eight rows above are deliberate, and a +gate that flattens them is the repo-wide behaviour change this audit refused to make. +What it can assert without ruling anything: + +1. **Every divergence is declared.** A `tsconfig.test.json` whose effective value on a + listed axis (`lib`, `types`, `include`, `exclude`) differs from the cohort's modal + value must carry a comment on that key. `permissions`, `app-shell`, `plugin-map` + already pass; `plugin-view`'s load-bearing `d.ts` exclusion and `i18n` / `plugin-form` + / `types` dropping `DOM.Iterable` do not. + **Would have caught**: this card, objectui#8691, and the false comment in + `plugin-designer`. +2. **No file's matchers may be typed only by another file's import.** The `leaning` + column must be 0 — fixed either by the importing file's package naming jest-dom in + `types`, or by each file importing it. **Would have caught**: 83 files across 4 + packages, today. +3. **Inert declarations fail.** An `include` pattern matching zero files, or an option + restating its own default, is deleted. **Would have caught**: 3 dead `d.ts` patterns, + 35 redundant `composite`/`declaration` lines. +4. **Ratchet, not big bang.** Every rule above has live violations; each needs a declared + allowlist that can only shrink, in the shape `check-type-check-coverage.mjs` already + uses. + +--- + +## Method — re-runnable + +```bash +pnpm build # PRECONDITION: unbuilt => TS2307 everywhere +pnpm census:tsconfig-test-parity # the matrix +pnpm census:tsconfig-test-parity -- --json # machine-readable + +# baseline: every test project compiles today +for f in $(git ls-files '*tsconfig.test.json'); do + ./node_modules/.bin/tsc -p "$f" > /tmp/b.log 2>&1 + printf '%-46s exit=%s errors=%s\n' "$f" "$?" "$(grep -c 'error TS' /tmp/b.log)" +done +``` + +## What this audit does NOT claim + +- **Not** that any of the 23 axes is wrong. It reports; triage rules. +- **Not** that the 38 are all the test type-programs — 4 more are fused into build + configs (see *Population*), and `scripts/__tests__/*.test.ts` are compiled by a 43rd + program, `tsconfig.scripts.json` (`lib: ES2022`, `types: ["node"]`). +- **Not** anything about `objectui#8710`'s module-resolution axis: that divergence is + between Vite's alias table and the `exports` map, not between two `tsconfig`s, so this + instrument cannot see it. `moduleResolution` is uniformly `bundler` across all 38. +- **Not** that a zero in the leakage column means a package is safe — it means no file + there uses a matcher without its own import *today*. diff --git a/package.json b/package.json index c5a4732083..bf9bcad980 100644 --- a/package.json +++ b/package.json @@ -51,6 +51,7 @@ "check:esm-specifiers": "node scripts/check-node-esm-load.mjs --specifiers-only", "check:node-esm-load": "node scripts/check-node-esm-load.mjs", "census:body-dialect": "node scripts/body-dialect-census.mjs", + "census:tsconfig-test-parity": "node scripts/tsconfig-test-parity-census.mjs", "check:control-bytes": "node scripts/check-control-bytes.mjs", "check:action-ref-convention": "node scripts/check-action-ref-convention.mjs", "check:published-dist": "node scripts/check-published-dist-tooling.mjs", diff --git a/scripts/__tests__/tsconfig-test-parity-census.test.ts b/scripts/__tests__/tsconfig-test-parity-census.test.ts new file mode 100644 index 0000000000..bb3beadb93 --- /dev/null +++ b/scripts/__tests__/tsconfig-test-parity-census.test.ts @@ -0,0 +1,166 @@ +/** + * ObjectUI + * Copyright (c) 2024-present ObjectStack Inc. + * + * This source code is licensed under the MIT license found in the + * LICENSE file in the root directory of this source tree. + */ + +/** + * Pins for `scripts/tsconfig-test-parity-census.mjs` (objectui#8714). + * + * The card this census answers exists because three divergences were each + * found by a developer tripping over them. The failure mode of the CURE is + * the same shape as the disease: a census that reads zero because its + * discovery glob is wrong is indistinguishable from a repo whose test + * type-programs agree, and it prints that lie with exit 0. + * + * So the population is asserted FIRST and the emptiness leg is OBSERVED here + * rather than reasoned about: `main` is run against a tree with no + * `tsconfig.test.json` in it and must return non-zero. Without that leg every + * other assertion in this file would still pass over an empty set. + * + * The rest pins the three readings a plain `diff` of the config text cannot + * make, each of which was measured on this tree before it was encoded here: + * spellings that are the same program, the jest-dom augmentation leaking + * program-wide from one file's import, and Node globals arriving through a + * `/// ` instead of `compilerOptions.types`. + */ + +import { describe, it, expect } from 'vitest'; +import { mkdtempSync, mkdirSync, writeFileSync, rmSync } from 'node:fs'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; + +import { + discoverTestProjects, + buildMatrix, + jestDomRoute, + nodeTypesRoute, + main, + FALSE_BY_DEFAULT, + NOT_AN_AXIS, + DEFAULT_ROOT, +} from '../tsconfig-test-parity-census.mjs'; + +function tree(files: Record): string { + const root = mkdtempSync(join(tmpdir(), 'parity-census-')); + for (const [rel, body] of Object.entries(files)) { + const abs = join(root, rel); + mkdirSync(join(abs, '..'), { recursive: true }); + writeFileSync(abs, body); + } + return root; +} + +describe('the population is real before anything is said about it', () => { + it('finds the workspace test projects, and names two the card turns on', () => { + const found = discoverTestProjects(DEFAULT_ROOT); + // A floor, not the exact count: packages are added often and this test is + // not the place to relitigate that. What it refuses is a collapse. + expect(found.length).toBeGreaterThan(20); + expect(found).toContain('packages/plugin-map/tsconfig.test.json'); + expect(found).toContain('packages/plugin-timeline/tsconfig.test.json'); + }); + + it('FAILS on an empty population instead of reporting a uniform repo', () => { + // The whole reason this file exists. Every other assertion here passes + // vacuously over an empty set; this is the one that does not. + const empty = tree({ 'packages/only/tsconfig.json': '{}' }); + try { + expect(discoverTestProjects(empty)).toEqual([]); + expect(main(['--root', empty])).toBe(1); + } finally { + rmSync(empty, { recursive: true, force: true }); + } + }); +}); + +describe('two spellings of the same program are not a divergence', () => { + const project = (pkg: string, options: Record) => ({ + project: `packages/${pkg}/tsconfig.test.json`, + package: `packages/${pkg}`, + options, + include: ['src/**/*.test.ts'], + exclude: undefined, + fileNames: [], + }); + + it('reads an omitted `composite` and an explicit `false` as one program', () => { + expect(FALSE_BY_DEFAULT.has('composite')).toBe(true); + const [axis] = buildMatrix([project('a', { composite: false }), project('b', {})]); + expect(axis.axis).toBe('composite'); + expect(axis.spellings).toBe(2); + expect(axis.effective).toBe(1); + }); + + it('reads `types` as a SET — sibling packages spell the same pair both ways', () => { + const axes = buildMatrix([ + project('a', { types: ['node', '@testing-library/jest-dom'] }), + project('b', { types: ['@testing-library/jest-dom', 'node'] }), + ]); + expect(axes.find((a) => a.axis === 'types')).toMatchObject({ spellings: 2, effective: 1 }); + }); + + it('does not report per-project paths as axes — they differ by construction', () => { + expect(NOT_AN_AXIS.has('configFilePath')).toBe(true); + const axes = buildMatrix([ + project('a', { configFilePath: '/x/a', lib: ['lib.es2020.d.ts'] }), + project('b', { configFilePath: '/x/b', lib: ['lib.es2022.d.ts'] }), + ]); + expect(axes.map((a) => a.axis)).toEqual(['lib']); + }); +}); + +describe('the routes that are not in the tsconfig text', () => { + it('counts files whose matchers are typed only by ANOTHER file s import', () => { + const root = tree({ + 'a.test.ts': "import '@testing-library/jest-dom';\nexpect(el).toBeInTheDocument();\n", + // No import of its own. It compiles today only because `a.test.ts` is in + // the same program — delete that file and this one turns TS2339. + 'b.test.ts': 'expect(el).toHaveTextContent("x");\n', + }); + try { + const p = { options: {}, fileNames: ['a.test.ts', 'b.test.ts'] }; + expect(jestDomRoute(root, p)).toMatchObject({ + route: 'side-effect import', + matcherFiles: 2, + importFiles: 1, + matcherWithoutOwnImport: 1, + }); + } finally { + rmSync(root, { recursive: true, force: true }); + } + }); + + it('does not count leaning when `compilerOptions.types` names jest-dom', () => { + const root = tree({ 'a.test.ts': 'expect(el).toBeInTheDocument();\n' }); + try { + const p = { options: { types: ['@testing-library/jest-dom'] }, fileNames: ['a.test.ts'] }; + expect(jestDomRoute(root, p)).toMatchObject({ + route: 'compilerOptions.types', + matcherWithoutOwnImport: 0, + }); + } finally { + rmSync(root, { recursive: true, force: true }); + } + }); + + it('sees Node globals arriving through a triple-slash directive', () => { + const root = tree({ + 'a.test.ts': '/// \nconst x = process.env.HOME;\n', + 'b.test.ts': 'const y = 1;\n', + }); + try { + const p = { options: {}, fileNames: ['a.test.ts', 'b.test.ts'] }; + expect(nodeTypesRoute(root, p)).toMatchObject({ + route: 'triple-slash reference in one file', + directiveFiles: ['a.test.ts'], + }); + const named = { options: { types: ['node'] }, fileNames: ['b.test.ts'] }; + expect(nodeTypesRoute(root, named).route).toBe('compilerOptions.types'); + } finally { + rmSync(root, { recursive: true, force: true }); + } + }); +}); diff --git a/scripts/tsconfig-test-parity-census.mjs b/scripts/tsconfig-test-parity-census.mjs new file mode 100644 index 0000000000..0d62b3b670 --- /dev/null +++ b/scripts/tsconfig-test-parity-census.mjs @@ -0,0 +1,263 @@ +#!/usr/bin/env node +/** + * ObjectUI + * Copyright (c) 2024-present ObjectStack Inc. + * + * This source code is licensed under the MIT license found in the + * LICENSE file in the root directory of this source tree. + */ + +/** + * Test-`tsconfig` parity census (objectui#8714). + * + * ## What this answers + * + * Every workspace package hand-maintains its own `tsconfig.test.json`, and + * until this script NOTHING compared them. Three divergences had been found, + * each by a developer tripping over it rather than by any instrument: + * + * - objectui#8691 — `lib` level (`packages/permissions` inherits `ES2020`, + * where `.at(-1)` is TS2550; `packages/app-shell` sets `ES2022`). + * - objectui#8710 — module resolution (Vite alias vs the `exports` map). + * - objectui#8714 — ambient types (`plugin-map` names + * `@testing-library/jest-dom`, `plugin-timeline` does not, so a matcher + * there is green under vitest and TS2339 under `tsc`). + * + * The shared failure shape is GREEN UNDER ONE TOOL, RED UNDER ANOTHER, so the + * author's evidence and CI's disagree and nothing in the package says why. + * This prints the whole matrix at once so the fourth axis is read off an + * instrument instead of being discovered next month by the same accident. + * + * ## This is a REPORT, not a gate + * + * It asserts nothing about which values are correct and it never fails on a + * divergence — some of them are deliberate and load-bearing (measured: adding + * `src/**\/*.d.ts` to `plugin-view`'s `include`, which six siblings do carry, + * turns that project RED with 7 errors, because its own `src/global.d.ts` + * redeclares `process`). Deciding which rows are defects is triage's, and a + * gate is a separate card. The one thing this DOES refuse is a silent zero: + * an empty population exits non-zero, because a census that reads nothing + * because it is blind is indistinguishable from a repo that is uniform. + * + * ## Why the tsconfig files are not the whole answer + * + * Two of the routes that decide a test program's ambient types are not in any + * tsconfig, and both were measured on this tree: + * + * - A bare `import '@testing-library/jest-dom'` registers the matcher + * augmentation PROGRAM-WIDE, so a file with no import of its own compiles + * because a DIFFERENT file in the same package has one. Four packages lean + * on that today for 54 files between them. + * - A `/// ` in one test file does the same for + * Node globals: `sdui-parser` and `react-runtime` both leave `types` + * unset, and `process` resolves in one and is TS2591 in the other. + * + * So the census reads those routes off the resolved program files, not off the + * config text. `scripts/__tests__/tsconfig-test-parity-census.test.ts` pins + * the instrument against throwaway trees, including the empty-population case. + * + * ## Boundary — 38 is not the number of test type-programs + * + * This reads `tsconfig.test.json` files. Four packages have no such file and + * compile their tests inside their ORDINARY `tsconfig.json` instead + * (`packages/cli`, `packages/data-objectstack`, `packages/test-support`, + * `apps/console`), so the workspace has 42 test type-programs and this census + * sees 38 of them. `scripts/check-type-check-coverage.mjs` is the instrument + * that asks whether every package's tests are compiled AT ALL; this one asks + * whether the ones with a dedicated project agree with each other. Reading 38 + * as "every test program" is the mistake this paragraph exists to prevent. + * + * Run: node scripts/tsconfig-test-parity-census.mjs [--json] [--root DIR] + */ + +import { readFileSync, existsSync } from 'node:fs'; +import { join, dirname, relative, sep } from 'node:path'; +import { fileURLToPath } from 'node:url'; +import { globSync } from 'node:fs'; +import ts from 'typescript'; + +const HERE = dirname(fileURLToPath(import.meta.url)); +export const DEFAULT_ROOT = join(HERE, '..'); + +/** + * Options whose absence is the same PROGRAM as an explicit `false`, so the two + * spellings must not be reported as a divergence. Every entry is a TypeScript + * default that does not depend on any other option (`useDefineForClassFields` + * is deliberately absent — its default follows `target`). + */ +export const FALSE_BY_DEFAULT = new Set([ + 'composite', + 'declaration', + 'declarationMap', + 'sourceMap', + 'allowJs', + 'checkJs', +]); + +/** + * Options TypeScript injects or resolves to an absolute path. They differ per + * project BY CONSTRUCTION — reporting them would bury the four real axes under + * 38 rows of "this file is at a different path than that file". + */ +export const NOT_AN_AXIS = new Set(['configFilePath', 'pathsBasePath', 'baseUrl', 'outDir', 'rootDir', 'typeRoots']); + +const JEST_DOM_MATCHER = + /\.(toBeInTheDocument|toHaveTextContent|toHaveClass|toHaveAttribute|toBeVisible|toBeDisabled|toBeEnabled|toBeChecked|toHaveValue|toHaveFocus|toBeEmptyDOMElement|toContainElement|toContainHTML|toHaveStyle|toBeRequired|toBeInvalid|toBeValid|toHaveAccessibleName|toHaveAccessibleDescription|toHaveDisplayValue|toBePartiallyChecked|toHaveErrorMessage|toHaveRole|toHaveAccessibleErrorMessage|toHaveSelection)\(/; +const JEST_DOM_IMPORT = /import\s+['"]@testing-library\/jest-dom(\/vitest)?['"]/; +const NODE_TRIPLE_SLASH = /\/\/\/\s*/; + +/** + * Every `tsconfig.test.json` under the workspace, repo-relative and sorted. + * + * Discovered from disk rather than listed, for the reason + * `check-type-check-coverage.mjs` records: a list is a thing to forget, and a + * package that lands outside the census reads exactly like a package that + * agrees with its siblings. + */ +export function discoverTestProjects(root = DEFAULT_ROOT) { + return globSync(['packages/*/tsconfig.test.json', 'apps/*/tsconfig.test.json', 'examples/*/tsconfig.test.json'], { + cwd: root, + }) + .map((p) => p.split(sep).join('/')) + .sort(); +} + +/** Flatten one project's `extends` chain through TypeScript's own resolver. */ +export function resolveProject(root, relPath) { + const abs = join(root, relPath); + const host = { + ...ts.sys, + onUnRecoverableConfigFileDiagnostic: (d) => { + throw new Error(ts.flattenDiagnosticMessageText(d.messageText, ' ')); + }, + }; + const parsed = ts.getParsedCommandLineOfConfigFile(abs, {}, host); + if (!parsed) throw new Error(`could not parse ${relPath}`); + return { + project: relPath, + package: dirname(relPath), + options: parsed.options, + include: parsed.raw?.include, + exclude: parsed.raw?.exclude, + fileNames: parsed.fileNames.map((f) => relative(root, f).split(sep).join('/')), + }; +} + +/** + * How this program gets jest-dom's matcher types, and how many of its files + * would lose them if the last importing file went away. + */ +export function jestDomRoute(root, project) { + const named = (project.options.types ?? []).includes('@testing-library/jest-dom'); + let matcherFiles = 0; + let importFiles = 0; + let matcherWithoutOwnImport = 0; + for (const f of project.fileNames) { + const p = join(root, f); + if (!existsSync(p)) continue; + const text = readFileSync(p, 'utf8'); + const usesMatcher = JEST_DOM_MATCHER.test(text); + const imports = JEST_DOM_IMPORT.test(text); + if (usesMatcher) matcherFiles += 1; + if (imports) importFiles += 1; + if (usesMatcher && !imports && !named) matcherWithoutOwnImport += 1; + } + const route = named ? 'compilerOptions.types' : importFiles > 0 ? 'side-effect import' : 'none'; + return { route, named, matcherFiles, importFiles, matcherWithoutOwnImport }; +} + +/** How this program gets Node globals: the `types` field, or one file's directive. */ +export function nodeTypesRoute(root, project) { + if ((project.options.types ?? []).includes('node')) return { route: 'compilerOptions.types', directiveFiles: [] }; + const directiveFiles = project.fileNames.filter((f) => { + const p = join(root, f); + return existsSync(p) && NODE_TRIPLE_SLASH.test(readFileSync(p, 'utf8')); + }); + if (project.options.types === undefined && directiveFiles.length > 0) { + return { route: 'triple-slash reference in one file', directiveFiles }; + } + return { route: 'none', directiveFiles }; +} + +const spell = (v) => (v === undefined ? '' : JSON.stringify(v)); + +/** Group projects by the value they give each axis; flag same-program spellings. */ +export function buildMatrix(projects) { + const keys = new Set(); + for (const p of projects) for (const k of Object.keys(p.options)) if (!NOT_AN_AXIS.has(k)) keys.add(k); + const axes = []; + const read = (p, k) => (k === 'include' || k === 'exclude' ? p[k] : p.options[k]); + for (const k of [...keys, 'include', 'exclude'].sort()) { + const byValue = new Map(); + for (const p of projects) { + const v = spell(read(p, k)); + if (!byValue.has(v)) byValue.set(v, []); + byValue.get(v).push(p.package); + } + if (byValue.size < 2) continue; + const effective = new Set( + [...byValue.keys()].map((v) => { + if (v === '') return FALSE_BY_DEFAULT.has(k) ? 'false' : v; + const parsed = JSON.parse(v); + return Array.isArray(parsed) ? JSON.stringify([...parsed].map(String).sort()) : v; + }), + ); + axes.push({ axis: k, spellings: byValue.size, effective: effective.size, byValue: [...byValue] }); + } + return axes.sort((a, b) => b.effective - a.effective || a.axis.localeCompare(b.axis)); +} + +/** CLI entry. Exported so the emptiness leg can be OBSERVED failing in a test. */ +export function main(argv) { + const rootFlag = argv.indexOf('--root'); + const root = rootFlag === -1 ? DEFAULT_ROOT : argv[rootFlag + 1]; + const files = discoverTestProjects(root); + if (files.length === 0) { + console.error( + 'tsconfig-test-parity-census: found NO tsconfig.test.json under packages/*, apps/* or examples/*.\n' + + 'That is a blind instrument, not a uniform repo — check --root and the workspace layout.', + ); + return 1; + } + const projects = files.map((f) => resolveProject(root, f)); + const axes = buildMatrix(projects); + const rows = projects.map((p) => ({ + package: p.package, + programFiles: p.fileNames.length, + lib: p.options.lib, + types: p.options.types, + jestDom: jestDomRoute(root, p), + node: nodeTypesRoute(root, p), + })); + + if (argv.includes('--json')) { + console.log(JSON.stringify({ population: files.length, axes, projects: rows }, null, 2)); + return 0; + } + + console.log(`# test-tsconfig parity census — ${files.length} projects, ${projects.reduce((a, p) => a + p.fileNames.length, 0)} program files\n`); + console.log(`## divergent axes (${axes.length})\n`); + for (const a of axes) { + const note = a.effective === 1 ? ' [same program, different spelling]' : ''; + console.log(`### ${a.axis} — ${a.spellings} spellings, ${a.effective} effective${note}`); + for (const [value, pkgs] of a.byValue.sort((x, y) => y[1].length - x[1].length)) { + console.log(` ${value} x${pkgs.length}`); + console.log(` ${pkgs.join(' ')}`); + } + console.log(''); + } + console.log('## ambient-type route per project (not visible in the tsconfig text)\n'); + console.log( + `${'package'.padEnd(32)}${'files'.padStart(6)} ${'jest-dom route'.padEnd(22)}${'leaning'.padStart(8)} node route`, + ); + for (const r of rows) { + console.log( + `${r.package.padEnd(32)}${String(r.programFiles).padStart(6)} ${r.jestDom.route.padEnd(22)}${String(r.jestDom.matcherWithoutOwnImport).padStart(8)} ${r.node.route}`, + ); + } + return 0; +} + +if (process.argv[1] && fileURLToPath(import.meta.url) === process.argv[1]) { + process.exit(main(process.argv.slice(2))); +} From 5c9ae708fdf602a21452b56d494bd655391b84bf Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 9 Sep 2026 01:57:11 +0000 Subject: [PATCH 2/2] fix(tooling): route the census entry guard through isEntrypoint `scripts/tsconfig-test-parity-census.mjs` hand-typed its entry guard as `process.argv[1] && fileURLToPath(import.meta.url) === process.argv[1]`, which `check:entry-guard` rejects. Node realpaths `import.meta.url` but leaves `argv[1]` as the caller typed it, so a script reached through a symlink compares two different paths, answers false, and does nothing -- exit 0, no output, indistinguishable from a pass to any wrapper holding only the exit status. Measured on this file, before and after. Old spelling: run directly it prints 25047 bytes of census; reached through a symlink it printed 0 bytes and exited 0. Through `isEntrypoint` both routes print the same 25047 bytes, and importing the module for its exports still prints nothing. The gate reported line 261 twice because it counts token occurrences rather than guard sites -- that one line spelled `process.argv[1]` twice, once as the truthiness leg and once as the comparand. Refs: objectui#8714 Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01YBWFb5YgMU5dw8p2VKj16S --- scripts/tsconfig-test-parity-census.mjs | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/scripts/tsconfig-test-parity-census.mjs b/scripts/tsconfig-test-parity-census.mjs index 0d62b3b670..f68b1f881e 100644 --- a/scripts/tsconfig-test-parity-census.mjs +++ b/scripts/tsconfig-test-parity-census.mjs @@ -75,6 +75,7 @@ import { join, dirname, relative, sep } from 'node:path'; import { fileURLToPath } from 'node:url'; import { globSync } from 'node:fs'; import ts from 'typescript'; +import { isEntrypoint } from './invoked-as.mjs'; const HERE = dirname(fileURLToPath(import.meta.url)); export const DEFAULT_ROOT = join(HERE, '..'); @@ -258,6 +259,6 @@ export function main(argv) { return 0; } -if (process.argv[1] && fileURLToPath(import.meta.url) === process.argv[1]) { +if (isEntrypoint(import.meta.url)) { process.exit(main(process.argv.slice(2))); }