Skip to content

test(plugin-dev): make the tenancy-posture clocked window load-independent - #19779

Merged
huangyiirene merged 2 commits into
mainfrom
claude/issue-19631-tenancy-posture-flake
Sep 23, 2026
Merged

huangyiirene merged 2 commits into
mainfrom
claude/issue-19631-tenancy-posture-flake

Conversation

@huangyiirene

Copy link
Copy Markdown
Collaborator

Fixes #19631

Clause-②: no

dev-plugin-tenancy-posture.test.ts timed out at 5000ms on Test Core (6/6) on an unrelated PR. The file's own header names a hazard; this PR removes the construct that made the clocked window depend on machine load, rather than raising the budget.

Premise check (done first, on origin/main)

The card's dependency claim holds. packages/plugins/plugin-dev/package.json declares 15 dependencies and 4 devDependencies and none of them is @objectstack/lint, so no byte of PR #19625 is imported, loaded or executed by the failing test. premise_still_valid: true.

What the clocked window actually contained

initUnder() boots DevPlugin.init(). Twelve workspace packages are vi.mocked with throwing factories precisely so the graceful-degradation branches run with no module resolution on the hot path. @objectstack/organizations was deliberately left out of that list, so five of the six cases each performed one real await import('@objectstack/organizations') inside the per-test window.

Three measurements, in the order they were taken, because the first reading was wrong:

  1. The failing resolution is not cached. 200 consecutive attempts from this file's own runner: min 0.765ms · p50 1.157ms · p99 5.972ms · max 8.037ms · total 265.4ms. None of them is free.
  2. It is not served by the vitest main process, so the transform queue the #3060 note blames is not this cost. With the main process blocked by ~45s of real transforms (eight generated 4MB modules, each measured at 4.6–6.1s of transform, imported from a sibling test file in the same run) the six cases still ran in 2–12ms. The cost is the worker's own resolver walking node_modules and raising ERR_MODULE_NOT_FOUND.
  3. That walk has a load-sensitive tail. The same 200 attempts under filesystem contention (six concurrent find node_modules -type f loops) keep p50 at 0.839ms but take max from 8.037ms to 28.216ms — the median is flat, the tail is not.

So each walled case drew from an unbounded, machine-load-dependent distribution, and a 5000ms per-test budget bounded the draw rather than any work this file is about. Everything else in that window is in-process CPU.

The fix

@objectstack/organizations now gets the same throwing ERR_MODULE_NOT_FOUND factory the other twelve packages get. The stage-1 catch in dev-plugin.ts still runs for real, still emits the same warning, and the assertions still key on branch entry — the only thing given up is that the absence is simulated here rather than real.

Nothing about the absence is given up by the suite:

  • dev-plugin-tenancy-failfast.test.ts still reads the real, unmocked resolution failure. Its subject is the absent-package path, so it keeps the property and is left untouched.
  • no-framework-dependents.pin.test.ts pins the ADR-0132 entitlement boundary that makes the package unresolvable from a framework package, mechanically and by manifest scan.

A mock that stopped matching reality would therefore redden there, not go unnoticed here. The file header carries this argument and all the figures above.

Before / after — same command, same load, N named

The failure is a load-dependent draw, so the reproduction scales the budget down instead of scaling the load up. Both sides run identically: same file, same six concurrent find node_modules -type f loops as filesystem contention, N = 10 runs per side, the per-test budget at 25ms.

Command, verbatim, run 10 times per side:

pnpm --filter @objectstack/plugin-dev exec vitest run --testTimeout=25 src/dev-plugin-tenancy-posture.test.ts
runs failed timeouts where the timeouts landed
before 6 of 10 10 6 on posture: isolated, 2 on posture: group, 2 on isolated + legacy false — all walled, all resolving
after 1 of 10 1 the single-org case, which resolves nothing

The case-level reading is the one that names the construct, and it comes with its own control: before, every one of the ten timeouts landed on a case that performs the real resolution and none on the single-org case — even though that case is the heaviest by CPU, running three init()s. After, the only remaining timeout is that single-org case, at 41ms against a 25ms budget: that is the scaled harness's own CPU floor, not this construct.

At the 5000ms budget CI actually uses, the whole window is now pure in-process CPU, measured at 1–9ms per case on this box.

What was deliberately not done

No .skip, no .todo, no quarantine, no deletion, no flaky-tolerance, no testTimeout raise. The residual 1-of-10 above is reported rather than papered over, and is explicitly marked in the file header as not a reason to reach for the budget.

The card's three questions

  1. Can branch entry be observed without paying a real, unbounded module resolution per case? Yes, and the file may pay it zero times rather than once: the warning that carries the signal is emitted by dev-plugin.ts's own catch, which runs identically on a simulated absence. The property the header defends is kept by the suite, in the file whose subject it is.
  2. Why does one case of five exceed 5000ms when the others do not? Because the resolution is not cached — measured above, 200 of 200 attempts paid — so each walled case is an independent draw from the same heavy-tailed distribution. There is no asymmetry between the cases to find; the asymmetry is in which draw is unlucky. The before-loop shows it moving between cases across runs.
  3. Is the sibling suite carrying the same exposure? dev-plugin-tenancy-failfast.test.ts carries the same construct, at roughly four times the count (about 20 real resolutions per run, 11 of them inside one truthiness case). It is deliberately not changed here: the real absent-package path is that file's entire subject, and mocking it would leave the repository with no runtime witness for the absence at all. dev-plugin-tenancy-mount-refusal.test.ts already mocks the package and carries none of this. dev-plugin.test.ts, dev-plugin-optional-load-failure.test.ts, dev-plugin-security-enforcement-warning.test.ts and dev-plugin-malformed-stack-posture.test.ts never enter the walled branch and never resolve it.

Verification, at 62d3ec8a3f

  • pnpm --filter @objectstack/plugin-dev testTest Files 8 passed (8), Tests 76 passed (76).
  • pnpm --filter @objectstack/plugin-dev run typecheck — clean; check:test-typecheck: OK, 0 files / 0 errors.
  • node scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstack --ran53 derived famil(ies) accounted for — 53 run, 0 NOT-MEASURED, every one exit 0.
  • pnpm lint (eslint . --no-inline-config, the whole repo, not a narrowing) — exit 0.
  • Dependency closure built first: pnpm --filter '@objectstack/plugin-dev^...' build.

Changeset

skip-changeset. The diff is one *.test.ts file. @objectstack/plugin-dev's files[] is dist, README.md, CHANGELOG.md; after pnpm --filter @objectstack/plugin-dev build, a string unique to the test file has zero hits under dist/ while the positive control DevPlugin hits all four emitted artifacts. No published byte changes.

Acceptance notes

  • packages/plugins/plugin-dev/src/dev-plugin-malformed-stack-posture.test.ts fails at collection on an unbuilt closure with Failed to resolve entry for package "@objectstack/runtime", where its siblings survive the same state through vitest.config.ts's source aliases. Not a defect reachable in CI, which builds the closure first; noted, not filed.

Generated by Claude Code

…posture clocked window

`dev-plugin-tenancy-posture.test.ts` left `@objectstack/organizations`
unmocked so the real resolution failure would be the signal for branch
entry. Five of its six cases therefore performed a real, uncached
`node_modules` walk inside the per-test clocked window, which is the only
thing in that window whose cost is a function of concurrent machine load.
Under a parallel shard one case exceeded the 5000ms budget.

Measured, since the first reading was wrong: the failing resolution is not
cached (200 consecutive attempts, p50 1.157ms, none free) and is not served
by the vitest main process (with that process blocked by ~45s of transforms
the cases still ran in 2-12ms), so it is the worker's own resolver; under
filesystem contention the same 200 attempts keep p50 at 0.839ms but take
max from 8.0ms to 28.2ms.

The package now gets the same throwing factory the other twelve get, which
is what this file's own `#3060` block always claimed to achieve ("zero
module resolution on the hot path"). The absence itself keeps its faithful
witness: `dev-plugin-tenancy-failfast.test.ts` still reads the real,
unmocked failure, and `no-framework-dependents.pin.test.ts` pins the
ADR-0132 entitlement boundary that makes the package unresolvable here.

Claude-Session: https://claude.ai/code/session_01AhQASwqJr2Z7XfGWUdvnbF
Co-authored-by: Claude <noreply@anthropic.com>
…mock count

The scaled repro's after-side is 1 of 10, not 0 of 10, and the one residual
timeout lands on the single-org case — which resolves nothing. That control
is worth more than a clean zero, so it is recorded as measured. The block
now also says "twelve OTHER packages", since organizations joined the list.

Claude-Session: https://claude.ai/code/session_01AhQASwqJr2Z7XfGWUdvnbF
Co-authored-by: Claude <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown
Contributor

📓 Docs Drift Check

Nothing in this diff resolved to a documentable surface (no symbol, route or SDK anchor derived from 0 changed package(s)), so this run has no opinion about the docs.

What this run could not see

Coarse fallback — 0 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): node scripts/docs-audit/affected-docs.mjs --json 67add1301aa2bce363953b3f424f416d43bf29f0packageMentionDocs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/s skip-changeset PR has no user-facing published change; bypasses the changeset gate tests

Projects

None yet

2 participants