Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
66 changes: 66 additions & 0 deletions packages/spec/scripts/check-browser-reachable-entries.ts
Original file line number Diff line number Diff line change
Expand Up @@ -152,6 +152,7 @@ import { tmpdir } from 'node:os';
import { dirname, join, relative, resolve } from 'node:path';
import { fileURLToPath } from 'node:url';

import { buildStamp } from '../../../scripts/check-regen-pending.mjs';
import { scanSource } from '../../../scripts/js-comment-mask.mjs';
import { inspectBundleFreshness } from './lib/dist-freshness';

Expand Down Expand Up @@ -927,6 +928,71 @@ function selfTest(): never {
!inspectBundleFreshness(fresh, 'check', RERUN).fresh,
mtimes(),
);

// ── The acquittal: an mtime accusation that CAN be answered (#16175) ────
// Every case above is the mtime rule convicting, and until #16175 there was
// nothing it would accept as an answer. So it also refused the tree a
// `git merge`, `git checkout` or `git worktree add` leaves behind: those
// re-check-out an UNCHANGED source file — bytes identical, mtime bumped —
// and the build that follows is a turbo cache hit that rewrites nothing, so
// every dist/ mtime stays where the previous build left it. Measured on the
// real tree: `touch packages/spec/src/data/query.zod.ts` with `git status`
// empty made this gate exit 1 and prescribe a multi-minute rebuild of
// bundles that were already exactly current.
//
// The evidence is `dist/.build-input-hash`, which every build of the package
// writes AFTER its unconditional `tsup` pass. It may only ever ACQUIT, and
// the two cases after this one are what keep that non-vacuous.
const stampFile = join(fresh, 'dist', '.build-input-hash');
// Two steps, for the reason `dist-freshness.test.ts` gives: `buildStamp`
// computes `actual` only when a syntactically valid digest is recorded, so
// seed a placeholder, read what the sources really hash to, then write that.
// Asked of the rule's own reader rather than hardcoded — the input set
// includes turbo.json's globalDependencies and every input's repo-relative
// path, so a literal here would rot into a `mismatch` that reads exactly
// like the refusal these cases exist to tell apart. The stamp is written
// last, so it records the sandbox as it now stands, tsup.config.ts included
// (that file is in the digest's input set as well as in the mtime rule's).
writeFileSync(stampFile, `${'0'.repeat(64)}\n`);
const digest = buildStamp(fresh).actual;
check(
'the sandbox build-input digest can be computed at all (the fixture is not vacuous)',
typeof digest === 'string' && /^[0-9a-f]{64}$/.test(digest),
JSON.stringify(digest),
);
writeFileSync(stampFile, `${digest}\n`);
check(
'ACQUITS an mtime-stale tree whose build stamp matches the sources (#16175)',
inspectBundleFreshness(fresh, 'check', RERUN).fresh,
mtimes(),
);

// The half that makes the case above non-vacuous. If the acquittal were
// keyed on the stamp's mere PRESENCE rather than on the digest, this would
// stay green — and that is #7122's false green restored, one axis over. The
// stamp is written first and the source edited after, so the recorded digest
// is genuinely stale rather than never-valid.
writeFileSync(srcFile, 'export const a = 2;\n');
stamp(srcFile, 0);
check(
'and CONVICTS the same tree the moment a source byte actually changes',
!inspectBundleFreshness(fresh, 'check', RERUN).fresh,
mtimes(),
);

// Absence of the freshness input is not licence to acquit (#4690), and
// neither is a truncated or half-flushed write. Anything that is not 64 hex
// characters is `unstamped`, which leaves the mtime verdict standing. The
// source is restored to the bytes the digest above was taken over first, so
// the ONLY reason this refuses is the stamp itself.
writeFileSync(srcFile, 'export const a = 1;\n');
stamp(srcFile, 0);
writeFileSync(stampFile, 'not-a-digest\n');
check(
'ignores a build stamp that is not a digest at all',
!inspectBundleFreshness(fresh, 'check', RERUN).fresh,
mtimes(),
);
} finally {
rmSync(fresh, { recursive: true, force: true });
}
Expand Down
94 changes: 78 additions & 16 deletions scripts/build-input-hash.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -34,13 +34,29 @@
*
* Both hold the SAME digest over the SAME inputs. The difference is which build
* writes them, and that difference is the whole reason the second one exists —
* see each constant's docblock, and `inspectDeclarationStamp` for the reader.
* see each constant's docblock, and the two `inspect*Stamp` readers at the
* bottom for what each stamp may and may not be believed about.
*/
import { createHash } from 'node:crypto';
import { existsSync, readdirSync, readFileSync } from 'node:fs';
import path from 'node:path';

/** Where a build records the hash of the inputs it was built from. */
/**
* Where a build records the hash of the inputs it was built from.
*
* Written by `--stamp` at the end of EVERY build of an amplifier package,
* `OS_SKIP_DTS=1` included. Two consumers read it, and the difference between
* what they may conclude is the whole reason the sibling stamp below exists:
*
* - `check-dev-prereqs.mjs`'s boot gate, for which "this dist was built from
* these sources" is the whole question (#5864);
* - `bundlesAreStale` in scripts/check-regen-pending.mjs, for which it is
* evidence about the emitted `.mjs`/`.js` ONLY — see `inspectBuildStamp`.
*
* ⛔ It says NOTHING about `dist/**\/*.d.ts`: the flag it is written under can
* skip the declaration pass entirely. That is #7122's rejected direction and it
* stays rejected; DTS_STAMP_BASENAME is the file that answers for those.
*/
export const STAMP_BASENAME = '.build-input-hash';

/**
Expand Down Expand Up @@ -166,28 +182,28 @@ export function buildInputHash(root, pkgDir) {
}

/**
* Did a declaration-emitting build produce THIS dist from THESE sources?
*
* The reader for DTS_STAMP_BASENAME, exported because the caller that needs it
* is `distIsStale` in scripts/check-regen-pending.mjs — and it has to be THIS
* function over THIS hash, or the comparison means nothing (the same argument
* that keeps `--stamp` in this file rather than in a script of its own).
* Read ONE of the two stamps and say what it vouches for. Shared by both
* readers below, because "the stamp and the reader must compute the same
* digest" is exactly as load-bearing between the two stamps as it is between a
* stamp and its reader: two copies of this comparison would drift, and the
* direction drift takes is the one that acquits.
*
* Three verdicts, and the asymmetry between them is deliberate:
*
* - `match` the recorded digest equals the inputs on disk right now, so
* the declarations describe exactly these sources. This is the
* ONLY verdict that may clear an mtime accusation.
* - `mismatch` a declaration-emitting build ran, and the sources have moved
* since. Nameable in a refusal message: this is not an mtime
* artefact, the content really did change.
* the artifact this stamp speaks for was emitted from exactly
* these sources. This is the ONLY verdict that may clear an
* mtime accusation.
* - `mismatch` a build that writes this stamp ran, and the sources have
* moved since. Nameable in a refusal message: this is not an
* mtime artefact, the content really did change.
* - `unstamped` NO EVIDENCE — no stamp, an unreadable one, a digest that
* cannot be computed, or a package whose build does not stamp
* at all. Every one of those collapses to the same answer on
* purpose: "cannot vouch" must never read as "vouched for"
* (#4690), and absence of the input is not licence to acquit.
*
* It never throws: it is called from inside a freshness predicate whose failure
* It never throws: it is called from inside freshness predicates whose failure
* direction is a silently wrong artifact, so an unreadable tree has to degrade
* to `unstamped` rather than take the caller down.
*
Expand All @@ -197,9 +213,9 @@ export function buildInputHash(root, pkgDir) {
* `actual` is computed only when there is a valid digest to compare it against,
* so the ~30ms hash stays off the path where no amplifier stamp exists at all.
*/
export function inspectDeclarationStamp(root, pkgDir) {
function inspectStamp(root, pkgDir, basename) {
const none = { state: 'unstamped', recorded: null, actual: null };
const stampFile = path.join(pkgDir, 'dist', DTS_STAMP_BASENAME);
const stampFile = path.join(pkgDir, 'dist', basename);
let recorded;
try {
if (!existsSync(stampFile)) return none;
Expand All @@ -216,3 +232,49 @@ export function inspectDeclarationStamp(root, pkgDir) {
}
return { state: recorded === actual ? 'match' : 'mismatch', recorded, actual };
}

/**
* Did a DECLARATION-emitting build produce THIS dist from THESE sources?
*
* The reader for DTS_STAMP_BASENAME, exported because the caller that needs it
* is `distIsStale` in scripts/check-regen-pending.mjs — and it has to be THIS
* function over THIS hash, or the comparison means nothing (the same argument
* that keeps `--stamp` in this file rather than in a script of its own).
*/
export function inspectDeclarationStamp(root, pkgDir) {
return inspectStamp(root, pkgDir, DTS_STAMP_BASENAME);
}

/**
* Did a BUNDLE-emitting build produce THIS dist from THESE sources?
*
* The reader for STAMP_BASENAME, exported for `bundlesAreStale` in
* scripts/check-regen-pending.mjs, which measures `dist/**\/*.mjs` and `*.js`
* — a different artifact from the declarations, produced by a different pass.
*
* ## Why reading STAMP_BASENAME is sound HERE and stays rejected next door
*
* #7122 proposed answering the DECLARATION rule with this stamp, and that was
* measured wrong in the dangerous direction: `--stamp` writes this file under
* `OS_SKIP_DTS=1`, the one build flag that emits JS and leaves whatever `.d.ts`
* was there before — so it says fresh over stale declarations. That rejection
* is pinned (`packages/spec/scripts/dist-freshness.test.ts`) and unchanged.
*
* On the BUNDLE axis the same fact points the other way. `OS_SKIP_DTS=1` emits
* every bundle this stamp would then vouch for, so the case that ruled the file
* out for declarations is not a hole here at all — it is the ordinary case.
* What makes the vouching sound is the build script's ORDER, not the flag:
* `packages/spec`'s `build` runs the unconditional `tsup` (the JS pass) before
* `--stamp` in the same `&&` chain, so this file is never written by a run that
* did not emit bundles. The declaration pass that follows is the one that can be
* skipped, and skipping it cannot refresh this stamp because the stamp is
* written after both either way.
*
* The same one-way property still governs: it may only ever ACQUIT a tree the
* mtime rule has already accused, never accuse one it cleared. The digest still
* cannot see a hand-edited dist, a toolchain change or dependency drift — the
* mtime rule remains the only thing that convicts.
*/
export function inspectBuildStamp(root, pkgDir) {
return inspectStamp(root, pkgDir, STAMP_BASENAME);
}
19 changes: 18 additions & 1 deletion scripts/check-regen-pending.d.mts
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@
// guard exists to prevent — would type-check clean.
//
// Declared rather than inferred (no `allowJs`) because the module sits at the
// repo root, outside the consuming program's `rootDir`. The surface is four
// repo root, outside the consuming program's `rootDir`. The surface is five
// functions with one optional argument; keep this file in step with them by
// hand, and keep it small enough that doing so stays trivial.

Expand Down Expand Up @@ -61,3 +61,20 @@ export function schemaTreeIsStale(specDir?: string): boolean;
* @param specDir Absolute path to the spec package; defaults to this repo's.
*/
export function bundlesAreStale(specDir?: string): boolean;

/**
* Did a bundle-emitting build produce `specDir/dist`'s `.mjs`/`.js` from the
* sources on disk right now? The bundle axis's counterpart to
* `declarationStamp`, reading the OTHER stamp file (`dist/.build-input-hash`,
* written by every build). `'match'` is the only verdict that clears an mtime
* accusation; `'unstamped'` is "no evidence". Read the function's own docblock
* before reusing it — the same file is deliberately NOT evidence about
* `dist/**\/*.d.ts`.
*
* @param specDir Absolute path to the spec package; defaults to this repo's.
*/
export function buildStamp(specDir?: string): {
state: 'match' | 'mismatch' | 'unstamped';
recorded: string | null;
actual: string | null;
};
59 changes: 57 additions & 2 deletions scripts/check-regen-pending.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,7 @@ import { dirname, join, resolve } from 'node:path';
import { fileURLToPath } from 'node:url';

import { PENDING_MARKER, entryForPath, ownerDir, ownerOf, ownerRunCommand } from './regen-artifacts.mjs';
import { inspectDeclarationStamp } from './build-input-hash.mjs';
import { inspectBuildStamp, inspectDeclarationStamp } from './build-input-hash.mjs';
import { isEntrypoint } from './invoked-as.mjs';
import {
EXIT_PREREQUISITE_NOT_MET,
Expand Down Expand Up @@ -156,6 +156,24 @@ export function declarationStamp(specDir = SPEC_DIR) {
}
}

/**
* Did a BUNDLE-emitting build produce `specDir/dist`'s `.mjs`/`.js` from the
* sources on disk right now? Same three verdicts, same one-way meaning, same
* wrapping argument as `declarationStamp` above — the repo root is supplied
* here so no caller can hash against the wrong one.
*
* The OTHER stamp file: `dist/.build-input-hash`, which every build writes.
* `inspectBuildStamp`'s docblock is the authority on why that is the right
* evidence for THIS axis and remains the wrong evidence for the declarations.
*/
export function buildStamp(specDir = SPEC_DIR) {
try {
return inspectBuildStamp(REPO_ROOT, specDir);
} catch {
return { state: 'unstamped', recorded: null, actual: null };
}
}

/**
* Is `packages/spec/dist` older than the sources it claims to describe? Missing
* counts as stale. Deliberately conservative: a false "stale" costs a build, a
Expand Down Expand Up @@ -276,6 +294,42 @@ export function schemaTreeIsStale(specDir = SPEC_DIR) {
* Missing counts as stale, and the direction is the same conservative one its
* two siblings take: a false "stale" costs a build, a false "fresh" costs a
* verdict nobody can trust.
*
* ## The mtime rule accuses; the BUILD stamp may acquit
*
* The blind spot `distIsStale` documents above is shared here, and was measured
* on this axis too: after a `git merge`, `git checkout` or `git worktree add`
* re-checks-out a source file with IDENTICAL bytes, the build that follows
* correctly does not run (turbo's cache hashes content, so it is a cache hit
* that rewrites nothing) and every `dist/` mtime stays put — so this rule
* refused `check:browser-reachable-entries` over bundles that were exactly
* current, and the only remedy on offer was a multi-minute rebuild under the
* shared verify lock.
*
* The evidence that answers it is `dist/.build-input-hash` — the file #7122
* proposed for the DECLARATION rule, where it was measured wrong and stays
* rejected. It is the right file HERE for a reason that is specific to this
* axis, not a relaxation of that ruling:
*
* - #7122's hole is that `--stamp` writes this file under `OS_SKIP_DTS=1`,
* which emits JS and skips the declarations. On this axis that flag emits
* exactly the artifact being vouched for — `inspectBundleFreshness`'s own
* refusal already tells the reader "OS_SKIP_DTS=1 is fine for THIS gate";
* - what makes it sound is the build script's ORDER rather than the flag:
* `packages/spec`'s `build` runs the unconditional `tsup` before `--stamp`
* in one `&&` chain, so nothing writes this stamp without having emitted
* bundles first. Only the declaration pass is conditional, and skipping it
* cannot refresh a stamp written after both;
* - the digest's input set is a strict SUPERSET of the source set measured
* above — every file under `src/` (`.test.ts` included) plus
* `tsup.config.ts` and the rest of `PACKAGE_BUILD_CONFIG` plus turbo's
* `globalDependencies` — so a `match` implies every input this rule counts
* is byte-identical to the one the bundles were emitted from. A superset
* can only ever withhold an acquittal, never grant one it should not.
*
* And it may only ACQUIT. `unstamped` — absent, unreadable, not 64 hex
* characters, or a package whose build does not stamp — leaves the mtime
* verdict standing (#4690), so nothing that passes today can start failing.
*/
export function bundlesAreStale(specDir = SPEC_DIR) {
const bundles = newestMtime(
Expand All @@ -289,7 +343,8 @@ export function bundlesAreStale(specDir = SPEC_DIR) {
);
const configPath = join(specDir, 'tsup.config.ts');
const config = existsSync(configPath) ? statSync(configPath).mtimeMs : 0;
return Math.max(src, config) > bundles;
if (Math.max(src, config) <= bundles) return false;
return buildStamp(specDir).state !== 'match';
}

/**
Expand Down
Loading