Skip to content

Commit 14cfc00

Browse files
os-steveclaude
andauthored
feat(spec): type-check packages/spec/src TSDoc examples as a third surface (#11433)
* feat(spec): type-check packages/spec/src TSDoc examples as a third surface `check:skill-examples` gains a `packages/spec/src/**` surface — the ADR-0033 authoring channel, where a schema's `@example` sits inches from the tombstone written for the same reader and, until now, was compiled by nothing. A new SURFACES entry, not a second extractor: the marker/tsc pipeline was already surface-parameterised and `commentPrefixed` already strips the JSDoc gutter. Compilation stays opt-in; six self-contained blocks are marked. Two things the surface needed: - Surfaces sharing a resolution dir must not share a throwaway build dir — it is wiped when written, so the second would delete the first's blocks. `buildDirName` + `findDuplicateBuildDir()`, pinned both directions. - The marker is a prose line, so `renderFileDescription` emitted it into generated MDX. MDX has no HTML comments, so a marker on a module-level docblock both published an internal annotation to a reference page and broke the docs build. Stripped at prose level during generation. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01T9cDbY2NBiVJWYx3BpWfH2 * test(spec): seed the spec-source example surface in the dist-freshness sandbox Adding a third check:skill-examples surface made its per-surface zero-block guard fire on the sandbox's stubbed `packages/spec/src`, ahead of the freshness refusal these cases exist to probe — the same interaction #10969 hit and solved with `seedClientSdkSurface()`. Seeded into the file `srcMtime` already governs, so the surface adds no second ungoverned mtime knob, and the surface count assertion moves 2 -> 3. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01T9cDbY2NBiVJWYx3BpWfH2 --------- Co-authored-by: Claude <noreply@anthropic.com>
1 parent d173125 commit 14cfc00

11 files changed

Lines changed: 369 additions & 10 deletions
Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
---
2+
'@objectstack/spec': patch
3+
---
4+
5+
Type-check the TSDoc code examples in `packages/spec/src` — the ADR-0033 authoring channel
6+
7+
`check:skill-examples` now scans a third surface: `packages/spec/src/**` TSDoc blocks,
8+
alongside `skills/` + `content/docs` and the client SDK sources. A schema's `@example`
9+
is what an AI author copies, and it sits inches from the tombstone written for that same
10+
reader — but nothing compiled it, so an example could name a retired key, a renamed
11+
export or a tightened union and stay green indefinitely. (`check:doc-formula-expressions`
12+
walks the same blocks but judges *formula expressions*, never TypeScript.)
13+
14+
This is a new `SURFACES` entry, not a second extractor: the existing marker/tsc pipeline
15+
was already surface-parameterised. Compilation stays **opt-in** via the `os:check`
16+
marker, which matters more on this root than anywhere else — of its 146 fenced ts blocks,
17+
128 carry no imports of their own and three more are ellipsis-placeholder prose
18+
(`defineStack({ ... })`) that is correct as documentation and can never compile. Six
19+
self-contained blocks are marked and now compile against the built declarations.
20+
21+
The marker is an inert HTML comment, as on the other surfaces, and is stripped from
22+
generated reference pages rather than published to them.

.gitignore

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -63,8 +63,10 @@ packages/spec/json-schema/
6363
# Generated bundle size reports
6464
packages/spec/bundle-size-report.json
6565

66-
# Throwaway build dir for `check:skill-examples` (extracted skill code blocks)
66+
# Throwaway build dirs for `check:skill-examples` — one per surface (a surface
67+
# is wiped when written, so surfaces sharing a resolution dir need their own).
6768
packages/spec/.examples-build/
69+
packages/spec/.examples-build-src/
6870

6971
# Generated SBOM (rebuilt during release)
7072
sbom.json

packages/spec/scripts/check-skill-examples.ts

Lines changed: 226 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -42,6 +42,18 @@
4242
* `content/docs` (a FAQ recommending `FieldSchema.extend()`, impossible since
4343
* FieldSchema became a ZodPipe); everything else was fragments.
4444
*
45+
* #10924's sweep of `packages/spec/src` re-measured both traps on a fresh
46+
* corpus and is worth recording, because the numbers argue the design: 146
47+
* fenced ts blocks, of which 128 carry no import of their own (fragments by
48+
* construction — a `columns: [...]` subtree, a `case 'in':` excerpt of a
49+
* consumer's switch). Of the 18 self-contained ones, trap 1 fired immediately:
50+
* three are ellipsis-placeholder prose (`defineStack({ ... })`, `{ ... }` in
51+
* argument position) whose TS1109 syntax errors suppressed the semantic pass
52+
* for the entire program — the first run reported 15 "clean" blocks that had
53+
* simply never been type-checked. Excluding those three and re-running turned
54+
* six of the remaining fifteen red on real semantics. So: mark deliberately,
55+
* and never read a run containing TS1xxx codes as evidence about rot.
56+
*
4557
* Each marked block is written verbatim to a throwaway build dir and type-checked
4658
* with `tsc --noEmit` against the built `@objectstack/spec` declarations — the
4759
* exact surface a consumer's `import { … } from '@objectstack/spec'` resolves to.
@@ -110,8 +122,9 @@
110122
* prose roots, its own throwaway build dir (which package's `node_modules`
111123
* a bare specifier like `react` or the surface's own package name resolves
112124
* against), and its own package(s) to derive a `paths` map from. Adding a
113-
* surface — #10924's `packages/spec/src/**` TSDoc `@example` channel is the
114-
* next candidate — is a new `SURFACES` entry, not a fork.
125+
* surface is a new `SURFACES` entry, not a fork — #10924's
126+
* `packages/spec/src/**` TSDoc channel was the next candidate and landed as
127+
* exactly that: one root, one surface entry, zero new extraction code.
115128
*
116129
* TWO differences from the original skills/docs surface, both load-bearing:
117130
*
@@ -239,6 +252,39 @@ const SKILLS_DOCS_ROOTS: SourceRoot[] = [
239252
/** `.test.ts(x)` / `.spec.ts(x)` files are not the documented SDK surface. */
240253
const isTestFile = (name: string): boolean => /\.(test|spec)\.tsx?$/.test(name);
241254

255+
/**
256+
* `packages/spec/src` — the schema package's OWN TSDoc code blocks (#10924).
257+
*
258+
* This is the ADR-0033 authoring channel: a schema's `@example` sits inches
259+
* from the tombstone written for the same reader, and is the text an AI author
260+
* copies. Until this root landed nothing compiled it — `check:doc-formula-
261+
* expressions` walks the same `@example`s but judges *formula expressions*
262+
* with `@objectstack/formula`, never TypeScript — so an example could name a
263+
* retired key, a renamed export or a tightened union and stay green
264+
* indefinitely. The measured specimen: `AgentSchema`'s own `@example` wrote
265+
* `knowledge: { … }`, a key the same file declares `retiredKey()`
266+
* (`z.never()`), which surfaces as `undefined` in `z.input` and so fails
267+
* `tsc` with TS2322 — a compile-only gate catches this class with no runner.
268+
*
269+
* Same shape as the client SDK roots above (JSDoc gutter, `commentPrefixed`),
270+
* and opt-in for the same reason, only more so: of the 146 fenced ts blocks in
271+
* this root, 128 do not even carry their own imports, and among the 18 that do,
272+
* three are ellipsis-placeholder fragments (`defineStack({ ... })`) that are
273+
* correct as prose and can never compile. A blanket sweep here would misfire on
274+
* every one of them; the marker is what separates "this is a claim" from "this
275+
* illustrates".
276+
*/
277+
const SPEC_SRC_ROOTS: SourceRoot[] = [
278+
{
279+
dir: path.resolve(SPEC_DIR, 'src'),
280+
ext: '.ts',
281+
label: 'spec',
282+
marker: '<!-- os:check -->',
283+
commentPrefixed: true,
284+
excludeFile: isTestFile,
285+
},
286+
];
287+
242288
/**
243289
* `packages/client-react/src` + `packages/client/src` — the SDK's own
244290
* hand-written TSDoc `@example` blocks (#10969). See the header comment's
@@ -278,18 +324,39 @@ interface Surface {
278324
/** Where the throwaway build dir + tsconfig.json live. Module resolution
279325
* for bare specifiers (react, workspace packages) walks up from here. */
280326
resolutionDir: string;
327+
/** Basename of the throwaway build dir inside `resolutionDir`. Two surfaces
328+
* MAY share a `resolutionDir` (spec's prose and its own source both resolve
329+
* against `packages/spec`) — they must NOT share a build dir: `writeBuildDir`
330+
* wipes it on entry, so the second surface would delete the first's blocks.
331+
* Sequential execution hides that today, but `--keep` would silently retain
332+
* only the last surface's dir. Defaults to `.examples-build`; every value
333+
* needs a `.gitignore` entry. `assertDistinctBuildDirs()` enforces it. */
334+
buildDirName?: string;
281335
/** package.json dirs whose `exports` become explicit `paths` entries —
282336
* what lets a block `import` its own surface's package by name. */
283337
selfPackages: string[];
284338
}
285339

340+
/** @see Surface.buildDirName */
341+
const DEFAULT_BUILD_DIR = '.examples-build';
342+
const buildDirOf = (s: Surface): string => path.join(s.resolutionDir, s.buildDirName ?? DEFAULT_BUILD_DIR);
343+
286344
const SURFACES: Surface[] = [
287345
{
288346
name: 'skills + docs (@objectstack/spec)',
289347
roots: SKILLS_DOCS_ROOTS,
290348
resolutionDir: SPEC_DIR,
291349
selfPackages: [SPEC_DIR],
292350
},
351+
{
352+
name: 'spec source TSDoc (@objectstack/spec)',
353+
roots: SPEC_SRC_ROOTS,
354+
// Same package as the prose surface above, so the same resolution env —
355+
// but its OWN build dir (see `Surface.buildDirName`).
356+
resolutionDir: SPEC_DIR,
357+
buildDirName: '.examples-build-src',
358+
selfPackages: [SPEC_DIR],
359+
},
293360
{
294361
name: 'client SDK (@objectstack/client-react, @objectstack/client)',
295362
roots: CLIENT_SDK_ROOTS,
@@ -976,6 +1043,84 @@ function selfTest(): never {
9761043
`marker must not be silently ignored`,
9771044
);
9781045

1046+
// ── commentPrefixed + a ```ts fence (#10924's spec-source surface). The
1047+
// tsx fixture above pins gutter-stripping against a ```tsx fence; this
1048+
// root is the other combination — a `.ts` source file whose docblock
1049+
// fences plain ```ts — and it is the one #10924's entire corpus uses.
1050+
// Without this, a regression that recognised a gutter-wrapped fence
1051+
// ONLY when its language was `tsx` would leave the whole spec-source
1052+
// surface extracting zero blocks, and the per-surface vacuous-green
1053+
// guard in `main()` is the only thing that would notice — a hard error
1054+
// a long way from its cause. Also pins that a marker sitting between
1055+
// `@example` and the fence still counts as adjacent: that is where
1056+
// every marker in the real spec corpus lives.
1057+
const specSrcRoot: SourceRoot = {
1058+
dir,
1059+
ext: '.ts',
1060+
label: 'spec',
1061+
marker: '<!-- os:check -->',
1062+
commentPrefixed: true,
1063+
excludeFile: isTestFile,
1064+
};
1065+
const specSrc = path.join(dir, 'schema.zod.ts');
1066+
fs.writeFileSync(
1067+
specSrc,
1068+
[
1069+
'// Copyright', // 1
1070+
'', // 2
1071+
'/**', // 3
1072+
' * A schema with a documented example.', // 4
1073+
' *', // 5
1074+
' * @example', // 6
1075+
' * <!-- os:check -->', // 7
1076+
' * ```ts', // 8
1077+
" * import { defineSkill } from '@objectstack/spec';", // 9
1078+
' *', // 10
1079+
" * const skill = defineSkill({ name: 'a' });", // 11
1080+
' * ```', // 12
1081+
' */', // 13
1082+
'export const SchemaLike = 1;', // 14
1083+
].join('\n'),
1084+
'utf8',
1085+
);
1086+
const specExtract = extractFromFile(specSrc, specSrcRoot);
1087+
check(
1088+
specExtract.examples.length === 1,
1089+
`spec-source fixture: extracted ${specExtract.examples.length} block(s), expected 1 — a gutter-wrapped ` +
1090+
'```ts fence is DORMANT, and every block in the spec-source corpus is exactly that shape',
1091+
);
1092+
check(
1093+
specExtract.orphans.length === 0,
1094+
`spec-source fixture: reported ${specExtract.orphans.length} orphan marker(s), expected 0 — a marker on the ` +
1095+
'line between `@example` and its fence IS adjacent',
1096+
);
1097+
if (specExtract.examples.length === 1) {
1098+
const ex = specExtract.examples[0];
1099+
check(
1100+
ex.fileName.endsWith('.ts') && !ex.fileName.endsWith('.tsx'),
1101+
`spec-source fixture: build file name "${ex.fileName}" should end in .ts for a \`\`\`ts fence`,
1102+
);
1103+
check(
1104+
ex.code.split('\n')[0] === "import { defineSkill } from '@objectstack/spec';",
1105+
`spec-source fixture: extracted body still carries a JSDoc gutter — ${JSON.stringify(ex.code.split('\n')[0])}`,
1106+
);
1107+
// body[0] is source line 9; `bodyStartLine` is what every diagnostic is
1108+
// remapped through, and an off-by-one here points authors at prose.
1109+
check(
1110+
ex.bodyStartLine === 9,
1111+
`spec-source fixture: bodyStartLine ${ex.bodyStartLine}, expected 9 — diagnostics would point at the wrong line`,
1112+
);
1113+
}
1114+
// A `.test.ts` sibling must be skipped by `excludeFile` even when it
1115+
// carries a perfectly good marked block: test fixtures are not the
1116+
// documented surface, and extracting them would type-check assertions.
1117+
const specTest = path.join(dir, 'schema.test.ts');
1118+
fs.writeFileSync(specTest, ['/**', ' * <!-- os:check -->', ' * ```ts', ' * const x = 1;', ' * ```', ' */'].join('\n'), 'utf8');
1119+
check(
1120+
sourceFiles([specSrcRoot]).every((f) => f.file !== specTest),
1121+
'spec-source fixture: a `.test.ts` file was scanned — `excludeFile` is not applied to this root',
1122+
);
1123+
9791124
// ── Fence-awareness (fenceSpans): the os:check convention has to be
9801125
// documentable in the very roots it governs. A marker shown as example
9811126
// text INSIDE some other fenced block (here a ```md illustration of
@@ -1021,6 +1166,40 @@ function selfTest(): never {
10211166
'STILL be one — a false orphan there is exactly the defect that makes this convention undocumentable, and a ' +
10221167
"silenced real orphan would weaken the gate's hard-error posture",
10231168
);
1169+
1170+
// ── Build-dir distinctness (#10924). The REAL surfaces are asserted on
1171+
// every run by `assertDistinctBuildDirs()`; these two fixtures pin the
1172+
// predicate underneath it in both directions, because a guard that can
1173+
// only ever return null is indistinguishable from a correct config —
1174+
// the dormant-checker failure this file's own docblocks keep naming.
1175+
const surfaceStub = (name: string, resolutionDir: string, buildDirName?: string): Surface => ({
1176+
name,
1177+
roots: [],
1178+
resolutionDir,
1179+
buildDirName,
1180+
selfPackages: [],
1181+
});
1182+
check(
1183+
findDuplicateBuildDir(SURFACES) === null,
1184+
'build-dir fixture: the REAL surfaces share a build dir — one of them would wipe the other',
1185+
);
1186+
const clash = findDuplicateBuildDir([
1187+
surfaceStub('alpha', SPEC_DIR),
1188+
surfaceStub('beta', SPEC_DIR), // same resolutionDir, both defaulting
1189+
]);
1190+
check(
1191+
clash !== null && clash.first === 'alpha' && clash.second === 'beta',
1192+
`build-dir fixture: two surfaces defaulting into one resolution dir were NOT reported (got ${JSON.stringify(clash)}) ` +
1193+
'— that is the collision the guard exists to catch',
1194+
);
1195+
check(
1196+
findDuplicateBuildDir([
1197+
surfaceStub('alpha', SPEC_DIR),
1198+
surfaceStub('beta', SPEC_DIR, '.examples-build-src'),
1199+
]) === null,
1200+
'build-dir fixture: a distinct `buildDirName` on a shared resolution dir was reported as a clash — ' +
1201+
'sharing a resolution dir is legitimate and must stay legal',
1202+
);
10241203
} finally {
10251204
fs.rmSync(dir, { recursive: true, force: true });
10261205
}
@@ -1036,11 +1215,52 @@ function selfTest(): never {
10361215
' JSDoc-gutter-wrapped ```tsx block (client SDK surface) extracts, strips and maps lines\n' +
10371216
' identically, and a misplaced gutter-wrapped marker is still caught as an orphan; a marker\n' +
10381217
' shown as example text inside another fenced block is not an orphan, while a genuine\n' +
1039-
' top-level misplaced marker still is.',
1218+
' top-level misplaced marker still is; a gutter-wrapped ```ts block (spec-source\n' +
1219+
' surface) extracts with the right build extension, body and line mapping, its\n' +
1220+
' `.test.ts` sibling is skipped, and two surfaces sharing one build dir are caught.',
10401221
);
10411222
process.exit(0);
10421223
}
10431224

1225+
/**
1226+
* No two surfaces may write into the same throwaway build dir.
1227+
*
1228+
* `writeBuildDir()` wipes its target on entry, so a shared dir means the
1229+
* second surface deletes the first's extracted blocks. Sequential execution
1230+
* makes that invisible today — each surface's `tsc` has already run by then —
1231+
* which is exactly why it is asserted rather than left to be noticed: the
1232+
* first symptom would be `--keep` retaining only the last surface's dir, and
1233+
* the first *real* symptom would be a surface silently type-checking another
1234+
* surface's blocks the day this loop is reordered or parallelised. Sharing a
1235+
* `resolutionDir` is legitimate and stays legal (#10924's spec-source surface
1236+
* resolves against `packages/spec` just as the prose surface does); sharing
1237+
* the dir underneath it is not.
1238+
*/
1239+
function findDuplicateBuildDir(
1240+
surfaces: Surface[],
1241+
): { dir: string; first: string; second: string } | null {
1242+
const seen = new Map<string, string>();
1243+
for (const surface of surfaces) {
1244+
const dir = buildDirOf(surface);
1245+
const first = seen.get(dir);
1246+
if (first) return { dir, first, second: surface.name };
1247+
seen.set(dir, surface.name);
1248+
}
1249+
return null;
1250+
}
1251+
1252+
function assertDistinctBuildDirs(): void {
1253+
const clash = findDuplicateBuildDir(SURFACES);
1254+
if (clash) {
1255+
fail(
1256+
`Surfaces "${clash.first}" and "${clash.second}" both use the build dir ${rel(clash.dir)}.\n\n` +
1257+
` A build dir is wiped when it is written, so the second surface would delete the\n` +
1258+
` first's extracted blocks. Give one of them a distinct \`buildDirName\` (and add it\n` +
1259+
` to .gitignore).`,
1260+
);
1261+
}
1262+
}
1263+
10441264
/** A package.json's own `name` field — used to look its self-entry up in the
10451265
* `paths` map `surfacePaths()` derived from it. */
10461266
function pkgName(pkgDir: string): string {
@@ -1050,6 +1270,8 @@ function pkgName(pkgDir: string): string {
10501270
function main() {
10511271
if (SELF_TEST) selfTest();
10521272

1273+
assertDistinctBuildDirs();
1274+
10531275
console.log(`🧪 Type-checking prose TypeScript examples (${SURFACES.map((s) => s.name).join(' · ')})...\n`);
10541276

10551277
const bySurface = SURFACES.map((surface) => {
@@ -1191,7 +1413,7 @@ function main() {
11911413
);
11921414
}
11931415

1194-
const buildDir = path.join(surface.resolutionDir, '.examples-build');
1416+
const buildDir = buildDirOf(surface);
11951417
buildDirs.push(buildDir);
11961418
writeBuildDir(buildDir, examples, paths);
11971419
const { code, output } = runTsc(buildDir);

packages/spec/scripts/dist-freshness-adoption.test.ts

Lines changed: 34 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -137,7 +137,33 @@ function declaredDts(spec: string): string[] {
137137
*/
138138
function seed(spec: string, { distMtime, srcMtime }: { distMtime: number; srcMtime: number }): void {
139139
fs.mkdirSync(path.join(spec, 'src'), { recursive: true });
140-
fs.writeFileSync(path.join(spec, 'src/index.ts'), 'export const live = 1;\n');
140+
// #10924 added `packages/spec/src` as check-skill-examples's THIRD surface, so
141+
// this seeded `src/` is now scanned for marked blocks too. Same reasoning as
142+
// `seedClientSdkSurface()` above, and the same failure it prevents: without a
143+
// marked block here that surface's zero-block guard fires on a directory this
144+
// sandbox merely stubs, and it fires BEFORE the freshness check — so the
145+
// staleness cases below would assert against "no marked examples found"
146+
// instead of the refusal they exist to probe.
147+
//
148+
// It goes in the SAME file `srcMtime` already governs rather than a sibling:
149+
// freshness compares dist against the newest source, so a second src file
150+
// would be a second, ungoverned mtime knob. The block imports nothing, which
151+
// is what lets it compile against the trivial `export {}` declarations
152+
// `declaredDts()` seeds and keeps the fresh-dist positive control honest.
153+
fs.writeFileSync(
154+
path.join(spec, 'src/index.ts'),
155+
[
156+
'/**',
157+
' * @example',
158+
' * <!-- os:check -->',
159+
' * ```ts',
160+
' * export const ok = 1;',
161+
' * ```',
162+
' */',
163+
'export const live = 1;',
164+
'',
165+
].join('\n'),
166+
);
141167
fs.utimesSync(path.join(spec, 'src/index.ts'), srcMtime, srcMtime);
142168

143169
for (const dts of declaredDts(spec)) {
@@ -304,9 +330,13 @@ describe('check:skill-examples refuses a stale dist (#7181)', () => {
304330
const run = runGate(tree.spec, SKILL);
305331
expect(run.status).toBe(0);
306332
// #10969: the surface-parameterised success message names surface COUNT,
307-
// not any one package -- it now covers this skills+docs surface AND the
308-
// client-SDK one `seedClientSdkSurface` populates in every case in this file.
309-
expect(run.stdout).toContain('prose examples type-check across 2 surface(s)');
333+
// not any one package -- it now covers this skills+docs surface, the
334+
// client-SDK one `seedClientSdkSurface` populates in every case in this
335+
// file, and (#10924) the spec-source one `seed()` populates. The count is
336+
// asserted literally, and moving it is the point: every surface added has
337+
// to be seeded here deliberately, or its zero-block guard fires ahead of
338+
// the refusal these cases probe.
339+
expect(run.stdout).toContain('prose examples type-check across 3 surface(s)');
310340
expect(run.stderr).not.toContain('OLDER than');
311341
}, 60_000);
312342

0 commit comments

Comments
 (0)