Skip to content

Commit 6d8e95d

Browse files
dmealingclaude
andcommitted
feat(cli): a libraries key, so a shipped library is loadable through the toolchain (#333)
`librarySources` was reachable only from `MetaDataLoader.fromDirectory`. No CLI command uses that factory — every one of them goes through `loadMemory` with a resolved file list — so `libraries` existed on the loader and reached nothing. The generators that consume a library ARE registered for the CLI, which is the sharp edge: the generator was reachable from the command line while its input was not, and an adopter following the documented `extends: "metaobjects::ai::LlmCallBase"` path got ERR_UNRESOLVED_SUPER pointing at their own metadata. `metaobjects.config.ts` gains `libraries?: readonly string[]`, beside `providers` because it answers the same shape of question — what does this project's model need in scope beyond the files it declares. Python took the same key on its project config, and the two ports agreeing on the name and the semantics is the point. Threaded to `loadMemory` by all eight load sites (gen, verify, docs, prompt-snapshot, and migrate's four) through one `loadMemoryOptionsFrom` helper rather than a spread pair copied eight times. That is the actual lesson of this bug: `providers` reached every command and `libraries` reached none, so the fix that leaves eight independent opportunities to thread one and forget the other has not fixed the class. Design points worth keeping: - Opt-in, never automatic. A library package registers real top-level nodes, and a project that never references one should not find them in its model, its generated output or its docs. The negative test arm exists for this: without the key the same model must still fail to load. A positive-only test keeps passing if the load is quietly made unconditional, which is exactly the regression that would matter. - An unknown name is a HARD config error naming the available packages, while `librarySources` keeps skipping one silently for a programmatic caller. An API caller asking for a package this version does not ship should still be able to load its own metadata; a name a human typed into a config file is a mistake worth failing on, because skipped it resurfaces later as ERR_UNRESOLVED_SUPER against the adopter's OWN metadata — the wrong place to send someone looking. Python draws the same line in the same place. - `librarySources` is imported lazily through a new `@metaobjectsdev/metadata/library` subpath, never from the root barrel. It reads `node:fs`, and a root-reachable static import drags Node built-ins into every consumer's module graph — the #287 bundle defect. The subpath exists for exactly the reason `./constants` does. - Library sources are PREPENDED, matching `fromDirectory`. Super resolution is order-independent, so this is determinism rather than correctness. `knownLibraryPackages()` is new, so the config validator can say what IS available instead of only what is not. cli: 630 pass / 3 skip / 0 fail. sdk: 278 pass / 0 fail. The help snapshot moved because `meta docs`' `<metadata>` line now states that passing it scopes the run (#327). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
1 parent 991ed9f commit 6d8e95d

14 files changed

Lines changed: 284 additions & 46 deletions

File tree

‎docs/features/cli.md‎

Lines changed: 39 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -105,6 +105,7 @@ for editor autocomplete and non-Python validation.
105105
metadata: metaobjects # optional, default "metaobjects" — relative to this file
106106
providers: # optional; "module:symbol" refs, resolved config-relative (no PYTHONPATH=)
107107
- my_project.providers:register_custom_types
108+
libraries: [ai] # optional; MetaObjects-shipped library packages (see below)
108109
targets:
109110
api:
110111
outDir: src/generated/api
@@ -142,6 +143,44 @@ targets:
142143
consulted. Config mode activates only when no positional `<metadata_dir>`
143144
is given.
144145

146+
## `libraries` — opting into a MetaObjects-shipped library package
147+
148+
MetaObjects ships a small set of standard metadata packages under `library/`. A project
149+
opts into one by name, and its nodes become available to `extends`:
150+
151+
```ts
152+
// metaobjects.config.ts (Node `meta`)
153+
export default defineConfig({
154+
libraries: ["ai"], // makes metaobjects::ai::LlmCallBase resolvable
155+
generators: [entityFile()],
156+
});
157+
```
158+
159+
```yaml
160+
# metaobjects.config.yaml (Python `metaobjects`)
161+
libraries: [ai]
162+
```
163+
164+
```jsonc
165+
// then, in your own metadata
166+
{ "object.entity": { "name": "AgentCall", "extends": "metaobjects::ai::LlmCallBase", ... } }
167+
```
168+
169+
- **Opt-in, never automatic.** A library package registers real top-level nodes. A project
170+
that never references one should not find them in its model, its generated output or its
171+
docs — so nothing is loaded until the key names it.
172+
- **An unknown name is a hard config error** that lists the packages this version ships.
173+
The programmatic loader API skips an unrecognised package instead, deliberately: an API
174+
caller asking for something a given version does not ship should still be able to load
175+
its own metadata, but a name a human typed into a config file is a mistake worth failing
176+
on — skipped, it resurfaces later as `ERR_UNRESOLVED_SUPER` pointing at the adopter's own
177+
metadata, which is the wrong place to send someone looking.
178+
- **Every command that loads metadata honours it** — `gen`, `verify`, `docs`, `migrate`,
179+
`prompt-snapshot`. It was previously reachable only from the programmatic
180+
`MetaDataLoader.fromDirectory`, which no CLI uses, so a generator that consumes a library
181+
was registered *for* the command line while its input was unreachable *through* it
182+
([#333](https://github.com/metaobjectsdev/metaobjects/issues/333)).
183+
145184
## `meta gen` / `meta verify` run an advisory anti-pattern pass (Node `meta`)
146185

147186
Both `meta verify` and a real `meta gen` write run (not `--dry-run`) end with a

‎server/typescript/packages/cli/src/commands/docs.ts‎

Lines changed: 8 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,7 @@
1414
import { resolve as resolvePath, basename } from "node:path";
1515
import { mkdir, writeFile } from "node:fs/promises";
1616
import { log } from "../lib/log.js";
17-
import { loadMetaobjectsConfig, resolveGenConfigDir } from "../lib/load-metaobjects-config.js";
17+
import { loadMemoryOptionsFrom, loadMetaobjectsConfig, resolveGenConfigDir } from "../lib/load-metaobjects-config.js";
1818
import { loadMemory, resolveCollection, resolveConfigDir, type Collection } from "@metaobjectsdev/sdk";
1919
import { existsSync } from "node:fs";
2020
import { join } from "node:path";
@@ -284,6 +284,10 @@ export async function docsCommand(args: string[], cwd: string): Promise<number>
284284
// if the metadata genuinely uses an unregistered type.
285285
let loadedConfig: Awaited<ReturnType<typeof loadMetaobjectsConfig>> | undefined;
286286
let configProviders: NonNullable<Awaited<ReturnType<typeof loadMetaobjectsConfig>>["providers"]> | undefined;
287+
// The same config's contributions to the metadata LOAD — providers plus the shipped
288+
// `libraries` a project opts into (#333). `configProviders` stays separate because the
289+
// site surface has its own loader and takes providers alone.
290+
let configLoadOptions: ReturnType<typeof loadMemoryOptionsFrom> = {};
287291
// hasConfig gates the api surface: api docs describe the GENERATED REST
288292
// surface, which only exists when there is a (loadable) gen config. A config
289293
// that EXISTS but fails to load degrades to model-only with a warning.
@@ -297,13 +301,15 @@ export async function docsCommand(args: string[], cwd: string): Promise<number>
297301
try {
298302
loadedConfig = await loadMetaobjectsConfig(genConfigDir);
299303
configProviders = loadedConfig.providers;
304+
configLoadOptions = loadMemoryOptionsFrom(loadedConfig);
300305
} catch (err) {
301306
log.warn(
302307
`docs: metaobjects.config.ts failed to load (${(err as Error).message}); ` +
303308
`generating docs without its providers`,
304309
);
305310
loadedConfig = undefined;
306311
configProviders = undefined;
312+
configLoadOptions = {};
307313
}
308314
}
309315

@@ -347,7 +353,7 @@ export async function docsCommand(args: string[], cwd: string): Promise<number>
347353
try {
348354
root = await loadMemory(collection.configDir, {
349355
files: collection.files,
350-
...(configProviders !== undefined ? { providers: configProviders } : {}),
356+
...configLoadOptions,
351357
});
352358
} catch (err) {
353359
log.error(`docs: failed to load metadata: ${(err as Error).message}`);

‎server/typescript/packages/cli/src/commands/gen.ts‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
import { relative } from "node:path";
22
import { parseGenArgs } from "../lib/args.js";
33
import { resolveGenConfig } from "../lib/config.js";
4-
import { loadMetaobjectsConfig, resolveGenConfigDir } from "../lib/load-metaobjects-config.js";
4+
import { loadMemoryOptionsFrom, loadMetaobjectsConfig, resolveGenConfigDir } from "../lib/load-metaobjects-config.js";
55
import { formatGenResult, formatGenResultToon, type GenFileEntry, type GenFileStatus } from "../lib/output.js";
66
import { formatGenResultJson } from "../lib/output-json.js";
77
import type { OutputFormat } from "../lib/format.js";
@@ -99,7 +99,7 @@ export async function genCommand(args: string[], cwd: string, fmt: OutputFormat
9999
try {
100100
metadata = await loadMemory(collection.configDir, {
101101
files: collection.files,
102-
...(forgeConfig.providers !== undefined ? { providers: forgeConfig.providers } : {}),
102+
...loadMemoryOptionsFrom(forgeConfig),
103103
});
104104
} catch (err) {
105105
log.error(`failed to load metadata: ${(err as Error).message}`);

‎server/typescript/packages/cli/src/commands/migrate.ts‎

Lines changed: 19 additions & 25 deletions
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,7 @@ import { toonEncode } from "../lib/format.js";
1212
import { buildKyselyFromUrl, redactUrl } from "../lib/kysely.js";
1313
import { log } from "../lib/log.js";
1414
import { loadMemory, resolveCollection, resolveConfigDir, type Collection } from "@metaobjectsdev/sdk";
15-
import { loadMetaobjectsConfig, resolveGenConfigDir } from "../lib/load-metaobjects-config.js";
15+
import { loadMemoryOptionsFrom, loadMetaobjectsConfig, resolveGenConfigDir } from "../lib/load-metaobjects-config.js";
1616
import { migrateScopeMismatch, outOfScopeNote } from "../lib/migrate-scope.js";
1717
import {
1818
buildExpectedSchemaWithProvenance,
@@ -601,12 +601,11 @@ export async function migrateCommand(
601601
// Best-effort load of metaobjects.config.ts to pick up consumer-supplied
602602
// providers. migrate's postgres/sqlite path also reads the config later
603603
// for columnNamingStrategy; we load it once here and reuse below.
604-
let postgresConfigProviders: readonly import("@metaobjectsdev/codegen-ts").MetaDataTypeProvider[] | undefined;
604+
let postgresLoadOptions: ReturnType<typeof loadMemoryOptionsFrom> = {};
605605
try {
606-
const forgeConfig = await loadMetaobjectsConfig(genRoot);
607-
postgresConfigProviders = forgeConfig.providers;
606+
postgresLoadOptions = loadMemoryOptionsFrom(await loadMetaobjectsConfig(genRoot));
608607
} catch {
609-
postgresConfigProviders = undefined;
608+
postgresLoadOptions = {};
610609
}
611610

612611
// Discovery and load are two separate failure modes, kept in separate try blocks
@@ -626,7 +625,7 @@ export async function migrateCommand(
626625
try {
627626
metadata = await loadMemory(collection.configDir, {
628627
files: collection.files,
629-
...(postgresConfigProviders !== undefined ? { providers: postgresConfigProviders } : {}),
628+
...postgresLoadOptions,
630629
});
631630
} catch (err) {
632631
log.error(`failed to load metadata: ${(err as Error).message}`);
@@ -1029,19 +1028,17 @@ export async function runBaseline(
10291028
`your metadata; for a new/empty database run \`${greenfieldCreateCmd(config.dialect)}\` instead.`,
10301029
);
10311030
let metadata;
1032-
// Load metaobjects.config.ts ONCE, up front, for BOTH the consumer providers
1031+
// Load metaobjects.config.ts ONCE, up front, for BOTH the consumer providers/libraries
10331032
// and the columnNamingStrategy — mirroring the DB path (and `meta gen`) so
10341033
// offline baseline resolves config-registered custom subtypes too (#157).
1035-
let baselineConfigProviders:
1036-
| readonly import("@metaobjectsdev/codegen-ts").MetaDataTypeProvider[]
1037-
| undefined;
1034+
let baselineLoadOptions: ReturnType<typeof loadMemoryOptionsFrom> = {};
10381035
let baselineStrategy: "snake_case" | "literal" | "kebab-case" = "snake_case";
10391036
try {
10401037
const cfg = await loadMetaobjectsConfig(genRoot);
1041-
baselineConfigProviders = cfg.providers;
1038+
baselineLoadOptions = loadMemoryOptionsFrom(cfg);
10421039
if (cfg.columnNamingStrategy) baselineStrategy = cfg.columnNamingStrategy;
10431040
} catch {
1044-
// config absent — no custom providers, default snake_case
1041+
// config absent — no custom providers, no libraries, default snake_case
10451042
}
10461043
// `baseline` records a STARTING POINT, so it is deliberately NOT scoped: the
10471044
// `--from-db` arm captures whatever the database holds (there is no provenance
@@ -1056,7 +1053,7 @@ export async function runBaseline(
10561053
const collection = await resolveCollection(metaRoot);
10571054
metadata = await loadMemory(collection.configDir, {
10581055
files: collection.files,
1059-
...(baselineConfigProviders !== undefined ? { providers: baselineConfigProviders } : {}),
1056+
...baselineLoadOptions,
10601057
});
10611058
} catch (err) {
10621059
log.error(`migrate baseline: failed to load metadata: ${(err as Error).message}`);
@@ -1176,19 +1173,17 @@ export async function runOfflineGenerate(
11761173
log.error(`migrate: --dialect required for offline generation (or use --from-db)`);
11771174
return 2;
11781175
}
1179-
// Load metaobjects.config.ts ONCE, up front, for BOTH the consumer providers
1176+
// Load metaobjects.config.ts ONCE, up front, for BOTH the consumer providers/libraries
11801177
// and the columnNamingStrategy — mirroring the DB path (and `meta gen`) so
11811178
// offline generate resolves config-registered custom subtypes too (#157).
1182-
let offlineConfigProviders:
1183-
| readonly import("@metaobjectsdev/codegen-ts").MetaDataTypeProvider[]
1184-
| undefined;
1179+
let offlineLoadOptions: ReturnType<typeof loadMemoryOptionsFrom> = {};
11851180
let offlineStrategy: "snake_case" | "literal" | "kebab-case" = "snake_case";
11861181
try {
11871182
const cfg = await loadMetaobjectsConfig(genRoot);
1188-
offlineConfigProviders = cfg.providers;
1183+
offlineLoadOptions = loadMemoryOptionsFrom(cfg);
11891184
if (cfg.columnNamingStrategy) offlineStrategy = cfg.columnNamingStrategy;
11901185
} catch {
1191-
// config absent — no custom providers, default snake_case
1186+
// config absent — no custom providers, no libraries, default snake_case
11921187
}
11931188

11941189
let metadata;
@@ -1197,7 +1192,7 @@ export async function runOfflineGenerate(
11971192
collection = await resolveCollection(metaRoot);
11981193
metadata = await loadMemory(collection.configDir, {
11991194
files: collection.files,
1200-
...(offlineConfigProviders !== undefined ? { providers: offlineConfigProviders } : {}),
1195+
...offlineLoadOptions,
12011196
});
12021197
} catch (err) {
12031198
log.error(`migrate: failed to load metadata: ${(err as Error).message}`);
@@ -1440,12 +1435,11 @@ async function runD1Migrate(
14401435

14411436
// 3. Load metadata. Best-effort config read for consumer providers; falls
14421437
// back to default core+forge bundle if metaobjects.config.ts is absent.
1443-
let d1ConfigProviders: readonly import("@metaobjectsdev/codegen-ts").MetaDataTypeProvider[] | undefined;
1438+
let d1LoadOptions: ReturnType<typeof loadMemoryOptionsFrom> = {};
14441439
try {
1445-
const forgeConfig = await loadMetaobjectsConfig(genRoot);
1446-
d1ConfigProviders = forgeConfig.providers;
1440+
d1LoadOptions = loadMemoryOptionsFrom(await loadMetaobjectsConfig(genRoot));
14471441
} catch {
1448-
d1ConfigProviders = undefined;
1442+
d1LoadOptions = {};
14491443
}
14501444

14511445
// Discovery and load are separate failure modes (the `meta gen` pattern);
@@ -1463,7 +1457,7 @@ async function runD1Migrate(
14631457
try {
14641458
metadata = await loadMemory(collection.configDir, {
14651459
files: collection.files,
1466-
...(d1ConfigProviders !== undefined ? { providers: d1ConfigProviders } : {}),
1460+
...d1LoadOptions,
14671461
});
14681462
} catch (err) {
14691463
log.error(`migrate: failed to load metadata: ${(err as Error).message}`);

‎server/typescript/packages/cli/src/commands/prompt-snapshot.ts‎

Lines changed: 9 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,7 @@ import { parsePromptSnapshotArgs } from "../lib/args.js";
1414
import { log } from "../lib/log.js";
1515
import { FileProvider } from "../lib/file-provider.js";
1616
import { snapshotPaths, unifiedDiff } from "../lib/snapshot.js";
17-
import { loadMetaobjectsConfig, resolveGenConfigDir } from "../lib/load-metaobjects-config.js";
17+
import { loadMemoryOptionsFrom, loadMetaobjectsConfig, resolveGenConfigDir } from "../lib/load-metaobjects-config.js";
1818
import { loadMemory, resolveCollection } from "@metaobjectsdev/sdk";
1919
import { TYPE_TEMPLATE, TEMPLATE_ATTR_TEXT_REF, TEMPLATE_ATTR_FORMAT } from "@metaobjectsdev/metadata";
2020
import { render, ESCAPERS, type RenderFormat } from "@metaobjectsdev/render";
@@ -53,28 +53,29 @@ export async function promptSnapshotCommand(args: string[], cwd: string): Promis
5353
// invocation that worked before metadata sources were resolvable at all.
5454
const projectRoot = collection.configDir;
5555

56-
// Best-effort load of metaobjects.config.ts to pick up consumer-supplied
57-
// providers. prompt-snapshot doesn't require codegen config; if it's absent
56+
// Best-effort load of metaobjects.config.ts to pick up consumer-supplied providers
57+
// and shipped `libraries`. prompt-snapshot doesn't require codegen config; if it's absent
5858
// or invalid, fall back to defaults — the loader still works for any
5959
// metadata that only uses core+forge subtypes.
60-
let configProviders: NonNullable<Awaited<ReturnType<typeof loadMetaobjectsConfig>>["providers"]> | undefined;
60+
let configLoadOptions: ReturnType<typeof loadMemoryOptionsFrom> = {};
6161
try {
6262
// `metaobjects.config.ts` gets its own nearest-ancestor walk, not `projectRoot`:
6363
// it answers a different question from `.metaobjects/config.json` (design §4.6)
6464
// and in a Maven- or pip-rooted monorepo the two legitimately sit in different
6565
// directories, so reading it from the collection's would silently drop this
6666
// package's providers (#326). Same path whenever the two files sit together.
67-
const forgeConfig = await loadMetaobjectsConfig(resolveGenConfigDir(cwd, projectRoot));
68-
configProviders = forgeConfig.providers;
67+
configLoadOptions = loadMemoryOptionsFrom(
68+
await loadMetaobjectsConfig(resolveGenConfigDir(cwd, projectRoot)),
69+
);
6970
} catch {
70-
configProviders = undefined;
71+
configLoadOptions = {};
7172
}
7273

7374
let root;
7475
try {
7576
root = await loadMemory(collection.configDir, {
7677
files: collection.files,
77-
...(configProviders !== undefined ? { providers: configProviders } : {}),
78+
...configLoadOptions,
7879
});
7980
} catch (err) {
8081
log.error(`failed to load metadata: ${(err as Error).message}`);

‎server/typescript/packages/cli/src/commands/verify.ts‎

Lines changed: 6 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -15,7 +15,7 @@ import { warnIfManifestIgnored } from "../lib/manifest-ignored-check.js";
1515
import { scanSourceForAntiPatterns } from "../lib/anti-patterns.js";
1616
import { FileProvider } from "../lib/file-provider.js";
1717
import { derivePayloadFieldTree } from "../lib/payload-field-tree.js";
18-
import { loadMetaobjectsConfig, resolveGenConfigDir } from "../lib/load-metaobjects-config.js";
18+
import { loadMemoryOptionsFrom, loadMetaobjectsConfig, resolveGenConfigDir } from "../lib/load-metaobjects-config.js";
1919
import { computeCodegenDrift } from "../lib/codegen-drift.js";
2020
import { checkRequirements, summariseRequirements } from "../lib/requirement-check.js";
2121
import { checkVerifiedBy } from "../lib/verified-by-scan.js";
@@ -204,15 +204,18 @@ export async function verifyCommand(
204204
} catch {
205205
forgeConfig = undefined;
206206
}
207-
const configProviders = forgeConfig?.providers;
207+
// Both of the gen config's contributions to the load, together — see
208+
// `loadMemoryOptionsFrom`. Threading `providers` and forgetting `libraries` is how
209+
// a shipped library became unloadable through the CLI (#333).
210+
const configLoadOptions = loadMemoryOptionsFrom(forgeConfig);
208211

209212
// ADR-0023 strict-by-default (#96): verify loads strict unless --lax is passed,
210213
// so an undeclared/typo'd own @attr fails verify (matching Java's Maven goal).
211214
let root: Awaited<ReturnType<typeof loadMemory>>;
212215
try {
213216
root = await loadMemory(collection.configDir, {
214217
files: collection.files,
215-
...(configProviders !== undefined ? { providers: configProviders } : {}),
218+
...configLoadOptions,
216219
strict: !flags.lax,
217220
});
218221
} catch (err) {

‎server/typescript/packages/cli/src/index.ts‎

Lines changed: 1 addition & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -41,9 +41,7 @@ EXPORT FLAGS:
4141
--out <file> Write output to a file (default: stdout)
4242
4343
DOCS FLAGS:
44-
<metadata> Project root to resolve metadata from. Passing it SCOPES the run to
45-
that directory's own sources; no ancestor .metaobjects/config.json is
46-
consulted. Omitted (default), the project is discovered by walking up.
44+
<metadata> Project root to resolve metadata from; passing it SCOPES the run (default: cwd)
4745
--out <dir>, -o Output directory for the pages (default: ./docs)
4846
--templates <dir> Project root to resolve adopter templates/ overrides (default: <metadata>)
4947
--prompts <dir> Extra dir holding prompt .mustache sources for --site (e.g. data/templates/)

0 commit comments

Comments
 (0)