diff --git a/.changeset/os-create-emits-an-installable-project.md b/.changeset/os-create-emits-an-installable-project.md new file mode 100644 index 0000000000..f4da992bd5 --- /dev/null +++ b/.changeset/os-create-emits-an-installable-project.md @@ -0,0 +1,29 @@ +--- +"@objectstack/cli": minor +--- + +`os create` now emits a project that installs outside this monorepo. + +Every project the command scaffolded declared its `@objectstack/*` dependencies +with pnpm's `workspace:*` protocol, extended a `tsconfig.json` two directories +above itself, and was written into this repository's own `packages/plugins/` or +`examples/` by default — so a developer following the documented command got a +project `pnpm install` refuses. The default emission is now standalone: + +- `@objectstack/*` dependencies are published semver ranges pinned to the + version of the CLI that generated them; +- the emitted `tsconfig.json` is self-contained and extends nothing; +- the project is written to `./` in the current directory (or `--dir`); +- a `pnpm-workspace.yaml` carries the build approvals a fresh `pnpm install` + needs on pnpm 11. + +The `plugin` template also emits `init` where it used to emit `initialize`. +`initialize` is not part of the `Plugin` contract, so the scaffold did not +type-check under its own `strict` config (TS7006 on the untyped `context` +parameter) and `kernel.use()` refused the plugin at load with +`Plugin init function is required` — a defect the kernel protocol docs +previously carried a warning about instead of a fix. + +The previous monorepo-internal placement is still available for ObjectStack +platform work as the explicit `--in-repo` flag, which keeps the `workspace:*` +specs and writes into `packages/plugins/` or `examples/`. diff --git a/.github/workflows/os-create-smoke.yml b/.github/workflows/os-create-smoke.yml new file mode 100644 index 0000000000..9dbebd34d8 --- /dev/null +++ b/.github/workflows/os-create-smoke.yml @@ -0,0 +1,123 @@ +# `os create` scaffold smoke — the emitted project installs and builds OUTSIDE +# this monorepo (#14824). +# +# ## What this gate holds +# +# `os create` is presented on four public documentation pages as a user-facing +# scaffolder, and every project it emitted was monorepo-shaped: `workspace:*` +# dependency specs, a `tsconfig.json` extending `'../../tsconfig.json'`, and a +# default output directory inside this repository. A reader who followed the +# docs got a project `pnpm install` refuses. The maintainer ruled that a +# documented developer-facing command must work for the developer who follows +# the docs, and attached an executable criterion: scaffold each template into a +# temporary directory outside the repository, install it with the registry, and +# boot / typecheck it — green outside the monorepo, ON CI, not on a developer +# box. `scripts/create-scaffold-smoke.sh` is that criterion; this workflow is +# the "on CI" half. +# +# ## Why paths-filtered rather than label-gated opt-in +# +# `pack-smoke-optin.yml` covers a different defect class — an author widening +# the unauthenticated surface — where the only reliable trigger is the author +# recognising their own change, so a label is the honest shape and its header +# forbids growing a `paths:` filter. This gate's defect class is the opposite: +# it can only be introduced by editing a bounded, nameable set of files, and +# those files are the `paths:` below. A label would mean a scaffolder change +# could be merged by anyone who did not think to apply it, which is precisely +# how the emitted contract drifted into being uninstallable in the first place. +# +# `init.ts` is in the set even though this gate does not test `os init`. The +# standalone emission CALLS that module — `getCliVersion`, `SCAFFOLD_PNPM_RANGE` +# and `renderPnpmWorkspaceYaml` are its exports — so its build approvals and its +# version resolution decide whether an `os create` scaffold installs. Naming the +# consumer and not the producer is the shape of coupling that lets a gate sit +# green through the change that breaks it. +# +# The nightly run is the backstop for everything the `paths:` set cannot name: a +# scaffolded project resolves the whole `@objectstack/*` graph, so a change in +# any of those packages can break its install or its boot without touching a +# single file listed here. +# +# ⛔ Not a required context — `scripts/check-required-contexts.mjs` owns that +# registry, and a paths-filtered job cannot be required: on a PR that does not +# trip the filter it never reports, and branch protection would block forever. +# It is advisory in the same way `scaffold-e2e.yml` is. +# +# Every step below is part of a build / scaffold / install / build pipeline, not +# a named local verification a dev pre-runs with `pnpm check:x`: +# dispatch-gates: no-check-families -- scaffold + install + build pipeline, no named local check family exists for it + +name: OS Create Smoke + +on: + pull_request: + branches: + - main + paths: + - 'packages/cli/src/commands/create.ts' + - 'packages/cli/src/commands/init.ts' + - 'scripts/create-scaffold-smoke.sh' + - 'scripts/publish-smoke-pack.mjs' + - '.github/workflows/os-create-smoke.yml' + schedule: + - cron: '41 4 * * *' + workflow_dispatch: + +permissions: + contents: read + +jobs: + create-scaffold-smoke: + name: Scaffold outside the monorepo, install, build + runs-on: ubuntu-latest + timeout-minutes: 60 + concurrency: + group: os-create-smoke-${{ github.event.pull_request.number || github.ref }} + cancel-in-progress: true + steps: + # No `ref:` — on `pull_request` the default checkout is `refs/pull/N/merge`, + # the merge preview, which is what `main` will actually contain. + - name: Checkout + uses: actions/checkout@v7 + + - name: Setup Node.js + uses: actions/setup-node@v7 + with: + node-version: '22' + + - name: Setup pnpm + uses: ./.github/actions/setup-pnpm + + - name: Get pnpm store directory + shell: bash + run: echo "STORE_PATH=$(pnpm store path --silent)" >> $GITHUB_ENV + + - name: Setup pnpm cache + uses: actions/cache@v6 + with: + path: ${{ env.STORE_PATH }} + key: ${{ runner.os }}-pnpm-store-v3-${{ hashFiles('**/pnpm-lock.yaml') }} + restore-keys: | + ${{ runner.os }}-pnpm-store-v3- + + - name: Setup turbo cache + uses: actions/cache@v6 + with: + path: .turbo/cache + key: ${{ runner.os }}-turbo-${{ github.job }}-${{ github.sha }} + restore-keys: | + ${{ runner.os }}-turbo-${{ github.job }}- + ${{ runner.os }}-turbo- + + - name: Install dependencies + run: pnpm install --frozen-lockfile + + # The smoke's own prerequisite, asserted by the script itself: it refuses + # to run without packages/cli/dist and the `os` bin. + - name: Build + run: pnpm run build + + # ⛔ Do not add flags or env here to make a red go away. A refusal this + # script reports is a refusal a developer following the docs would get. + - name: Scaffold smoke (packed tarballs, outside the repo) + run: bash scripts/create-scaffold-smoke.sh diff --git a/content/docs/deployment/cli.mdx b/content/docs/deployment/cli.mdx index cb77f1e46a..c2237a5367 100644 --- a/content/docs/deployment/cli.mdx +++ b/content/docs/deployment/cli.mdx @@ -1190,7 +1190,7 @@ only. | Command | Alias | Description | |---------|-------|-------------| | `os generate ` | `os g` | Generate metadata files | -| `os create [name]` | | Create a new package from template | +| `os create [name]` | | Scaffold a standalone plugin or example project | #### `os generate` (alias: `os g`) @@ -1263,13 +1263,32 @@ third-party extension primitive, authored as `src/skills/.skill.ts` with #### `os create` -Creates new packages from built-in templates (for monorepo-level scaffolding): +Scaffolds a **standalone** project — a plugin, or an example application — into +the current directory: ```bash -os create plugin analytics # Create packages/plugins/plugin-analytics -os create example my-app # Create examples/my-app +os create plugin analytics # Create ./plugin-analytics +os create example my-app # Create ./my-app + +cd plugin-analytics +pnpm install +pnpm build ``` +The emitted `package.json` declares its `@objectstack/*` dependencies as +published semver ranges pinned to the version of the CLI that generated it, and +the emitted `tsconfig.json` is self-contained, so the project installs and +builds anywhere — a workspace around it is neither needed nor assumed. + +**Options:** +- `-d, --dir ` — Write the project here instead of `./` +- `--in-repo` — Scaffold **inside an ObjectStack monorepo checkout** instead + (`packages/plugins/` for a plugin, `examples/` for an example), + with `workspace:*` dependencies and a `tsconfig.json` that extends the + repository root config. For ObjectStack platform work only: the project it + writes installs nowhere else, and the command refuses the flag when the + current directory is not a pnpm workspace root. + ### Quality | Command | Description | diff --git a/content/docs/plugins/index.mdx b/content/docs/plugins/index.mdx index 9fd7aaa43e..348016bd24 100644 --- a/content/docs/plugins/index.mdx +++ b/content/docs/plugins/index.mdx @@ -78,18 +78,30 @@ Plugins follow a strict three-phase lifecycle (`init()` → `start()` → `destr The fastest way to create a plugin is with the CLI scaffolding: ```bash -# Create a new plugin project +# Create a new plugin project in the current directory os create plugin my-feature # This creates: -# packages/plugins/plugin-my-feature/ +# plugin-my-feature/ # ├── package.json # ├── tsconfig.json # ├── README.md +# ├── pnpm-workspace.yaml # └── src/ # └── index.ts + +cd plugin-my-feature +pnpm install +pnpm build ``` +The scaffold is a **standalone** project: its `package.json` depends on the +published `@objectstack/*` releases that match the CLI which generated it, and +its `tsconfig.json` extends nothing outside the project — so it installs and +builds wherever you put it. Add `--in-repo` only when you are scaffolding into +a checkout of the ObjectStack monorepo itself; that placement emits +`workspace:*` dependencies and installs nowhere else. + For the full walkthrough — implementing the `Plugin` interface, registering services and hooks, testing, and registering with the kernel — see the [Plugin Development](/docs/plugins/development) tutorial. --- diff --git a/content/docs/protocol/kernel/index.mdx b/content/docs/protocol/kernel/index.mdx index 79b1e6ccec..bcf15ffe23 100644 --- a/content/docs/protocol/kernel/index.mdx +++ b/content/docs/protocol/kernel/index.mdx @@ -381,24 +381,31 @@ All system configuration lives in Git: ### Plugin Development ```bash -# Scaffold new plugin (created under packages/plugins/plugin-/) +# Scaffold new plugin (created as ./plugin-/ in the current directory) os create plugin slack-integration # Generated structure: -packages/plugins/plugin-slack-integration/ - package.json # name, version, dependencies (@objectstack/spec, zod) - tsconfig.json +plugin-slack-integration/ + package.json # name, version, dependencies (@objectstack/spec, zod) + tsconfig.json # self-contained, extends nothing outside the project + pnpm-workspace.yaml # the pnpm build approvals a fresh install needs src/ - index.ts # default-export Plugin object (name, version, initialize, destroy) + index.ts # default-export Plugin object (name, version, init, destroy) README.md ``` - - The scaffold still emits an `initialize` method. The kernel's plugin contract - only invokes `init` / `start` / `destroy`, and `init` is **required** — rename - `initialize` to `init` in the generated `src/index.ts` or `kernel.use()` - rejects the plugin outright with +The emitted dependencies are published semver ranges pinned to the CLI that +generated them, so `pnpm install && pnpm build` works in the new directory +without a workspace around it. `os create plugin --in-repo` is the opt-in for +platform work inside an ObjectStack checkout; it emits `workspace:*` instead. + + + The scaffold emits `init` and `destroy`. `init` is the **required** phase — + `kernel.use()` rejects a plugin without it, with `Failed to load plugin: slack-integration - Plugin init function is required`. + `start` is optional and is called after every plugin has initialized; add it + when your plugin needs the rest of the graph to be up first. See + [Plugin Anatomy](/docs/plugins/anatomy#plugin-lifecycle) for the phase model. ### Configuration Management diff --git a/content/docs/protocol/kernel/plugin-spec.mdx b/content/docs/protocol/kernel/plugin-spec.mdx index b23570ee05..84f933667d 100644 --- a/content/docs/protocol/kernel/plugin-spec.mdx +++ b/content/docs/protocol/kernel/plugin-spec.mdx @@ -839,25 +839,31 @@ describe('CRM Workflow', () => { os create plugin crm 📁 Creating plugin: crm -📂 Location: packages/plugins/plugin-crm +📂 Location: /home/you/projects/plugin-crm ✓ Created package.json ✓ Created tsconfig.json ✓ Created src/index.ts ✓ Created README.md +✓ Created pnpm-workspace.yaml ✅ Project created successfully! Next steps: - cd packages/plugins/plugin-crm + cd plugin-crm pnpm install pnpm build ``` +The project is standalone — published dependency ranges pinned to the CLI that +generated it, and a self-contained `tsconfig.json` — so it installs outside any +workspace. Pass `--in-repo` only to scaffold into an ObjectStack monorepo +checkout. + ### 2. Develop Locally ```bash -cd packages/plugins/plugin-crm +cd plugin-crm # Watch mode (auto-rebuild on changes) npm run dev diff --git a/packages/cli/src/commands/create.ts b/packages/cli/src/commands/create.ts index 779e4fecfd..3beda2c13a 100644 --- a/packages/cli/src/commands/create.ts +++ b/packages/cli/src/commands/create.ts @@ -1,50 +1,239 @@ // Copyright (c) 2025 ObjectStack. Licensed under the Apache-2.0 license. +/** + * `os create [name]` — scaffold a plugin or an example application. + * + * ## What this command emits, and why it has two shapes + * + * ObjectStack is a developer tool, so a documented developer-facing command has + * to work for the developer who follows the docs. This command is documented on + * four public doc pages (`deployment/cli`, `plugins/index`, the two + * `protocol/kernel` pages) and, until #14824, every one of those readers got a + * project that CANNOT INSTALL: + * + * - the emitted `package.json` declared `@objectstack/spec` and + * `@objectstack/cli` as `workspace:*`, a pnpm protocol that resolves only + * inside a workspace that already contains those packages; + * - the emitted `tsconfig.json` declared `extends: '../../tsconfig.json'`, + * a file that exists in no directory the scaffold lands in (measured: for + * the `plugin` template it did not resolve even INSIDE this monorepo — + * `packages/plugins//../../tsconfig.json` is `packages/tsconfig.json`, + * which does not exist; every real plugin here spells `../../../`); + * - and the default output location was this repo's own `packages/plugins/` + * or `examples/`, so the command only did anything sensible when it was run + * from a checkout of ObjectStack itself. + * + * A fourth followed from making the emission real: the `plugin` template wrote + * an `initialize` method, which is not part of the `Plugin` contract. `Plugin` + * carries an index signature, so the excess property was accepted but got no + * contextual type — the scaffold failed its own `strict` type-check with TS7006 + * — and the kernel loader refuses a plugin without `init` outright. It emits + * `init` now; the warning the kernel protocol docs carried about renaming it is + * gone with the defect. + * + * The fix is not to narrow the promise but to deliver it, so the DEFAULT is now + * a standalone project: + * + * `standalone` (default) every `@objectstack/*` dependency is a PUBLISHED + * semver range pinned to the running CLI's own + * version, the `tsconfig.json` is self-contained, + * a `pnpm-workspace.yaml` carries the build + * approvals a fresh `pnpm install` needs, and the + * project lands in the developer's own directory. + * `in-repo` (--in-repo) the platform-work shape: `workspace:*` deps, a + * `tsconfig.json` that extends this repo's root + * config, landing under `packages/plugins/` or + * `examples/`. Explicit and documented, never the + * default — its output installs nowhere else. + * + * ## The version the standalone shape pins + * + * Every `@objectstack/*` package in this monorepo is released together on one + * version, so the range that is guaranteed to exist and to be mutually + * compatible is the CLI's own. `getCliVersion()` (owned by `init.ts`, which + * has pinned scaffolded deps this way since long before this command did) reads + * it from the CLI package's own manifest; the range is imported rather than + * re-derived so the two scaffolders cannot drift on the one value that decides + * whether a scaffold resolves at all. + * + * ## Why the standalone shape reuses `init`'s renderers + * + * `renderPnpmWorkspaceYaml()` and `SCAFFOLD_PNPM_RANGE` are `init.ts`'s, and + * they are CALLED here rather than restated. A restatement is the two-producer + * defect `test/scaffold-workspace-consistency.test.ts` exists to catch, and it + * has already been paid for once in this repo: the build-approval block landed + * in one scaffold path and not the other, and one of them shipped the pre-fix + * shape for months. + * + * ## The pin + * + * `scripts/create-scaffold-smoke.sh` scaffolds every template in `templates` + * into a temporary directory OUTSIDE this repository, installs it from packed + * tarballs (the honest stand-in for a registry install of an unreleased + * version), and runs the project's own `build` and `typecheck`. It is wired + * into `.github/workflows/os-create-smoke.yml`, path-filtered onto this file + * and the templates, so it runs on exactly the changes that can break it. + */ + import { Args, Command, Flags } from '@oclif/core'; import chalk from 'chalk'; import fs from 'fs'; import path from 'path'; -import { execSync } from 'child_process'; import { PROTOCOL_MAJOR } from '@objectstack/spec/kernel'; -import { sanitizeNamespace } from './init.js'; +import { + getCliVersion, + renderPnpmWorkspaceYaml, + sanitizeNamespace, + SCAFFOLD_PNPM_RANGE, +} from './init.js'; + +/** + * Where the scaffold is going to live, which is the only thing the emitted + * dependency ranges and `tsconfig.json` differ on. + */ +export type ScaffoldPlacement = 'standalone' | 'in-repo'; + +/** ⛔ Never `in-repo` — that placement emits a project that installs nowhere. */ +export const DEFAULT_PLACEMENT: ScaffoldPlacement = 'standalone'; + +/** + * The dependency spec every `@objectstack/*` entry in an emitted manifest gets. + * + * `standalone` is caret-pinned to the CLI's own version — a published range + * that npm, pnpm, yarn and bun all resolve. `workspace:*` is emitted ONLY for + * the in-repo placement, where a workspace really does supply those names. + */ +export function objectstackDependencySpec(placement: ScaffoldPlacement): string { + return placement === 'in-repo' ? 'workspace:*' : `^${getCliVersion()}`; +} + +/** + * The `extends` an in-repo scaffold needs to reach this repo's root + * `tsconfig.json`, DERIVED from where the template lands rather than written + * down. Writing it down is how the `plugin` template came to declare + * `'../../tsconfig.json'` from a directory two levels below `packages/`, which + * resolves to a file that does not exist. + */ +export function rootTsconfigExtends(inRepoDir: string, projectDirName: string): string { + const depth = path.posix.join(inRepoDir, projectDirName).split('/').filter(Boolean).length; + return `${'../'.repeat(depth)}tsconfig.json`; +} + +/** + * The compiler options a standalone scaffold carries in full, because it + * extends nothing. Deliberately the same set `objectstack init` writes: two + * scaffolders that disagree about `moduleResolution` is a support question + * nobody can answer, and `bundler` is what resolves the `exports` subpaths + * (`@objectstack/spec/contracts`, `/kernel`) the templates import. + */ +const STANDALONE_COMPILER_OPTIONS = { + target: 'ES2022', + module: 'ESNext', + moduleResolution: 'bundler', + strict: true, + esModuleInterop: true, + skipLibCheck: true, +} as const; + +/** A rendered file: JSON objects are stringified on write, strings land as-is. */ +type FileRenderer = (name: string) => unknown; + +export interface CreateTemplate { + description: string; + /** Directory, relative to the monorepo root, the `--in-repo` placement uses. */ + inRepoDir: string; + /** The project directory's own name, in either placement. */ + dirName: (name: string) => string; + /** Every file this template emits for a given placement, keyed by its path. */ + filesFor: (placement: ScaffoldPlacement) => Record; + /** + * The DEFAULT (standalone) file map — what `os create ` writes + * when nobody passes a flag. Kept as a plain property so a caller that only + * cares about the default shape (the manifest-schema sweep in + * `test/scaffold-manifest-schema.test.ts`) reads it without knowing about + * placements at all. + */ + files: Record; +} + +function defineTemplate(t: Omit): CreateTemplate { + return { + ...t, + get files() { + return t.filesFor(DEFAULT_PLACEMENT); + }, + }; +} + +function toCamelCase(str: string): string { + return str.replace(/-([a-z])/g, (g) => g[1].toUpperCase()); +} + +const PLUGIN_IN_REPO_DIR = 'packages/plugins'; +const EXAMPLE_IN_REPO_DIR = 'examples'; -export const templates = { - plugin: { +export const templates: Record = { + plugin: defineTemplate({ description: 'Create a new ObjectStack plugin', - files: { - 'package.json': (name: string) => ({ - name: `@objectstack/plugin-${name}`, - version: '0.1.0', - description: `ObjectStack Plugin: ${name}`, - main: 'dist/index.js', - types: 'dist/index.d.ts', - scripts: { - build: 'tsc', - dev: 'tsc --watch', - test: 'vitest', - }, - keywords: ['objectstack', 'plugin', name], - author: '', - license: 'MIT', - dependencies: { - '@objectstack/spec': 'workspace:*', - zod: '^4.3.6', - }, - devDependencies: { - '@types/node': '^22.0.0', - typescript: '^5.8.0', - vitest: '^4.0.0', - }, - }), - 'tsconfig.json': () => ({ - extends: '../../tsconfig.json', - compilerOptions: { - outDir: 'dist', - rootDir: 'src', - }, - include: ['src/**/*'], - }), - 'src/index.ts': (name: string) => `import type { Plugin } from '@objectstack/spec/contracts'; + inRepoDir: PLUGIN_IN_REPO_DIR, + dirName: (name: string) => `plugin-${name}`, + filesFor: (placement: ScaffoldPlacement) => { + const standalone = placement === 'standalone'; + const files: Record = { + 'package.json': (name: string) => ({ + name: `@objectstack/plugin-${name}`, + version: '0.1.0', + description: `ObjectStack Plugin: ${name}`, + // `tsc` emits ES modules under the compiler options below, so the + // manifest has to declare the project as ESM or Node refuses the + // emitted `dist/index.js`. The in-repo placement inherits its module + // semantics from the root config it extends, so it does not. + ...(standalone ? { type: 'module' } : {}), + main: 'dist/index.js', + types: 'dist/index.d.ts', + // Not a build-script allowlist (that is pnpm-workspace.yaml) — the + // minimum pnpm that reads that file at all. + ...(standalone ? { engines: { pnpm: SCAFFOLD_PNPM_RANGE } } : {}), + scripts: { + build: 'tsc', + dev: 'tsc --watch', + test: 'vitest', + typecheck: 'tsc --noEmit', + }, + keywords: ['objectstack', 'plugin', name], + author: '', + license: 'MIT', + dependencies: { + '@objectstack/spec': objectstackDependencySpec(placement), + zod: '^4.3.6', + }, + devDependencies: { + '@types/node': '^22.0.0', + typescript: '^5.8.0', + vitest: '^4.0.0', + }, + }), + 'tsconfig.json': (name: string) => + standalone + ? { + compilerOptions: { + ...STANDALONE_COMPILER_OPTIONS, + outDir: 'dist', + rootDir: 'src', + declaration: true, + }, + include: ['src/**/*'], + exclude: ['dist', 'node_modules'], + } + : { + extends: rootTsconfigExtends(PLUGIN_IN_REPO_DIR, `plugin-${name}`), + compilerOptions: { + outDir: 'dist', + rootDir: 'src', + }, + include: ['src/**/*'], + }, + 'src/index.ts': (name: string) => `import type { Plugin } from '@objectstack/spec/contracts'; /** * ${name} Plugin for ObjectStack @@ -53,7 +242,7 @@ export const ${toCamelCase(name)}Plugin: Plugin = { name: '${name}', version: '0.1.0', - async initialize(context) { + async init(context) { console.log('Initializing ${name} plugin...'); // Plugin initialization logic }, @@ -66,7 +255,7 @@ export const ${toCamelCase(name)}Plugin: Plugin = { export default ${toCamelCase(name)}Plugin; `, - 'README.md': (name: string) => `# @objectstack/plugin-${name} + 'README.md': (name: string) => `# @objectstack/plugin-${name} ObjectStack Plugin: ${name} @@ -93,37 +282,55 @@ export default { MIT `, + }; + + // pnpm does not run dependency build scripts unless they are approved in + // this file, and pnpm 11 made the omission a HARD ERROR — without it a + // fresh `pnpm install` on the scaffold exits 1. ⛔ Never emitted for the + // in-repo placement: a `pnpm-workspace.yaml` inside a workspace declares + // the directory its OWN workspace root, which severs `workspace:*`. + if (standalone) { + files['pnpm-workspace.yaml'] = () => renderPnpmWorkspaceYaml(); + } + return files; }, - }, - - example: { + }), + + example: defineTemplate({ description: 'Create a new ObjectStack example application', - files: { - 'package.json': (name: string) => ({ - name: `@example/${name}`, - version: '0.1.0', - private: true, - description: `ObjectStack Example: ${name}`, - scripts: { - build: 'objectstack compile', - dev: 'objectstack dev', - test: 'vitest', - }, - dependencies: { - '@objectstack/spec': 'workspace:*', - '@objectstack/cli': 'workspace:*', - zod: '^4.3.6', - }, - devDependencies: { - '@types/node': '^22.0.0', - tsx: '^4.21.0', - typescript: '^5.8.0', - vitest: '^4.0.0', - }, - }), - 'objectstack.config.ts': (name: string) => { - const namespace = sanitizeNamespace(name); - return `import { defineStack } from '@objectstack/spec'; + inRepoDir: EXAMPLE_IN_REPO_DIR, + dirName: (name: string) => name, + filesFor: (placement: ScaffoldPlacement) => { + const standalone = placement === 'standalone'; + const files: Record = { + 'package.json': (name: string) => ({ + name: `@example/${name}`, + version: '0.1.0', + private: true, + ...(standalone ? { type: 'module' } : {}), + description: `ObjectStack Example: ${name}`, + ...(standalone ? { engines: { pnpm: SCAFFOLD_PNPM_RANGE } } : {}), + scripts: { + build: 'objectstack compile', + dev: 'objectstack dev', + test: 'vitest', + typecheck: 'tsc --noEmit', + }, + dependencies: { + '@objectstack/spec': objectstackDependencySpec(placement), + '@objectstack/cli': objectstackDependencySpec(placement), + zod: '^4.3.6', + }, + devDependencies: { + '@types/node': '^22.0.0', + tsx: '^4.21.0', + typescript: '^5.8.0', + vitest: '^4.0.0', + }, + }), + 'objectstack.config.ts': (name: string) => { + const namespace = sanitizeNamespace(name); + return `import { defineStack } from '@objectstack/spec'; // Barrel imports — add more as you create new type folders // import * as objects from './src/objects'; @@ -157,14 +364,17 @@ export default defineStack({ ], }); `; - }, - 'README.md': (name: string) => `# ${name} Example + }, + 'README.md': (name: string) => `# ${name} Example ObjectStack example application: ${name} ## Quick Start \`\`\`bash +# Install dependencies +pnpm install + # Build the configuration pnpm build @@ -179,27 +389,45 @@ pnpm dev ## Learn More -- [ObjectStack Documentation](../../content/docs) -- [Examples](../) -`, - 'tsconfig.json': () => ({ - extends: '../../tsconfig.json', - compilerOptions: { - outDir: 'dist', - rootDir: '.', - }, - include: ['*.ts', 'src/**/*'], - }), +${ + standalone + ? '- [ObjectStack Documentation](https://objectstack.ai/docs)\n' + + '- [CLI Reference](https://objectstack.ai/docs/deployment/cli)\n' + : '- [ObjectStack Documentation](../../content/docs)\n- [Examples](../)\n' +}`, + 'tsconfig.json': (name: string) => + standalone + ? { + compilerOptions: { + ...STANDALONE_COMPILER_OPTIONS, + outDir: 'dist', + rootDir: '.', + declaration: true, + }, + include: ['*.ts', 'src/**/*'], + exclude: ['dist', 'node_modules'], + } + : { + extends: rootTsconfigExtends(EXAMPLE_IN_REPO_DIR, name), + compilerOptions: { + outDir: 'dist', + rootDir: '.', + }, + include: ['*.ts', 'src/**/*'], + }, + }; + + if (standalone) { + files['pnpm-workspace.yaml'] = () => renderPnpmWorkspaceYaml(); + } + return files; }, - }, + }), }; -function toCamelCase(str: string): string { - return str.replace(/-([a-z])/g, (g) => g[1].toUpperCase()); -} - export default class Create extends Command { - static override description = 'Create a new package, plugin, or example from template'; + static override description = + 'Create a new standalone plugin or example project from a built-in template'; static override args = { type: Args.string({ description: 'Type of project to create (plugin, example)', required: true }), @@ -207,7 +435,16 @@ export default class Create extends Command { }; static override flags = { - dir: Flags.string({ char: 'd', description: 'Target directory' }), + dir: Flags.string({ + char: 'd', + description: 'Target directory (default: ./ in the current directory)', + }), + 'in-repo': Flags.boolean({ + default: false, + description: + 'Scaffold INSIDE an ObjectStack monorepo checkout (packages/plugins/ or examples/) with ' + + 'workspace:* dependencies. For platform work only — the emitted project installs nowhere else.', + }), }; async run(): Promise { @@ -230,15 +467,31 @@ export default class Create extends Command { const template = templates[args.type as keyof typeof templates]; const cwd = process.cwd(); - + const placement: ScaffoldPlacement = flags['in-repo'] ? 'in-repo' : DEFAULT_PLACEMENT; + const projectDirName = template.dirName(args.name); + + // Refuse `--in-repo` outside a workspace rather than emit the one thing + // this command is no longer allowed to emit: a project that cannot install. + if (placement === 'in-repo' && !fs.existsSync(path.join(cwd, 'pnpm-workspace.yaml'))) { + console.error(chalk.red('\n❌ --in-repo needs to run from a pnpm workspace root')); + console.log( + chalk.dim( + ` No pnpm-workspace.yaml in ${cwd}. --in-repo emits workspace:* dependencies, which\n` + + ' resolve only inside a workspace that already provides @objectstack/*.\n' + + ' Drop the flag to scaffold a standalone project that installs from the registry.', + ), + ); + process.exit(1); + } + // Determine target directory let targetDir: string; if (flags.dir) { targetDir = path.resolve(cwd, flags.dir); + } else if (placement === 'in-repo') { + targetDir = path.join(cwd, template.inRepoDir, projectDirName); } else { - const baseDir = args.type === 'plugin' ? 'packages/plugins' : 'examples'; - const projectName = args.type === 'plugin' ? `plugin-${args.name}` : args.name; - targetDir = path.join(cwd, baseDir, projectName); + targetDir = path.join(cwd, projectDirName); } // Check if directory already exists @@ -249,6 +502,9 @@ export default class Create extends Command { console.log(`📁 Creating ${args.type}: ${chalk.blue(args.name)}`); console.log(`📂 Location: ${chalk.dim(targetDir)}`); + if (placement === 'in-repo') { + console.log(chalk.yellow('⚠️ --in-repo: workspace:* dependencies — this project installs only in this monorepo')); + } console.log(''); try { @@ -256,7 +512,7 @@ export default class Create extends Command { fs.mkdirSync(targetDir, { recursive: true }); // Create files from template - for (const [filePath, contentFn] of Object.entries(template.files)) { + for (const [filePath, contentFn] of Object.entries(template.filesFor(placement))) { const fullPath = path.join(targetDir, filePath); const dir = path.dirname(fullPath); @@ -267,7 +523,7 @@ export default class Create extends Command { const content = contentFn(args.name); const fileContent = typeof content === 'string' ? content - : JSON.stringify(content, null, 2); + : JSON.stringify(content, null, 2) + '\n'; fs.writeFileSync(fullPath, fileContent); console.log(chalk.green(`✓ Created ${filePath}`)); diff --git a/packages/cli/test/create-plugin-docs-parity.test.ts b/packages/cli/test/create-plugin-docs-parity.test.ts new file mode 100644 index 0000000000..f75e33305d --- /dev/null +++ b/packages/cli/test/create-plugin-docs-parity.test.ts @@ -0,0 +1,106 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +/** + * The `plugin` template's SHAPE against what the docs promise (#14824). + * + * ## Why this pin exists + * + * `os create plugin ` is shown on three public pages, and each of them + * prints the file tree the reader is told to expect — a tutorial's whole value + * is that the listing matches what appears on disk. `test/create.test.ts` pins + * what the template EMITS and `test/scaffold-manifest-schema.test.ts` pins that + * what it emits LOADS, but neither can fail when the docs and the template + * disagree, and the `plugin` template is the one `ManifestSchema` does not + * govern at all (it emits no `objectstack.config.ts`). So the docs were the + * only statement of its shape, and nothing held them to it. + * + * The maintainer's ruling on #14824 is that a documented developer-facing + * command must work for the developer who follows the docs. This file is the + * half of that which no install can check: that the listing the developer READS + * is the listing they GET. + * + * ## Both sides are derived + * + * The expected set comes from `templates.plugin.files` — the default (standalone) + * emission, which is what a reader of these pages runs. The actual set is + * harvested from the fenced blocks that contain the `os create plugin` command, + * by file-extension shape rather than by a transcription of today's tree. So a + * template that grows a file reddens all three pages until they say so, and a + * page that invents a file reddens too. + * + * ⛔ Never satisfy a red here by deleting the listing from a page. The listing + * is the promise; the point is to keep it true. + */ + +import { describe, it, expect } from 'vitest'; +import { readFileSync } from 'node:fs'; +import { basename } from 'node:path'; +import { resolve } from 'node:path'; +import { fileURLToPath } from 'node:url'; +import { templates } from '../src/commands/create.js'; + +const HERE = resolve(fileURLToPath(import.meta.url), '..'); + +// One `resolve(HERE, …)` call per line and nothing split across lines: +// `check:cross-package-test-inputs` reconstructs these reads by SOURCE SCAN, +// and a spelling it cannot parse leaves the glob declared and held by nothing. +// All three are declared for `@objectstack/cli` in +// scripts/cross-package-test-inputs.mjs and mirrored into turbo.json. +const PLUGINS_INDEX = resolve(HERE, '../../..', 'content/docs/plugins/index.mdx'); +const KERNEL_INDEX = resolve(HERE, '../../..', 'content/docs/protocol/kernel/index.mdx'); +const PLUGIN_SPEC = resolve(HERE, '../../..', 'content/docs/protocol/kernel/plugin-spec.mdx'); + +/** The pages that print a file tree for `os create plugin`. */ +const DOC_SITES: Record = { + 'content/docs/plugins/index.mdx': PLUGINS_INDEX, + 'content/docs/protocol/kernel/index.mdx': KERNEL_INDEX, + 'content/docs/protocol/kernel/plugin-spec.mdx': PLUGIN_SPEC, +}; + +/** Fenced code blocks whose body invokes `os create plugin`. */ +function scaffoldFences(mdx: string): string[] { + const fences = [...mdx.matchAll(/^```[^\n]*\n([\s\S]*?)^```/gm)].map((m) => m[1]); + return fences.filter((body) => /^[^\n]*\bos create plugin\b/m.test(body)); +} + +/** + * The file names a fence promises. Harvested by extension shape — the three + * pages spell their trees three different ways (a box-drawing tree, an indented + * listing, a CLI transcript), and the one thing all three share is that a file + * is named with its extension. + */ +function promisedFiles(fence: string): Set { + const tokens = fence.match(/[A-Za-z0-9_.-]+(?:\/[A-Za-z0-9_.-]+)*\.(?:json|md|ts|yaml|yml)\b/g) ?? []; + return new Set(tokens.map((t) => basename(t))); +} + +/** What `os create plugin ` actually writes, by file name. */ +const EMITTED = new Set(Object.keys(templates.plugin.files).map((p) => basename(p))); + +describe('the `plugin` template emits what the docs promise', () => { + it('emits something to compare, so an empty template cannot pass vacuously', () => { + expect(EMITTED.size).toBeGreaterThan(0); + expect([...EMITTED]).toContain('package.json'); + }); + + it.each(Object.keys(DOC_SITES))('%s prints a scaffold listing', (site) => { + const fences = scaffoldFences(readFileSync(DOC_SITES[site], 'utf8')); + expect( + fences.length, + `${site} no longer shows an \`os create plugin\` block — the promise this pin holds is gone`, + ).toBeGreaterThan(0); + expect(promisedFiles(fences.join('\n')).size, `${site} names no files`).toBeGreaterThan(0); + }); + + it.each(Object.keys(DOC_SITES))('%s promises exactly the files the template writes', (site) => { + const promised = promisedFiles(scaffoldFences(readFileSync(DOC_SITES[site], 'utf8')).join('\n')); + const missing = [...EMITTED].filter((f) => !promised.has(f)).sort(); + const invented = [...promised].filter((f) => !EMITTED.has(f)).sort(); + expect( + { missing, invented }, + `${site} disagrees with the \`plugin\` template:\n` + + ` emitted but NOT documented: ${missing.join(', ') || '(none)'}\n` + + ` documented but NOT emitted: ${invented.join(', ') || '(none)'}`, + ).toEqual({ missing: [], invented: [] }); + }); +}); diff --git a/packages/cli/test/create.test.ts b/packages/cli/test/create.test.ts index c67371536b..dd14af4de8 100644 --- a/packages/cli/test/create.test.ts +++ b/packages/cli/test/create.test.ts @@ -1,25 +1,203 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +/** + * `os create`'s emitted contract — the pin for #14824. + * + * ## The defect this file used to certify + * + * Until #14824 the only assertion here was + * `expect(packageJson.dependencies['@objectstack/cli']).toBe('workspace:*')` — + * a test that PASSED on the defect, and would have gone red on the fix. Every + * project `os create` emitted declared its `@objectstack/*` dependencies with + * pnpm's workspace protocol and extended a `tsconfig.json` two directories up, + * so it resolved nothing outside this monorepo; the four public doc pages that + * present `os create` as a user-facing command were therefore teaching a + * command whose output cannot install. The maintainer ruled that a documented + * developer-facing command must work for the developer who follows the docs, + * so the default emission is now standalone and this file pins that shape. + * + * ## What is asserted here, and what is asserted elsewhere + * + * These are the STATIC properties of the emission — the ones a unit test can + * decide from the rendered files alone. That an emitted project actually + * installs and builds from a registry-shaped source is not one of them, and it + * is not asserted here: `scripts/create-scaffold-smoke.sh` scaffolds every + * template into a temp directory OUTSIDE this repository, installs it from + * packed tarballs and runs its `build` and `typecheck`. A unit test that + * claimed the stronger property would be the same shape of comfort the + * `workspace:*` assertion above was. + * + * The sweep is DERIVED from the template map, never a list of `plugin` and + * `example`: a third template must arrive already covered. + */ + import { describe, it, expect } from 'vitest'; -import { templates } from '../src/commands/create'; - -describe('Create Command Templates', () => { - describe('Example Template', () => { - it('should generate correct package.json scripts', () => { - const packageJsonFn = templates.example.files['package.json']; - const packageJson = packageJsonFn('test-app'); - - expect(packageJson.scripts.dev).toBe('objectstack dev'); - expect(packageJson.scripts.build).toBe('objectstack compile'); - expect(packageJson.dependencies['@objectstack/cli']).toBe('workspace:*'); - }); - }); - - describe('Plugin Template', () => { - it('should generate correct dependencies', () => { - const packageJsonFn = templates.plugin.files['package.json']; - const packageJson = packageJsonFn('test-plugin'); - - expect(packageJson.dependencies).toHaveProperty('@objectstack/spec'); - expect(packageJson.keywords).toContain('test-plugin'); - }); +import { + templates, + objectstackDependencySpec, + rootTsconfigExtends, + DEFAULT_PLACEMENT, + type ScaffoldPlacement, +} from '../src/commands/create.js'; +import { getCliVersion, SCAFFOLD_PNPM_RANGE } from '../src/commands/init.js'; + +const TEMPLATE_KEYS = Object.keys(templates); +const PROJECT = 'my-thing'; + +/** Render one template's whole emission for a placement, path → content. */ +function render(key: string, placement: ScaffoldPlacement): Record { + const out: Record = {}; + for (const [filePath, fn] of Object.entries(templates[key].filesFor(placement))) { + out[filePath] = fn(PROJECT); + } + return out; +} + +/** The bytes that land on disk, which is where `workspace:` has to be absent. */ +function serialize(content: unknown): string { + return typeof content === 'string' ? content : JSON.stringify(content, null, 2); +} + +function pkgJson(key: string, placement: ScaffoldPlacement): Record { + return render(key, placement)['package.json'] as Record; +} + +function tsconfig(key: string, placement: ScaffoldPlacement): Record { + return render(key, placement)['tsconfig.json'] as Record; +} + +/** Every `@objectstack/*` entry the template declares, both dep sections. */ +function objectstackDeps(pkg: Record): Record { + const all = { ...(pkg.dependencies ?? {}), ...(pkg.devDependencies ?? {}) }; + return Object.fromEntries( + Object.entries(all).filter(([name]) => name.startsWith('@objectstack/')), + ) as Record; +} + +describe('os create: the sweep covers every shipped template', () => { + it('derives its population from the template map', () => { + expect(TEMPLATE_KEYS.length).toBeGreaterThan(0); + // The two reported in #14824, named so a rename is loud rather than silent. + expect(TEMPLATE_KEYS).toEqual(expect.arrayContaining(['plugin', 'example'])); + }); + + it('defaults to the standalone placement', () => { + expect(DEFAULT_PLACEMENT).toBe('standalone'); + }); + + it('`files` is the default placement, so a caller that ignores placements gets it', () => { + for (const key of TEMPLATE_KEYS) { + expect(Object.keys(templates[key].files).sort()).toEqual( + Object.keys(templates[key].filesFor(DEFAULT_PLACEMENT)).sort(), + ); + } }); }); + +describe.each(TEMPLATE_KEYS)('os create %s — the standalone (default) emission', (key) => { + it('declares every @objectstack dependency as a published range pinned to this CLI', () => { + const deps = objectstackDeps(pkgJson(key, 'standalone')); + expect(Object.keys(deps).length).toBeGreaterThan(0); + for (const [name, spec] of Object.entries(deps)) { + expect(spec, `${key}: ${name}`).toBe(`^${getCliVersion()}`); + // A published range, spelled the way npm/pnpm/yarn/bun all resolve it. + expect(spec, `${key}: ${name}`).toMatch(/^\^\d+\.\d+\.\d+/); + } + }); + + it('emits no `workspace:` dependency protocol in ANY file it writes', () => { + // A dependency SPEC, not the word: `pnpm-workspace.yaml` explains itself in + // prose that says "workspace" repeatedly, and a substring rule would red on + // the file whose presence is part of the fix. A spec is always quoted — + // `"@objectstack/spec": "workspace:*"` in JSON, `'workspace:*'` in a + // TypeScript template — so the quote is what separates the two. + for (const [filePath, content] of Object.entries(render(key, 'standalone'))) { + expect(serialize(content), `${key}: ${filePath}`).not.toMatch(/["']workspace:/); + } + }); + + it('emits a self-contained tsconfig.json — nothing to extend outside the project', () => { + const cfg = tsconfig(key, 'standalone'); + expect(cfg.extends, `${key}: tsconfig.json still extends something`).toBeUndefined(); + // Self-contained means the options are actually THERE, not merely unextended. + expect(cfg.compilerOptions.target).toBeDefined(); + expect(cfg.compilerOptions.module).toBeDefined(); + expect(cfg.compilerOptions.strict).toBe(true); + // The `exports` subpaths the templates import (`@objectstack/spec/kernel`, + // `/contracts`) resolve only under a subpath-aware resolution mode. + expect(['bundler', 'node16', 'nodenext', 'NodeNext', 'Node16']).toContain( + cfg.compilerOptions.moduleResolution, + ); + }); + + it('carries the pnpm build approvals a fresh install needs', () => { + const files = render(key, 'standalone'); + const yaml = files['pnpm-workspace.yaml']; + expect(yaml, `${key}: no pnpm-workspace.yaml`).toBeDefined(); + // Without an approval key a fresh `pnpm install` exits 1 on pnpm 11 + // (ERR_PNPM_IGNORED_BUILDS) — the scaffold would not install at all. + expect(String(yaml)).toMatch(/^\s*(allowBuilds|onlyBuiltDependencies)\s*:/m); + expect((pkgJson(key, 'standalone') as any).engines?.pnpm).toBe(SCAFFOLD_PNPM_RANGE); + }); + + it('names a build script, the second command its own output tells the user to run', () => { + expect(pkgJson(key, 'standalone').scripts?.build).toBeTruthy(); + expect(pkgJson(key, 'standalone').scripts?.typecheck).toBeTruthy(); + }); + + it('emits no monorepo-relative path into the project it hands the developer', () => { + for (const [filePath, content] of Object.entries(render(key, 'standalone'))) { + // `../../content/docs` and friends: links that resolve only from inside + // this checkout. Relative paths that stay INSIDE the project (`./src`) + // are fine, so only the ascending form is refused. + expect(serialize(content), `${key}: ${filePath}`).not.toMatch(/\.\.\/\.\.\//); + } + }); +}); + +describe.each(TEMPLATE_KEYS)('os create %s --in-repo — the platform-work emission', (key) => { + it('keeps the workspace protocol, which is what that placement is for', () => { + const deps = objectstackDeps(pkgJson(key, 'in-repo')); + expect(Object.keys(deps).length).toBeGreaterThan(0); + for (const [name, spec] of Object.entries(deps)) { + expect(spec, `${key}: ${name}`).toBe('workspace:*'); + } + expect(objectstackDependencySpec('in-repo')).toBe('workspace:*'); + }); + + it('extends a tsconfig that resolves to the monorepo ROOT from where it lands', () => { + const t = templates[key]; + const cfg = tsconfig(key, 'in-repo'); + expect(cfg.extends, `${key}: --in-repo tsconfig extends nothing`).toBeTruthy(); + // The defect this replaces: `packages/plugins/plugin-x/../../tsconfig.json` + // is `packages/tsconfig.json`, which does not exist — the `plugin` + // template's `extends` did not resolve even inside this monorepo. Resolved + // arithmetic, not a transcription: a template that moves takes its own + // `extends` with it. + const landedIn = `${t.inRepoDir}/${t.dirName(PROJECT)}`; + expect(normalizeJoin(landedIn, cfg.extends)).toBe('tsconfig.json'); + }); + + it('emits no pnpm-workspace.yaml, which would sever the workspace it joins', () => { + expect(Object.keys(templates[key].filesFor('in-repo'))).not.toContain('pnpm-workspace.yaml'); + }); +}); + +describe('rootTsconfigExtends derives the ascent from where a template lands', () => { + it('counts the project directory itself', () => { + expect(rootTsconfigExtends('packages/plugins', 'plugin-x')).toBe('../../../tsconfig.json'); + expect(rootTsconfigExtends('examples', 'my-app')).toBe('../../tsconfig.json'); + }); +}); + +/** posix `a/b` + `../../x` → `x`, with no filesystem access. */ +function normalizeJoin(dir: string, rel: string): string { + const parts = `${dir}/${rel}`.split('/'); + const out: string[] = []; + for (const p of parts) { + if (p === '' || p === '.') continue; + if (p === '..') out.pop(); + else out.push(p); + } + return out.join('/'); +} diff --git a/scripts/check-ci-filter-parity.mjs b/scripts/check-ci-filter-parity.mjs index 00bf58fd26..26563d289c 100644 --- a/scripts/check-ci-filter-parity.mjs +++ b/scripts/check-ci-filter-parity.mjs @@ -676,14 +676,18 @@ export async function selfTest() { // is the one of the three no earlier declaration had reached. Its two // siblings move nothing here -- `content/**` and `skills/**` were already // unique members of this set from #10015 and #12201. - // Ten plus one plus two plus one plus one plus one: the rollback now uncovers - // sixteen. This pin is judged over the LIVE declaration table on + // Plus, since #14824, the three doc pages that card declared for + // @objectstack/cli: `os create`'s emitted plugin shape is stated nowhere but + // its documentation, so the pin holding the template to it reads all three + // pages, each covered only through the `content/**` root #10015 added. + // Ten plus one plus two plus one plus one plus one plus three: the rollback + // now uncovers nineteen. This pin is judged over the LIVE declaration table on // purpose: a declaration added under a root the rollback keeps leaves the // count alone, one under a new root moves it and is recorded here by name. const preFix = judge(fixtureWorkflow({ core: real.filters?.core, crosspkg: ['scripts/**'] }), CROSS_PACKAGE_TEST_INPUTS); assert( - new Set(uncoveredGlobs(preFix)).size === 16, - `rolling \`crosspkg\` back to its pre-#10015 list uncovers the ten it fixed plus #10848's one plus #10178's two plus #12201's one plus #12924's one plus #14561's one -- got ${new Set(uncoveredGlobs(preFix)).size}`, + new Set(uncoveredGlobs(preFix)).size === 19, + `rolling \`crosspkg\` back to its pre-#10015 list uncovers the ten it fixed plus #10848's one plus #10178's two plus #12201's one plus #12924's one plus #14561's one plus #14824's three -- got ${new Set(uncoveredGlobs(preFix)).size}`, ); assert( uncoveredGlobs(preFix).includes('skills/**'), @@ -709,6 +713,16 @@ export async function selfTest() { uncoveredGlobs(preFix).includes('docs/**'), `-- and #14561 added the authored-prose root the discovered teaching-site population reads, by name`, ); + for (const page of [ + 'content/docs/plugins/index.mdx', + 'content/docs/protocol/kernel/index.mdx', + 'content/docs/protocol/kernel/plugin-spec.mdx', + ]) { + assert( + uncoveredGlobs(preFix).includes(page), + `-- and #14824 added the \`os create plugin\` scaffold-listing page ${page}, by name`, + ); + } // ── (7) WIRING: the gate and its self-test really run in CI ────────────── battery('(7) WIRING: the gate and its self-test really run in CI'); @@ -780,7 +794,7 @@ export async function selfTest() { `same-root-different-file case observed failing and then covered by naming the file, a glob covered by ` + `\`core\`, one covered only by \`crosspkg\` and one covered by neither judged separately in one table, the ` + `stale-entry direction, seven refusals over subjects that could not be read, the checked-in ci.yml, the ` + - `pre-#10015 rollback uncovering the ten it fixed plus #10848's one plus #10178's two plus #12201's one plus #12924's one plus #14561's one, ` + + `pre-#10015 rollback uncovering the ten it fixed plus #10848's one plus #10178's two plus #12201's one plus #12924's one plus #14561's one plus #14824's three, ` + `and the CI wiring read out of lint.yml.`, ); selfTestReachedVerdict = true; diff --git a/scripts/create-scaffold-smoke.sh b/scripts/create-scaffold-smoke.sh new file mode 100755 index 0000000000..1371fded1d --- /dev/null +++ b/scripts/create-scaffold-smoke.sh @@ -0,0 +1,257 @@ +#!/usr/bin/env bash +# `os create` scaffold smoke — prove the emitted project installs and builds +# OUTSIDE this monorepo (#14824). +# +# ## What this gate is for +# +# `os create` is documented on four public pages as a user-facing scaffolder, +# and until #14824 every project it emitted was monorepo-shaped: `workspace:*` +# dependency specs, a `tsconfig.json` extending `'../../tsconfig.json'`, and a +# default output directory inside this repository. A reader who followed the +# docs got a project that `pnpm install` refuses. The maintainer's ruling is +# that a documented developer-facing command must work for the developer who +# follows the docs, and the executable criterion attached to it is this script: +# +# scaffold each template into a temporary directory OUTSIDE the repository, +# install it with the registry, and boot / typecheck it — green outside the +# monorepo, on CI, not on a developer box. +# +# ## Why packed tarballs are the honest stand-in for "the registry" +# +# The scaffold pins `@objectstack/*` to `^`, and on +# a pull request that version is by definition NOT published yet — a literal +# registry install could only ever test the PREVIOUS release, i.e. not the +# change under review. `pnpm pack` applies the same manifest rewrites as +# `pnpm publish`, so a tarball is what npm would hand a downstream installer. +# The tarballs are wired in through the PROJECT'S OWN pnpm overrides, which +# redirect resolution while leaving the emitted dependency SPECS untouched — +# the specs are the thing under test and must not be rewritten to make the +# install work. ⛔ Never substitute a `file:` or `link:` dependency for them: +# that would pin exactly the monorepo-shaped success this gate exists to end. +# +# Anything NOT in the override map (zod, typescript, vitest, every transitive) +# resolves from the real registry, exactly as it would for a user. +# +# ## Why the pack step is shared and the glue is not +# +# `scripts/publish-smoke-pack.mjs` is called rather than reimplemented: it owns +# the publishable population (`private !== true`, re-asserted every run) and a +# curated closure would rot. The ~30 lines of override-append and leak-check +# glue below are deliberately NOT shared with `scripts/publish-smoke.sh`: +# factoring them out would mean editing a release gate to serve a PR gate, and +# the two have different projects, different assertions and different failure +# vocabularies. The duplication is the cheaper risk, and it is stated here so a +# future reader does not "discover" it as an oversight. +# +# ## Usage +# bash scripts/create-scaffold-smoke.sh +# Env: +# SMOKE_ROOT work dir (default: mktemp -d) +# SMOKE_KEEP 1 = keep the work dir (default: 0, auto-clean) + +set -euo pipefail + +REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" +SMOKE_KEEP="${SMOKE_KEEP:-0}" +SMOKE_ROOT="${SMOKE_ROOT:-$(mktemp -d "${TMPDIR:-/tmp}/objectstack-create-smoke.XXXXXX")}" +CLI_BIN="$REPO_ROOT/packages/cli/bin/run.js" + +log() { printf '\n== %s\n' "$*"; } +fail() { printf '::error::%s\n' "$*" >&2; exit 1; } + +cleanup() { + local code=$? + if [ "$SMOKE_KEEP" = "1" ]; then + printf '\nSMOKE_KEEP=1 — work dir preserved: %s\n' "$SMOKE_ROOT" + else + rm -rf "$SMOKE_ROOT" + fi + exit "$code" +} +trap cleanup EXIT INT TERM + +# The work dir must not be inside the repository, or the project would inherit +# this workspace's pnpm settings and the whole measurement would be void. +case "$SMOKE_ROOT/" in + "$REPO_ROOT"/*) fail "SMOKE_ROOT ($SMOKE_ROOT) is inside the repository — that is the one place this gate must not test" ;; +esac + +[ -d "$REPO_ROOT/packages/cli/dist" ] || fail "packages/cli/dist missing — run 'pnpm build' first" +[ -f "$CLI_BIN" ] || fail "$CLI_BIN missing — run 'pnpm build' first" + +# ── 0. the templates, DERIVED from the shipped command ────────────────────── +# Never a hand list: a template added later is smoked the day it is added. +log "Enumerating templates from the built CLI" +TEMPLATE_KEYS="$(node --input-type=module -e " + const m = await import('file://$REPO_ROOT/packages/cli/dist/commands/create.js'); + const keys = Object.keys(m.templates ?? {}); + if (keys.length === 0) throw new Error('the built CLI exports no create templates'); + console.log(keys.join(' ')); +")" +echo " templates: $TEMPLATE_KEYS" + +# ── 1. pack the publishable population ────────────────────────────────────── +log "Packing publishable packages (pnpm pack == publish-time manifests)" +node "$REPO_ROOT/scripts/publish-smoke-pack.mjs" "$SMOKE_ROOT/tarballs" + +# ── 2. scaffold, install, build, typecheck — one template at a time ───────── +for KEY in $TEMPLATE_KEYS; do + WORK="$SMOKE_ROOT/scaffold-$KEY" + mkdir -p "$WORK" + NAME="smoke-$KEY" + + # No `--dir`: the DEFAULT output location is part of what is under test. A + # scaffold that still wrote into `packages/plugins/` or `examples/` would + # land two levels down and be caught by the depth assertion below. + log "os create $KEY $NAME (from $WORK, default location)" + (cd "$WORK" && node "$CLI_BIN" create "$KEY" "$NAME") + + # Counted with `wc -l` and read back as one string rather than into an array: + # the shell floor here is bash 3.2 (what macOS ships), where the array + # builtins this would otherwise reach for do not exist and an empty array + # under `set -u` is itself an error. `pnpm check:bash32-floor` holds the floor. + ENTRY_COUNT="$(cd "$WORK" && ls -A | wc -l | tr -d ' ')" + if [ "$ENTRY_COUNT" -ne 1 ]; then + fail "os create $KEY wrote $ENTRY_COUNT top-level entries ($(cd "$WORK" && ls -A | tr '\n' ' ')), expected exactly one project directory" + fi + APP_DIR="$WORK/$(cd "$WORK" && ls -A)" + [ -d "$APP_DIR" ] || fail "os create $KEY did not create a directory (${ENTRIES[0]})" + [ -f "$APP_DIR/package.json" ] || fail "os create $KEY wrote no package.json into $APP_DIR — the default location is still not the developer's directory" + echo " scaffolded → $APP_DIR" + + # ── 2a. the emitted BYTES, before anything installs ────────────────────── + # Asserted on what landed on disk rather than on the renderer, because this + # is the only place the two can be compared. The unit pin in + # packages/cli/test/create.test.ts reads the renderer. + log "Asserting the emitted manifest is registry-shaped ($KEY)" + node - "$APP_DIR" <<'EOF' +const { existsSync, readFileSync } = require('node:fs'); +const { join } = require('node:path'); +const appDir = process.argv[2]; +const pkg = JSON.parse(readFileSync(join(appDir, 'package.json'), 'utf8')); +const deps = { ...(pkg.dependencies ?? {}), ...(pkg.devDependencies ?? {}) }; +const workspace = Object.entries(deps).filter(([, spec]) => String(spec).startsWith('workspace:')); +if (workspace.length > 0) { + console.error('::error::the scaffold declares workspace-protocol dependencies, which resolve nowhere outside this monorepo:'); + for (const [n, s] of workspace) console.error(` ${n}: ${s}`); + process.exit(1); +} +const os = Object.entries(deps).filter(([n]) => n.startsWith('@objectstack/')); +if (os.length === 0) { + console.error('::error::the scaffold declares no @objectstack/* dependency at all — nothing to resolve, so this smoke would prove nothing'); + process.exit(1); +} +for (const [n, s] of os) { + if (!/^\^?\d+\.\d+\.\d+/.test(String(s))) { + console.error(`::error::${n} is declared as "${s}", which is not a published semver range`); + process.exit(1); + } +} +const tsconfigPath = join(appDir, 'tsconfig.json'); +if (existsSync(tsconfigPath)) { + // JSON5-ish: the scaffold writes plain JSON, so a plain parse is right. + const ts = JSON.parse(readFileSync(tsconfigPath, 'utf8')); + if (ts.extends) { + console.error(`::error::the scaffold's tsconfig.json extends ${JSON.stringify(ts.extends)} — a path that exists only inside this monorepo`); + process.exit(1); + } +} +console.log(` ok — ${os.length} @objectstack dependency spec(s), all published ranges; tsconfig extends nothing`); +EOF + + # ── 2b. pin every publishable package to its tarball ───────────────────── + # Appended to the file the TEMPLATE ships, never written from scratch: the + # build-approval block is part of what a user gets, so it has to be part of + # what this gate tests. + log "Pinning @objectstack/* to local tarballs via the project's own overrides ($KEY)" + node - "$SMOKE_ROOT/tarballs/overrides.json" "$APP_DIR/pnpm-workspace.yaml" <<'EOF' +const { existsSync, readFileSync, writeFileSync } = require('node:fs'); +const [overridesPath, wsPath] = process.argv.slice(2); +const overrides = JSON.parse(readFileSync(overridesPath, 'utf8')); +if (!existsSync(wsPath)) { + console.error( + '::error::the scaffold wrote no pnpm-workspace.yaml. Without it a fresh ' + + '`pnpm install` exits 1 on pnpm 11 (ERR_PNPM_IGNORED_BUILDS) for every ' + + 'user. Fix the `os create` template — not this script.', + ); + process.exit(1); +} +const base = readFileSync(wsPath, 'utf8').replace(/\s*$/, ''); +if (!/^\s*(allowBuilds|onlyBuiltDependencies)\s*:/m.test(base)) { + console.error( + '::error::the scaffolded project declares no pnpm build approvals ' + + '(allowBuilds / onlyBuiltDependencies). A fresh `pnpm install` will exit 1 ' + + 'on pnpm 11. Fix the `os create` template — not this script.', + ); + process.exit(1); +} +const lines = [ + base, + '', + '# ── appended by scripts/create-scaffold-smoke.sh ─────────────────────────', + '# every publishable package redirected to its about-to-publish tarball. The', + '# dependency SPECS in package.json are untouched — they are what is on trial.', + 'overrides:', + ...Object.entries(overrides).map(([name, spec]) => ` '${name}': '${spec}'`), +]; +writeFileSync(wsPath, lines.join('\n') + '\n'); +console.log(` wrote ${wsPath} (${Object.keys(overrides).length} overrides, template settings preserved)`); +EOF + + log "Installing outside the monorepo ($KEY)" + (cd "$APP_DIR" && pnpm install --no-frozen-lockfile) + + log "Asserting no pinned package leaked to the registry ($KEY)" + node - "$SMOKE_ROOT/tarballs/overrides.json" "$APP_DIR/pnpm-lock.yaml" <<'EOF' +const { readFileSync } = require('node:fs'); +const [overridesPath, lockPath] = process.argv.slice(2); +const names = Object.keys(JSON.parse(readFileSync(overridesPath, 'utf8'))); +const lock = readFileSync(lockPath, 'utf8').split(/\r?\n/); +const esc = (s) => s.replace(/[.*+?^${}()|[\]\\]/g, '\\$&'); +const leaked = new Set(); +const pinned = new Set(); +for (const name of names) { + const key = new RegExp(`^\\s*'?${esc(name)}@([^']+?)'?:\\s*$`); + for (const line of lock) { + const m = key.exec(line); + if (!m) continue; + if (m[1].startsWith('file:')) pinned.add(name); + else if (/^[0-9]/.test(m[1])) leaked.add(`${name}@${m[1]}`); + } +} +if (leaked.size > 0) { + console.error('::error::these PINNED packages resolved from the npm registry:'); + for (const l of [...leaked].sort()) console.error(` ${l}`); + console.error('The smoke tested PUBLISHED code instead of the change under review.'); + process.exit(1); +} +console.log(` ok — ${pinned.size} pinned package(s) resolved from tarballs, 0 registry leaks`); +EOF + + # ── 2c. build and typecheck — the "boot / typecheck it" half ───────────── + # `build` is the command the scaffolder's own output tells the user to run. + # For a template whose build is `objectstack compile`, running it IS the boot: + # the config is loaded through the real runtime and its manifest parsed by + # `ManifestSchema`, outside this monorepo, from published-shaped packages. + log "Building the scaffolded project ($KEY)" + (cd "$APP_DIR" && pnpm run build) + + log "Type-checking the scaffolded project ($KEY)" + (cd "$APP_DIR" && pnpm run typecheck) + + BUILD_SCRIPT="$(node -e "console.log(JSON.parse(require('fs').readFileSync(process.argv[1],'utf8')).scripts?.build ?? '')" "$APP_DIR/package.json")" + case "$BUILD_SCRIPT" in + *"objectstack compile"*) + [ -f "$APP_DIR/dist/objectstack.json" ] \ + || fail "$KEY: 'objectstack compile' exited 0 but wrote no dist/objectstack.json — the stack never loaded" + echo " ok — dist/objectstack.json written: the stack loaded and its manifest parsed outside this monorepo" + ;; + *) + [ -d "$APP_DIR/dist" ] && [ -n "$(ls -A "$APP_DIR/dist")" ] \ + || fail "$KEY: 'pnpm run build' exited 0 but produced no dist/ output" + echo " ok — dist/ populated by '$BUILD_SCRIPT'" + ;; + esac +done + +log "os create scaffold smoke passed for: $TEMPLATE_KEYS" diff --git a/scripts/cross-package-test-inputs.mjs b/scripts/cross-package-test-inputs.mjs index 75ee0619f0..c33dda69a1 100644 --- a/scripts/cross-package-test-inputs.mjs +++ b/scripts/cross-package-test-inputs.mjs @@ -264,6 +264,18 @@ export const CROSS_PACKAGE_TEST_INPUTS = { // any package here, and a subtree glob would put cli's e2e suite on every // documentation PR. // + // The three pages added for #14824 are read by + // test/create-plugin-docs-parity.test.ts, which holds the `plugin` template's + // emitted file set equal to the tree each page PRINTS. They are the only + // statement of that template's shape -- it emits no `objectstack.config.ts`, + // so `ManifestSchema` does not govern it and the manifest sweep cannot see + // it. The coupling runs both ways and so must the re-run: a template that + // grows a file must redden the pages that no longer list it, and a page + // rewrite must redden if it drops or invents one. Undeclared, a + // documentation-only PR would leave cli outside the affected set and the + // merge queue would be the first signal -- the shape the three e2e pages + // above were declared for. + // // `connector-mcp-plugin.ts` is read by test/serve-capability-identity.test.ts, // which pins that the connector still registers the name the #7652 repro uses // rather than importing the class. It surfaced with the three above and has the @@ -323,6 +335,9 @@ export const CROSS_PACKAGE_TEST_INPUTS = { 'content/docs/deployment/cli.mdx', 'content/docs/deployment/index.mdx', 'content/docs/permissions/authentication.mdx', + 'content/docs/plugins/index.mdx', + 'content/docs/protocol/kernel/index.mdx', + 'content/docs/protocol/kernel/plugin-spec.mdx', 'scripts/check-nul-bytes.mjs', // This gate's OWN script, the third entry of the mention shape on this // package: test/scaffold-workspace-consistency.test.ts quotes it while diff --git a/turbo.json b/turbo.json index fc134881be..8a076868d3 100644 --- a/turbo.json +++ b/turbo.json @@ -98,6 +98,9 @@ "$TURBO_ROOT$/content/docs/deployment/cli.mdx", "$TURBO_ROOT$/content/docs/deployment/index.mdx", "$TURBO_ROOT$/content/docs/permissions/authentication.mdx", + "$TURBO_ROOT$/content/docs/plugins/index.mdx", + "$TURBO_ROOT$/content/docs/protocol/kernel/index.mdx", + "$TURBO_ROOT$/content/docs/protocol/kernel/plugin-spec.mdx", "$TURBO_ROOT$/scripts/check-nul-bytes.mjs", "$TURBO_ROOT$/scripts/js-comment-mask.mjs", "$TURBO_ROOT$/scripts/js-comment-mask.d.mts",