From 52a721ee84d39856dfa98a1989cf44561898c501 Mon Sep 17 00:00:00 2001 From: Mehdi Shahdoost Date: Mon, 17 Aug 2026 23:42:28 +0200 Subject: [PATCH 1/7] feat(schema): add layered global overrides --- docs/cli.md | 70 ++- docs/customization.md | 125 +++- docs/opsx.md | 5 +- docs/troubleshooting.md | 18 + .../add-global-schema-overlays/.openspec.yaml | 2 + .../add-global-schema-overlays/design.md | 210 +++++++ .../add-global-schema-overlays/proposal.md | 40 ++ .../specs/artifact-graph/spec.md | 128 +++++ .../specs/cli-artifact-workflow/spec.md | 23 + .../specs/schema-override-command/spec.md | 68 +++ .../specs/schema-resolution/spec.md | 82 +++ .../specs/schema-validate-command/spec.md | 45 ++ .../specs/schema-which-command/spec.md | 39 ++ .../add-global-schema-overlays/tasks.md | 44 ++ src/commands/schema.ts | 369 +++++++++++- src/commands/workflow/schemas.ts | 2 + src/commands/workflow/templates.ts | 55 +- src/core/artifact-graph/index.ts | 26 +- src/core/artifact-graph/instruction-loader.ts | 29 +- src/core/artifact-graph/resolver.ts | 541 +++++++++--------- src/core/artifact-graph/schema.ts | 157 ++++- src/core/artifact-graph/types.ts | 104 ++++ src/core/completions/command-registry.ts | 14 + src/core/validation/validator.ts | 9 +- test/commands/schema-overlay.test.ts | 247 ++++++++ .../schema-overlay.integration.test.ts | 154 +++++ .../artifact-graph/schema-override.test.ts | 259 +++++++++ 27 files changed, 2496 insertions(+), 369 deletions(-) create mode 100644 openspec/changes/add-global-schema-overlays/.openspec.yaml create mode 100644 openspec/changes/add-global-schema-overlays/design.md create mode 100644 openspec/changes/add-global-schema-overlays/proposal.md create mode 100644 openspec/changes/add-global-schema-overlays/specs/artifact-graph/spec.md create mode 100644 openspec/changes/add-global-schema-overlays/specs/cli-artifact-workflow/spec.md create mode 100644 openspec/changes/add-global-schema-overlays/specs/schema-override-command/spec.md create mode 100644 openspec/changes/add-global-schema-overlays/specs/schema-resolution/spec.md create mode 100644 openspec/changes/add-global-schema-overlays/specs/schema-validate-command/spec.md create mode 100644 openspec/changes/add-global-schema-overlays/specs/schema-which-command/spec.md create mode 100644 openspec/changes/add-global-schema-overlays/tasks.md create mode 100644 test/commands/schema-overlay.test.ts create mode 100644 test/core/artifact-graph/schema-overlay.integration.test.ts create mode 100644 test/core/artifact-graph/schema-override.test.ts diff --git a/docs/cli.md b/docs/cli.md index d17c6d662f..407bc679c3 100644 --- a/docs/cli.md +++ b/docs/cli.md @@ -15,7 +15,7 @@ The OpenSpec CLI (`openspec`) provides terminal commands for project setup, vali | **Validation** | `validate` | Check changes and specs for issues | | **Lifecycle** | `archive` | Finalize completed changes | | **Workflow** | `new change`, `status`, `instructions`, `templates`, `schemas` | Artifact-driven workflow support | -| **Schemas** | `schema init`, `schema fork`, `schema validate`, `schema which` | Create and manage custom workflows | +| **Schemas** | `schema init`, `schema fork`, `schema override`, `schema validate`, `schema which` | Create and manage custom workflows | | **Config** | `config` | View and modify settings | | **Utility** | `feedback`, `completion` | Feedback and shell integration | @@ -887,12 +887,14 @@ openspec templates --json ``` Schema: spec-driven +Source: package + user overlay -Templates: - proposal → ~/.openspec/schemas/spec-driven/templates/proposal.md - specs → ~/.openspec/schemas/spec-driven/templates/specs.md - design → ~/.openspec/schemas/spec-driven/templates/design.md - tasks → ~/.openspec/schemas/spec-driven/templates/tasks.md +proposal: + Path: /usr/local/lib/node_modules/@fission-ai/openspec/schemas/spec-driven/templates/proposal.md + Source: package +tasks: + Path: /home/user/.local/share/openspec/schemas/spec-driven/templates/tasks.md + Source: user ``` --- @@ -1019,6 +1021,39 @@ openspec schema fork [name] [options] openspec schema fork spec-driven my-workflow ``` +When the source has an active layered user override, the fork materializes the effective schema and effective user/package templates into a self-contained project bundle. + +--- + +### `openspec schema override` + +Create a layered user override for a packaged schema. Unlike `schema fork`, this command does not copy `schema.yaml` or its templates; packaged updates remain active for fields and templates you do not override. + +``` +openspec schema override [options] +``` + +**Arguments:** + +| Argument | Required | Description | +|----------|----------|-------------| +| `name` | Yes | Packaged schema to customize | + +**Options:** + +| Option | Description | +|--------|-------------| +| `--force` | Replace an existing layered override with a new starter file | +| `--json` | Output `created`, `schema`, `path`, and `basePath` as JSON | + +**Example:** + +```bash +openspec schema override spec-driven +``` + +This creates `schema.override.yaml` in the user OpenSpec data directory. See [Global Overrides](customization.md#global-overrides) for the patch format, template fallback, and platform-specific paths. + --- ### `openspec schema validate` @@ -1082,18 +1117,29 @@ openspec schema which [name] [options] openspec schema which spec-driven ``` -**Output:** +**Package-only output:** ``` -spec-driven resolves from: package - Source: /usr/local/lib/node_modules/@fission-ai/openspec/schemas/spec-driven +Schema: spec-driven +Source: package +Path: /usr/local/lib/node_modules/@fission-ai/openspec/schemas/spec-driven +``` + +**Layered output:** + +``` +Schema: spec-driven +Source: package + user overlay +Path: /usr/local/lib/node_modules/@fission-ai/openspec/schemas/spec-driven +Overlay: /home/user/.local/share/openspec/schemas/spec-driven/schema.override.yaml ``` **Schema precedence:** -1. Project: `openspec/schemas//` -2. User: `~/.local/share/openspec/schemas//` -3. Package: Built-in schemas +1. Complete project schema: `openspec/schemas//schema.yaml` +2. Complete user schema: `~/.local/share/openspec/schemas//schema.yaml` +3. Packaged schema plus optional user `schema.override.yaml` +4. Packaged schema unchanged --- diff --git a/docs/customization.md b/docs/customization.md index b1143b9276..20639a0a90 100644 --- a/docs/customization.md +++ b/docs/customization.md @@ -342,7 +342,130 @@ Path: /path/to/project/openspec/schemas/my-workflow --- -> **Note:** OpenSpec also supports user-level schemas at `~/.local/share/openspec/schemas/` for sharing across projects, but project-level schemas in `openspec/schemas/` are recommended since they're version-controlled with your code. +## Global Overrides + +OpenSpec has two user-level customization modes. Choose one mode per schema name: + +| Mode | File | Update behavior | +|------|------|-----------------| +| **Layered override** | `schema.override.yaml` | Keeps packaged fields and templates unless explicitly changed | +| **Complete replacement** | `schema.yaml` plus `templates/` | Freezes a self-contained copy that fully shadows the package | + +User schemas live under the OpenSpec data directory: + +- `$XDG_DATA_HOME/openspec/schemas/` when `XDG_DATA_HOME` is set +- `~/.local/share/openspec/schemas/` on Unix and macOS by default +- `%LOCALAPPDATA%\openspec\schemas\` on Windows by default + +Project schemas in `openspec/schemas/` remain higher priority than either user mode because they represent version-controlled team intent. + +### Layered Global Customization + +Create a starter override for a packaged schema: + +```bash +openspec schema override spec-driven +``` + +This creates only: + +```text +~/.local/share/openspec/schemas/spec-driven/ +└── schema.override.yaml +``` + +The packaged `schema.yaml` remains the base. For example, keep OpenSpec's task guidance and append personal rules: + +```yaml +patchVersion: 1 + +artifacts: + tasks: + instruction: + append: | + Additional rules: + - Include verification commands in every task group. + - Mention the affected package in each task. +``` + +Text fields use explicit operations: + +```yaml +artifacts: + tasks: + instruction: + prepend: Read the repository contribution guide first. + append: Include focused and full verification commands. +``` + +`prepend` and `append` may be combined. Use `replace` by itself only when you intend to discard the packaged instruction: + +```yaml +artifacts: + tasks: + instruction: + replace: Write tasks using my personal workflow. +``` + +Plain `description`, `generates`, and `template` values replace the matching packaged value. Dependency lists use explicit operations: + +```yaml +artifacts: + tasks: + requires: + remove: [design] + add: [proposal] +``` + +Use either `replace`, or `add`/`remove`, for one dependency field. Layered overrides modify existing artifacts only; use a complete schema fork to add, remove, or reorder artifacts. + +### Optional Template Overrides + +You do not need to copy the packaged templates. Add only the templates you want to replace: + +```text +~/.local/share/openspec/schemas/spec-driven/ +├── schema.override.yaml +└── templates/ + └── tasks.md # user version +``` + +For a layered schema, each template resolves independently: + +1. User `templates/` +2. Packaged `templates/` + +In this example, `tasks.md` is user-owned while proposal, specs, and design templates continue following package updates. Template files are whole-file replacements; their Markdown contents are not merged. + +### Complete Global Replacement + +The existing global replacement behavior remains available. Copy a complete schema bundle to the user data directory: + +```text +~/.local/share/openspec/schemas/spec-driven/ +├── schema.yaml +└── templates/ + ├── proposal.md + ├── spec.md + ├── design.md + └── tasks.md +``` + +This directory is self-contained. It receives no packaged schema or template updates, and missing templates do not fall back to the package. You must manually compare and rebase it when OpenSpec changes the built-in workflow. + +Do not put `schema.yaml` and `schema.override.yaml` in the same user schema directory. OpenSpec rejects that conflict instead of guessing which customization you intended. + +### Inspect and Validate + +```bash +openspec schema which spec-driven +openspec schema validate spec-driven +openspec templates --schema spec-driven +``` + +For a layered schema, `which` reports both package and user paths, validation checks the composed schema and every effective template, and `templates` shows whether each concrete template came from the user or package directory. + +If you later need structural changes, `openspec schema fork spec-driven my-workflow` materializes the effective packaged-plus-user schema and templates into a self-contained project schema. --- diff --git a/docs/opsx.md b/docs/opsx.md index c1b2dba37f..124436fca6 100644 --- a/docs/opsx.md +++ b/docs/opsx.md @@ -583,6 +583,9 @@ openspec schema init my-workflow # Or fork an existing schema as a starting point openspec schema fork spec-driven my-workflow +# Or add personal guidance while retaining packaged updates +openspec schema override spec-driven + # Validate your schema structure openspec schema validate my-workflow @@ -590,7 +593,7 @@ openspec schema validate my-workflow openspec schema which my-workflow ``` -Schemas are stored in `openspec/schemas/` (project-local, version controlled) or `~/.local/share/openspec/schemas/` (user global). +Schemas are stored in `openspec/schemas/` (project-local, version controlled) or `~/.local/share/openspec/schemas/` (user global). A user `schema.yaml` is a complete replacement; a user `schema.override.yaml` is layered over the packaged schema and may provide only selected templates. See [Customization](customization.md#global-overrides). **Schema structure:** ``` diff --git a/docs/troubleshooting.md b/docs/troubleshooting.md index 1097aaadc0..b38313284c 100644 --- a/docs/troubleshooting.md +++ b/docs/troubleshooting.md @@ -166,6 +166,24 @@ openspec schema init # create a custom one See [Customization](customization.md#custom-schemas). +### My global schema stopped receiving built-in updates + +A user-level `schema.yaml` is a complete replacement. Its schema and templates intentionally shadow the packaged bundle, so OpenSpec cannot automatically apply newer built-in guidance. + +If you only need additive personal customization, use a layered override instead: + +```bash +openspec schema override spec-driven +openspec schema validate spec-driven +openspec schema which spec-driven +``` + +Move your changes into `schema.override.yaml`, keep only templates you intentionally replace, and remove the complete user `schema.yaml` after preserving any custom content you still need. See [Global Overrides](customization.md#global-overrides). + +### Complete replacement conflicts with layered override + +One user schema directory cannot contain both `schema.yaml` and `schema.override.yaml`. Keep `schema.yaml` for a self-contained frozen workflow, or keep `schema.override.yaml` to inherit packaged updates. `openspec schema which ` shows which source is active. + ## Migration from the legacy workflow ### "Legacy files detected in non-interactive mode" diff --git a/openspec/changes/add-global-schema-overlays/.openspec.yaml b/openspec/changes/add-global-schema-overlays/.openspec.yaml new file mode 100644 index 0000000000..149631464a --- /dev/null +++ b/openspec/changes/add-global-schema-overlays/.openspec.yaml @@ -0,0 +1,2 @@ +schema: spec-driven +created: 2026-08-17 diff --git a/openspec/changes/add-global-schema-overlays/design.md b/openspec/changes/add-global-schema-overlays/design.md new file mode 100644 index 0000000000..3c137e022d --- /dev/null +++ b/openspec/changes/add-global-schema-overlays/design.md @@ -0,0 +1,210 @@ +## Context + +Schema resolution currently returns one winning directory in this order: project-local, user-level, then package. `resolveSchema()` reads only that directory's `schema.yaml`, and template loading reads only its `templates/` directory. This makes every override a complete bundle and prevents packaged changes from flowing into a user customization. + +The requested behavior is narrower than general schema inheritance. A user wants to keep the packaged `spec-driven` schema, append personal guidance to an existing artifact such as `tasks`, and optionally replace individual templates. The design must remain deterministic for ordered artifact and dependency arrays, preserve the existing full-replacement contract, and make the effective result debuggable. + +## Goals / Non-Goals + +**Goals:** + +- Let users layer small, global customizations over packaged schemas without copying the complete bundle. +- Preserve packaged schema and template updates unless a field or template is explicitly customized. +- Give text and array fields explicit, unsurprising merge operations. +- Keep project-local schemas authoritative over personal global customization. +- Preserve complete user schema replacements without changing their template behavior. +- Validate the effective schema and report every source involved in resolution. + +**Non-Goals:** + +- Project-local overlay files in v1; projects continue to use complete, version-controlled schemas and project `rules`. +- Arbitrary `extends` chains or inheritance between user-created schemas. +- Adding, removing, or reordering artifacts through an overlay. Structural workflow forks continue to use a complete schema. +- Merging Markdown template contents. A user template replaces one packaged template as a whole. +- Automatically converting or rebasing existing complete user schemas. +- Treating project `rules` as schema fields; they remain separately injected artifact constraints. + +## Decisions + +### 1. Add an overlay beside, not instead of, complete user schemas + +The layered file is: + +```text +${XDG_DATA_HOME}/openspec/schemas//schema.override.yaml +``` + +The existing platform fallbacks from `getGlobalDataDir()` remain authoritative. An optional `templates/` directory in the same user schema directory contains whole-file template overrides. + +The existing file keeps its existing meaning: + +```text +${XDG_DATA_HOME}/openspec/schemas//schema.yaml +``` + +It is a complete user schema replacement with self-contained templates. Reinterpreting that file as a partial document would silently change current users' behavior. + +If both `schema.yaml` and `schema.override.yaml` exist in the same user schema directory, resolution fails with a conflict diagnostic when the user layer would otherwise be active. Direct user-overlay creation and validation also reject the conflict. A higher-priority project schema continues to resolve normally, while diagnostics may report the inactive conflicting user sources. The user must choose complete replacement or layered customization before the user layer can become active; silently selecting either user file would make edits appear ineffective. + +### 2. Preserve project intent and layer only onto packaged schemas + +Resolution for a name is: + +```text +1. project schema.yaml complete project bundle +2. user schema.yaml complete user bundle +3. package schema.yaml + user schema.override.yaml +4. package schema.yaml unchanged fallback +``` + +A project schema suppresses the personal overlay because version-controlled team intent remains highest priority. A user overlay without a packaged base is invalid and is not treated as a new schema; custom schemas use the existing complete-bundle format. + +Calls without `projectRoot` continue to omit project discovery, then follow steps 2-4. + +### 3. Use a dedicated, strict overlay format + +The overlay is not parsed as `SchemaYaml`. It has its own versioned, strict shape: + +```yaml +patchVersion: 1 + +description: My global additions to the packaged workflow + +artifacts: + tasks: + instruction: + append: | + Additional rules: + - Include verification commands in every task group. + - Mention the affected package in each task. + requires: + remove: [design] + add: [proposal] + +apply: + instruction: + prepend: | + Read the repository contribution guide before implementation. +``` + +Supported artifact patches are: + +- `description`, `generates`, and `template`: a supplied scalar replaces the base value. +- `instruction`: explicit text operation. +- `requires`: explicit collection operation. + +Supported apply patches are: + +- `tracks`: a supplied string or `null` replaces the base value. +- `instruction`: explicit text operation. +- `requires`: explicit collection operation. + +The overlay cannot set schema `name` or `version`, change an artifact `id`, or introduce an unknown artifact ID. Unknown keys are rejected so misspellings do not become silent no-ops. + +### 4. Define deterministic text and collection operations + +Text operations support: + +```yaml +instruction: + prepend: text before the packaged instruction + append: text after the packaged instruction +``` + +`prepend` and `append` may be used together. Non-empty segments are joined with one blank line. `replace` is mutually exclusive with both: + +```yaml +instruction: + replace: complete replacement text +``` + +Collection operations support either `replace`, or `remove` followed by `add`: + +```yaml +requires: + remove: [design] + add: [proposal] +``` + +Removal preserves the relative order of remaining base values. Additions are appended in declaration order and cannot introduce duplicates. `replace` is mutually exclusive with `add` and `remove`. Duplicate entries, an ID present in both `add` and `remove`, and removal of a value not present in the base are validation errors rather than silent behavior. + +After applying the overlay, the existing full-schema parser validates required fields, dependency references, and cycles. Artifact declaration order remains the packaged order. + +### 5. Layer templates only for composed schemas + +For a package schema with a user overlay, each referenced template resolves independently: + +```text +1. user /templates/