Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 28 additions & 0 deletions config/models.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
# Valid provider and model identifiers for this module.
# Qualifier directory names under rules/<provider>/ must match these IDs exactly.
# Update when providers release new models.

claude:
- claude-fable-5
- claude-opus-4-8
- claude-opus-4-7
- claude-opus-4-6
- claude-opus-4-5
- claude-sonnet-4-6
- claude-sonnet-4-5
- claude-haiku-4-5

codex:
- gpt-5.3-codex
- gpt-5.4
- gpt-5.5

gemini:
- gemini-2.5-pro
- gemini-2.5-flash
- gemini-2.0-flash

opencode:
- claude-fable-5
- claude-opus-4-8
- claude-sonnet-4-6
7 changes: 5 additions & 2 deletions docs/decisions/MVPR-0003 Dedicated Release Bundles.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,10 +8,11 @@ tags:
- deployment
status: proposed
created: 2026-03-16
updated: 2026-03-30
updated: 2026-06-11
author: "@N4M3Z"
project: forge-core
related: []
related:
- "PROV-0005 Qualifier Directories for Model Targeting.md"
responsible: ["@N4M3Z"]
accountable: ["@N4M3Z"]
consulted: []
Expand Down Expand Up @@ -43,6 +44,8 @@ releases/
forge-core-opencode-v0.6.0.tar.gz
```

Where model-level qualifier variants exist ([PROV-0005](PROV-0005%20Qualifier%20Directories%20for%20Model%20Targeting.md)), the release matrix extends per model: one bundle per provider and model ID, assembled with `--model <model-id>` and archived as `<module>-<provider>-<model>-v<version>.tar.gz`. A bundle built without `--model` uses base resolution. The bundle's `.manifest` records the target provider and model, so integrity checks also verify which model variant a deployment received.

Each bundle contains:

```
Expand Down
64 changes: 64 additions & 0 deletions docs/decisions/MVPR-0004 Model Baseline Context Evaluation.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
---
title: Model Baseline Context Evaluation
description: The running model self-assesses loaded context against its own baseline, with verification rails and qualifier-directory parking
type: adr
category: architecture
tags:
- architecture
- prompts
status: accepted
created: 2026-06-12
updated: 2026-06-12
author: "@N4M3Z"
project: forge-core
related:
- "MVPR-0001 Minimum Viable Prompt.md"
- "MVPR-0002 Prompt Minimalization Metrics.md"
- "PROV-0005 Qualifier Directories for Model Targeting.md"
responsible: ["@N4M3Z"]
accountable: ["@N4M3Z"]
consulted: []
informed: []
upstream: []
---

# Model Baseline Context Evaluation

## Context and Problem Statement

Always-on instructions accumulate for the weakest model that ever needed them. As frontier models internalize behaviors through training and harness prompts (read-before-assert, verify-before-done, scope discipline), the rules written to compensate become permanent token cost with no behavioral effect, and nothing in the toolchain distinguishes "still steering" from "internalized." MVPR-0001 demands a minimum viable prompt per model; MVPR-0002 supplies static scans and on-demand ablation. Neither answers the operational question a new flagship raises: which loaded instructions can this model drop, and where does the dropped content go so weaker models keep it?

## Considered Options

- **Manual curation per model release** — re-read every rule by hand; does not scale and has no test
- **Static scan only** — MVPR-0002 scan mode; catches structure, staleness, and conflicts but cannot measure whether the running model needs an instruction
- **Ablation for everything** — PromptFoo with/without per rule per model; accurate but slow and expensive as a default path
- **Baseline self-assessment with verification rails** — the running model evaluates its own loaded context, bounded by rails that counter its bias

## Decision Outcome

Chosen option: **baseline self-assessment with verification rails**, implemented as the OptimizeContext skill.

The running model applies a behavioral test to each loaded instruction: "if this instruction were absent, would my unprompted output already satisfy it?" Knowing is not complying; the test is on behavior, not comprehension. Content divides into three kinds with different verdict spaces:

| Kind | Test | Verdicts |
| ------------------- | ----------------------------------------------- | ------------------------------------- |
| Capability steering | behavioral test against the session baseline | offload, keep |
| Preference, policy | user choices no model can infer | slim, stale-fix, keep (never removed) |
| Knowledge reference | does this belong in always-on context at all? | relocate to a lazy skill, stale-fix, keep |

The rails are part of the decision, not implementation detail:

- **Nothing is destroyed by capability reasoning.** Offload parks the full text in model qualifier directories ([PROV-0005](PROV-0005 Qualifier Directories for Model Targeting.md)) for the models that still need it; deletion requires the content to be wrong or stale, and explicit confirmation.
- **The evaluator is biased toward a smaller context.** Verdicts below high confidence escalate to ablation ([MVPR-0002](MVPR-0002 Prompt Minimalization Metrics.md)) instead of acting; every duplication or staleness claim is verified against the repository before application; every proposal is confirmed individually by the user.
- **Agents are evaluated against their pinned model tier, never the session model.** A fast-tier agent still needs scaffolding the session model does not.
- **Rules are evaluated first.** They cost tokens in every session; skill bodies are lazy and are trimmed only for dead weight, while their reference and policy content is protected.
- **Knowledge migrates down the loading ladder.** Reference material found in always-on rules relocates into the owning skill, loading only when the skill fires.

## Consequences

- Positive: context shrinks as models improve instead of growing monotonically
- Positive: older models keep their full instruction sets through qualifier parking
- Positive: reference content moves from always-on rules to lazy skills, sharpening the rules/skills split
- Tradeoff: self-assessment is a weak signal; the confidence bar and ablation escalation are load-bearing, not optional
- Tradeoff: the evaluation recurs at every model generation; it is an operating practice, not a one-time cleanup
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ tags:
- targeting
status: accepted
created: 2026-03-15
updated: 2026-03-30
updated: 2026-06-11
author: "@N4M3Z"
project: forge-core
related: []
Expand All @@ -27,7 +27,7 @@ Models evolve under your feet. Instructions essential for one model version beco

## Considered Options

- **Qualifier directories** — `rules/claude/opus4.5/Rule.md` overrides the base `rules/Rule.md`
- **Qualifier directories** — `rules/claude/claude-opus-4-6/Rule.md` overrides the base `rules/Rule.md`
- **Frontmatter targets** — `targets: [claude]` for include/exclude filtering
- **Config-driven** — `defaults.yaml` enumerates which rules deploy to which providers
- **Filename convention** — `Rule.claude.md` suffix-based variants
Expand All @@ -36,7 +36,7 @@ Models evolve under your feet. Instructions essential for one model version beco

Chosen option: **qualifier directories + frontmatter targets**, because they solve different problems and compose cleanly.

**Qualifier directories** handle content variants. Same filename in a subdirectory named after a provider or model overrides the base. Resolution precedence: `user/` > `provider/model/` > `provider/` > base. Valid qualifier names come from `defaults.yaml` providers config; `user/` is always valid (gitignored, personal overrides).
**Qualifier directories** handle content variants. Same filename in a subdirectory named after a provider or model overrides the base. Resolution precedence: `user/` > `provider/model/` > `provider/` > base. Valid provider qualifiers come from the providers config; valid model qualifiers are the exact model IDs listed in `config/models.yaml`, so the directory name is the model ID itself (`rules/claude/claude-opus-4-6/Rule.md`). No aliases, no segment matching. `user/` is always valid (gitignored, personal overrides). An unrecognized subdirectory inside a qualifier directory is a validation error: silent omission would drop content from deployment without trace.

```
rules/
Expand All @@ -46,6 +46,8 @@ rules/
user/AgentTeams.md # personal override
```

**Skills** carry qualifier overlays inside the skill directory, reusing the `user/` flatten mechanics: `skills/Name/<provider>/<model>/SKILL.md` overrides `skills/Name/SKILL.md` per file at assembly. Top-level qualifier directories under `skills/` are not valid; the skill directory name must always match the skill's `name:` field.

**Frontmatter targets** handle include/exclude. A rule that applies to some providers but not others declares `targets:` in frontmatter, stripped at deploy. This also serves as a reconstructibility record — if qualifier directories are lost, the frontmatter documents which providers the rule was meant for.

```yaml
Expand Down
6 changes: 6 additions & 0 deletions docs/todos/2026-06-12.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
# Todos — 2026-06-12

- [ ] Run PromptAnalysis ablation on the rules escalated by the model baseline scan [priority:: medium] [id:: 0001] #rules #mvpr
- Self-assessment scored these medium confidence; per MVPR-0004 they act only after behavioral confirmation.
- Candidates: AvoidDuplication, LessIsMore, NullableTypeSafety, PosixTrailingNewline, TestCorrectness, plus slim proposals for BehavioralSteering, HashVerifiedExecution, InstallInstructions, ShellAliases, VerifyClaims.
- Acceptance: each candidate has an ablation verdict (redundant or load-bearing) recorded, and confirmed-redundant ones are offloaded per PROV-0005.
9 changes: 0 additions & 9 deletions rules/AgentStructure.md

This file was deleted.

12 changes: 0 additions & 12 deletions rules/ArtifactComposition.md

This file was deleted.

16 changes: 2 additions & 14 deletions rules/BehavioralSteering.md
Original file line number Diff line number Diff line change
@@ -1,20 +1,8 @@
These thoughts mean STOP — you're rationalizing a bad decision:

| Thought | Reality |
| ---------------------------------------- | ---------------------------------------------------------------- |
| "This is just a small change" | Small changes have big blast radii. Verify before claiming done. |
| "I'll clean this up while I'm here" | Scope creep. Do what was asked, then stop. |
| "I know what this code does" | Read it. Memory lies. Verify before asserting. |
| "Tests aren't needed for this" | If it can break, it needs a test. |
| "Let me add error handling just in case" | Don't guard against scenarios that can't happen. |
| "I'll create a helper for this" | Three similar lines beat a premature abstraction. |
| "I'll write a function to do X" | Check if X already exists. Scaffold around it instead of reimplementing. |
| "The user probably wants me to also..." | Do what was asked. Ask before expanding. |
| "I can skip reading the file" | Read before claiming. Every shortcut is a future bug. |
| "Quick fix for now, investigate later" | Root cause first. Symptom fixes create new bugs. |
| "I'm confident it works" | Confidence is not evidence. Run the command. |
| Thought | Reality |
| ---------------------------------------- | ------------------------------------------------------------------------ |
| "Propose the full framework, then trim" | First pass gets heavily cut. Start minimal; scope up on demand. |
| "I know how this tool or format works" | External-behavior claims need evidence. Read docs or say you're unsure. |
| "I'll design what should exist here" | Scan existing skills and modules first. New usually collides with old. |
| "This short message clearly means X" | Short messages are easy to misread. Re-read before acting on new work. |
| "More detail is safer than less" | Bloat is unsafe. Fewer structured points beat a longer response. |
Expand Down
11 changes: 0 additions & 11 deletions rules/ClaudeCoworkLimitations.md

This file was deleted.

13 changes: 0 additions & 13 deletions rules/CrossProviderAssembly.md

This file was deleted.

8 changes: 0 additions & 8 deletions rules/DeployManifest.md

This file was deleted.

18 changes: 18 additions & 0 deletions rules/Deslop.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
Write like a human talking to a colleague. Slop is text that performs the act of answering instead of informing: it pads, inflates, and hedges. Strip it everywhere: prose, docs, commit messages, reviews, READMEs.

Vocabulary tells ([Wikipedia catalog][WPSIGNS]): delve, tapestry, testament, underscore, pivotal, crucial, robust, vibrant, landscape, intricate, meticulous, boasts, showcase, harness, leverage, seamless, cutting-edge, game-changer. If a sentence needs one of these, it usually needs a fact instead.

Construction tells:

- Negative parallelism ("It's not just X, it's Y") manufacturing fake epiphany
- Rule-of-three adjective stacks ("fast, reliable, and scalable")
- Copula avoidance ("serves as", "stands as", "marks") where "is" works
- Participle tails attaching unverified significance ("...creating a vibrant community")
- Bold-every-keyword formatting and bullet lists where each bullet restates its heading
- Throat-clearing openers ("Certainly!", "Great question") and canned outros ("In conclusion", "I hope this helps")
- Weasel attribution ("experts argue", "observers note") without a named source
- Emoji in technical content, commits, code, or documentation

Human test: read it aloud. If you would not say it to a colleague, rewrite it. If deleting the sentence loses nothing, delete it.

[WPSIGNS]: https://en.wikipedia.org/wiki/Wikipedia:Signs_of_AI_writing
8 changes: 0 additions & 8 deletions rules/DriftComparison.md

This file was deleted.

22 changes: 0 additions & 22 deletions rules/GitCryptUserData.md

This file was deleted.

10 changes: 1 addition & 9 deletions rules/GitWorktrees.md
Original file line number Diff line number Diff line change
@@ -1,11 +1,3 @@
Use git worktrees for parallel feature work instead of stashing or switching branches. Each worktree gets its own working directory with a shared `.git` store -- no context switching, no stash conflicts.

```sh
git worktree add .worktrees/feature-branch feature-branch
git worktree remove .worktrees/feature-branch
git worktree list
```

Worktrees live in `.worktrees/<branch-name>` inside the repo, ignored by `.gitignore`. Add `.worktrees/` to `.gitignore` before creating the first worktree so the parallel checkouts do not appear in `git status` of the main tree. The sibling-directory pattern (`../repo-branchname`) also works but pollutes the parent directory and breaks paths in IDE workspaces and editor projects.
Use git worktrees for parallel feature work instead of stashing or switching branches. Worktrees live in `.worktrees/<branch-name>` inside the repo; add `.worktrees/` to `.gitignore` before creating the first worktree. Setup, safety checks, and cleanup mechanics: [GitWorktrees companion](../skills/VersionControl/GitWorktrees.md).

When spawning agents for parallel implementation, use `isolation: "worktree"` so each agent works on an isolated copy without conflicting with the main tree or other agents.
Loading
Loading