Skip to content

docs: make the setup prompt, FAQ, plugin README and bench text match what onsystem does - #241

Merged
dgesteves merged 2 commits into
mainfrom
docs/os-text-followups-copy
Oct 28, 2026
Merged

dgesteves merged 2 commits into
mainfrom
docs/os-text-followups-copy

Conversation

@dgesteves

@dgesteves dgesteves commented Oct 27, 2026 •

Copy link
Copy Markdown
Owner

Roadmap item: os-text-followups (review follow-ups from OS2-28 #185, OS2-44 #192 and the bench wording). Docs and copy only, so no changeset.

Each point was checked against current main, after #199, #203 and #204 merged. Point 6 was already fixed on main, so it is dropped (see below). The rest still reproduced.

What was wrong

  1. /setup.md (site/lib/llms.ts:98) told Claude Code "Do not add onsystem to .mcp.json as well". Since docs(clients): lead with the setup; add Antigravity, Gemini folder trust, Copilot via the marketplace and one .mcp.json #185, docs/clients.md offers one committed .mcp.json for the team, and Claude Code keeps the plugin and adds "disabledMcpjsonServers": ["onsystem"] to .claude/settings.json (docs/rollout.md says the same). The Codex CLI step listed only "the MCP server and the skill". The plugin Codex installs also runs the hook on apply_patch (plugins/onsystem/hooks/check-ui.mjs:283, since 0.8.0, fix(hook): check the files a Codex apply_patch changes; init offers an AGENTS.md section #182), once the hook is trusted in /hooks.

  2. plugins/onsystem-agent/README.md:25 gave copilot plugin install dgesteves/onsystem:plugins/onsystem-agent, the direct install that Copilot CLI has deprecated. docs/clients.md gives the marketplace route.

  3. FAQ (site/lib/faq.ts:45): "The hook ... ships for Claude Code only so far." The llms.txt summary (site/lib/llms.ts:34) said the same ("In Claude Code, a hook ..."). docs/clients.md says Claude Code and Codex CLI both run it.

  4. Antigravity has a setup block in docs/clients.md, but docs/quickstart.md:42 and the /docs/clients description in site/lib/docs.ts left it out.

  5. docs/tools.md linked to clients.md with no anchor. docs(clients): lead with the setup; add Antigravity, Gemini folder trust, Copilot via the marketplace and one .mcp.json #185 added ## How the server finds your project there.

  6. bench/v2:

    • LABELLING.md:3 said "two automatic scorers". Scoring lists four measures, and three of them are automatic: tsc, onsystem and @shadcn/lint.
    • README.md:93 said "prepare checks that the scorers can be trusted". runControls in lib/repo.mjs checks tsc and onsystem only.
    • README.md:20 said "onsystem ships its hook only for Claude Code" in the present tense.
    • scripts/corpus-lib.ts:283 said "so not an estimate" on the audit tally. The report prints only that tally's counts, never a rate.
  7. llms-full.txt: these lines start with # inside code blocks:

    • from docs/tools.md's get_component example: # Badge, ## Props and ## Variants
    • from docs/ci.md's shell block: # Baseline: 1,340 findings ...

    A reader who splits the file at its # lines got a second top-level title and two extra ## sections. The Tools section ended at ## Props.

What changed

  • site/lib/llms.ts:
    • The Claude Code step says not to add a .mcp.json entry for Claude Code. When the repository already commits one with onsystem for the other clients, it says to add "disabledMcpjsonServers": ["onsystem"] to the same .claude/settings.json, as docs/clients.md does.
    • The Codex CLI step names the hook and says it runs once trusted in /hooks.
    • The summary says "In Claude Code and Codex CLI, a hook ...".
    • In llmsFullTxt, a code block that has a line starting with # is indented two spaces. All other code blocks are unchanged.
  • site/lib/faq.ts: the hook runs in Claude Code and Codex CLI (trusted in /hooks). It says nothing about other agents: per review, Copilot CLI's changelog says it runs Claude-format plugin hooks, and check-ui.mjs (lines 272-279) already reads Copilot's and VS Code's edit-tool fields, but the hook has not been run in either (follow-up below).
  • site/lib/docs.ts: Antigravity is in the /docs/clients description. A comment next to it says it must name every client and stay at 160 characters (151 now), and what to shorten before adding the next one.
  • site/app/page.tsx (the Works with list only): Antigravity after Gemini CLI.
  • docs/quickstart.md: Antigravity is in the client list. The list now matches the README's.
  • docs/tools.md: the link goes to clients.md#how-the-server-finds-your-project.
  • plugins/onsystem-agent/README.md:
    • Copilot CLI installs from the marketplace, and the README notes that Copilot CLI has deprecated installing straight from a repository. That gives it the Claude Code plugin's server and skill, which are the same as this package's: both run npx -y onsystem@0.10.1, and the two SKILL.md files are byte-identical.
    • VS Code loads this folder through chat.pluginLocations, as docs/clients.md says. The intro says "for VS Code and any other client of the standard".
    • The no-hook line says that Codex CLI installs the Claude Code plugin too.
  • bench/v2/LABELLING.md: names the three automatic scorers and links Scoring. The labeller says what each scorer missed, and which of onsystem's findings are wrong: labels.csv counts the violations in each file, and findings.csv has a verdict for each onsystem finding only (labelSummary in lib/stats.ts compares the labels with onsystem alone).
  • bench/v2/README.md:
  • scripts/corpus-lib.ts (comment only): the audit labels were picked because they looked suspicious, so their share of false positives says nothing about the rate, and the report prints only their counts.

Different from the brief

  • Point 6 is dropped, because main already says both cases. docs/required-check.md:49: "The action runs the onsystem your project installed ... Without an install, each release of the action runs the CLI of the same version (npx onsystem@<version>)." The step comment in action.yml:42-45 says the same. Both came from d81ac48, in fix(baseline): follow moved files, accept findings off the changed lines first, and run the installed CLI in the Action #192's own fix round. I checked them against action/check.mjs: lines 235-295 find the install and fall back to npx at the action's version, and lines 362-365 print the version notice. Nothing to change.
  • Point 8: indented, not demoted. Demoting # Badge to #### Badge would change get_component's real answer. The fix indents the whole block (fences included) by two spaces. CommonMark strips that indent again (a fence indented N spaces removes up to N from each content line). I checked with markdown-it 14.1.0, installed with --ignore-scripts in a scratch folder. It parses main's and this branch's llms-full.txt to the same 96 code blocks with identical content and the same 119 headings. The shell comment in docs/ci.md is covered by the same rule.
  • The /docs/clients description dropped "without the plugin". Adding "Antigravity, " made it 170 characters. site/test/shell.test.ts caps docs descriptions at 160. It now ends "... Antigravity, Grok Build and Claude Code." (151).
  • Extra edits in this PR's files, for the same contradiction:
    • the Codex step in /setup.md
    • the llms.txt summary
    • the intro and no-hook line of the Agent Plugins README
    • bench/v2/README.md:20

How it was tested

Two new test files. Every test in them fails on main and passes here (checked by putting main's versions of the other changed files back and running both files: 6 of 6 and 6 of 6 failed). They check what the pages say, not their exact sentences: terms in the same paragraph of docs/clients.md, the Scoring titles that name each scorer, and the clause that says what prepare checks.

  • test/docs-agree.test.ts, 6 tests, all 6 fail on main:
    • the quickstart lists every client under Other clients in docs/clients.md
    • the Agent Plugins README uses the marketplace commands that docs/clients.md gives, with no direct install
    • docs/tools.md links to the anchor, and the heading exists
    • LABELLING.md names the three automatic scorers that Scoring lists
    • the prepare paragraph names tsc and onsystem as what it checks, and runControls in lib/repo.mjs scores its controls with typecheck and onsystemCheck only
    • the bench README no longer says the hook is Claude Code only
  • site/test/docs-agree.test.ts, 6 tests, all 6 fail on main:
    • /setup.md's Claude Code step gives the disabledMcpjsonServers line from docs/clients.md
    • its Codex step names the hook and /hooks
    • the FAQ and llms.txt name Codex CLI with the hook
    • the /docs/clients description names every client with a setup block
    • the home page's Works with list names every client with a setup block
    • no line inside a code block of llms-full.txt starts with #, # onsystem is the only # line, and the Tools section, cut at ## lines, runs to the check_ui example

Gate, all passing on 0ac646c (main is still 42bf9d6):

Command Result
pnpm install --frozen-lockfile --prefer-offline ok
pnpm format:check ok
pnpm lint ok
pnpm typecheck ok
pnpm build ok, schema.json unchanged
pnpm test 103 files passed, 1 skipped; 1296 tests passed, 3 skipped
pnpm --filter site generate ok
pnpm --filter site test 21 files, 291 tests passed
pnpm --filter site typecheck ok
pnpm --filter site lint ok
pnpm --filter site build ok
pnpm corpus "Counts match corpus/snapshot.json", no diff under corpus/

pnpm smoke was not run: no MCP server or tool code changed.

I checked the pages this changes in the built output (.next/server/app): setup.md, llms.txt, llms-full.txt, /, /docs, /docs/clients, /docs/faq and /docs/tools. The home page lists Antigravity between Gemini CLI and Grok Build. I also diffed the generated llms.txt, llms-full.txt and setup.md against main's. The only changes are the lines above and the two indented blocks.

Follow-ups (outside this PR's files)

  • Running the hook in Copilot CLI and VS Code. Per review, Copilot CLI's changelog says it runs Claude-format plugin hooks, and plugins/onsystem/hooks/check-ui.mjs reads Copilot's and VS Code's edit-tool fields, but no one has run it there. docs/clients.md's support tiers are unchanged here.
  • The VS Code double start that docs: support tiers for every client setup, and a four-week integration freeze #204 left open: a team can get both the root .mcp.json and .vscode/mcp.json. That needs a run in VS Code (docs/clients.md, docs/rollout.md). /setup.md's VS Code step is unchanged here until that is settled.

…what onsystem does

/setup.md told Claude Code never to add onsystem to .mcp.json, while
docs/clients.md offers one committed .mcp.json for the team and turns it
off for Claude Code with "disabledMcpjsonServers". The setup prompt now
says the same, and gives Codex CLI the hook it installs (trusted in
/hooks). The FAQ and the llms.txt summary said the hook is Claude Code
only; it runs in Codex CLI too.

- plugins/onsystem-agent/README.md: Copilot CLI installs from the
  marketplace, not by the deprecated direct install; VS Code loads the
  folder through chat.pluginLocations.
- Antigravity joins the client lists in docs/quickstart.md and the
  /docs/clients description.
- docs/tools.md links to the "How the server finds your project" anchor.
- bench/v2: LABELLING.md names the three automatic scorers, the README
  says prepare checks tsc and onsystem only, and that the hook was
  Claude Code only when the benchmark ran (0.5.0; Codex since 0.8.0).
- scripts/corpus-lib.ts: the audit labels' comment says what the report
  does with them.
- llms-full.txt: a code block with a line that starts with # (the
  get_component answer, a shell comment) is indented two spaces, so the
  file's own outline has only its sections. CommonMark parses the same
  code blocks.

New checks in test/docs-agree.test.ts and site/test/docs-agree.test.ts.

os-text-followups
…st content not sentences

- FAQ: say only that Claude Code and Codex CLI run the hook.
- Agent Plugins README: keep the deprecation, drop "as this folder would need".
- LABELLING.md: the labeller says what each scorer missed, and which of onsystem's findings are wrong.
- Home page: Antigravity in the Works with list, with a test that the list names every client in docs/clients.md.
- docs-agree tests assert what the pages say, not their exact sentences.
- The /docs/clients description notes its 160-character cap and what to shorten first.
dgesteves added a commit that referenced this pull request Oct 28, 2026
#201 shipped in 0.9.1, #202 in 0.10.0 and #199 in 0.10.1, as the
changelog shows; #200, #203 and #204 are website and docs changes, live
since they merged. Main is at 0.10.1 with no pending changesets, so
nothing is merged and waiting for a release. In review now lists the
open pull requests #241 and #242.
@dgesteves
dgesteves merged commit 01981be into main Oct 28, 2026
19 checks passed
@dgesteves
dgesteves deleted the docs/os-text-followups-copy branch October 28, 2026 08:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant