chore(claude-ops): validate the native-surface registry against Claude Code 2.1.287 - #5731
Merged
Merged
Conversation
…e Code 2.1.287 Bump the inventory's VALIDATED_AGAINST to 2.1.287: every lane extracts ok on that build. The hidden built-in /plugin-types is gone from the 2.1.287 binary, so its orphaned dismissal against code-metrics:audit-type-debt is removed from the native-surfaces store and the registry view is regenerated. claude-ops 0.80.1. Closes #5730 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
kyle-sexton
marked this pull request as ready for review
October 1, 2026 18:33
Contributor
|
Claude finished @kyle-sexton's task in 49s —— View job Security review complete
No security issues found. This PR is a routine per-release inventory validation bump for
No injection surfaces, auth/access-control logic, secrets handling, or executable code paths are touched. The diff carries no I also checked the |
Contributor
|
Claude finished @kyle-sexton's task in 1m 21s —— View job Claude's code-review (
|
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
kyle-sexton
enabled auto-merge (squash)
October 1, 2026 18:36
kyle-sexton
added a commit
that referenced
this pull request
Oct 1, 2026
claude-ops 0.80.1 landed on main (#5731); this change moves to 0.80.2. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
kyle-sexton
added a commit
that referenced
this pull request
Oct 2, 2026
) Closes #5733 ## Summary The inventory now lists Claude Code's built-in plugins (`cc-plugin-*@builtin`) in a new `builtin_plugins` lane. Before this, only `security-review` was visible (under `plugin_backed`), so the native-surface registry and overlap detection could not see these plugins' skills, agents or commands. How the binary registers them (2.1.285 through 2.1.287): - **Loader.** A function guarded by the `builtinPluginsInitialized` latch calls `L("cc-plugin-x", () => import.meta.require("/$bunfs/root/chunk-….js"))` once per plugin, some of them under `CLAUDE_CODE_ENTRYPOINT !== "local-agent"` and `!c7r()` conditions. - **Registrar.** Each plugin's module calls the function whose body is `x().builtinPlugins.set(e.name, e)`. It passes an object literal, sometimes a `...descriptor` spread, with `name`, `description`, `version`, `isAvailable` (usually a `tengu_*` flag check), `defaultEnabled`, `enabledFromPolicyOnly`, `enabledFromTrustedSettingsOnly`, `skills`, `hooksModule`, `mcpServers` and `userConfig`. - **Components.** Skills come from the `skills` array: an inline object, or a module factory `X("run", {...})`. Agents are markdown embedded as strings in the hooks-module manifest (`files: {"agents/author.md": ...}`). Commands are registered at runtime through the hooks API (`command.register`). Hook events come from the manifest's `scan.hooks`. The definitions are inline objects in modules that the binary embeds as `/$bunfs/` chunks. The PR also records the operator's 2026-10-01 rulings on the candidates the new lane surfaced: - **`defer` row**: `claude-test` (a skill of the built-in `cc-plugin-claude-test` plugin) against `testing:run-e2e`, integration `route`. Both verify a running web app in a browser with screenshot evidence, but the native side is gated off by default (`tengu_mellow_hollerith`, default false). Re-rule as complementary/route when the flag defaults on. - **`complementary` row**: the hand-seeded pair `cc-plugin-agents-md` against `instruction-placement:migrate`, integration `route`. The plugin loads AGENTS.md; the skill migrates content into it. `/instruction-placement:migrate` gets a `## Boundary` section for the plugin and a dated record in `reference/sources.md`. instruction-placement goes to 0.16.9. - **15 dismissals** of shared-word pairs. - **4 refreshed dismissals**: their component descriptions changed when #5721 declared node, so they were resurfacing. Same ruling, updated dependency lists. ## Fix - `inventory.py`: `extract_builtin_plugins` finds the registrar, the loader and every registration by property names and string literals, never by minified callee names. Per plugin it reports `id`, `aliases`, `description`/`version` with `_source`, `load` and `load_guards`, `default_enabled` with its source, the policy-only and trusted-settings-only flags, `gated` and `gate_flags`, `skills`, `agents`, `commands`, `hook_events`, `mcp_servers` and `user_config`. Anything it cannot resolve goes into `partial` and degrades the lane. `check_integrity` adds the lane, with canaries `cc-plugin-sec-default` and `cc-plugin-agents-md`. `--self-check` prints it. - `overlap.py detect`: flattens the lane into plugin-backed-built-in surfaces, one for each plugin and one for each skill, agent and command, scored after every other lane. A name another lane already holds (such as `diff`) is scored only in that lane. - How each `builtin_plugins` field reports a failed read. Absent by proof stays absent. A failed read puts the field's name in the record's `partial`, and any non-empty `partial` degrades the lane. A key written before an unresolved `...spread` counts as unknown, since the spread may override it. | Field | Unresolved case reported as | |---|---| | `name` | registration skipped; `unresolved_names` lane advisory (list is a floor), including a name written before an unresolved spread. A one-letter parameter is `factory_registrations` | | `id` | `partial: id` when the marketplace id does not resolve to one value from the `${name}@${M}` template in the function that walks `builtinPlugins` | | `aliases` | floor by construction (`builtin_plugin_notes.floors`) | | `description`, `version` | `partial` when present but unresolved, or absent or written before an unresolved spread | | `load`, `load_guards` | `partial: load` when the loader requires the plugin but the walk cannot place the call: after an exit it does not recognize as always taken, after a compound latch test, or in a position it does not read. A recognized exit (`return`, `return v`, `throw`, a block ending in one) adds `!(cond)` to later calls | | `in_loader` | false: `registered_not_loaded` lane advisory and left out of overlap detection; null: "loader not located" advisory | | `default_enabled`, `enabled_from_policy_only`, `enabled_from_trusted_settings_only` | `partial` when not a `!0`/`!1` literal, or absent or written before an unresolved spread (`default_enabled` also when the `??!0` consumer rule is missing) | | `gated` | `partial` when `isAvailable` is absent or written before an unresolved spread | | `gate_flags` | `partial` when the gate does not read or a flag default does not resolve; otherwise a floor (`floors`) | | `skills` (field) | `partial` when the field is not an array, an element does not resolve, or a skill's name, description or `user_invocable` does not resolve | | `agents`, `commands`, `skills` (embedded files) | `partial` when the `files` value or a path is not a literal, a file's text is not a literal or holds a `${...}` substitution, or its frontmatter has no `name` or an unparsed `description` (plain scalars fold across indented lines; block scalars parse) | | `commands` (registered) | `partial` when `calls` is not a literal array, `command.register` is declared and nothing resolves, or a registered object, name or description does not resolve | | `agents`, `commands`, `skills`, `hook_events` | `partial` when a hooks module (or an unresolved spread) has no single readable manifest | | `hook_events`, `commands` | `partial` when the manifest's `hooks` or `calls` is not a literal array, or the manifest or its `scan`/`shipped` object holds a spread | | `hooks_module`, `user_config`, `classic_hooks`, `mcp_servers` | `partial` when absent behind an unresolved spread; `classic_hooks` also when present and not an object literal | | `registration` (whole record) | `partial: registration` plus a lane advisory when one name is registered twice; the registrar is a `Map.set` and run order is not read | | lane | advisories for unresolved names, loaded-not-registered, registered-not-loaded, duplicate registrations and a missing loader | Further reader corner cases go to #5640 (the parser replacement), not to this PR. - Docs: `reference/extraction.md` section 11, plus rows in the integrity and when-a-build-changes tables. The inventory `SKILL.md` gets the lane in its routing table, report structure, three-switches guidance and a dated upstream-claim row. The audit-native-overlap `SKILL.md` gets the lane note. claude-ops goes to 0.81.0 with a CHANGELOG entry, and the catalog is regenerated. ## Verification - On 2.1.285, 2.1.286 and 2.1.287, `--binary-only` output for every existing lane is identical to origin/main's. A JSON diff script compared everything except `builtin_plugins`, `builtin_plugin_notes` and the run's elapsed time: 0 differences on all three builds. - `inventory.py --self-check` on 2.1.287: `OK`, with all 7 lanes `ok`. - 2.1.287 lane output: 11 plugins, 11 registrations resolved (plus 1 test-seat factory), every `partial` empty. Skills: plugin-authoring 1, claude-test 3. Agents: claude-test 3. Commands: diff 1 (`/diff`). Every plugin except plugin-authoring declares hook events. 2.1.285 reads 10 plugins (no you-should-know), 2.1.286 reads 11. - `python3 -m unittest test_inventory`: 260 tests OK, including synthetic-fixture tests for each unresolved case in the table above. `python3 -m unittest test_overlap`: 190 tests OK. - `scripts/run-ruff.sh check` and `format --check` on both script directories: clean. `check-changed-skills.sh origin/main`: 2 skills PASS. Changelog parity checks pass. markdownlint: 0 issues. - Before the rulings, `overlap.py detect` on the 2.1.287 inventory returned 16 new discovered candidates. After them, `detect` on the 2.1.287 inventory exits 0 with discovered 0, resurfaced 0, dismissals orphaned 0 (24 seeded, 110 suppressed, 34 existing). On the 2.1.285 and 2.1.286 inventories one discovered candidate appears, `plugin-types` against `code-metrics:audit-type-debt`. It appears on origin/main's extraction of those builds too: the hidden `/plugin-types` command exists there and its dismissal was removed when 2.1.287 dropped it. It does not come from this lane. - `overlap.py generate --check`: in sync (71 rows, 110 dismissals). `node scripts/generate-catalog.mjs --check`: in sync. `overlap.py self-check`: degraded, only from the same two advisories origin/main reports (older recorded extraction versions, no `--upstream-sha`). - `check-changed-skills.sh origin/main`: 3 skills PASS (audit-native-overlap, inventory, migrate). `cutover-check.test.sh`: 73 checks pass. ## Related - #5731: the 2.1.287 per-release pass that found the gap. - #5640: the JS parser for the bundle reader. - #5721: the node declarations that changed the four refreshed dismissals' component descriptions. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
kyle-sexton
added a commit
that referenced
this pull request
Oct 3, 2026
…de Code 2.1.288 (#5957) No related issue: every drift item the 2.1.288 native-drift pass found is resolved in this PR (keys listed under Verification), so none was filed. ## Summary Per-release native-surface pass for Claude Code 2.1.288. Every inventory lane extracts ok and the regex and parser readers agree, so the inventory is revalidated against 2.1.288. The one native surface that moved is the hidden, gated built-in `/update`, renamed `/restart` with `update` kept as an alias. Detect surfaced 14 overlap candidates; each is ruled here by operator decision (2026-10-02, all recommended). The Explore and Plan verification records also named too few disallowed tools. ## Fix - `VALIDATED_AGAINST` in `plugins/harness-ops/skills/inventory/scripts/inventory.py` is now `2.1.288`. - `docs/native-surfaces/records.json`: the two `update` dismissals (vs `playbooks:update`, `firecrawl:update`) are orphaned by the rename, so they are removed and re-recorded against `restart`. Twelve more dismissals: - Three resurfaced pairs re-dismissed with their original reason: `claude-test` vs `prototype:pressure-test` and `testing:test-value` (native description now "Run tests on this machine (local)"), and `code-review` vs `review:security-review` (native gained `--max-findings`). - Shared-word pairs: `claude-test` vs `mutation-testing:audit`, `rename` vs `docs-hygiene:rename-references`, `agents` vs `multi-agent:assess` and `multi-agent:route`, `workflow-subagent` vs `multi-agent:assess`, and `worker` vs `discovery:sweep-worker`. - `Agent` vs `multi-agent:assess` and `multi-agent:route`: ours decide how the Agent tool is used and launch nothing. - `cc-plugin-claude-test` vs `testing:run-e2e`: covered by the existing `claude-test` -> `testing:run-e2e` defer row. - `docs/native-surfaces.md` regenerated with `overlap.py generate`; `node scripts/generate-catalog.mjs` reported the catalog already in sync. - `plugins/discovery/skills/explore/reference/native-explore.md` and `plugins/planning/skills/plan/reference/native-plan-agent.md`: on 2.1.288 both agents disallow Agent, Artifact, ArtifactComments, ArtifactData, ArtifactCheck, ExitPlanMode, Edit, Write and NotebookEdit (source `literal`). The records named four (Explore) and five (Plan) tools from the 2.1.285 extraction; they now state what that means for each skill and point at `builtin_agents.<agent>.disallowed_tools` instead of copying the list (Codex review). - Versions: harness-ops 2.5.2 -> 2.5.3, discovery 0.26.1 -> 0.26.2, planning 0.62.1 -> 0.62.2 (main took the earlier numbers while this PR was open), each with a CHANGELOG entry. ## Verification - Rename confirmed by string literal in the binaries: `name:"update"` occurs once in 2.1.287 and 0 times in 2.1.288; `name:"restart"` 0 then 1. The 2.1.288 extraction reads `restart` with `aliases: ["update"]`, `hidden: true`, `gated: true`. - `inventory.py --self-check`: `DEGRADED: cli 2.1.288, validated against 2.1.287` (exit 3, all seven lanes ok) before the bump; `OK: cli 2.1.288, validated against 2.1.288` after. - `inventory.py --reader compare --self-check`: `reader compare: ok, 2167 of 2167 modules parse`: no value->value divergence. A plain `--reader compare` run shows 4 wrong->unresolved entries: the parser reads the Explore and Plan `disallowed_tools` as partial, missing the four Artifact tools, as on 2.1.284-2.1.287 (#5901). This PR records the literal regex values, which the live Plan agent roster confirms. - `native_drift.py diff` against the 2.1.287 summary: surfaces added/removed/reclassified none, renamed `update -> restart`; invocability and markers none; fired triggers none; unresolved descriptions unchanged (`design`). The 15 items: the 14 `native-drift:candidate:*` keys above plus `native-drift:inventory-degraded:2.1.288:inventory`. - `overlap.py detect` on the new store: discovered 0, resurfaced 0, orphaned dismissals 0, 120 suppressed. - `overlap.py self-check`: exit 3, only the 2 standing advisories (older recorded extraction versions on rows; upstream SHA not locally decidable). 71 rows checked. - `TestInstalledBuilds` now covers 2.1.288 (passed locally with acorn, 88.8 s); `extraction.md` names 2.1.288 in its record. - Tests: inventory 362 OK (161 skipped without acorn), audit-native-overlap 195 OK, changelog 40 OK. Pinned ruff on `inventory.py`: all checks passed. ## Related - #5731: the same pass for 2.1.287. - #5901: the parser default flip; this pass took values from the default regex reader and used the parser only as a guard. - #5890 and #5921 added `multi-agent` and `discovery:sweep-worker`, the source of several new candidates. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #5730
Summary
Per-release native-surface pass for Claude Code 2.1.287. Every inventory lane extracts ok on 2.1.287, so the inventory is revalidated against it. The one native surface that moved is the hidden built-in command
/plugin-types, which the 2.1.287 build no longer ships. Its dismissal againstcode-metrics:audit-type-debtwas orphaned and is removed.Fix
VALIDATED_AGAINSTinplugins/claude-ops/skills/inventory/scripts/inventory.pyis now2.1.287.plugin-types/code-metrics:audit-type-debtdismissal is removed fromdocs/native-surfaces/records.json.overlap.pyhas no undismiss command, so the record was deleted from the store anddocs/native-surfaces.mdwas regenerated withoverlap.py generate.node scripts/generate-catalog.mjsreported the catalog already in sync.Verification
/plugin-typesoccurs 10 times andWrite claude-code.d.ts2 times. In 2.1.287 both occur 0 times. The onlyplugin-typesleft in 2.1.287 is the CSP directive name inside a list of*-srcdirectives.plugin-typesremoved). Nothing was added or renamed in any lane. The Explore and Plandisallowed_toolsnow readliteralwith the Artifact tools included. That comes from the extractor fix in fix(claude-ops): resolve the Explore and Plan disallowed-tools spread in its own scope #5717, not from a change in the binary.inventory.py --self-check:OK: cli 2.1.287, validated against 2.1.287, all six lanes ok, exit 0. It wasDEGRADED(exit 3) before the bump.overlap.py detecton the final extraction: exit 0, integrity ok, discovered 0, resurfaced 0, orphaned dismissals 0, 95 suppressed.overlap.py self-check: exit 3, degraded only by the 2 standing advisories it also reports on main (older recorded extraction versions on rows, upstream SHA not locally decidable). 69 rows checked, 0 problems.test_inventory.py: 215 tests OK.test_overlap.py: 184 tests OK. Pinned ruff passes oninventory.py.Related
native-drift:inventory-degraded:2.1.287:inventory, filed by this pass and closed by this PR.🤖 Generated with Claude Code