[objectstack-fleet]

Requested by 田騏 · project thread
Fixes #27455

Clause-②: no
Metadata-protocol: none

维护者速读

• 改了什么:skills/objectstack-platform/evals/config-plugins-ops.json 四个用例(1、2、4、7)的标准答案与自身的 must_not_contain 断言不再互斥——用例 1 的断言收窄为键拼法 tableName:;用例 2、4、7 的标准答案改写警句,不再逐字拼出被拒字串。另加一张 @objectstack/skills 的 patch changeset。
• 为什么改:这份评测文件既是标准答案又是评分器;四个用例的标准答案逐字命中自己的拒收串,逐字写出理想答案会被判错,平台技能在配置、插件、运维题上的质量信号因此是噪声。
• 风险与回滚:只动一份评测 fixture 与一张 changeset,不动 SKILL.md、规则、spec 或任何运行时;无断言被删除,文件缩 13 token、天花板不动。回滚 = revert 这一个 commit。
• 先看:skills/objectstack-platform/evals/config-plugins-ops.json 用例 7 的 expected_output(两处改写:role 去冒号、具名平台集改为「no platform set」),其次用例 1 的 tableName: 收窄,再次用例 2。
• 两分钟验证:仓根跑 python3 -c 'import json;d=json.load(open("skills/objectstack-platform/evals/config-plugins-ops.json"));[print(e["id"],[s for s in e["assertions"].get("must_not_contain",[]) if s in e["expected_output"]]) for e in d["evals"]]',预期九行、每行的列表都是空的 [](main 上第 1、2、4、7 行非空)。
• 需要你做的:Tier H(skills/**)——GOVERNED_APPROVERS 账号一次 APPROVED review,其后由席位落地。

What changes

One shipped fixture, skills/objectstack-platform/evals/config-plugins-ops.json (ships in @objectstack/skills), and one patch changeset. Each of the four cases was judged from the skill text, per the card's Wanted: drop the refused literal from the reference answer when the answer only names it to warn, or narrow the assertion to the wrong form it was meant to catch.

eval	refused string spelled in its own answer	judged from	fix
1	tableName	SKILL.md line 116 ("never set tableName"; also AGENTS.md Prime Directive 6) — the skill's own warning uses the bare word; the wrong form is the object key	assertion narrowed to the key spelling tableName:, the form its siblings workflows: / approvals: already use; reference answer unchanged
2	data:beforeInsert, onInstall	references/plugin-hooks.md lines 73–76 (a 'data:beforeInsert' handler on the kernel bus registers and silently never fires); AGENTS.md Plugin pattern (the contract is init / start / destroy; no install / enable / disable hooks) — both named only to warn, and the wrong forms have several spellings (method, property, call site), so no single narrowing holds	reference rephrased: "a data:* kernel hook registers and never fires", "No install / enable / disable hooks."; assertions unchanged
4	@objectstack/adapter-	SKILL.md line 275 ("There are no @objectstack/adapter-* packages") — named only to warn; narrowing to a single-quoted import would miss the double-quoted package.json dependency form	reference rephrased: "no adapter-* packages exist"; assertion unchanged
7	role:, admin_full_access	DemoStaffSchema refuses role as an alias of position (packages/spec/src/system/demo-staff.zod.ts line 71); SKILL.md lines 438–440 ("a platform set such as admin_full_access is refused") — role: is already the key spelling; admin_full_access is a value with the same quote-style leak as eval 4	reference names the alias as role (no colon) and says "no platform set" (the skill's own term); assertions unchanged

• No assertion deleted, no case added or removed. Evals 3, 5, 6, 8 and 9, including the predecessor PR docs(skills): objectstack-data, -automation and -platform teach ownership queues and assignment rules (ADR-0142 phase 2) with one eval each #27426 (merged 501bfc69) eval 9, are unchanged byte for byte.
• .changeset/27455-platform-eval-self-agreement.md: @objectstack/skills patch, carrying the two declaration lines above.
• Not done, by ruling (triage comment 6491392918): no lint over evals — a self-check that would catch the class is a separate card. The file's ceiling row in scripts/check-skills-token-ratchet.mjs is not lowered either: that script is outside this card's declared file surface, so the new headroom (14 tokens) is reported here, not spent or banked.

Tests

No test file changes: nothing in the repo executes this fixture; its consumer is the skills eval grader that applies must_contain / must_not_contain as substrings to an answer (the card's repro models that). The diff touches no workspace package, so no package build or test is owed (local verification scope ① and ② are empty).

The card's repro, before (origin/main ded67bf) and after (6f2257a)

$ python3 -c 'import json;d=json.load(open("skills/objectstack-platform/evals/config-plugins-ops.json"));[print(e["id"],[s for s in e["assertions"].get("must_not_contain",[]) if s in e["expected_output"]]) for e in d["evals"]]'
# before
1 ['tableName']
2 ['data:beforeInsert', 'onInstall']
3 []
4 ['@objectstack/adapter-']
5 []
6 []
7 ['role:', 'admin_full_access']
8 []
9 []
# after
1 []
2 []
3 []
4 []
5 []
6 []
7 []
8 []
9 []

Per-case cross-check after (no must_not_contain string in its own answer, every must_contain string present, JSON parses)

1 neg-in-answer: [] pos-missing: []
2 neg-in-answer: [] pos-missing: []
3 neg-in-answer: [] pos-missing: []
4 neg-in-answer: [] pos-missing: []
5 neg-in-answer: [] pos-missing: []
6 neg-in-answer: [] pos-missing: []
7 neg-in-answer: [] pos-missing: []
8 neg-in-answer: [] pos-missing: []
9 neg-in-answer: [] pos-missing: []
cases: 9 failures: 0

Acceptance notes

Tier H (skills/**) readings, both required:

• Changed file, whole file: 95 → 95 lines; 6756 → 6703 bytes; 1689 → 1676 tokens (scripts/check-skills-token-ratchet.mjs, ceiling 1690 unchanged, headroom 1 → 14).
• Whole package: skills/objectstack-platform/** 88041 → 87988 bytes; the sum of every skills/*/SKILL.md 4610 → 4610 lines (no SKILL.md touched); the whole authored bundle 207870 → 207857 tokens (ceiling sum 217474).

Observations, noted and not filed:

• No gate in either root checks a must_not_contain string against the same case's expected_output (ui/scripts/check-skill-eval-tokens.mjs deliberately shape-checks must_not_contain only). Triage ruled that lint a separate card; the seat carries it.
• Eval 3 also refuses @objectstack/adapter-; its answer never spelled the string, so it is untouched.

Verification

Readings taken on the final head 6f2257ad (git rev-parse --short HEAD), in the worktree, after pnpm install --frozen-lockfile.

• node scripts/pm/dispatch-gates.mjs --commands --repo objectstack-ai/objectstack derived 42 commands from the diff against the merge base ded67bfb. Each ran with its exit code captured before any pipe: 37 exit 0; 5 exit 3 (PREREQUISITE NOT MET — no package dist/ in the tree; the diff touches no package, so the closure build is not owed locally and CI runs these): pnpm --filter @objectstack/lint run check:doc-formula-expressions, pnpm check:dts-closure, pnpm check:dual-build-cjs-loads, pnpm check:lean-entry-closure, pnpm check:sourcemap-no-sources-content. Reconciliation node scripts/pm/dispatch-gates.mjs --ran FILE --repo objectstack-ai/objectstack: "42 derived famil(ies) accounted for — 37 run, 5 NOT-MEASURED (5 DERIVED from a recorded exit 3)", exit 0.
• node scripts/check-skills-token-ratchet.mjs: exit 0 — "skills/objectstack-platform/evals/config-plugins-ops.json is 1676 tokens (ceiling 1690; headroom 14)"; --self-test: exit 0, "79 cases pass".
• Control-byte self-scan over both touched files: none; pnpm check:nul-bytes exit 0.
• Dogfood battery: n/a — no ! in the title and no (narrowing) arm.
• CI check-runs: in progress at the time of opening; the seat reads them.

Gate ledger (exit code, command) on 6f2257a

0	node scripts/check-adr-0087-registration.mjs --base origin/main
0	node scripts/check-adr-0087-registration.mjs --self-test
0	node scripts/check-changeset-no-major.mjs --base origin/main
0	node scripts/check-closing-keyword-parity.mjs
0	node scripts/check-closing-keyword-parity.mjs --self-test
0	node scripts/check-comment-mask-corpus.mjs
0	node scripts/check-doc-route-spelling.mjs --advisory
0	node scripts/check-doc-route-spelling.mjs --self-test
0	node scripts/check-dts-emitted.mjs --self-test
0	node scripts/check-empty-changeset.mjs --base origin/main
0	node scripts/check-empty-changeset.mjs --self-test
0	node scripts/check-skills-token-ratchet.mjs
0	node scripts/check-skills-token-ratchet.mjs --self-test
0	node scripts/ci/assert-turbo-remote-cache.mjs --self-test
0	node scripts/ci/plan-test-core-shards.mjs --self-test
0	node scripts/ci/prune-turbo-cache.mjs --self-test
0	node scripts/pm/release-rehearsal-clone.mjs --self-test
0	node scripts/release-pending-publish.mjs --self-test
3	pnpm --filter @objectstack/lint run check:doc-formula-expressions   (PREREQUISITE NOT MET: @objectstack/lint not built)
0	pnpm check:agent-test-spelling
0	pnpm check:changeset-gate-self-tests
0	pnpm check:ci-filter-parity
0	pnpm check:closed-ceremony-text
0	pnpm check:corpus-claim-drift
0	pnpm check:cross-package-test-inputs
0	pnpm check:doc-authoring
0	pnpm check:driver-memory-census
3	pnpm check:dts-closure   (PREREQUISITE NOT MET: no dist/)
3	pnpm check:dual-build-cjs-loads   (PREREQUISITE NOT MET: no dist/)
0	pnpm check:gitlink-declared
3	pnpm check:lean-entry-closure   (PREREQUISITE NOT MET: no dist/)
0	pnpm check:nul-bytes
0	pnpm check:pm-changeset-deadline-census
0	pnpm check:pm-governed-merges
0	pnpm check:published-files
0	pnpm check:refd-timer-probe
0	pnpm check:role-word
0	pnpm check:skill-compatibility
0	pnpm check:skill-frame-sync
0	pnpm check:skill-identifier-liveness
3	pnpm check:sourcemap-no-sources-content   (PREREQUISITE NOT MET: no dist/)
0	pnpm check:watch-hint-literal

---

Generated by Claude Code

[objectstack-fleet]

domain:skills#1 · marchtian · session_017ihL2dPRZXfgE1A2Vct3FU

Contract review

Served-tier: CONTRACT_REVIEW_TIER
Head-sha: 6f2257adda5f95f7105bed948477c094c3532d11
Local-runs: probe — the card's one-line python repro (plus a must_contain-present cross-check) re-run against git show 6f2257ad:skills/objectstack-platform/evals/config-plugins-ops.json into the scratchpad; nothing built, no gate re-run
Scope: skills/objectstack-platform/evals/config-plugins-ops.json + skills/objectstack-platform/SKILL.md (lines 116, 265–279, 425–447), skills/objectstack-platform/references/plugin-hooks.md (lines 62–76), skills/objectstack-platform/references/plugin-development.md (lines 8, 135–141), skills/objectstack-platform/rules/plugin-lifecycle.md, packages/spec/src/system/demo-staff.zod.ts (line 71), packages/spec/src/shared/strict-object.ts (aliases contract), packages/adapters/hono/package.json, AGENTS.md (lines 200, 763–764), scripts/check-skills-token-ratchet.mjs (row at line 1073; discipline header), .changeset/27455-platform-eval-self-agreement.md (read for ②). The changeset is otherwise the lane seat's ACCEPT at TIER_DEFAULT.

Reviewer: an independent agent at CONTRACT_REVIEW_TIER (not the drafter), fed the card, the PR and the check-runs only, adopted by the domain:skills#1 seat.

① Derived judgments

• Accept set: two files — the hits fixture (4 lines changed, 95 → 95 lines, 6756 → 6703 bytes) and one new patch changeset. No SKILL.md, rule, reference, spec, runtime or script moves. reviewFace answers face rules, hits = the fixture alone; the record stands at CONTRACT_REVIEW_TIER.
• Public surface: the fixture ships in @objectstack/skills (packages/skills/scripts/sync-catalog.mjs copies skills/** byte-identical into dist/skills): a published-content change with no API, key or schema effect.
• Eval 1 assertion tableName → tableName:: a narrowing of the grader, judged right (④). Evals 2, 4, 7: expected_output rephrased, assertions byte-identical. Evals 3, 5, 6, 8, 9 unchanged. No assertion deleted, no case added or removed — confirmed per case against origin/main ded67bfb.
• Token ratchet: ceil(6703/4) = 1676 under the unchanged 1690 ceiling (scripts/check-skills-token-ratchet.mjs:1073); budget "net growth ≤ 1 token", actual −13. Lint & Repo Gates (carries the ratchet) is success on the head.
• The card's repro against the head: all nine rows []; every must_contain string present in its own expected_output; JSON parses. On origin/main the same probe prints the card's four rows.
• Check-runs on 6f2257ad at review time: no failure, no cancellation; Test Core (1/8) and Type Check · consumer gates in progress (the seat reads them before the hand-off).

② Semver level

• '@objectstack/skills': patch — right: published content of that package, no export, key or behaviour added, removed or renamed. check-changeset-no-major and Check Changeset green.
• Clause-②: no on the PR body and in the changeset — right: a grader string narrowed from tableName to tableName: is the fixture agreeing with its own reference answer, not a spec narrowing. Metadata-protocol: none — right.
• The changeset's Before/Now account matches the diff line for line.

③ Boundary flags

Dev report = card comment 6494906404.

1. 5 of 42 derived gate families NOT MEASURED locally (exit 3, no dist/). Answered: no workspace package in the diff; Build Core success, Type Check · consumer gates carries them in CI.
2. PR-body footer in the AGENTS.md single-line form (open question 1): answered A — AGENTS.md pins one footer line, PR-body guards success.
3. os-dev-report on line 2 of the report comment (post-stamped's signature rule): a writer constraint, not a content defect.
4. test:repo not run locally: Test Core (2/8) success; the diff names no test and no package.
5. Ratchet row not lowered to 1676 (open question 2): answered B — lowering is encouraged, not required; outside this PR's two-file surface.
6. No label write: nothing owed.

• Out-of-scope: the missing must_not_contain vs expected_output lint is triage 6491392918's separate card; eval 3's @objectstack/adapter- refusal never matched its answer — unchanged. No flag escalates.

④ Design judgment

hits = skills/objectstack-platform/evals/config-plugins-ops.json, judged per case against the eval prompt and the skill text on main (substring grader semantics).

• Eval 1 — SKILL.md:116 "never set tableName" and AGENTS.md:200 use the bare word, so the ideal answer must be allowed to say it; the key spelling tableName: still catches tableName: 'crm_account', matching the siblings workflows: / approvals:. Residual (quoted key) is the siblings' residual on main, not introduced here. Holds.
• Eval 2 — "a data:* kernel hook registers and never fires" is references/plugin-hooks.md:62–76's own wording, generalised correctly; "No install / enable / disable hooks." names the AGENTS.md:763–764 phantoms without literals. ctx.hook('data:beforeInsert', …) and onInstall() still caught; every must_contain still present. Holds (and clearer, since SKILL.md:388–397 teaches a real config-level onEnable).
• Eval 4 — SKILL.md:275 "no @objectstack/adapter-* packages"; tree: only @objectstack/hono under packages/adapters/. Imports and package.json dependencies on @objectstack/adapter-… still caught. Holds.
• Eval 7 — demo-staff.zod.ts:71 role: 'position' in DemoStaffSchema.aliases (curated near-miss rejection per strict-object.ts), so "no role alias" is exact; "no platform set" is SKILL.md:437–440's term. role: … and permissionSets: ['admin_full_access'] still caught. Holds.
• Ruling 「⛔ no assertion is deleted」: must_not_contain counts main → head 3→3, 2→2, 2→2, 2→2; one string rewritten (eval 1), none removed. Honoured.
• Attempted break that did not land: an answer quoting a phantom literal inside its own warning is still graded wrong — the trade-off the card's Wanted chose, as in the automation-skill repair.

Must-fix: none. Should-fix: none in the hits file.

Implemented-by: claude/issue-27455-platform-evals-agree-with-answers
Reviewed-by: session_017ihL2dPRZXfgE1A2Vct3FU

VERDICT: PASS

[objectstack-fleet]

domain:skills#1 · marchtian · session_017ihL2dPRZXfgE1A2Vct3FU

席位核对 — ACCEPT · #27455 · head 6f2257adda5f95f7105bed948477c094c3532d11 · 2026-10-29T11:13Z

维护者速读

• 改了什么:objectstack-platform 技能的配置/插件/运维评测里有 4 条用例,禁用词恰好出现在自己的标准答案里;这次逐条对照技能原文改正——1 条把禁用词收紧为键写法 tableName:,3 条把标准答案换一种说法而不再写出那个禁用字面量。
• 为什么改:评测既是标准答案又是评分器;照抄标准答案的 AI 回答也会被判错,这份评测因此分不出好坏答案,技能质量信号失真。
• 风险与回滚:只动一个评测夹具文件(4 行)和一个 patch changeset,不改任何技能正文、spec 或运行时;断言一条未删,文件 1689 → 1676 token(上限 1690 不变)。回滚即 revert。
• 先看:skills/objectstack-platform/evals/config-plugins-ops.json 的 eval 1、2、4、7。
• 两分钟验证:跑卡片里那行 python(卡 skills(objectstack-platform): four eval cases in config-plugins-ops.json refuse strings their own expected answer contains, so a verbatim ideal answer grades as wrong #27455 的 Repro),main 上打印 4 行冲突,本分支 9 行全空。
• 需要你做的:批准本 PR(Tier H,skills/**)。契约复核 PASS 在案:评论 6495077720,Head-sha: 6f2257adda5f95f7105bed948477c094c3532d11;批准后本席落地。

席位意见:改法符合分诊裁决 6491392918 与卡片 Wanted 两臂之一,逐条有技能原文出处;独立 Fable 完整复核(①–④)无 must-fix、无 should-fix。

席位核对

1. 形态 — 2 files, +16/−4: the fixture (evals 1, 2, 4, 7 only; 95 → 95 lines) and .changeset/27455-platform-eval-self-agreement.md (@objectstack/skills patch). Inside the claimed surface (6494361803). Tier H (skills/**).
2. 前提实测 — the card's repro on origin/main ded67bfb printed the four rows (seat, claim); at this head all nine rows are empty (independent reviewer's probe, record 6495077720). must_not_contain counts per case main → head 3→3, 2→2, 2→2, 2→2: no assertion deleted.
3. 门禁 check-runs — at this record: 25 success, 14 skipped, Test Core (1/8) in progress; none failed. Landing waits for all green or expected-skip (check-expected-skips.mjs).
4. 接受的偏差 — 5 of 42 gate families NOT MEASURED locally (exit 3, no dist/; no workspace package in the diff; CI's Build Core / Type Check carry them); PR-body footer in AGENTS.md's single-line form (accepted, PR-body guards green); report marker on line 2 behind the signature (writer rule).
5. 范围外去向 — (a) a lint cross-checking each eval case's must_not_contain against its own expected_output: triage 6491392918 already ruled it a separate card, not a gate here; per SKILL.md 「新增门禁默认否」 the seat does not file a new gate card without the maintainer's word — dropped — new gate, default no. (b) Lowering the fixture's ratchet row 1690 → 1676: dropped — optional (the ratchet header calls it encouraged, not owed); the next PR touching that row may bank it. Skills-docs: n/a (Metadata-protocol: none).
6. 落地条款 — Tier H: hand-off to GOVERNED_APPROVERS (os-zhuang, hotlong) via pr_request_reviewers with this comment; needs-user-decision on the PR while it waits. After an APPROVED review and every check green: strip the label, check-governed-merges test with the paths, relay pr_ready + automerge_enable, confirm MERGED, close skills(objectstack-platform): four eval cases in config-plugins-ops.json refuse strings their own expected answer contains, so a verbatim ideal answer grades as wrong #27455 if Fixes does not fire.