feat(copilot): add GitHub Copilot as a first-class host platform

[ShawnXxy]

Summary

Makes GitHub Copilot a first-class, evidence-safe Better Harness host alongside Qoder, Codex, Claude Code, Cursor, and Qwen Code.

Copilot already loads the canonical better-harness Skill today, but only by accident: it resolves plugin manifests in the order .plugin/plugin.json → plugin.json → .github/plugin/plugin.json → .claude-plugin/plugin.json, so it falls through to the Claude Code shell. That fallback is undocumented, gives Copilot no shell of its own, and makes load order depend on which other host shells happen to exist.

The larger gap was evidence: every provider enum rejected copilot, so the workflow could not inventory Copilot's configured assets or read Copilot session transcripts. A Copilot user could run the Skill but could not get a Copilot-scoped Harness report.

Spec: docs/specs/2026-07-29-copilot-host-support.md

What's in it

Native shell — .github/plugin/plugin.json + .github/plugin/marketplace.json, version-pinned to package.json (0.3.0) like every other host manifest. Copilot now resolves its own shell instead of borrowing Claude Code's.

Asset provider — scripts/agent-customize/providers/copilot.mjs (new, 426 lines). Adds --provider copilot / --copilot-home, inventorying plugin, user (~/.copilot), and project (.github) scopes with the same scope separation as existing providers.

Session platform — scripts/session-analysis/platforms/copilot.mjs (new, 571 lines). Adds --platform copilot, normalizing ~/.copilot/session-state/<id>/events.jsonl + workspace.yaml. Copilot's transcript is the richest of any supported host — it records hook, subagent, plan, compaction, and permission lifecycle events directly. Unrecognized event types degrade to metadata.<rawType> rather than being dropped, so future Copilot events never silently vanish.

Evidence honesty — Copilot facts report usageFieldsObserved: false because the transcript records no per-response token usage. The adapter reports partial coverage rather than inventing it, and rejects foreign-workspace sessions.

Provider enum registration — copilot is now accepted across evidence-bundle contract/CLI, report-quality, task-loop, agent-lint, the better-harness-cli registry, coding-agent-practices inventory/baseline/integrity, and pack:verify.

Documentation routing fixes — several agent-facing routing docs enumerated providers as <cursor|qoder|codex|claude>, silently omitting both Qwen and Copilot. An agent reading those docs could never discover --provider copilot (or qwen). Corrected in the agent-customize inventory CLI flags, the global-assets scope + home list, the routing index, the ADR host-adapter comment, and the session-evidence --provider enum.

Commits

Commit	Scope	Δ
93abeec feat:	Adapters, plugin manifests, provider enums, CLI contract fixtures	29 files, +1118/−43
ffd6ab6 feat:	Deterministic Copilot fixtures & coverage	3 files, +277/−1
8f362ea docs:	Spec, Copilot asset route, README/architecture, routing-enum fixes	19 files, +455/−60

Total: 51 files, +1850/−104.

Each slice was verified standing alone by materializing the index read-only and running the suite against it, so the series is bisectable — no commit in the middle is red.

Verification

• npm test — 872 / 872 pass, 0 fail
• npm run pack:verify — PASS (npm 314 entries, runtime zip 338)
• node scripts/doc-link-graph/cli.mjs skills/better-harness — 34 files / 50 links, zero drift
• Live smoke: harness evidence-bundle --platform copilot returns all three lanes as available; --copilot-home override routes correctly

Acceptance criteria

All nine confirmed: CHS-AC-1 (native shell) · AC-2 (version alignment) · AC-3 (configured assets) · AC-4 (session evidence) · AC-5 (evidence boundaries) · AC-6 (bundle propagation) · AC-7 (host routing) · AC-8 (provider behavior) · AC-9 (documentation integrity).

Non-goals

No changes to other hosts' adapters, no new report formats, no changes to coverage semantics. Copilot is added alongside existing hosts, not in place of any.

Open questions for reviewers

1. plugin.json schema — does Copilot publish a versioned JSON schema the manifest test should validate against? Currently the test asserts structure by hand. (tracked as [NEEDS CLARIFICATION] in the spec)
2. ~/.copilot/session-state stability — is the layout stable enough to promote from observed to documented in the host matrix? (tracked as [NEEDS CLARIFICATION] in the spec)
3. Install command — the README documents copilot plugin marketplace add QoderAI/better-harness. I could not verify the owner/repo shorthand against Copilot CLI's real marketplace-add grammar. Please confirm before merge.

Reviewer notes

• The .github/plugin/ location is not arbitrary — it is the third entry in Copilot's manifest resolution order and the first that doesn't collide with another host's shell. See the resolution order in scripts/agent-customize/providers/copilot.mjs.
• test/scripts-refactor-contract.test.mjs changes only two SHA-256 constants. That is a lockstep artifact of the CLI help text gaining copilot, not new test logic — which is why it ships in the feat commit rather than the test one.
• Conversely, test/agent-customize-architecture.test.mjs, better-harness-skill, coding-agent-platform-notes, and style-templates assert on docs prose, so they ship in the docs commit.

---

Reviewed with .agents/skills/change-traceability-review/SKILL.md (Review Readiness Check mode).

[phodal]

Blocking on evidence correctness found with native GitHub Copilot CLI 1.0.74 data. The cross-platform CI suite and deterministic fixtures pass, and isolated marketplace installation works, but the public home override is ignored and real session usage, permission lifecycle, and coverage fields are misreported. Please address the four inline findings and add CLI/native-shape regression coverage before merge.

[phodal]

[P1] Forward --copilot-home into the collector. The help now advertises this flag, but the options object passed to collectAgentCustomizeInventory never includes copilotHome. In a local repro, requesting an isolated temp home still resolved /Users/phodal/.copilot and scanned the real user's assets. That breaks AC-6 and can cross the caller's intended authorization boundary. Please pass copilotHome: options["copilot-home"] and add a CLI-level regression test that asserts the resolved home and inventory both come from the override.

[phodal]

[P1] Preserve the per-message output usage that Copilot actually records. In native 1.0.74 transcripts, 498 of 608 matching assistant messages carried numeric data.outputTokens (365,290 total), but this adapter keeps only model and emits type: "assistant". session-efficiency.mjs excludes ordinary assistant events from model responses, so the same data produces responseCount: 0 and tokenTotals: null. Please model these as partial response usage (without inventing input tokens or exact cost), retain a response/request identity for deduplication, and update AC-5/docs/tests to describe partial rather than wholly unobserved per-response usage.

[phodal]

Rechecked at 58b93fd with the same native Copilot data. The companion events do recover 498 responses and 365,290 outputTokens, but the public usage-summary now emits {"inputTokens":0,"outputTokens":365290,"cacheReadInputTokens":0,"cacheCreationInputTokens":0}. Those zeroes come from the generic aggregation/serialization path, so CHS-AC-5’s “omit input/cache tokens rather than reporting them as zero” is still not true at the user-facing contract. Please preserve per-field observation through aggregation (for example, omit unobserved fields or expose an explicit observed-field mask without serializing missing fields as measured zero) and add a usage-summary regression that asserts only output usage is observed and exact cost remains unavailable.

[phodal]

[P2] Normalize Copilot's actual permission lifecycle event names. The local transcripts contain 65 permission.requested and 65 permission.completed records; they do not use session.permissions_changed. Both currently fall through to metadata.*, so permission approvals/denials cannot contribute bounded operation-control or friction evidence even though the PR describes permission lifecycle support. Please map the request/result pair without retaining prompt or command payloads and cover the observed event shape in tests.

[phodal]

[P2] Align this coverage payload with the public session-core-facts contract. This provider emits withTranscript, withoutTranscript, and emptyTranscripts, while safeSourceCoverage accepts withConversation, withRequest, unreadable, and outOfWindowSessions. In a native run with 21 matching workspace sessions, 16 transcripts, and 5 transcript-less sessions, the public facts therefore reported withConversation: 0 and withRequest: 0 and dropped the structured missing-transcript count. Please either populate the canonical fields here or extend the bounded facts schema and tests so unknown/unmapped evidence is not serialized as confirmed zero.

[phodal]

Rechecked public facts at 58b93fd against the same native workspace: it reports workspaceSessions: 21, inWindowSessions: 16, outOfWindowSessions: 0, timeUnobservedSessions: 0, relevantSessions: 16, and unreadable: 0, although 5 matching session directories still have no events.jsonl. facts always supplies an until, so requestedWindow is true; relevant narrows to inWindowProbes, while timeUnobserved counts only probes that already have a transcript and outOfWindowSessions subtracts all time-unobserved probes. The 5 missing transcripts therefore remain unaccounted for in every canonical counter. Please keep them visible as time-unobserved/unreadable evidence (or extend the bounded schema) and cover the actual public facts result, not only the internal factsSourceCoverage object.

[ShawnXxy]

Thanks for the detailed review @phodal — all four findings are fixed. Summary of what changed and why:

1. --copilot-home was ignored (P1)

Confirmed: scripts/agent-customize/cli.mjs parsed the flag but never put copilotHome into the collector options, so the CLI always inventoried the real ~/.copilot. The collector API path was already correct, so only the CLI half of CHS-AC-6 was broken. Fixed by forwarding the key, plus a CLI regression test that asserts the inventory resolves under the fixture home and that no evidence.path points at the real user home. Reverting the one-line fix reproduces exactly the symptom you described, so the test bites.

2. Per-response usage was dropped (P1)

Confirmed and quantified against 491 local Copilot sessions: 10,462 of 11,313 assistant.message events carry a non-zero outputTokens (10.7M tokens total), and messageId/requestId are present on 100% / 99.2%.

isModelRequestEvent excludes assistant-typed events, so usage cannot ride the message itself. Each assistant message that reports usage now emits a companion model.response.completed carrying outputTokens, messageId as responseId, and requestId. Input and cache tokens stay absent, not zero — no invented coverage. Roughly 7.5% of messages legitimately have no usage, so the framing stays partial: the warning code is renamed copilot-per-response-usage-partial and CHS-AC-5 plus Session Diagnostics say so explicitly.

3. Permission lifecycle (P2)

permission.requested / permission.completed now normalize into the shared control.permission lifecycle. Retained: requestId, request kind, result decision. Never retained: intention, path, paths, commands, prompt text. The test includes a payload-leak scan over the serialized events.

Two design calls worth flagging, since they read as omissions otherwise:

• The decision rides the result event only. permissionObservation counts any event carrying a decision, and Copilot emits a request even when policy auto-approves. Putting a decision on both would double-count every permission and mark all 1,177 requests as prompted friction. permissionMode is left unset so it defaults to unknown, which is already in ROUTINE_PERMISSION_MODES — no false escalation. Request kind goes into a separate permissionKind field rather than permissionBoundary.
• toolInvocationId is deliberately unset. dedupeEvents keys on toolInvocationId + lifecyclePhase. Across the 491 sessions, 5 sessions contain 32 distinct toolCallIds that repeat across separate permission.requested events — setting the field would have silently dropped those. Pairing is done on permissionRequestId instead. There's an inline comment recording this.

session.permissions_changed is kept; the new mapping is additive.

4. Coverage contract (P2)

Source coverage now emits the canonical session-core-facts fields that safeSourceCoverage whitelists (inWindowSessions, outOfWindowSessions, timeUnobservedSessions, relevantSessions, withConversation, withRequest, terminalOnly, unreadable), keeping the Copilot-specific transcript counters beside them because analysisWarnings reads the raw scope payload.

Fixing this surfaced two latent bugs that were hiding real signal:

• Coverage read probe fields (transcriptAvailable, conversationRecords, requestRecords) off session descriptors, which never carried them — so withRequest was always zero. Probes are now threaded through separately.
• The relevant-set ladder narrowed even when no window was requested, and Copilot's inWindow additionally filters on transcriptAvailable, so transcript-less sessions vanished instead of being reported as unreadable — the exact signal this finding exists to preserve.

Validation

npm test 874/874 (was 872, +2 new tests) · npm run pack:verify PASS (npm 314 / runtime zip 338) · node --test test/doc-link-graph.test.mjs 6/6 with no mermaid drift.

Spec updated in the same commit: CHS-AC-5 and CHS-AC-6 in docs/specs/2026-07-29-copilot-host-support.md.

All of the above is in 58b93fd.