Epic: AMS parity for the ORB backtest/calibration system (blocked until ORB side is complete)

[JSONbored]

Problem

Everything in #8082 — the calibration ground-truth wiring, the backtest primitives, the advisory CI check, the confirmed-signal design — is ORB-only, by explicit stated scope in every one of its sub-issues. AMS has its own class of deterministic/policy decisions (eligibility exclusions — the kind #7982's own AMS adapter, packages/loopover-miner/lib/signal-tracking-store.ts, already records fired events for via discover-cli.ts) that deserve the same backtest-before-ship discipline eventually. Building both sides blind, in parallel, risks duplicating design mistakes the ORB build will surface first — #8082 itself has already gone through one real supersession (#8101 → #8104) from research turning up a better design mid-flight.

Area

Mirrors #8082's own shape, applied to packages/loopover-miner/packages/loopover-engine's AMS-facing surfaces instead of src/rules/**/src/review/**.

Proposal

Once #8082 is fully complete (every sub-issue closed, including the confirmed-signal design and the Phase-2 merge-gate decision) and has run in production long enough to surface real lessons, decompose this epic into AMS-mirrored sub-issues. The likely shape (named here so the roadmap is legible now — NOT precisely scoped, see Deliverables/Boundaries):

1. AMS ground-truth wiring (mirrors calibration: wire linked_issue_scope_mismatch into the shared signal-tracking module (fired + reversal) #8101/calibration: wire recordRuleFired/recordHumanOverride for the remaining isConfiguredGateBlocker codes (excludes linked_issue_scope_mismatch, covered by #8101) #8104) — record fired + override events for AMS's own eligibility/policy exclusion codes, via the already-built packages/loopover-miner/lib/signal-tracking-store.ts adapter. AMS's "reversal" equivalent needs its own real-precedent audit at that time (AMS has no PR-reopen concept the way ORB does — what a miner or maintainer overriding an AMS exclusion actually looks like needs checking against AMS's real event-ledger vocabulary then, not guessed at now).
2. AMS advisory CI check (mirrors Design + wire an advisory-only backtest check for PRs touching ORB/AMS deterministic rules #8089) — backtest a proposed change to an AMS policy/eligibility heuristic before it ships, reusing calibration: pure BacktestCase corpus builder from RuleFiredEvent/HumanOverrideEvent pairs #8083-calibration: render a backtest score/comparison report as Markdown #8088's primitives directly (they're host-agnostic pure functions — likely zero changes needed).
3. AMS's own confirmed-signal design (mirrors Design a positive-confirmation ('this automated decision was correct') signal #8106) — only needed if Design a positive-confirmation ('this automated decision was correct') signal #8106's ORB-side decision doesn't generalize cleanly to AMS's different action shape (a miner's own submission being excluded, vs. a PR being blocked).
4. AMS trend/docs (mirrors Feed rule/AI-judgment calibration data into an accuracy trend view #8113/Document the backtest/calibration system once #8138 ships #8114) — once 1-2 have real data.

Deliverables

Deliberately none precisely scoped yet — the numbered list above is a roadmap sketch, not build-ready issue bodies. Decomposing this into the exact-file/exact-function precision #8082's own sub-issues required, before the ORB side is proven, would risk scoping against code that will have moved by the time this is actually unblocked (the ORB epic itself needed one real correction, #8101 → #8104, from research turning up a better design mid-build — the AMS side deserves the same chance to be built with the ORB side's finished lessons in hand, not guessed at now).

Resources

• Epic: verifiable pre-merge backtest for ORB/AMS deterministic rules #8082 (the ORB-side epic this mirrors and is blocked by)
• packages/loopover-miner/lib/signal-tracking-store.ts (AMS's own already-built calibration adapter — the AMS-side counterpart to ORB's signal-tracking-wire.ts)

Boundaries

No sub-issues get filed under this epic, and no code gets written against it, until #8082 is fully closed. This is a deliberate, explicit hold — not an oversight if it sits untouched for a while.

maintainer-only — cross-cutting review-infrastructure work, blocked pending #8082, not a build task for contributors.

[linear-code]

JSO-5595