feat(calibration): backfill the corpus from historical review_targets decisions

[JSONbored]

Summary

• Ships calibration: backfill the corpus from the 10k+ historical review ledger (labels + firings; raw context phase 2) #8157 phase 1 — and the backfill is already applied to production (per the issue's dry-run-report-then-apply protocol, both recorded on the thread): the pure core synthesizes the signal.rule_fired/signal.human_override pairs the calibration: wire linked_issue_scope_mismatch into the shared signal-tracking module (fired + reversal) #8101 capture writers would have produced from historical review_targets decisions — close verdicts with decision-level confidence, labeled by terminal outcome, mapped per the ratified option (a) onto ai_consensus_defect (one rule id, never duplicated), every row tagged backfilled: true + provenance.
• Integrity properties, each pinned by tests: never fabricates (non-close verdicts, missing confidence, non-terminal rows are counted and skipped; modelResponseText never synthesized), idempotent by construction (deterministic ids + chunked INSERT OR IGNORE; re-runs and partial-apply overlaps are no-ops), latest-terminal-decision-wins dedupe, and both SQLite/ISO timestamp eras normalized. A buildBacktestCorpus round-trip test proves synthesized rows become real labeled BacktestCases.
• Production result: 460 eligible decisions → 920 rows applied; verified live by backtest-corpus-export round-tripping 460 labeled cases (checksum bd92485cea22…). The threshold backtest and the calibration trend now start from real history instead of an empty corpus.

Advances #8157 (phase 2 — GitHub-truth reversal labels + raw-context re-fetch — is being split to its own issue; #8157 closes once this merges).

Scope

• The PR title follows type(scope): short summary Conventional Commit format, for example fix(api): restore profile access checks.
• This PR is focused and does not mix unrelated backend, UI, MCP, docs, dependency, and deploy changes.
• This follows CONTRIBUTING.md and does not reintroduce GitHub Pages, VitePress, site/, or CNAME.
• I linked a currently open issue this PR resolves (e.g. Closes #123) — a linked open issue is required for every contributor PR.

Validation

• git diff --check
• npm run actionlint
• npm run typecheck
• npm run test:coverage locally; codecov/patch requires ≥99% coverage of the lines AND branches you changed (aim for 100% on your diff so CI variance does not fail near the threshold). Global coverage is a non-blocking trend with a loose 90% backstop, not the gate.
• npm run test:workers
• npm run build:mcp
• npm run test:mcp-pack
• npm run ui:openapi:check
• npm run ui:lint
• npm run ui:typecheck
• npm run ui:test
• npm run ui:build
• npm audit --audit-level=moderate
• New or changed behavior has unit/integration tests for new branches, fallback paths, and sanitizer boundaries

If any required check was skipped, explain why:

• Full npm run test:ci ran green through the whole suite (1108 test files, 0 failures); its ui:version-audit failure was purely the branch's stale fork point, resolved by rebasing onto current main (audit re-ran green, plus typecheck + the new suite). The diff is three standalone scripts/** + test/unit/** files — outside coverage.include per the established scripts exemption, with 100% line AND branch coverage on the pure core anyway (10 tests).

Safety

• No secrets, wallet details, hotkeys, coldkeys, user PATs, private keys, raw trust scores, private rankings, or private maintainer evidence are exposed.
• Public GitHub text stays sanitized, low-noise, and does not imply compensation guarantees or optimization tactics.
• Auth, cookie, CORS, GitHub App, Cloudflare, or session changes include negative-path tests.
• API/OpenAPI/MCP behavior is updated and tested where needed.
• UI changes use live API data or real empty/error/loading states, not production mock/demo fallbacks.
• Visible UI changes include a UI Evidence section below with JPG/JPEG or PNG screenshots arranged as organized, captioned, clickable thumbnails. SVG screenshots are not used as review evidence. Review-only screenshots or recordings are not committed to the repository.
• Public docs/changelogs are updated where needed; changelogs are only edited for release-prep PRs.

Backfilled rows carry only what the ledger already held (verdict, confidence, timestamps) plus tags — no raw content of any kind is synthesized or copied.

UI Evidence

Not applicable — no UI change.

Notes

• Reversible by tag: DELETE FROM audit_events WHERE json_extract(metadata_json,'$.backfilled') removes the era cleanly if ever needed.

[superagent-security]

Superagent didn't find any vulnerabilities or security issues in this PR.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 92.06%. Comparing base (87e8a89) to head (fc2b53b).
⚠️ Report is 3 commits behind head on main.
✅ All tests successful. No failed tests found.

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #8169   +/-   ##
=======================================
  Coverage   92.06%   92.06%           
=======================================
  Files         768      768           
  Lines       77838    77838           
  Branches    23516    23517    +1     
=======================================
  Hits        71663    71663           
  Misses       5062     5062           
  Partials     1113     1113           

Flag	Coverage Δ
shard-1	56.99% <ø> (-0.23%)	⬇️
shard-2	53.52% <ø> (+0.98%)	⬆️
shard-3	51.99% <ø> (-1.73%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

[loopover-orb]

Caution

🛑 LoopOver review result - fixes required

_{Review updated: 2026-07-23 08:24:30 UTC}

3 files · 1 AI reviewer · 2 blockers · CI failing · blocked

🛑 Suggested Action - Manual Review

Review summary
Adds a pure-core + thin-IO CLI pair that reconstructs signal.rule_fired/signal.human_override audit rows from historical review_targets close decisions, mapping them onto a single ai_consensus_defect rule id per the ratified option (a), with a 150-line test suite exercising every eligibility branch (wrong verdict, missing/NaN confidence, non-terminal status, duplicate-target dedupe, both timestamp eras, and a round-trip through buildBacktestCorpus). The core logic is correct: eligibility filtering, descending sort by normalized terminal timestamp, and latest-wins dedupe by targetKey all match the stated integrity rules, and the CLI is a straightforward dry-run-by-default wrapper mirroring the existing backtest-corpus-export.ts split. The most notable subtlety is that idempotency across separate invocations relies on a targetKey-only deterministic id combined with INSERT OR IGNORE, so if a target's terminal decision changes between two backfill runs, the second run's INSERT is silently dropped rather than updating the existing row — acceptable for a one-time historical backfill of already-terminal PRs but worth calling out explicitly.

Nits — 5 non-blocking

• scripts/backfill-calibration-corpus-core.ts: the synthesized row id (`backfill:${rule}:${targetKey}:fired/override`) is keyed only on targetKey, not on the decision content, so if review_targets' terminal decision for a given PR changes between two backfill invocations, the second run's INSERT OR IGNORE silently keeps the stale first-run row instead of updating it — worth a one-line comment noting this is safe only because the source data is already terminal and won't change.
• scripts/backfill-calibration-corpus-core.ts:160 and scripts/backfill-calibration-corpus.ts:39/42 — the chunk size (50), maxBuffer (256*1024*1024), and similar constants are unnamed magic numbers; a named constant would make the intent (wrangler --command size limits) clearer.
• scripts/backfill-calibration-corpus.ts:d1Execute — JSON.parse(result.stdout) has no try/catch, so a malformed/truncated wrangler JSON response throws an unhandled SyntaxError instead of the intended fail-loud Error with stderr context.
• The two failing CI checks (`validate`, `validate-tests (1)`) are reported with no detail — I can't verify the cause from what's given here; worth checking the actual failure output before merging.
• Consider naming the magic numbers (50-row chunk size, 256MB buffer) as exported constants for discoverability, per scripts/backfill-calibration-corpus-core.ts:160 and scripts/backfill-calibration-corpus.ts:39.

Why this is blocked

• No linked issue detected: No closing reference or linked issue number was found in the PR metadata/body. — If this PR is intended to solve an issue, link it explicitly in the PR body.
• Maintainer requires a linked issue: This repo's maintainer focus manifest requires every PR to reference a tracked issue. — Link the relevant issue (for example Closes #123) before opening the PR.

📋 Copy for AI agents — paste into your coding agent

Fix the following blocker(s) from this PR review:

1. No linked issue detected: No closing reference or linked issue number was found in the PR metadata/body. — If this PR is intended to solve an issue, link it explicitly in the PR body.

2. Maintainer requires a linked issue: This repo's maintainer focus manifest requires every PR to reference a tracked issue. — Link the relevant issue (for example `Closes #123`) before opening the PR.

CI checks failing

• validate
• validate-tests (1)

Decision drivers

• ❌ Code review — 2 blockers (1 reviewer)
• ❌ Gate result — Blocking (Repo-configured hard blocker found.)

Context & advisory signals — never blocks the verdict

Signal	Result	Evidence
Linked issue	⚠️ Missing	No linked issue or no-issue rationale found.
Related work	✅ No active overlap found	No same-issue or scoped active PR overlap found.
Change scope	✅ 20/20	Low review scope from cached public metadata (no linked issue context).
Validation posture	✅ 25/25	PR body includes validation/test evidence.
Contributor workload	✅ 10/10	Author activity: 14 registered-repo PR(s), 14 merged, 253 issue(s).
Contributor context	✅ Confirmed Gittensor contributor	JSONbored; Gittensor profile; 14 PR(s), 253 issue(s).
Improvement	✅ Minor	risk: clean · value: minor

Review context

• Author: JSONbored
• Role context: owner (maintainer lane)
• Public audience mode: oss maintainer
• Lane context: Repository is configured for direct PR review.
• Public profile languages: Python, TypeScript, Ruby, Go, JavaScript, MDX, Shell, Solidity
• Official Gittensor activity: 14 PR(s), 253 issue(s).
• PR-specific overlap: none found.

Contributor next steps

• Start here: Treat this as maintainer-lane context rather than normal contributor-lane activity.
• Then work through the remaining 2 steps in the Signals table above.

Signal definitions

• Related work = same linked issue, overlapping active PRs, or title/path similarity.
• Change scope = cached public metadata such as size labels, draft state, and review-burden hints.
• Validation posture = whether the PR provides enough public validation/test evidence for maintainer review.
• Contributor workload = public contributor activity and cleanup pressure, not a repo-wide quality failure.
• Contributor context = public GitHub/Gittensor identity context; non-Gittensor status is not a blocker.

🧪 Chat with LoopOver

Ask LoopOver a question about this PR directly in a comment — grounded only in the same cached, public-safe facts shown above, never a new claim.

• @loopover ask <question> answers contribution-quality Q&A with source citations and freshness.
• @loopover chat <question> answers in natural prose from cached decision-pack facts via local inference (maintainer/collaborator; read-only).
• A plain-language @loopover mention with a real question is routed to the closest matching read-only command automatically — no exact syntax required.

Full command reference: https://loopover.ai/docs/loopover-commands

🧪 Experimental — new and may change.

_{🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed}

---

💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →.

Checked by LoopOver, a quiet PR intelligence layer for OSS maintainers.

• Re-run LoopOver review

[JSONbored]

Good catch by the review on the INSERT OR IGNORE idempotency subtlety — addressed with a real fix rather than a comment: buildBackfillInsertStatements now emits ON CONFLICT(id) DO UPDATE on the decision-bearing columns (detail, metadata_json, created_at), so a target whose terminal decision changes between runs gets its pair updated (matching the transform's own latest-terminal-wins dedupe), while identical re-runs remain effective no-ops and the deterministic ids/provenance are unchanged. Test updated to pin the conflict clause. The already-applied production rows are unaffected (a re-run would update them to identical values).