feat(gate): config-driven backtest-regression gate mode, default advisory#8162
Merged
Conversation
…sory (#8105) Resolves #8105's Phase-2 decision as a CONDITIONAL policy: the full gating mechanism ships now as config-as-code, defaulting to the shipped advisory-only behavior, so the deploy is behavior-identical and the future flip to block is a one-line config change made only once #8140's persisted track record supports it -- honoring the issue's own do-not-gate-before-data boundary while completing the epic's mechanical scope end to end. backtestRegressionGateMode (off|advisory|block; DB column + dashboard field + .loopover.yml gate.backtestRegression override) is wired through every per-repo-setting site: types, schema + migration 0174, repositories (defaults/parse/normalize/both write paths), settings-preview, focus-manifest (host + engine manifest schema), openapi schemas + regenerated artifact, routes PATCH schema, gate-check policy passthrough, isConfiguredGateBlocker (host + engine gate-decision twin), and the example yml. Enforcement lives in resolveThresholdBacktestAdvisory: off silences the advisory entirely; block pushes a backtest_regression finding when any comparison is REGRESSED; the finding only exists in block mode, so advisory stays byte-identical. Advances #8105.
Deploying with
|
| Status | Name | Latest Commit | Preview URL | Updated (UTC) |
|---|---|---|---|---|
| ✅ Deployment successful! View logs |
loopover-ui | 4d05fc5 | Commit Preview URL Branch Preview URL |
Jul 23 2026, 07:15 AM |
Contributor
|
Superagent didn't find any vulnerabilities or security issues in this PR. |
Contributor
Logic backtestReplayed 0 historical case(s) for Backtest comparison:
|
Bundle ReportChanges will increase total bundle size by 1.91kB (0.03%) ⬆️. This is within the configured threshold ✅ Detailed changes
Affected Assets, Files, and Routes:view changes for bundle: loopover-uiAssets Changed:
|
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #8162 +/- ##
=======================================
Coverage 92.05% 92.06%
=======================================
Files 766 766
Lines 77761 77772 +11
Branches 23485 23494 +9
=======================================
+ Hits 71586 71597 +11
Misses 5062 5062
Partials 1113 1113
Flags with carried forward coverage won't be shown. Click here to find out more.
|
loopover-orb
Bot
added
the
gittensor:bug
Contributor
|
Tip ✅ LoopOver review result - approve/merge recommendedReview updated: 2026-07-23 07:29:50 UTC
Review summary Nits — 5 non-blocking
Decision drivers
Context & advisory signals — never blocks the verdict
Linked issue satisfactionAddressed Review context
Contributor next steps
Signal definitions
🧪 Chat with LoopOverAsk LoopOver a question about this PR directly in a comment — grounded only in the same cached, public-safe facts shown above, never a new claim.
Full command reference: https://loopover.ai/docs/loopover-commands 🧪 Experimental — new and may change. Visual preview
Click any thumbnail to open the full-size screenshot. Before = production · After = this PR's preview deploy. Scroll preview
A short scroll-through clip (desktop) — click either thumbnail to open the full animation. Evidence for scroll-linked behavior a single screenshot can't show. 🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed 💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →. Checked by LoopOver, a quiet PR intelligence layer for OSS maintainers.
|
![loopover-orb[bot]](https://avatars.githubusercontent.com/in/4139483?s=60&v=4){kind=small}
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
backtestRegressionGateMode: off | advisory | block— defaulting toadvisory, the shipped behavior, so this deploy is behavior-identical and the future flip toblockis a one-line config change made when calibration: compute REGRESSED-verdict track-record statistics from ORB's persisted backtest results #8140's persisted track record supports it. This honors the issue's own do-not-gate-before-data boundary (0 calibration rows exist today) while completing the epic's mechanical scope end to end.RepositorySettings+ Drizzle column + migration0174(default'advisory'), repositories defaults/parse/normalize/both write paths, settings-preview (39-field parity), focus-manifest host apply + engine manifest schema (gate.backtestRegression, all 4 sites), OpenAPI zod schemas ×2 + regenerated artifact (118-field parity), routes PATCH schema, gate-check policy passthrough,isConfiguredGateBlockerin both gate-decision twins (hostadvisory.ts+ enginegate-advisory.ts— parity checks green),.loopover.yml.example+ the byte-matchedconfig/examples/loopover.full.ymltemplate, the docs-driftGATE_MODE_MANIFESTrow, and real docs coverage (tuning.mdxbullet + the backtest-calibration page's callout updated from "advisory-only today" to the configurable truth).resolveThresholdBacktestAdvisory(Scaffold the advisory backtest CI workflow + threshold-only comparison, and persist each run's result #8138's resolver):offsilences the advisory entirely (no D1 read, no persist, no section);blockpushes abacktest_regressionfinding when any comparison is REGRESSED. The finding only exists in block mode, soadvisoryremains byte-identical to today, and the blocker branch is defense-in-depth mirroringcontent_lane_deliverable_missing's split.Advances #8105 — the written decision (conditional: advisory until the track record justifies gating; mechanism ready) will be posted on the issue at merge.
Scope
type(scope): short summaryConventional Commit format, for examplefix(api): restore profile access checks.CONTRIBUTING.mdand does not reintroduce GitHub Pages, VitePress,site/, orCNAME.Closes #123) — a linked open issue is required for every contributor PR.Validation
git diff --checknpm run actionlintnpm run typechecknpm run test:coveragelocally;codecov/patchrequires ≥99% coverage of the lines AND branches you changed (aim for 100% on your diff so CI variance does not fail near the threshold). Global coverage is a non-blocking trend with a loose 90% backstop, not the gate.npm run test:workersnpm run build:mcpnpm run test:mcp-packnpm run ui:openapi:checknpm run ui:lintnpm run ui:typechecknpm run ui:testnpm run ui:buildnpm audit --audit-level=moderateIf any required check was skipped, explain why:
npm run test:ciran green through the full suite (1108 test files) up toui:version-audit, which failed only on npm's second mid-session publish (3.5.0, fixed on main in chore(ui): sync MCP_PACKAGE_KNOWN_LATEST_VERSION with npm dist-tags.latest 3.5.0 #8155); after rebasing onto that fix, the audit and every remaining step (docs/branding/manifest/engine-parity/nvmrc/release-manifest/command-reference+ui:lint/ui:typecheck/ui:test/ui:build) re-ran green individually. New coverage: mode behavior (off/advisory-default/advisory-explicit/block-regressed/block-improved/block-without-sink) in the wiring suite, blocker-policy branches in BOTH twins' suites, storage round-trip + junk-value fallback, yml→effective-settings mapping, docs-drift fixtures.Safety
UI Evidencesection below with JPG/JPEG or PNG screenshots arranged as organized, captioned, clickable thumbnails. SVG screenshots are not used as review evidence. Review-only screenshots or recordings are not committed to the repository.Gate-authority change is opt-in only: an unrecognized stored mode degrades to
advisory(never silently toblock), and thebacktest_regressionfinding's public text carries only rule ids and backtest language — the leak-guard conventions hold.UI Evidence
Not applicable — docs-content-only MDX edits (two pages), no layout/visual change.
Notes
gate.backtestRegression: blockin.loopover.yml(or the dashboard field) — no deploy needed.