Skip to content

feat(review): surface backtest-cleared loosening proposals in the tuning advisor#8177

Merged
JSONbored merged 1 commit into
mainfrom
feat/loosening-advisor-recs
Jul 23, 2026
Merged

feat(review): surface backtest-cleared loosening proposals in the tuning advisor#8177
JSONbored merged 1 commit into
mainfrom
feat/loosening-advisor-recs

Conversation

@JSONbored

Copy link
Copy Markdown
Owner

Summary

  • Closes calibration: surface backtest-cleared loosening proposals in the tuning advisor with evidence payloads #8160: the tuning advisor finally sees the backtest-gated loosening loop. buildSatisfactionFloorLooseningRecs (pure, 100% branch-covered) turns the loop's state into ranked TuningRecs — a backtest-cleared proposal surfaces as a good-severity rec carrying both split verdicts, sample sizes, and the precision movement inline, with the action line matched to the flag state; a recently applied loosening surfaces as info pointing at the calibration: operator visibility for applied loosenings — live floor + history on the internal calibration surface #8161 status surface.
  • Hard boundary, pinned by test: these recs never carry an overridePayload — that field is the tightening-only auto-apply channel, so the advisor's apply path provably cannot promote a loosening.
  • loadSatisfactionFloorRecState (evaluate-ONLY, fail-safe) feeds it from the same corpus + current floor the applying tick would use; runSelfTune appends the recs exactly once per pass on the first repo's iteration (deployment-global state — pinned by a spy test over a two-repo pass).

Closes #8160

Scope

  • The PR title follows type(scope): short summary Conventional Commit format, for example fix(api): restore profile access checks.
  • This PR is focused and does not mix unrelated backend, UI, MCP, docs, dependency, and deploy changes.
  • This follows CONTRIBUTING.md and does not reintroduce GitHub Pages, VitePress, site/, or CNAME.
  • I linked a currently open issue this PR resolves (e.g. Closes #123) — a linked open issue is required for every contributor PR.

Validation

  • git diff --check
  • npm run actionlint
  • npm run typecheck
  • npm run test:coverage locally; codecov/patch requires ≥99% coverage of the lines AND branches you changed (aim for 100% on your diff so CI variance does not fail near the threshold). Global coverage is a non-blocking trend with a loose 90% backstop, not the gate.
  • npm run test:workers
  • npm run build:mcp
  • npm run test:mcp-pack
  • npm run ui:openapi:check
  • npm run ui:lint
  • npm run ui:typecheck
  • npm run ui:test
  • npm run ui:build
  • npm audit --audit-level=moderate
  • New or changed behavior has unit/integration tests for new branches, fallback paths, and sanitizer boundaries

If any required check was skipped, explain why:

Safety

  • No secrets, wallet details, hotkeys, coldkeys, user PATs, private keys, raw trust scores, private rankings, or private maintainer evidence are exposed.
  • Public GitHub text stays sanitized, low-noise, and does not imply compensation guarantees or optimization tactics.
  • Auth, cookie, CORS, GitHub App, Cloudflare, or session changes include negative-path tests.
  • API/OpenAPI/MCP behavior is updated and tested where needed.
  • UI changes use live API data or real empty/error/loading states, not production mock/demo fallbacks.
  • Visible UI changes include a UI Evidence section below with JPG/JPEG or PNG screenshots arranged as organized, captioned, clickable thumbnails. SVG screenshots are not used as review evidence. Review-only screenshots or recordings are not committed to the repository.
  • Public docs/changelogs are updated where needed; changelogs are only edited for release-prep PRs.

Advisor output is numbers, verdicts, and env-var/endpoint names only. The apply-channel separation (no payload, ever) is the load-bearing safety property and has its own test.

UI Evidence

Not applicable — no UI change.

Notes

@superagent-security

Copy link
Copy Markdown
Contributor

Superagent didn't find any vulnerabilities or security issues in this PR.

@codecov

codecov Bot commented Jul 23, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 89.58%. Comparing base (87e8a89) to head (a50b863).
⚠️ Report is 3 commits behind head on main.
✅ All tests successful. No failed tests found.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #8177      +/-   ##
==========================================
+ Coverage   89.54%   89.58%   +0.03%     
==========================================
  Files          97      100       +3     
  Lines       22622    22712      +90     
  Branches     3866     3889      +23     
==========================================
+ Hits        20257    20346      +89     
  Misses       2187     2187              
- Partials      178      179       +1     
Flag Coverage Δ
shard-1 28.88% <40.00%> (?)
shard-2 91.11% <72.00%> (?)
shard-3 7.77% <4.00%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
src/review/loosening-recs.ts 100.00% <100.00%> (ø)
src/review/selftune-wire.ts 96.77% <100.00%> (ø)
src/services/satisfaction-floor-loosening-run.ts 100.00% <100.00%> (ø)

@JSONbored JSONbored self-assigned this Jul 23, 2026
…ing advisor (#8160)

computeTuningRecommendations still emits loosening advice as prose with no
payload -- and the #8121 loop that can now actually measure a loosening was
invisible to the advisor. buildSatisfactionFloorLooseningRecs (pure) turns the
loop's state into ranked TuningRecs: a backtest-cleared proposal surfaces as a
good-severity rec carrying both split verdicts, sample sizes, and the
precision movement inline, with the action line matched to the flag state; a
recently applied loosening surfaces as info pointing at the #8161 status
surface. HARD BOUNDARY, tested: these recs never carry an overridePayload --
that field is the tightening-only auto-apply channel, so the advisor's apply
path provably cannot promote a loosening.

loadSatisfactionFloorRecState (evaluate-ONLY, fail-safe) feeds it from the
same corpus + current floor the applying tick would use, and runSelfTune
appends the recs exactly once per pass on the first repo's iteration
(deployment-global state, never once per repo -- pinned by a spy test).

100% line+branch coverage on the new pure module; state-read paths (override-
adjusted floor, empty corpus, broken DB, flag-off advice) all pinned.

Closes #8160.
@JSONbored
JSONbored force-pushed the feat/loosening-advisor-recs branch from d957ab8 to a50b863 Compare July 23, 2026 08:48
@JSONbored
JSONbored merged commit a41206a into main Jul 23, 2026
6 checks passed
@JSONbored
JSONbored deleted the feat/loosening-advisor-recs branch July 23, 2026 08:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

calibration: surface backtest-cleared loosening proposals in the tuning advisor with evidence payloads

1 participant