Skip to content

feat(admin-ui): A/B experiments console card + Phase-5 close-out (Step 5.7d) - #147

Merged
officialCodeWork merged 1 commit into
mainfrom
build/phase-5/step-5.7d-experiments-console
Jun 7, 2026
Merged

feat(admin-ui): A/B experiments console card + Phase-5 close-out (Step 5.7d)#147
officialCodeWork merged 1 commit into
mainfrom
build/phase-5/step-5.7d-experiments-console

Conversation

@officialCodeWork

Copy link
Copy Markdown
Owner

Summary

Step 5.7d — the final slice of Step 5.7 (A/B testing & shadow mode) and the Phase-5 close-out.

  • A/B experiments card on the admin Live Status page (apps/admin-ui, /status) surfacing GET /v1/status/experiments — one row per running experiment with:
    • the lift (relative change of candidate vs control), colour-coded
    • a verdict badge: candidate wins (significant + positive), candidate worse (significant + negative), no change (analysed, not significant), or collecting (insufficient data)
    • the control→candidate means, the confidence interval on the difference, and per-arm sample counts
    • live-wired via useLive (fetchExperiments + seedExperiments) with a seed/demo fallback when no gateway is configured
  • New ABAnalysis TS type mirrors rag_core.eval.ABAnalysisResult.
  • No backend change — the /v1/status/experiments endpoint shipped in 5.7a.

Phase-5 close-out

Step 5.7 is now complete, so Phase 5 — Eval & Observability is done (7 / 7). The TRACKER milestone counts move to 64 / 84, and the next action points to Phase 6 (Step 6.1 — Logical multi-tenancy). Deferred items remain documented (per-tenant drift / per-dimension embedding PSI; feedback/breaker/quota Grafana export + Loki-events dashboard; gRPC proto mirror of corpus_decision + experiment; sequential / multi-metric experiments).

Documentation

Test plan

  • admin-ui tsc --noEmit clean
  • admin-ui eslint clean (changed files)
  • admin-ui 34 vitest pass — incl. a new spec asserting the card renders the significant win (router_v2, +24.0%, "candidate wins") and the collecting run (reranker_v3, "21 samples")
  • admin-ui next build succeeds (/status prerenders)
  • browser-verified on the dev server: the card renders both states with correct styling/colour; no card-related console errors (the pre-existing relTime() hydration warning on seed timestamps is unrelated)

🤖 Generated with Claude Code

…p 5.7d)

Add an "A/B experiments" card to the admin Live Status page surfacing
GET /v1/status/experiments — one row per running experiment with the lift
(relative, colour-coded), a verdict badge (candidate wins / candidate worse /
no change / collecting), control->candidate means, the confidence interval, and
per-arm sample counts. Live-wired via useLive (fetchExperiments + seedExperiments)
with a seed/demo fallback when no gateway is configured. New ABAnalysis TS type
mirrors rag_core.eval.ABAnalysisResult. No backend change — the endpoint shipped
in 5.7a.

Phase-5 close-out: Step 5.7 is complete, so Phase 5 — Eval & Observability is done
(7/7); TRACKER milestone counts updated (64/84), next action points to Phase 6
(6.1 — Logical multi-tenancy). Deferred items remain documented.

admin-ui gates green: tsc, eslint, 34 vitest (incl. the new card test), next build;
browser-verified both the significant-win and insufficient-data states render.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
@officialCodeWork
officialCodeWork merged commit 4162edb into main Jun 7, 2026
12 of 17 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant