feat(admin-ui): A/B experiments console card + Phase-5 close-out (Step 5.7d) - #147
Merged
officialCodeWork merged 1 commit intoJun 7, 2026
Merged
Conversation
…p 5.7d) Add an "A/B experiments" card to the admin Live Status page surfacing GET /v1/status/experiments — one row per running experiment with the lift (relative, colour-coded), a verdict badge (candidate wins / candidate worse / no change / collecting), control->candidate means, the confidence interval, and per-arm sample counts. Live-wired via useLive (fetchExperiments + seedExperiments) with a seed/demo fallback when no gateway is configured. New ABAnalysis TS type mirrors rag_core.eval.ABAnalysisResult. No backend change — the endpoint shipped in 5.7a. Phase-5 close-out: Step 5.7 is complete, so Phase 5 — Eval & Observability is done (7/7); TRACKER milestone counts updated (64/84), next action points to Phase 6 (6.1 — Logical multi-tenancy). Deferred items remain documented. admin-ui gates green: tsc, eslint, 34 vitest (incl. the new card test), next build; browser-verified both the significant-win and insufficient-data states render. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Step 5.7d — the final slice of Step 5.7 (A/B testing & shadow mode) and the Phase-5 close-out.
apps/admin-ui,/status) surfacingGET /v1/status/experiments— one row per running experiment with:useLive(fetchExperiments+seedExperiments) with a seed/demo fallback when no gateway is configuredABAnalysisTS type mirrorsrag_core.eval.ABAnalysisResult./v1/status/experimentsendpoint shipped in 5.7a.Phase-5 close-out
Step 5.7 is now complete, so Phase 5 — Eval & Observability is done (7 / 7). The TRACKER milestone counts move to 64 / 84, and the next action points to Phase 6 (Step 6.1 — Logical multi-tenancy). Deferred items remain documented (per-tenant drift / per-dimension embedding PSI; feedback/breaker/quota Grafana export + Loki-events dashboard; gRPC proto mirror of
corpus_decision+experiment; sequential / multi-metric experiments).Documentation
Test plan
tsc --noEmitcleaneslintclean (changed files)router_v2, +24.0%, "candidate wins") and the collecting run (reranker_v3, "21 samples")next buildsucceeds (/statusprerenders)relTime()hydration warning on seed timestamps is unrelated)🤖 Generated with Claude Code