diff --git a/CHALLENGE.md b/CHALLENGE.md new file mode 100644 index 0000000..62a4e5b --- /dev/null +++ b/CHALLENGE.md @@ -0,0 +1,174 @@ +# GPUSTACK Causal Mission Control + +**WebMCP Challenge 2026 entry** + +GPUSTACK is a virtual AI datacenter whose full Python model connects 1,517 variables across training, kernels, memory, interconnects, thermals, power delivery, economics, semiconductor devices, lithography, materials, and physical constants. The browser publishes immutable experiment artifacts and a 700-node dependency cone rather than running that model. Causal Mission Control turns the existing observatory into a shared evidence-audit surface: an agent handles the breadth of the published evidence, while the human owns the scientific conclusion. + +- **Live application:** +- **Public source:** +- **Pre-WebMCP baseline:** [`3d7339a87e13c4f809ed223c2aa299fb3f631799`](https://github.com/Cuuper22/gpu_stack-/commit/3d7339a87e13c4f809ed223c2aa299fb3f631799) +- **Baseline ref:** [`pre-webmcp-baseline`](https://github.com/Cuuper22/gpu_stack-/tree/pre-webmcp-baseline) (public branch; also an annotated tag in the development checkout) + +## Challenge-window disclosure + +GPUSTACK is an existing project. The baseline commit above, merged August 13, 2026, is the exact state immediately before the WebMCP Challenge extension. It is preserved as a public immutable comparison ref so reviewers can separate the new work from the underlying project. + +### Existing before the challenge extension + +- The Python registry-backed equation graph and resolver +- The virtual datacenter, scenario runners, and preregistered research program +- Existing experiment results and evidence JSON +- The Causal Observatory, its visual design, charts, data loaders, and normal human controls +- The public GitHub Pages deployment + +### Added for the WebMCP Challenge after August 25, 2026 + +- Top-level imperative WebMCP registration for eight domain-specific tools +- A bounded adapter between WebMCP tool calls and the observatory's published, immutable artifacts +- A generated scalar-only projection of the existing E001-SC1 raw trace, so an agent can inspect bounded epoch slices without loading the roughly 72 MB authoritative artifact +- Visible causal spotlight, agent activity, pending-review, receipt, and undo surfaces +- A local review workflow with a distinct human approval boundary +- Structured results carrying affected model IDs and current state version +- Contract tests for registration, schemas, state effects, and failure behavior +- This challenge disclosure, a root open-source license, and the WebMCP eval suite + +The challenge contribution can be inspected directly with: + +```bash +git diff 3d7339a87e13c4f809ed223c2aa299fb3f631799..HEAD +git log --oneline 3d7339a87e13c4f809ed223c2aa299fb3f631799..HEAD +``` + +Only that post-baseline work is presented as the WebMCP Challenge submission. + +## Why WebMCP is load-bearing + +A screenshot-based agent sees windows, labels, and charts. It does not naturally know that `training.tokens_per_sec` is a typed model variable, which published dependencies lead to it, which quantities are measured or modeled, which experiment artifact supports a claim, or whether an operation merely inspects evidence or records a review decision. + +WebMCP exposes those domain objects and authority boundaries directly. The tools read the same immutable artifacts shown in the page, and each meaningful call leaves a visible selection, highlight, comparison, or review receipt. The result is not a chat layer over GPUSTACK: it is a semantic audit plane for evidence that would otherwise require laborious manual traversal. + +The extension deliberately does **not** pretend the static browser is a simulator. It cannot recompute the Python model, invent an intervention, or turn one small-model experiment into a frontier-scale claim. Its job is narrower and more defensible: find the relevant published evidence, preserve its boundary, and help a person judge what conclusion it supports. + +## Architecture + +| Layer | Path | Responsibility | +|---|---|---| +| WebMCP adapter | `docs/webmcp-tools.js` | Feature detection, JSON Schemas, imperative `document.modelContext.registerTool(...)` calls, bounded results, and abort handling | +| Mission handlers | `docs/webmcp-mission.js` via `window.GPUStackMission.invoke(name, args, { signal })` | Indexed, bounded reads over the shipped artifacts plus local review state and receipts | +| View bridge | `docs/observatory.js` via `window.GPUStackObservatory` | Applies tool-driven selections and causal highlights to the same observatory the human sees | +| Published data | `docs/data/*.json` | Immutable registry-cone and experiment evidence loaded by the observatory | +| Run projection | `scripts/generate_webmcp_projection.py` and `docs/data/webmcp-run-projection-v1.json.gz` | Reproducible, scalar-only view of all 56 E001-SC1 runs and 12,981 epochs; source hashes and omissions are explicit | +| Shared interface | `docs/observatory.html` and `docs/styles/99-webmcp-mission.css` | Human-visible selections, paths, pending reviews, receipts, and approval controls | +| Contract tests | `tests/test_webmcp_contract.py` | Registration, schema, tool inventory, safety boundary, and integration checks | +| Agent evals | `evals/webmcp-evals.json` | Official experimental WebMCP call-selection and trajectory cases | + +Registration occurs in the top-level document. If `document.modelContext` is unavailable, the observatory keeps working normally; the application does not install a fake compatibility object. Tool `execute` callbacks validate their inputs again in code and return compact, JSON-serializable objects. + +## Tools and authority + +| Tool | Mode | What it contributes to the shared page | +|---|---|---| +| `get_observatory_state` | Read-only | Returns the active immutable artifact, registered IDs, evidence boundary, visible selection, and current review state so the agent does not guess identifiers | +| `compare_stress_families` | Read-only | Compares all six held-out E001-SC1 families, or a named subset, across the registered learning, infrastructure, work, and abstention fields | +| `inspect_stress_family` | Read-only | Selects and explains one held-out family, its adaptive-versus-comparator deltas, uncertainty regions, abstention reason, and linked run IDs | +| `inspect_run` | Read-only | Opens an exact run, accepts a request of up to 20 epochs, and returns at most 6 scalar-projection rows per compact result while preserving the authoritative-trace binding | +| `trace_causal_path` | Read-only | Finds and highlights a bounded path through the seven-node conceptual evidence graph without collapsing its branches or evidence classes | +| `open_evidence` | Read-only | Opens a registered artifact, source result, assumption, uncertainty item, or missing-evidence boundary at a chosen semantic depth | +| `compare_policies` | Read-only | Compares up to three registered policies; by default it uses `observable_adaptive` and the calibration-frozen `periodic_local` comparator | +| `stage_conclusion` | Staging write | Places a supported, qualified, or abstain claim plus one to eight evidence IDs in the pending tray; it cannot approve or commit the claim | + +The first seven tools are annotated read-only. `stage_conclusion` affects only local pending-review state, never the source experiment JSON. No WebMCP tool can approve, reject, or undo a conclusion: those actions are page-only human controls. The agent can explore broadly and prepare a coherent evidence bundle, but it must stop at the judgment boundary. + +## Testing + +Run the WebMCP contract tests: + +```bash +python -m pytest tests/test_webmcp_contract.py -q +``` + +Run the complete Python suite: + +```bash +python -m pytest -q +``` + +Serve the static site locally: + +```bash +python -m http.server 8000 --directory docs +``` + +Then open in a WebMCP-enabled browser. The application must remain fully usable in an ordinary browser where WebMCP is absent. + +The eval file follows the official experimental [`webmcp-evals`](https://github.com/GoogleChromeLabs/webmcp-tools/tree/main/webmcp-evals) format. With the page already served, deterministic calls can be checked without an API key: + +```bash +npx webmcp-evals smoke \ + -u http://localhost:8000/observatory.html \ + -e evals/webmcp-evals.json \ + -v +``` + +For probabilistic tool selection and multi-step journeys: + +```bash +npx webmcp-evals browser \ + -u https://cuuper22.github.io/gpu_stack-/observatory.html \ + -e evals/webmcp-evals.json \ + --open +``` + +Final acceptance is performed against the deployed top-level page in ChatGPT desktop and a WebMCP-enabled Chrome build. Each demonstrated call must update the visible shared state before returning. + +## Judging-criteria map + +| Criterion | What to inspect | +|---|---| +| WebMCP leverage | Eight typed domain operations replace brittle UI-coordinate automation; calls operate on the published evidence graph and produce visible state transitions. The human/agent authority boundary depends on staged semantic actions. | +| Execution | The shipped surface contains a 700-node dependency cone and compact E001-SC1 summaries for 56 runs backed by 12,981 raw epochs. Inputs are validated, results are bounded, artifacts stay immutable, and local decisions leave receipts with undo. | +| Potential impact | Researchers can audit a result across stress families and evidence classes faster without giving an agent authority to upgrade an experiment into a stronger claim. The pattern applies to other evidence-heavy technical reviews. | +| Creativity and ambition | The browser becomes an evidence court: the agent traverses a dense causal record, but the human decides whether the claim survives its falsifier. | + +## Demo prompt + +> Audit whether E001-SC1's observable adaptive controller deserves a transferable win claim over `periodic_local`. Compare all six held-out stress families, inspect the failure, trace the evidence boundary, and stage the scientifically honest conclusion. Do not approve it for me. + +The demo is anchored to values already serialized in `docs/data/e001-semantic-consistency-v1.json`: + +| Audit fact | Published value | +|---|---| +| Experiment | `E001-SC1` | +| Candidate | `observable_adaptive` | +| Frozen comparator | `periodic_local`; selected on the calibration split before evaluation | +| Untouched evaluation families | `E1` through `E6` | +| Out-of-support abstentions | 104 total: 32 in `E2`, 48 in `E4`, and 24 in `E6` | +| Decisive E6 held-out NLL | adaptive `1.063824194483459`; comparator `0.9984574504196644` | +| Serialized conclusion | `abstain_without_policy_claim` | +| Evidence boundary | measured small-model learning, exact accounting, modeled infrastructure; frontier-scale transfer remains unresolved | + +A complete demo should make one coherent loop visible: + +1. Open E001-SC1 and compare the controller with `periodic_local` across its six untouched evaluation families. +2. Inspect the compact run ledger, then drill into a bounded slice of the 12,981-epoch scalar projection instead of dumping the roughly 72 MB authoritative raw artifact into context. +3. Surface the decisive failure: on `E6-repeated-membership-loss`, the adaptive policy reaches 1.0638 held-out NLL versus 0.9985 for `periodic_local`. +4. Trace one directed route from site availability to time to target through the published seven-node, eight-edge causal DAG, then open the measured/modeled/unresolved boundary. Present the returned route as one path through a branching graph, not as the entire DAG. +5. Stage `abstain_without_policy_claim`, linked to the evidence, in the pending-review surface. +6. Have the human alone approve or reject it, then show the decision receipt and undo. + +The strongest moment is not a list of tool names. It is the point where the agent finds the tempting win, finds the falsifying family, and voluntarily stages the narrower conclusion already warranted by the record—then stops for human judgment. + +## Known limits + +- WebMCP is experimental and availability depends on the host browser or agent. +- The challenge extension reads the shipped registry-cone export and experiment artifacts; it does not execute the Python simulator, fabricate measurements, or claim live access to a physical datacenter. +- Scenario comparisons are only as strong as their declared assumptions and evidence. The interface keeps those boundaries visible rather than hiding them behind a single recommendation. +- The static public deployment has no account system or server-side persistence. Review state is local to the current browser session, while source evidence remains immutable. + +## References + +- [OpenAI WebMCP Challenge](https://openai.com/webmcp-challenge/) +- [Official challenge rules and judging criteria](https://webmcp.devpost.com/rules) +- [WebMCP Community Group draft](https://webmachinelearning.github.io/webmcp/) +- [Chrome imperative WebMCP API](https://developer.chrome.com/docs/ai/webmcp/imperative-api) +- [Chrome WebMCP eval guidance](https://developer.chrome.com/docs/ai/webmcp/evals) diff --git a/LICENSE b/LICENSE new file mode 100644 index 0000000..48ff7ea --- /dev/null +++ b/LICENSE @@ -0,0 +1,21 @@ +MIT License + +Copyright (c) 2026 gpu_stack contributors / Cuuper22 + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. diff --git a/README.md b/README.md index 5b18f7c..c906c34 100644 --- a/README.md +++ b/README.md @@ -7,6 +7,12 @@ **Repository:**
**Research program:** [RESEARCH.md](RESEARCH.md) +## WebMCP Challenge 2026 + +GPUSTACK's [Causal Observatory](https://cuuper22.github.io/gpu_stack-/observatory.html) is now a shared evidence-audit surface for people and browser agents. Eight WebMCP tools let an agent compare the immutable E001-SC1 results, inspect bounded run traces, follow claims through the causal evidence graph, and stage a review conclusion while the human keeps control of approval and undo. + +See [CHALLENGE.md](CHALLENGE.md) for the exact pre-challenge baseline, challenge-window work, tool contracts, architecture, tests, and demo path. + `gpu_stack` started as a curiosity project in the overlap between my AI work and my physics brain. The question was simple enough to be annoying: if frontier training is supposedly "more GPUs, more data, more money," where does that sentence actually bottom out? diff --git a/docs/data/webmcp-run-projection-v1.json.gz b/docs/data/webmcp-run-projection-v1.json.gz new file mode 100644 index 0000000..3857a93 Binary files /dev/null and b/docs/data/webmcp-run-projection-v1.json.gz differ diff --git a/docs/observatory.html b/docs/observatory.html index 46c9860..b075565 100644 --- a/docs/observatory.html +++ b/docs/observatory.html @@ -15,9 +15,12 @@ + + + - +
@@ -67,6 +70,85 @@ +
+ + + + +
+ Objective: + Audit whether adaptive training deserves a transferable win claim +
+ +
+ Policy: + Human approval required +
+ +
+ Live status: + Connecting to the observatory… + +
+ +
+ + +
+
+ + +
diff --git a/docs/observatory.js b/docs/observatory.js index b00b6f6..287a4c0 100644 --- a/docs/observatory.js +++ b/docs/observatory.js @@ -212,6 +212,8 @@ let inspectorHidden = false; let siteRailInitialized = false; let resizeFrame = 0; + let resolveObservatoryReady; + const observatoryReady = new Promise((resolve) => { resolveObservatoryReady = resolve; }); class ArtifactContractError extends Error {} @@ -1060,6 +1062,8 @@ loadCheckpointEnergyArtifact(); loadRackDephasingArtifact(); loadSemanticConsistencyArtifact(); + resolveObservatoryReady(); + document.dispatchEvent(new CustomEvent("gpustack:observatory-ready")); } document.addEventListener("DOMContentLoaded", init, { once: true }); @@ -1088,7 +1092,7 @@ renderRackDephasingV3(); renderSemanticConsistencyV1(); renderExperimentView(); - if (state.experiment === "E001-SC1" && state.depth === "full_trace" && semanticConsistencyArtifact && !semanticConsistencyRawArtifact && !semanticConsistencyRawLoad && !semanticConsistencyRawArtifactError) { + if (state.experiment === "E001-SC1" && state.depth === "full_trace" && dom.semanticconsistencyrawdetails?.open && semanticConsistencyArtifact && !semanticConsistencyRawArtifact && !semanticConsistencyRawLoad && !semanticConsistencyRawArtifactError) { loadSemanticConsistencyRawArtifact().catch(() => {}); } } @@ -2713,6 +2717,7 @@ const stateLabel = passed === true ? "PASS" : passed === false ? "FAIL" : "UNRESOLVED"; const card = element("article", "equal-work-effect-card"); card.dataset.passed = passed === null || passed === undefined ? "unresolved" : String(passed); + card.dataset.effectId = effect.effect_id || ""; const header = element("header"); header.append(element("h4", "", effect.label), element("span", "equal-work-effect-status", stateLabel)); card.append( @@ -2944,6 +2949,7 @@ dom.semanticconsistencyfamilybody.replaceChildren(); semanticFamilyResults().forEach((family) => { const row = element("tr"); + row.dataset.familyId = family.family_id || ""; const stateCell = element("td", "", `${semanticRankingStateLabel(family.ranking_state)}${family.abstention_reason ? ` · ${family.abstention_reason}` : ""}`); stateCell.dataset.state = family.ranking_state || "unmeasured"; row.append( @@ -4028,6 +4034,8 @@ class: `causal-edge${selectedId && (edge.source === selectedId || edge.target === selectedId) ? " is-selected" : ""}`, d: causalEdgePath(source, target, mobile), "marker-end": "url(#causal-arrow)", + "data-source": edge.source, + "data-target": edge.target, }); svg.append(path); if (!mobile) { @@ -4052,6 +4060,7 @@ const kind = normalizedEvidence(node.evidence_class); const group = svgElement("g", { class: `causal-node node--${kind}${state.node === node.node_id ? " is-selected" : ""}`, + "data-node-id": node.node_id, transform: `translate(${box.x} ${box.y})`, role: "button", tabindex: 0, @@ -4201,6 +4210,7 @@ const orderedRuns = [...artifactRuns()].sort((a, b) => POLICY_ORDER.indexOf(a.policy) - POLICY_ORDER.indexOf(b.policy)); orderedRuns.forEach((run) => { const row = element("tr"); + row.dataset.policyId = run.policy || ""; row.classList.toggle("is-selected", run.policy === state.policy); const policyCell = element("td"); const policyButton = element("button", "policy-select", policyLabel(run.policy)); @@ -5005,4 +5015,52 @@ dom.rawtracejson.textContent = JSON.stringify(artifact, null, 2); } + const observatoryBridge = Object.freeze({ + version: "1.0.0", + whenReady() { + return observatoryReady; + }, + getState() { + return { ...state }; + }, + getArtifactStatus() { + return { + screening: artifact ? "ready" : artifactError ? "error" : "loading", + semanticConsistency: semanticConsistencyArtifact ? "ready" : semanticConsistencyArtifactError ? "error" : "loading", + semanticRaw: semanticConsistencyRawArtifact ? "ready" : semanticConsistencyRawArtifactError ? "error" : "not_loaded", + }; + }, + async selectView(patch, options = {}) { + await observatoryReady; + commitState(patch, { replace: Boolean(options.replace) }); + return { ...state }; + }, + async focusCausalPath(nodeIds, edges = []) { + await observatoryReady; + const pathNodes = Array.isArray(nodeIds) ? nodeIds.filter((value) => typeof value === "string") : []; + const terminalNode = pathNodes[pathNodes.length - 1] || "time_to_target"; + commitState({ experiment: "E001", depth: "researcher", node: terminalNode }); + const selectedNodes = new Set(pathNodes); + const selectedEdges = new Set(edges.map((edge) => `${edge.source}>${edge.target}`)); + document.querySelectorAll("#causal-svg [data-node-id]").forEach((node) => { + node.classList.toggle("is-mission-path", selectedNodes.has(node.dataset.nodeId)); + }); + document.querySelectorAll("#causal-svg [data-source][data-target]").forEach((edge) => { + edge.classList.toggle("is-mission-path", selectedEdges.has(`${edge.dataset.source}>${edge.dataset.target}`)); + }); + byId("causal-field")?.scrollIntoView({ behavior: "smooth", block: "center" }); + return { ...state }; + }, + announce(message) { + announce(String(message || "")); + }, + }); + + Object.defineProperty(window, "GPUStackObservatory", { + configurable: false, + enumerable: true, + writable: false, + value: observatoryBridge, + }); + })(); diff --git a/docs/styles/99-webmcp-mission.css b/docs/styles/99-webmcp-mission.css new file mode 100644 index 0000000..c58b0c4 --- /dev/null +++ b/docs/styles/99-webmcp-mission.css @@ -0,0 +1,769 @@ +/* GPUSTACK WebMCP mission layer. + This file intentionally loads last: it extends the existing CuperOS + observatory without changing the evidence grammar underneath it. */ + +:root { + --mission-rail: clamp(330px, 24vw, 390px); + --mission-bar-height: 50px; + --mission-ready: oklch(0.55 0.17 142); + --mission-wait: oklch(0.55 0.14 88); + --mission-cyan-wash: oklch(0.94 0.045 195); + --mission-gold-wash: oklch(0.95 0.075 88); + --mission-red-wash: oklch(0.94 0.04 28); +} + +body[data-mission-mode="active"] .obs-desktop { + width: min(100% - 20px, 1660px); +} + +/* Reserve a real column for the rail. It never floats over evidence. */ +.obs-window-content { + display: grid; + grid-template-columns: minmax(0, 1fr) var(--mission-rail); + align-items: start; +} + +.obs-header, +.mission-control, +.obs-footer { + grid-column: 1 / -1; +} + +.obs-header { + grid-row: 1; +} + +.mission-control { + grid-row: 2; +} + +#observatory-main { + grid-row: 3; + grid-column: 1; + width: 100%; + min-width: 0; + border-right: 2px solid var(--inset-dark); +} + +.mission-sidecar { + grid-row: 3; + grid-column: 2; +} + +.obs-footer { + grid-row: 4; +} + +/* ------------------------------------------------------------------ */ +/* Mission bar */ +/* ------------------------------------------------------------------ */ + +.mission-control { + position: sticky; + z-index: 38; + top: var(--obs-header-height); + display: grid; + grid-template-columns: auto minmax(220px, 1fr) auto minmax(300px, 1.25fr) auto; + min-height: var(--mission-bar-height); + margin: 0; + border-top: 2px solid var(--inset-light); + border-bottom: 2px solid var(--inset-dark); + background: var(--pane-bright); + box-shadow: 0 2px 0 oklch(0.18 0.006 250 / 0.14); + color: var(--text-dark); + font: 600 13px/1.2 var(--font-display); +} + +.mission-control > div { + min-width: 0; + min-height: calc(var(--mission-bar-height) - 4px); + padding: 8px 13px; + display: flex; + align-items: center; + gap: 8px; +} + +.mission-control > div + div { + border-left: 2px solid var(--inset-dark); + box-shadow: inset 2px 0 0 var(--inset-light); +} + +.mission-identity { + text-transform: uppercase; + letter-spacing: 0.035em; + white-space: nowrap; +} + +.mission-signal { + position: relative; + width: 13px; + height: 13px; + flex: 0 0 auto; + border: 2px solid var(--accent-cyan-deep); + transform: rotate(45deg); + background: var(--pane-bright); +} + +.mission-signal::after { + position: absolute; + inset: 3px; + content: ""; + background: var(--accent-cyan-deep); +} + +.mission-bar-label { + flex: 0 0 auto; + color: var(--text-soft); + font-family: var(--font-mono); + text-transform: uppercase; + letter-spacing: 0.035em; +} + +.mission-objective-block strong, +.mission-live-copy { + min-width: 0; + overflow: hidden; + text-overflow: ellipsis; + white-space: nowrap; +} + +.mission-objective-block strong { + font-weight: 700; +} + +.mission-policy strong { + color: var(--danger-red); + font-weight: 700; + text-transform: uppercase; + white-space: nowrap; +} + +.mission-live-block { + overflow: hidden; +} + +.mission-live-copy { + flex: 1 1 auto; +} + +body[data-mission-status="working"] .mission-live-copy { + color: var(--title-bar); +} + +body[data-mission-status="waiting"] .mission-live-copy, +body[data-mission-status="waiting"] .mission-signal { + color: var(--mission-wait); + border-color: var(--mission-wait); +} + +body[data-mission-status="waiting"] .mission-signal::after { + background: var(--mission-wait); +} + +body[data-mission-status="error"] .mission-live-copy, +body[data-mission-status="error"] .mission-signal { + color: var(--danger-red); + border-color: var(--danger-red); +} + +body[data-mission-status="error"] .mission-signal::after { + background: var(--danger-red); +} + +body[data-mission-status="fallback"] .mission-live-copy { + color: var(--text-soft); +} + +.mission-meter { + display: grid; + grid-template-columns: repeat(6, 6px); + gap: 3px; + flex: 0 0 auto; + margin-left: auto; +} + +.mission-meter i { + width: 6px; + height: 10px; + border: 1px solid oklch(0.31 0.01 250); + background: var(--mission-ready); + box-shadow: inset 1px 1px 0 oklch(0.86 0.12 142); +} + +.mission-meter i:nth-child(5), +.mission-meter i:nth-child(6) { + animation: mission-meter-pulse 1400ms steps(2, end) infinite; +} + +body[data-mission-status="waiting"] .mission-meter i { + background: var(--mission-wait); +} + +body[data-mission-status="error"] .mission-meter i { + background: var(--danger-red); +} + +body[data-mission-status="fallback"] .mission-meter i:nth-child(n + 3) { + background: var(--chrome-well); + box-shadow: none; +} + +.mission-utility-actions { + gap: 5px !important; + padding-right: 8px !important; + padding-left: 8px !important; +} + +@keyframes mission-meter-pulse { + 50% { + background: var(--chrome-well); + box-shadow: none; + } +} + +/* ------------------------------------------------------------------ */ +/* Human decision rail */ +/* ------------------------------------------------------------------ */ + +.mission-sidecar { + position: sticky; + z-index: 18; + top: calc(var(--obs-header-height) + var(--mission-bar-height)); + align-self: start; + display: grid; + gap: 8px; + width: 100%; + min-width: 0; + max-height: calc(100vh - var(--obs-header-height) - var(--mission-bar-height) - 68px); + padding: 8px; + overflow: auto; + overscroll-behavior: contain; + background: var(--window-chrome); +} + +.mission-panel { + min-width: 0; + border: 2px outset var(--button-face); + background: var(--pane-bright); + box-shadow: 1px 1px 0 oklch(0.16 0.008 250 / 0.28); +} + +.mission-panel-header { + min-height: 34px; + padding: 6px 9px; + display: flex; + align-items: center; + justify-content: space-between; + gap: 10px; + border-bottom: 2px solid var(--inset-dark); + background: linear-gradient(180deg, var(--pane-bright), var(--chrome-well)); +} + +.mission-panel-header h2 { + margin: 0; + color: var(--text-dark); + font: 700 14px/1.1 var(--font-display); + letter-spacing: 0.035em; + text-transform: uppercase; +} + +.mission-count { + flex: 0 0 auto; + color: var(--text-soft); + font: 500 12px/1 var(--font-mono); +} + +.mission-list, +.receipt-list { + min-width: 0; + margin: 0; + padding: 7px; + display: grid; + gap: 7px; + list-style: none; + background: var(--chrome-well-bright); +} + +.mission-list { + max-height: min(34vh, 330px); + overflow: auto; +} + +.receipt-list { + max-height: min(32vh, 300px); + overflow: auto; +} + +.mission-empty { + min-height: 76px; + padding: 12px; + display: grid; + grid-template-columns: 16px minmax(0, 1fr); + align-items: start; + gap: 10px; + border: 1px dashed var(--chrome-border); + background: var(--pane-bright); + color: var(--text-soft); +} + +.mission-empty--receipt { + min-height: 70px; +} + +.mission-empty-mark { + width: 11px; + height: 11px; + margin-top: 3px; + border: 2px solid var(--chrome-border); + transform: rotate(45deg); +} + +.mission-empty strong { + display: block; + margin-bottom: 3px; + color: var(--text-dark); + font: 700 12px/1.25 var(--font-display); + text-transform: uppercase; +} + +.mission-empty p { + margin: 0; + font: 500 12px/1.35 var(--font-display); +} + +/* The bridge can render either class names or semantic data attributes. + Both receive the same compact, evidence-first card treatment. */ +.mission-change, +.pending-change, +#pending-changes > [data-change-id] { + padding: 10px 11px; + border: 1px solid var(--accent-cyan-deep); + background: var(--mission-cyan-wash); + box-shadow: inset 0 0 0 1px oklch(0.92 0.035 195); + color: var(--text-dark); +} + +.mission-change[data-kind="assumption"], +.pending-change[data-kind="assumption"], +#pending-changes > [data-change-kind="assumption"], +.mission-change[data-kind="conclusion"], +.pending-change[data-kind="conclusion"], +#pending-changes > [data-change-kind="conclusion"] { + border-color: var(--gold-line); + background: var(--mission-gold-wash); + box-shadow: inset 0 0 0 1px oklch(0.94 0.065 88); +} + +.mission-change[aria-selected="true"], +.pending-change[aria-selected="true"], +#pending-changes > [data-selected="true"] { + outline: 2px solid var(--title-bar); + outline-offset: -3px; +} + +.mission-change h3, +.pending-change h3, +#pending-changes > [data-change-id] h3 { + margin: 0 0 8px; + font: 700 13px/1.25 var(--font-display); +} + +.mission-change p, +.pending-change p, +#pending-changes > [data-change-id] p { + margin: 4px 0; + font: 500 12px/1.35 var(--font-display); + overflow-wrap: anywhere; +} + +.mission-change code, +.pending-change code, +#pending-changes > [data-change-id] code { + font: 500 14px/1 var(--font-mono); +} + +.mission-claim-editor { + display: block; + width: 100%; + min-height: 118px; + max-height: 240px; + margin: 6px 0 8px; + padding: 8px 9px; + resize: vertical; + border: 2px inset var(--window-chrome); + border-radius: 0; + background: var(--pane-bright); + color: var(--text-dark); + font: 500 13px/1.45 var(--font-display); +} + +.mission-claim-editor:focus-visible { + outline: 2px solid var(--accent-gold); + outline-offset: 2px; +} + +.mission-decision-actions, +.mission-receipt-actions { + padding: 7px; + display: grid; + gap: 7px; + border-top: 2px solid var(--inset-light); + background: var(--window-chrome); +} + +.mission-decision-actions { + grid-template-columns: repeat(3, minmax(0, 1fr)); +} + +.mission-receipt-actions { + grid-template-columns: minmax(0, 1fr); +} + +.mission-button { + min-width: 0; + min-height: 32px; + padding: 5px 8px; + border: 2px outset var(--button-face); + border-radius: 0; + background: var(--button-face); + color: var(--text-dark); + font: 700 12px/1 var(--font-display); + letter-spacing: 0.025em; + text-transform: uppercase; +} + +.mission-button:hover:not(:disabled) { + background: var(--button-hover); + color: var(--title-bar); +} + +.mission-button:active:not(:disabled) { + border-style: inset; +} + +.mission-button:disabled { + cursor: not-allowed; + color: oklch(0.50 0.004 250); + text-shadow: 1px 1px 0 var(--inset-light); + opacity: 0.72; +} + +.mission-button--compact { + min-height: 28px; + padding: 4px 7px; + font-size: 11px; +} + +.mission-button--reject:not(:disabled) { + color: var(--danger-red); +} + +.mission-button--approve:not(:disabled) { + color: oklch(0.42 0.16 142); +} + +.mission-human-boundary { + margin: 0; + padding: 6px 9px; + border-top: 2px solid var(--inset-dark); + background: var(--chrome-well); + color: var(--text-dark); + font: 600 11px/1.3 var(--font-display); + text-align: center; +} + +/* ------------------------------------------------------------------ */ +/* Receipts */ +/* ------------------------------------------------------------------ */ + +.receipt-list > li:not(.mission-empty), +.webmcp-receipt, +.receipt-item { + position: relative; + min-width: 0; + padding: 9px 10px 9px 27px; + border: 1px solid var(--chrome-border); + background: var(--pane-bright); + color: var(--text-dark); + font: 500 12px/1.35 var(--font-display); +} + +.receipt-list > li:not(.mission-empty)::before, +.webmcp-receipt::before, +.receipt-item::before { + position: absolute; + top: 12px; + left: 10px; + width: 9px; + height: 9px; + content: ""; + border: 2px solid var(--mission-ready); + transform: rotate(45deg); +} + +.receipt-list > li[data-status="pending"] { + border-color: var(--gold-line); + background: var(--mission-gold-wash); +} + +.receipt-list > li[data-status="pending"]::before { + border-color: var(--mission-wait); +} + +.receipt-list > li[data-status="rejected"], +.receipt-list > li[data-status="failed"] { + border-color: var(--danger-red); + background: var(--mission-red-wash); +} + +.receipt-list > li[data-status="rejected"]::before, +.receipt-list > li[data-status="failed"]::before { + border-color: var(--danger-red); +} + +.receipt-list strong { + font-weight: 700; +} + +.receipt-list code, +.receipt-list time, +.receipt-id, +.receipt-delta { + font-family: var(--font-mono); +} + +.receipt-list p { + margin: 3px 0 0; + overflow-wrap: anywhere; +} + +.receipt-evidence { + color: var(--text-soft); + font: 500 13px/1.25 var(--font-mono); +} + +/* Agent-read paths are a temporary cyan spotlight, never a new evidence + class. The node's original evidence label and glyph remain untouched. */ +#causal-svg .causal-node.is-mission-path .node-frame { + fill: var(--mission-cyan-wash); + stroke: var(--accent-cyan-deep); + stroke-width: 3; + filter: drop-shadow(2px 2px 0 oklch(0.28 0.06 195 / 0.24)); +} + +#causal-svg .causal-edge.is-mission-path { + stroke: var(--accent-cyan-deep); + stroke-width: 3; + stroke-dasharray: 8 4; + animation: mission-path-flow 900ms linear infinite; +} + +@keyframes mission-path-flow { + to { + stroke-dashoffset: -12; + } +} + +/* Family rows and effect cards opened by an agent keep their original + PASS/FAIL and evidence state; this gold focus band only shows location. */ +.is-mission-evidence:not(tr) { + position: relative; + z-index: 1; + outline: 3px solid var(--accent-gold); + outline-offset: 3px; + scroll-margin-top: calc(var(--obs-header-height) + var(--mission-bar-height) + 18px); + animation: mission-evidence-arrive 700ms steps(2, end) 2; +} + +tr.is-mission-evidence > th, +tr.is-mission-evidence > td { + background: var(--mission-gold-wash); + box-shadow: inset 0 3px 0 var(--accent-gold), inset 0 -3px 0 var(--accent-gold); +} + +tr.is-mission-evidence > :first-child { + box-shadow: inset 3px 0 0 var(--accent-gold), inset 0 3px 0 var(--accent-gold), inset 0 -3px 0 var(--accent-gold); +} + +tr.is-mission-evidence > :last-child { + box-shadow: inset -3px 0 0 var(--accent-gold), inset 0 3px 0 var(--accent-gold), inset 0 -3px 0 var(--accent-gold); +} + +@keyframes mission-evidence-arrive { + 50% { + outline-color: var(--title-bar); + } +} + +/* ------------------------------------------------------------------ */ +/* Responsive */ +/* ------------------------------------------------------------------ */ + +@media (max-width: 1380px) { + .mission-control { + grid-template-columns: auto minmax(200px, 1fr) auto minmax(250px, 1fr) auto; + } + + .mission-control > div { + padding-right: 9px; + padding-left: 9px; + } + + .mission-meter { + display: none; + } +} + +/* Below a wide desktop, an inline rail gives the dense observatory its full + width. This deliberately switches before any research grid can collide. */ +@media (max-width: 1280px) { + .obs-window-content { + grid-template-columns: minmax(0, 1fr); + } + + .obs-header, + .mission-control, + .mission-sidecar, + #observatory-main, + .obs-footer { + grid-column: 1; + } + + .obs-header { + grid-row: 1; + } + + .mission-control { + grid-row: 2; + } + + .mission-sidecar { + position: static; + grid-row: 3; + display: grid; + grid-template-columns: repeat(2, minmax(0, 1fr)); + max-height: none; + padding: 8px; + overflow: visible; + border-bottom: 2px solid var(--inset-dark); + } + + #observatory-main { + grid-row: 4; + border-right: 0; + } + + .obs-footer { + grid-row: 5; + } + + .mission-list, + .receipt-list { + max-height: 250px; + } +} + +@media (max-width: 900px) { + .mission-control { + position: static; + grid-template-columns: minmax(0, 1fr) auto; + min-height: 0; + } + + .mission-control > div { + min-height: 42px; + } + + .mission-identity { + grid-column: 1; + } + + .mission-objective-block { + grid-column: 1 / -1; + grid-row: 2; + border-top: 2px solid var(--inset-dark); + border-left: 0 !important; + box-shadow: inset 0 2px 0 var(--inset-light) !important; + } + + .mission-policy { + grid-column: 1 / -1; + grid-row: 3; + border-top: 2px solid var(--inset-dark); + border-left: 0 !important; + box-shadow: inset 0 2px 0 var(--inset-light) !important; + } + + .mission-live-block { + grid-column: 1 / -1; + grid-row: 4; + border-top: 2px solid var(--inset-dark); + border-left: 0 !important; + box-shadow: inset 0 2px 0 var(--inset-light) !important; + } + + .mission-utility-actions { + grid-column: 2; + grid-row: 1; + } + + .mission-sidecar { + grid-template-columns: minmax(0, 1fr); + } + + .mission-panel { + box-shadow: none; + } +} + +@media (max-width: 620px) { + body[data-mission-mode="active"] .obs-desktop { + width: min(100% - 10px, 1660px); + } + + .mission-control { + font-size: 12px; + } + + .mission-control > div { + padding: 7px 8px; + } + + .mission-objective-block, + .mission-policy, + .mission-live-block { + display: grid !important; + grid-template-columns: 1fr; + gap: 3px !important; + } + + .mission-objective-block strong, + .mission-live-copy, + .mission-policy strong { + overflow: visible; + white-space: normal; + } + + .mission-sidecar { + padding: 5px; + } + + .mission-decision-actions { + gap: 5px; + } + + .mission-button { + min-height: 36px; + padding-right: 5px; + padding-left: 5px; + font-size: 11px; + } +} + +@media (prefers-reduced-motion: reduce) { + .mission-meter i, + #causal-svg .causal-edge.is-mission-path, + .is-mission-evidence:not(tr) { + animation: none; + } +} diff --git a/docs/webmcp-mission.js b/docs/webmcp-mission.js new file mode 100644 index 0000000..96d687b --- /dev/null +++ b/docs/webmcp-mission.js @@ -0,0 +1,1039 @@ +(() => { + "use strict"; + + const SEMANTIC_URL = "data/e001-semantic-consistency-v1.json"; + const SCREENING_URL = "data/e001-screening-v1.json"; + const PROJECTION_URL = "data/webmcp-run-projection-v1.json.gz"; + const STORAGE_KEY = "gpustack.webmcp.mission.v1"; + const MAX_RECEIPTS = 40; + const REGISTERED_METRICS = Object.freeze([ + "final_held_out_nll", + "modeled_completion_seconds", + "inter_site_payload_bytes", + "abstention_count", + "replayed_tokens", + "divergence_count", + ]); + + const EMPTY_PENDING_HTML = ` +
+ +
+ No conclusion is staged +

Agent evidence bundles land here before they can become a recorded decision.

+
+
`; + + class MissionError extends Error { + constructor(code, message, extra = {}) { + super(message); + this.name = "MissionError"; + this.code = code; + this.extra = extra; + } + } + + const dom = {}; + let semanticPromise = null; + let screeningPromise = null; + let projectionPromise = null; + let tourRunning = false; + let editingProposal = false; + let missionState = loadMissionState(); + + function initialMissionState() { + return { + stateVersion: 1, + nextReceipt: 1, + nextProposal: 1, + pending: null, + approved: [], + receipts: [], + }; + } + + function isRecord(value) { + return value !== null && typeof value === "object" && !Array.isArray(value); + } + + function loadMissionState() { + const fallback = initialMissionState(); + try { + const parsed = JSON.parse(window.sessionStorage.getItem(STORAGE_KEY) || "null"); + if (!isRecord(parsed)) return fallback; + return { + stateVersion: Number.isInteger(parsed.stateVersion) && parsed.stateVersion > 0 ? parsed.stateVersion : 1, + nextReceipt: Number.isInteger(parsed.nextReceipt) && parsed.nextReceipt > 0 ? parsed.nextReceipt : 1, + nextProposal: Number.isInteger(parsed.nextProposal) && parsed.nextProposal > 0 ? parsed.nextProposal : 1, + pending: isRecord(parsed.pending) ? parsed.pending : null, + approved: Array.isArray(parsed.approved) ? parsed.approved.slice(-8) : [], + receipts: Array.isArray(parsed.receipts) ? parsed.receipts.slice(-MAX_RECEIPTS) : [], + }; + } catch (_error) { + return fallback; + } + } + + function persist() { + try { + window.sessionStorage.setItem(STORAGE_KEY, JSON.stringify(missionState)); + } catch (_error) { + // The mission stays fully functional when storage is disabled. + } + } + + function abortIfNeeded(signal) { + if (signal && signal.aborted) throw signal.reason || new DOMException("Aborted", "AbortError"); + } + + async function fetchArtifact(url, signal) { + abortIfNeeded(signal); + const response = await fetch(url, { cache: "no-store", headers: { Accept: "application/json" }, signal }); + if (!response.ok) throw new MissionError("ARTIFACT_UNAVAILABLE", `${url} returned HTTP ${response.status}.`); + return response.json(); + } + + async function fetchGzipArtifact(url, signal) { + abortIfNeeded(signal); + const response = await fetch(url, { cache: "no-store", headers: { Accept: "application/gzip" }, signal }); + if (!response.ok) throw new MissionError("ARTIFACT_UNAVAILABLE", `${url} returned HTTP ${response.status}.`); + if (typeof DecompressionStream !== "function") { + throw new MissionError("DECOMPRESSION_UNAVAILABLE", "This browser cannot open the bounded gzip projection."); + } + const compressed = await response.arrayBuffer(); + const bytes = new Uint8Array(compressed); + if (bytes[0] !== 0x1f || bytes[1] !== 0x8b) { + return JSON.parse(new TextDecoder().decode(bytes)); + } + const stream = new Blob([compressed]).stream().pipeThrough(new DecompressionStream("gzip")); + return new Response(stream).json(); + } + + function semanticArtifact(signal) { + semanticPromise ||= fetchArtifact(SEMANTIC_URL, signal).then((value) => { + if (value?.schema !== "gpu-stack.causal-observatory.e001-semantic-consistency.v1") { + throw new MissionError("ARTIFACT_INVALID", "The E001-SC1 compact artifact has an unsupported schema."); + } + return value; + }).catch((error) => { + semanticPromise = null; + throw error; + }); + return semanticPromise; + } + + function screeningArtifact(signal) { + screeningPromise ||= fetchArtifact(SCREENING_URL, signal).then((value) => { + if (value?.schema !== "gpu-stack.causal-observatory.e001.v1") { + throw new MissionError("ARTIFACT_INVALID", "The E001 screening artifact has an unsupported schema."); + } + return value; + }).catch((error) => { + screeningPromise = null; + throw error; + }); + return screeningPromise; + } + + function runProjection(signal) { + projectionPromise ||= fetchGzipArtifact(PROJECTION_URL, signal).then((value) => { + if (value?.schema !== "gpustack.webmcp-run-projection.v1" || !Array.isArray(value.epoch_columns)) { + throw new MissionError("PROJECTION_INVALID", "The bounded epoch projection has an unsupported schema."); + } + return value; + }).catch((error) => { + projectionPromise = null; + throw error; + }); + return projectionPromise; + } + + async function observatory() { + const bridge = window.GPUStackObservatory; + if (!bridge || typeof bridge.whenReady !== "function") { + throw new MissionError("OBSERVATORY_UNAVAILABLE", "The visible observatory bridge is not ready."); + } + await bridge.whenReady(); + return bridge; + } + + function setStatus(message, state = "ready") { + if (!dom.status) return; + dom.status.textContent = message; + document.body.dataset.missionStatus = state; + } + + function evidenceIdsFromReceipt(receipt) { + return Array.isArray(receipt.evidenceIds) ? receipt.evidenceIds.slice(0, 4) : []; + } + + function addReceipt(tool, status, summary, options = {}) { + const receipt = { + receiptId: `wmcp-${String(missionState.nextReceipt).padStart(4, "0")}`, + tool, + status, + summary: String(summary || "Action completed.").slice(0, 260), + evidenceIds: Array.isArray(options.evidenceIds) ? options.evidenceIds.slice(0, 8) : [], + delta: options.delta ? String(options.delta).slice(0, 180) : "No model mutation", + origin: options.origin || "webmcp", + timestamp: new Date().toISOString(), + }; + missionState.nextReceipt += 1; + missionState.receipts.push(receipt); + missionState.receipts = missionState.receipts.slice(-MAX_RECEIPTS); + persist(); + renderMission(); + return receipt; + } + + function resultWithReceipt(result, receipt) { + return { + ...result, + state_version: missionState.stateVersion, + receipt_id: receipt.receiptId, + human_approval_required: Boolean(missionState.pending), + }; + } + + function familyList(semantic) { + return Array.isArray(semantic?.researcher?.family_results) ? semantic.researcher.family_results : []; + } + + function runLedger(semantic) { + return Array.isArray(semantic?.full_trace?.run_ledger) ? semantic.full_trace.run_ledger : []; + } + + function totalAbstentions(semantic) { + return runLedger(semantic) + .filter((run) => run.split === "evaluation" && run.policy_id === "observable_adaptive") + .reduce((total, run) => total + Number(run.abstention_count || 0), 0); + } + + function adaptiveRunForFamily(semantic, familyId) { + return runLedger(semantic).find( + (run) => run.family_or_stratum_id === familyId && run.policy_id === "observable_adaptive" && run.split === "evaluation", + ); + } + + function scrollToId(id) { + const target = document.getElementById(id); + if (target) target.scrollIntoView({ behavior: "smooth", block: "center" }); + } + + function clearHighlights() { + document.querySelectorAll(".is-mission-evidence").forEach((node) => node.classList.remove("is-mission-evidence")); + } + + function highlight(selector) { + clearHighlights(); + const target = document.querySelector(selector); + if (target) { + target.classList.add("is-mission-evidence"); + target.scrollIntoView({ behavior: "smooth", block: "center" }); + } + } + + function compactFamily(family, semantic) { + const adaptiveRun = adaptiveRunForFamily(semantic, family.family_id); + return { + family_id: family.family_id, + ranking_state: family.ranking_state, + learning_delta: roundNumber(family.learning_delta), + completion_ratio: roundNumber(family.completion_ratio), + wan_ratio: roundNumber(family.wan_ratio), + abstentions: adaptiveRun ? adaptiveRun.abstention_count : 0, + }; + } + + function roundNumber(value, digits = 6) { + return Number.isFinite(value) ? Number(value.toFixed(digits)) : null; + } + + async function getObservatoryState(_args, context) { + const [semantic, screening, bridge] = await Promise.all([ + semanticArtifact(context.signal), + screeningArtifact(context.signal), + observatory(), + ]); + abortIfNeeded(context.signal); + const effects = semantic.researcher.paired_effects || []; + const families = familyList(semantic); + const view = bridge.getState(); + const receipt = addReceipt("get_observatory_state", "complete", "Read the immutable audit state and registered evidence IDs.", { + evidenceIds: [semantic.artifact_sha256], + origin: context.origin, + }); + return resultWithReceipt({ + ok: true, + active_view: { + experiment: view.experiment, + depth: view.depth, + selected_family: view.semanticFamily || null, + selected_run: view.semanticRun || null, + }, + artifact: { + experiment_id: semantic.experiment_id, + sha256: semantic.artifact_sha256, + raw_sha256: semantic.full_trace.raw_trace_artifact.artifact_sha256, + families: families.length, + runs: runLedger(semantic).length, + epochs: semantic.full_trace.raw_trace_artifact.epoch_count || 12981, + }, + evidence_boundary: "Measured learning + exact accounting; modeled infrastructure; frontier transfer unresolved.", + frozen_result: { + conclusion: semantic.status.conclusion, + all_falsifiers_pass: semantic.status.all_falsifiers_pass, + failed_gate_count: effects.filter((effect) => effect.passed === false).length, + abstentions: totalAbstentions(semantic), + }, + registered_ids: { + families: families.map((family) => family.family_id), + policies: [...new Set(runLedger(semantic).map((run) => run.policy_id))], + causal_nodes: screening.causal_graph.nodes.map((node) => node.node_id), + effect_ids: effects.map((effect) => effect.effect_id), + }, + pending_proposal: missionState.pending ? missionState.pending.proposalId : null, + suggested_next: "compare_stress_families", + }, receipt); + } + + async function compareStressFamilies(args, context) { + const semantic = await semanticArtifact(context.signal); + abortIfNeeded(context.signal); + const families = familyList(semantic); + const selectedIds = args.family_ids || families.map((family) => family.family_id); + const unknown = selectedIds.find((id) => !families.some((family) => family.family_id === id)); + if (unknown) { + throw new MissionError("UNKNOWN_FAMILY", `No held-out family named ${unknown}.`, { + available_ids: families.map((family) => family.family_id), + }); + } + const rows = selectedIds.map((id) => compactFamily(families.find((family) => family.family_id === id), semantic)); + const bridge = await observatory(); + await bridge.selectView({ experiment: "E001-SC1", depth: "researcher", semanticFamily: rows[0].family_id, semanticRun: "" }); + scrollToId("semantic-consistency-v1"); + const receipt = addReceipt("compare_stress_families", "complete", `Compared ${rows.length} held-out stress families; none establishes an adaptive transferable win.`, { + evidenceIds: selectedIds, + delta: `Visible family → ${rows[0].family_id}`, + origin: context.origin, + }); + return resultWithReceipt({ + ok: true, + comparator: semantic.comparison.selected_fixed_policy_id, + candidate: "observable_adaptive", + families: rows, + shared_fields: { + replayed_work: "0 adaptive / 0 fixed tokens in every held-out family", + energy: "not measured for both paired runs", + }, + counts: rows.reduce((result, row) => { + result[row.ranking_state] = (result[row.ranking_state] || 0) + 1; + return result; + }, {}), + aggregate_conclusion: semantic.status.conclusion, + }, receipt); + } + + async function inspectStressFamily(args, context) { + const semantic = await semanticArtifact(context.signal); + const family = familyList(semantic).find((entry) => entry.family_id === args.family_id); + if (!family) { + throw new MissionError("UNKNOWN_FAMILY", `No held-out family named ${args.family_id}.`, { + available_ids: familyList(semantic).map((entry) => entry.family_id), + }); + } + const linkedRuns = runLedger(semantic) + .filter((run) => run.family_or_stratum_id === family.family_id && run.split === "evaluation") + .filter((run) => ["observable_adaptive", semantic.comparison.selected_fixed_policy_id].includes(run.policy_id)) + .map((run) => run.run_id); + const bridge = await observatory(); + await bridge.selectView({ experiment: "E001-SC1", depth: "researcher", semanticFamily: family.family_id, semanticRun: "" }); + highlight(`[data-family-id="${CSS.escape(family.family_id)}"]`); + const result = compactFamily(family, semantic); + result.abstention_reason = family.abstention_reason || null; + result.replayed_work = family.replayed_work_display; + result.energy = family.energy_display; + if (args.include_regions) { + result.regions = (family.regions || []).slice(0, 8).map((region) => ({ + region_id: region.region_id, + state: region.state, + bandwidth_x: roundNumber(region.coordinates?.bandwidth_realization_multiplier), + compute_x: roundNumber(region.coordinates?.compute_rate_realization_multiplier), + rtt_s: roundNumber(region.coordinates?.wan_round_trip_seconds), + })); + } + const receipt = addReceipt("inspect_stress_family", "complete", `Opened ${family.family_id}: ${family.ranking_state}, ${result.abstentions} adaptive abstentions.`, { + evidenceIds: [family.family_id, ...linkedRuns.slice(0, 2)], + delta: `Visible family → ${family.family_id}`, + origin: context.origin, + }); + return resultWithReceipt({ ok: true, family: result, linked_run_ids: linkedRuns }, receipt); + } + + function projectedEpochRows(projection, runId, offset, requestedLimit) { + const run = projection.runs.find((entry) => entry.run_id === runId); + if (!run) return null; + const columnIndexes = [0, 1, 3, 5, 6, 7, 9, 14, 23, 28]; + const returnedLimit = Math.min(requestedLimit, 6); + return { + columns: columnIndexes.map((index) => projection.epoch_columns[index]), + rows: run.epochs.slice(offset, offset + returnedLimit).map((row) => columnIndexes.map((index) => row[index])), + returned: Math.min(returnedLimit, Math.max(0, run.epoch_count - offset)), + next_offset: offset + returnedLimit < run.epoch_count ? offset + returnedLimit : null, + total_epochs: run.epoch_count, + context_limit_applied: requestedLimit > returnedLimit, + }; + } + + async function inspectRun(args, context) { + const [semantic, projection] = await Promise.all([ + semanticArtifact(context.signal), + runProjection(context.signal), + ]); + abortIfNeeded(context.signal); + const run = runLedger(semantic).find((entry) => entry.run_id === args.run_id); + if (!run) { + throw new MissionError("UNKNOWN_RUN", `No compact run named ${args.run_id}.`, { + hint: "Call get_observatory_state or inspect_stress_family for exact run IDs.", + }); + } + if (args.epoch_offset >= run.epoch_count) { + throw new MissionError("EPOCH_OFFSET_OUT_OF_RANGE", `epoch_offset must be below ${run.epoch_count}.`); + } + if (projection.source_artifact_sha256 !== semantic.full_trace.raw_trace_artifact.artifact_sha256) { + throw new MissionError("HASH_MISMATCH", "The bounded projection is not bound to this compact artifact's raw trace."); + } + const page = projectedEpochRows(projection, run.run_id, args.epoch_offset, args.epoch_limit); + const bridge = await observatory(); + await bridge.selectView({ + experiment: "E001-SC1", + depth: "researcher", + semanticFamily: run.family_or_stratum_id, + semanticRun: run.run_id, + }); + scrollToId("semantic-consistency-timeline-title"); + const receipt = addReceipt("inspect_run", "complete", `Opened ${run.run_id} and epoch page ${args.epoch_offset}–${args.epoch_offset + page.returned - 1}.`, { + evidenceIds: [run.run_id, projection.source_artifact_sha256], + delta: `Visible run → ${run.run_id}`, + origin: context.origin, + }); + return resultWithReceipt({ + ok: true, + run: { + run_id: run.run_id, + family_id: run.family_or_stratum_id, + policy_id: run.policy_id, + split: run.split, + seed: run.seed, + epoch_count: run.epoch_count, + abstention_count: run.abstention_count, + ood_epoch_count: run.out_of_distribution_epoch_count, + final_held_out_nll: run.final_held_out_nll, + completion_seconds: run.modeled_infrastructure?.completion_seconds, + inter_site_payload_bytes: run.modeled_infrastructure?.inter_site_payload_bytes, + }, + epoch_page: page, + source_raw_sha256: projection.source_artifact_sha256, + projection_boundary: "Bounded scalar projection; the raw SHA-256 remains authoritative.", + }, receipt); + } + + function shortestCausalPath(graph, fromNode, toNode) { + const queue = [[fromNode, [], [fromNode]]]; + const visited = new Set([fromNode]); + while (queue.length) { + const [current, pathEdges, pathNodes] = queue.shift(); + for (const edge of graph.edges.filter((candidate) => candidate.source === current)) { + const nextEdges = [...pathEdges, edge]; + const nextNodes = [...pathNodes, edge.target]; + if (edge.target === toNode) return { edges: nextEdges, nodeIds: nextNodes }; + if (!visited.has(edge.target)) { + visited.add(edge.target); + queue.push([edge.target, nextEdges, nextNodes]); + } + } + } + return null; + } + + async function traceCausalPath(args, context) { + const screening = await screeningArtifact(context.signal); + const graph = screening.causal_graph; + const nodes = new Map(graph.nodes.map((node) => [node.node_id, node])); + if (!nodes.has(args.from_node) || !nodes.has(args.to_node)) { + const unknown = !nodes.has(args.from_node) ? args.from_node : args.to_node; + throw new MissionError("UNKNOWN_CAUSAL_NODE", `No conceptual evidence node named ${unknown}.`, { + available_ids: [...nodes.keys()], + }); + } + const path = shortestCausalPath(graph, args.from_node, args.to_node); + if (!path) throw new MissionError("NO_CAUSAL_PATH", `No directed path connects ${args.from_node} to ${args.to_node}.`); + if (path.nodeIds.length > args.max_nodes) { + throw new MissionError("PATH_LIMIT_EXCEEDED", `The shortest path needs ${path.nodeIds.length} nodes; max_nodes is ${args.max_nodes}.`); + } + const bridge = await observatory(); + await bridge.focusCausalPath(path.nodeIds, path.edges); + const nodeRows = path.nodeIds.map((id) => { + const node = nodes.get(id); + return { node_id: id, label: node.label, evidence_class: node.evidence_class }; + }); + const receipt = addReceipt("trace_causal_path", "complete", `Highlighted the ${path.nodeIds.length}-node path from ${args.from_node} to ${args.to_node}.`, { + evidenceIds: path.nodeIds, + delta: `${path.nodeIds.length} causal nodes highlighted`, + origin: context.origin, + }); + return resultWithReceipt({ + ok: true, + nodes: nodeRows, + edges: path.edges, + boundary: "A directed evidence path is not proof that every downstream quantity is measured.", + }, receipt); + } + + async function resolveEvidence(evidenceId, depth, navigate, signal) { + const [semantic, screening] = await Promise.all([semanticArtifact(signal), screeningArtifact(signal)]); + const bridge = await observatory(); + const artifactAliases = new Set([semantic.artifact_sha256, `sha256:${semantic.artifact_sha256}`, `artifact:${semantic.artifact_sha256}`]); + const rawHash = semantic.full_trace.raw_trace_artifact.artifact_sha256; + const rawAliases = new Set([rawHash, `sha256:${rawHash}`, `artifact:${rawHash}`]); + if (artifactAliases.has(evidenceId)) { + if (navigate) { + await bridge.selectView({ experiment: "E001-SC1", depth }); + scrollToId("semantic-consistency-v1"); + } + return { + evidence_id: evidenceId, + kind: "compact_artifact", + sha256: semantic.artifact_sha256, + schema: semantic.schema, + conclusion: semantic.status.conclusion, + boundary: semantic.evidence_boundary?.plain_boundary, + }; + } + if (rawAliases.has(evidenceId)) { + if (navigate) { + await bridge.selectView({ experiment: "E001-SC1", depth: "researcher" }); + scrollToId("semantic-consistency-raw-details"); + } + return { + evidence_id: evidenceId, + kind: "raw_artifact_binding", + sha256: rawHash, + schema: semantic.full_trace.raw_trace_artifact.schema, + epoch_count: semantic.full_trace.raw_trace_artifact.epoch_count || 12981, + boundary: "Use inspect_run for bounded rows; the authoritative raw artifact is not dumped into agent context.", + }; + } + const effect = semantic.researcher.paired_effects.find((entry) => entry.effect_id === evidenceId); + if (effect) { + if (navigate) { + await bridge.selectView({ experiment: "E001-SC1", depth: depth === "freshman" ? "researcher" : depth }); + highlight(`[data-effect-id="${CSS.escape(evidenceId)}"]`); + } + return { + evidence_id: evidenceId, + kind: "frozen_gate", + label: effect.label, + value: effect.display_value, + interval: effect.interval_display, + boundary: effect.boundary, + passed: effect.passed, + evidence_class: effect.evidence_class, + }; + } + const family = familyList(semantic).find((entry) => entry.family_id === evidenceId); + if (family) { + if (navigate) { + await bridge.selectView({ experiment: "E001-SC1", depth: "researcher", semanticFamily: family.family_id, semanticRun: "" }); + highlight(`[data-family-id="${CSS.escape(evidenceId)}"]`); + } + return { evidence_id: evidenceId, kind: "held_out_family", ...compactFamily(family, semantic) }; + } + const run = runLedger(semantic).find((entry) => entry.run_id === evidenceId); + if (run) { + if (navigate) { + await bridge.selectView({ experiment: "E001-SC1", depth: "researcher", semanticFamily: run.family_or_stratum_id, semanticRun: run.run_id }); + scrollToId("semantic-consistency-timeline-title"); + } + return { + evidence_id: evidenceId, + kind: "run", + family_id: run.family_or_stratum_id, + policy_id: run.policy_id, + final_held_out_nll: run.final_held_out_nll, + completion_seconds: run.modeled_infrastructure?.completion_seconds, + abstentions: run.abstention_count, + work_contract_violations: run.exact_accounting?.work_contract_violations || [], + }; + } + const observation = (screening.observations || []).find((entry) => entry.observation_id === evidenceId); + if (observation) { + if (navigate) { + await bridge.selectView({ experiment: "E001", depth }); + scrollToId("source-observations-title"); + } + return { + evidence_id: evidenceId, + kind: "source_observation", + citation: observation.provenance?.citation, + uri: observation.provenance?.uri, + license: observation.provenance?.license, + measured_values: Object.fromEntries(Object.entries(observation.measured_values || {}).map(([metric, record]) => [metric, { + value: record.value, + unit: record.unit, + lower_bound: record.uncertainty?.lower_bound, + upper_bound: record.uncertainty?.upper_bound, + }])), + scope: observation.provenance?.notes?.[0], + }; + } + const causalNode = screening.causal_graph.nodes.find((entry) => entry.node_id === evidenceId); + if (causalNode) { + if (navigate) await bridge.focusCausalPath([causalNode.node_id], []); + return { + evidence_id: evidenceId, + kind: "causal_node", + label: causalNode.label, + evidence_class: causalNode.evidence_class, + explanation: causalNode[depth], + }; + } + const ledgers = [...(semantic.full_trace.assumptions || []), ...(semantic.full_trace.missing_evidence || [])]; + const ledgerEntry = ledgers.find((entry) => [entry.assumption_id, entry.evidence_id, entry.id].includes(evidenceId)); + if (ledgerEntry) { + if (navigate) { + await bridge.selectView({ experiment: "E001-SC1", depth: "researcher" }); + scrollToId("semantic-consistency-assumptions"); + } + return { evidence_id: evidenceId, kind: "evidence_boundary", entry: ledgerEntry }; + } + throw new MissionError("UNKNOWN_EVIDENCE", `No registered evidence named ${evidenceId}.`, { + hint: "Call get_observatory_state for registered family, run, causal-node, and effect IDs.", + }); + } + + async function openEvidence(args, context) { + const evidence = await resolveEvidence(args.evidence_id, args.semantic_depth, true, context.signal); + const receipt = addReceipt("open_evidence", "complete", `Opened ${evidence.kind}: ${args.evidence_id}.`, { + evidenceIds: [args.evidence_id], + delta: `Evidence focus → ${args.evidence_id}`, + origin: context.origin, + }); + return resultWithReceipt({ ok: true, evidence }, receipt); + } + + function median(values) { + const ordered = values.filter(Number.isFinite).sort((a, b) => a - b); + if (!ordered.length) return null; + const middle = Math.floor(ordered.length / 2); + return ordered.length % 2 ? ordered[middle] : (ordered[middle - 1] + ordered[middle]) / 2; + } + + function metricValue(run, metricId) { + const values = { + final_held_out_nll: run.final_held_out_nll, + modeled_completion_seconds: run.modeled_infrastructure?.completion_seconds, + inter_site_payload_bytes: run.modeled_infrastructure?.inter_site_payload_bytes, + abstention_count: run.abstention_count, + replayed_tokens: run.exact_accounting?.replayed_tokens, + divergence_count: run.diverged ? 1 : 0, + }; + return values[metricId]; + } + + async function comparePolicies(args, context) { + const semantic = await semanticArtifact(context.signal); + const ledger = runLedger(semantic); + const availablePolicies = [...new Set(ledger.map((run) => run.policy_id))]; + const policyIds = args.policy_ids || ["observable_adaptive", semantic.comparison.selected_fixed_policy_id]; + const metricIds = args.metric_ids || REGISTERED_METRICS.slice(0, 5); + const unknownPolicy = policyIds.find((id) => !availablePolicies.includes(id)); + const unknownMetric = metricIds.find((id) => !REGISTERED_METRICS.includes(id)); + if (unknownPolicy) throw new MissionError("UNKNOWN_POLICY", `No registered policy named ${unknownPolicy}.`, { available_ids: availablePolicies }); + if (unknownMetric) throw new MissionError("UNKNOWN_METRIC", `No registered comparison metric named ${unknownMetric}.`, { available_ids: REGISTERED_METRICS }); + const rows = policyIds.map((policyId) => { + const runs = ledger.filter((run) => run.split === "evaluation" && run.policy_id === policyId); + const metrics = {}; + metricIds.forEach((metricId) => { + const values = runs.map((run) => Number(metricValue(run, metricId))).filter(Number.isFinite); + metrics[metricId] = ["abstention_count", "divergence_count", "replayed_tokens"].includes(metricId) + ? values.reduce((sum, value) => sum + value, 0) + : median(values); + }); + return { policy_id: policyId, evaluation_runs: runs.length, metrics }; + }); + const bridge = await observatory(); + await bridge.selectView({ experiment: "E001-SC1", depth: "researcher" }); + scrollToId("semantic-consistency-effects-title"); + const receipt = addReceipt("compare_policies", "complete", `Compared ${policyIds.join(" vs ")} on ${metricIds.length} registered metrics.`, { + evidenceIds: semantic.researcher.paired_effects.map((effect) => effect.effect_id), + delta: "Aggregate gate evidence in view", + origin: context.origin, + }); + return resultWithReceipt({ + ok: true, + evaluation_split_only: true, + comparator_frozen_before_evaluation: semantic.comparison.selection.frozen_before_evaluation, + policies: rows, + frozen_gates: semantic.researcher.paired_effects.map((effect) => ({ + effect_id: effect.effect_id, + passed: effect.passed, + })), + conclusion: semantic.status.conclusion, + }, receipt); + } + + async function stageConclusion(args, context) { + const semantic = await semanticArtifact(context.signal); + if (args.expected_state_version !== undefined && args.expected_state_version !== missionState.stateVersion) { + throw new MissionError("STALE_STATE", `Expected state version ${args.expected_state_version}, but current version is ${missionState.stateVersion}.`, { + current_state_version: missionState.stateVersion, + }); + } + if (args.confidence === "supported" && semantic.status.all_falsifiers_pass !== true) { + throw new MissionError("EVIDENCE_CONFLICT", "A supported conclusion is not admissible: all four frozen aggregate gates failed. Use qualified or abstain and cite the failed gates.", { + frozen_conclusion: semantic.status.conclusion, + recommended_confidence: "abstain", + }); + } + const resolved = []; + for (const evidenceId of args.evidence_ids) { + resolved.push(await resolveEvidence(evidenceId, "researcher", false, context.signal)); + } + const proposal = { + proposalId: `proposal-${String(missionState.nextProposal).padStart(3, "0")}`, + claim: args.claim, + evidenceIds: [...args.evidence_ids], + evidenceKinds: resolved.map((entry) => entry.kind), + confidence: args.confidence, + frozenConclusion: semantic.status.conclusion, + createdAt: new Date().toISOString(), + }; + missionState.nextProposal += 1; + missionState.stateVersion += 1; + missionState.pending = proposal; + editingProposal = false; + persist(); + renderMission(); + scrollToId("pending-changes-title"); + const receipt = addReceipt("stage_conclusion", "pending", `Staged ${proposal.proposalId}; human approval remains required.`, { + evidenceIds: proposal.evidenceIds, + delta: `Pending proposal → ${proposal.proposalId}`, + origin: context.origin, + }); + return resultWithReceipt({ + ok: true, + proposal_id: proposal.proposalId, + status: "pending_human_review", + confidence: proposal.confidence, + evidence_ids: proposal.evidenceIds, + frozen_conclusion: proposal.frozenConclusion, + next_action: "A human must approve, edit, or reject in the visible pending tray.", + }, receipt); + } + + const HANDLERS = Object.freeze({ + get_observatory_state: getObservatoryState, + compare_stress_families: compareStressFamilies, + inspect_stress_family: inspectStressFamily, + inspect_run: inspectRun, + trace_causal_path: traceCausalPath, + open_evidence: openEvidence, + compare_policies: comparePolicies, + stage_conclusion: stageConclusion, + }); + + async function invoke(toolName, args = {}, options = {}) { + const handler = HANDLERS[toolName]; + if (!handler) return { ok: false, code: "UNKNOWN_TOOL", message: `No mission handler named ${toolName}.` }; + const context = { signal: options.signal, origin: options.origin || "webmcp" }; + setStatus(`${toolName.replaceAll("_", " ")}…`, "working"); + try { + abortIfNeeded(context.signal); + const result = await handler(args, context); + abortIfNeeded(context.signal); + refreshRegistrationStatus(); + return result; + } catch (error) { + if (error?.name === "AbortError") throw error; + const code = error instanceof MissionError ? error.code : "MISSION_ERROR"; + const message = error instanceof Error ? error.message : "Mission execution failed."; + const receipt = addReceipt(toolName, "failed", message, { origin: context.origin }); + refreshRegistrationStatus("error"); + return { + ok: false, + code, + message, + ...(error instanceof MissionError ? error.extra : {}), + state_version: missionState.stateVersion, + receipt_id: receipt.receiptId, + }; + } + } + + function escapeText(value) { + return String(value ?? ""); + } + + function renderPending() { + if (!dom.pending) return; + const proposal = missionState.pending; + dom.pending.replaceChildren(); + if (!proposal) { + dom.pending.innerHTML = EMPTY_PENDING_HTML; + } else { + const card = document.createElement("article"); + card.className = "mission-change"; + card.dataset.kind = "conclusion"; + card.dataset.changeId = proposal.proposalId; + card.setAttribute("aria-selected", "true"); + const title = document.createElement("h3"); + title.textContent = `${proposal.proposalId} · ${proposal.confidence}`; + const claim = editingProposal ? document.createElement("textarea") : document.createElement("p"); + if (editingProposal) { + claim.id = "mission-claim-editor"; + claim.className = "mission-claim-editor"; + claim.value = proposal.claim; + claim.maxLength = 600; + claim.setAttribute("aria-label", "Edit staged conclusion"); + } else { + claim.textContent = proposal.claim; + } + const evidence = document.createElement("p"); + const evidenceLabel = document.createElement("strong"); + evidenceLabel.textContent = "Evidence: "; + evidence.append(evidenceLabel, document.createTextNode(proposal.evidenceIds.join(" · "))); + const boundary = document.createElement("p"); + boundary.textContent = `Frozen result: ${proposal.frozenConclusion}. Approval has not occurred.`; + card.append(title, claim, evidence, boundary); + dom.pending.append(card); + } + dom.pendingCount.textContent = proposal ? "1 staged" : "0 staged"; + [dom.approve, dom.reject, dom.edit].forEach((button) => { button.disabled = !proposal; }); + dom.edit.textContent = editingProposal ? "Save" : "Edit"; + } + + function renderReceipts() { + if (!dom.receipts) return; + dom.receipts.replaceChildren(); + if (!missionState.receipts.length) { + const item = document.createElement("li"); + item.className = "mission-empty mission-empty--receipt"; + item.innerHTML = `
No WebMCP calls yet

Semantic calls appear with evidence IDs and visible state deltas.

`; + dom.receipts.append(item); + } else { + [...missionState.receipts].reverse().forEach((receipt) => { + const item = document.createElement("li"); + item.className = "webmcp-receipt"; + item.dataset.status = receipt.status; + const heading = document.createElement("strong"); + heading.textContent = `${receipt.tool.replaceAll("_", " ")} · ${receipt.status}`; + const summary = document.createElement("p"); + summary.textContent = receipt.summary; + const metadata = document.createElement("p"); + metadata.className = "receipt-delta"; + const time = document.createElement("time"); + time.dateTime = receipt.timestamp; + time.textContent = new Date(receipt.timestamp).toLocaleTimeString([], { hour: "2-digit", minute: "2-digit", second: "2-digit" }); + metadata.append(document.createTextNode(`${receipt.receiptId} · ${receipt.delta} · `), time); + item.append(heading, summary, metadata); + const evidenceIds = evidenceIdsFromReceipt(receipt); + if (evidenceIds.length) { + const evidence = document.createElement("p"); + evidence.className = "receipt-evidence"; + evidence.textContent = `Evidence: ${evidenceIds.join(" · ")}`; + item.append(evidence); + } + dom.receipts.append(item); + }); + } + dom.receiptCount.textContent = `${missionState.receipts.length} item${missionState.receipts.length === 1 ? "" : "s"}`; + dom.undo.disabled = missionState.approved.length === 0; + } + + function renderMission() { + renderPending(); + renderReceipts(); + } + + async function approvePending() { + if (!missionState.pending) return; + const approved = { ...missionState.pending, approvedAt: new Date().toISOString() }; + missionState.approved.push(approved); + missionState.approved = missionState.approved.slice(-8); + missionState.pending = null; + missionState.stateVersion += 1; + editingProposal = false; + persist(); + addReceipt("human_approve", "approved", `Human recorded ${approved.proposalId}.`, { + evidenceIds: approved.evidenceIds, + delta: `Recorded conclusion → ${approved.proposalId}`, + origin: "human", + }); + setStatus("Human decision recorded · undo available", "ready"); + window.GPUStackObservatory?.announce("Staged conclusion approved and recorded by the human reviewer."); + } + + function rejectPending() { + if (!missionState.pending) return; + const rejected = missionState.pending; + missionState.pending = null; + missionState.stateVersion += 1; + editingProposal = false; + persist(); + addReceipt("human_reject", "rejected", `Human rejected ${rejected.proposalId}.`, { + evidenceIds: rejected.evidenceIds, + delta: `Rejected proposal → ${rejected.proposalId}`, + origin: "human", + }); + setStatus("Proposal rejected · evidence remains visible", "ready"); + } + + function editPending() { + if (!missionState.pending) return; + if (!editingProposal) { + editingProposal = true; + renderMission(); + document.getElementById("mission-claim-editor")?.focus(); + return; + } + const editor = document.getElementById("mission-claim-editor"); + const claim = editor ? editor.value.trim() : ""; + if (!claim) { + setStatus("A staged conclusion cannot be empty", "error"); + return; + } + missionState.pending.claim = claim.slice(0, 600); + missionState.stateVersion += 1; + editingProposal = false; + persist(); + addReceipt("human_edit", "pending", `Human edited ${missionState.pending.proposalId}; approval is still required.`, { + evidenceIds: missionState.pending.evidenceIds, + delta: `Edited proposal → ${missionState.pending.proposalId}`, + origin: "human", + }); + } + + function undoApproved() { + const undone = missionState.approved.pop(); + if (!undone) return; + missionState.stateVersion += 1; + persist(); + addReceipt("human_undo", "complete", `Human undid recorded decision ${undone.proposalId}.`, { + evidenceIds: undone.evidenceIds, + delta: `Removed recorded conclusion → ${undone.proposalId}`, + origin: "human", + }); + setStatus("Last recorded conclusion undone", "ready"); + } + + async function resetMission() { + missionState = initialMissionState(); + editingProposal = false; + try { window.sessionStorage.removeItem(STORAGE_KEY); } catch (_error) { /* no-op */ } + renderMission(); + clearHighlights(); + const bridge = await observatory(); + await bridge.selectView({ experiment: "E001-SC1", depth: "freshman", semanticFamily: "", semanticRun: "" }, { replace: true }); + refreshRegistrationStatus(); + } + + function pause(milliseconds) { + return new Promise((resolve) => window.setTimeout(resolve, milliseconds)); + } + + async function runTour() { + if (tourRunning) return; + tourRunning = true; + dom.tour.disabled = true; + try { + await invoke("get_observatory_state", {}, { origin: "local_tour" }); + await pause(220); + await invoke("compare_stress_families", {}, { origin: "local_tour" }); + await pause(220); + await invoke("inspect_stress_family", { family_id: "E6-repeated-membership-loss", include_regions: false }, { origin: "local_tour" }); + await pause(220); + await invoke("trace_causal_path", { from_node: "site_availability", to_node: "time_to_target", max_nodes: 7 }, { origin: "local_tour" }); + await pause(220); + await invoke("open_evidence", { evidence_id: "adaptive_minus_best_fixed_final_nll", semantic_depth: "researcher" }, { origin: "local_tour" }); + await pause(220); + await invoke("stage_conclusion", { + claim: "The observable adaptive controller does not earn a transferable winner claim: it abstained under out-of-distribution stress, and every frozen aggregate gate failed.", + evidence_ids: [ + "E6-repeated-membership-loss", + "adaptive_minus_best_fixed_final_nll", + "adaptive_to_best_fixed_inter_site_payload_ratio", + "adaptive_to_best_fixed_modeled_completion_time_ratio", + ], + confidence: "abstain", + expected_state_version: missionState.stateVersion, + }, { origin: "local_tour" }); + setStatus("Audit staged · waiting for human approval", "waiting"); + } finally { + tourRunning = false; + dom.tour.disabled = false; + } + } + + function cacheDOM() { + dom.status = document.getElementById("mission-status"); + dom.pending = document.getElementById("pending-changes"); + dom.pendingCount = document.getElementById("pending-change-count"); + dom.receipts = document.getElementById("webmcp-receipts"); + dom.receiptCount = document.getElementById("webmcp-receipt-count"); + dom.approve = document.getElementById("mission-approve"); + dom.reject = document.getElementById("mission-reject"); + dom.edit = document.getElementById("mission-edit"); + dom.undo = document.getElementById("mission-undo"); + dom.reset = document.getElementById("mission-reset"); + dom.tour = document.getElementById("mission-tour"); + } + + function refreshRegistrationStatus(forcedState) { + const webmcp = window.GPUStackWebMCP; + if (webmcp?.supported) { + webmcp.ready.then((status) => { + const failed = status.failed?.length || 0; + setStatus(failed ? `${status.registered.length} tools ready · ${failed} failed` : `${status.registered.length} WebMCP tools ready`, failed ? "error" : (forcedState || "ready")); + }); + } else { + setStatus("Manual audit ready · WebMCP unavailable in this browser", forcedState || "fallback"); + } + } + + function bindUI() { + dom.approve.addEventListener("click", approvePending); + dom.reject.addEventListener("click", rejectPending); + dom.edit.addEventListener("click", editPending); + dom.undo.addEventListener("click", undoApproved); + dom.reset.addEventListener("click", () => { resetMission().catch(() => setStatus("Reset failed", "error")); }); + dom.tour.addEventListener("click", () => { runTour().catch((error) => setStatus(error.message || "Tour failed", "error")); }); + window.addEventListener("gpustack:webmcp-ready", () => refreshRegistrationStatus()); + window.addEventListener("gpustack:webmcp-unavailable", () => refreshRegistrationStatus()); + } + + async function init() { + document.body.dataset.missionMode = "active"; + cacheDOM(); + bindUI(); + renderMission(); + refreshRegistrationStatus(); + try { + await Promise.all([semanticArtifact(), screeningArtifact(), observatory()]); + refreshRegistrationStatus(); + } catch (error) { + setStatus(`Evidence unavailable · ${error.message}`, "error"); + } + } + + window.GPUStackMission = Object.freeze({ + version: "1.0.0", + invoke, + getState() { + return JSON.parse(JSON.stringify(missionState)); + }, + }); + + if (document.readyState === "loading") document.addEventListener("DOMContentLoaded", init, { once: true }); + else init(); +})(); diff --git a/docs/webmcp-tools.js b/docs/webmcp-tools.js new file mode 100644 index 0000000..179fc15 --- /dev/null +++ b/docs/webmcp-tools.js @@ -0,0 +1,556 @@ +(() => { + "use strict"; + + /** + * WebMCP adapter for GPUSTACK's immutable E001-SC1 evidence observatory. + * + * The page owns the artifacts, visible selection, staged conclusion, and + * human approval/rejection controls. It exposes one bridge to this adapter: + * + * window.GPUStackMission.invoke(toolName, validatedArgs, { signal }) + * + * The bridge may be installed after this file is evaluated. Every invocation + * resolves it from `window` at call time. It must update the human-visible UI + * before resolving and return a compact JSON-serializable object. WebMCP can + * stage a conclusion, but approval remains an explicit page-only human act. + */ + + const MAX_ID_LENGTH = 180; + const MAX_RESULT_CHARS = 1500; + const ID_PATTERN = /^[A-Za-z0-9][A-Za-z0-9_.:/-]{0,179}$/; + const SEMANTIC_DEPTHS = ["freshman", "researcher", "full_trace"]; + const CONFIDENCE_LEVELS = ["supported", "qualified", "abstain"]; + + class ArgumentError extends Error { + constructor(field, message, expected) { + super(message); + this.name = "ArgumentError"; + this.field = field; + this.expected = expected; + } + } + + const objectSchema = (properties, required = []) => ({ + type: "object", + properties, + required, + additionalProperties: false, + }); + + const idSchema = (description) => ({ + type: "string", + description, + minLength: 1, + maxLength: MAX_ID_LENGTH, + pattern: "^[A-Za-z0-9][A-Za-z0-9_.:/-]{0,179}$", + }); + + const idArraySchema = (description, maxItems, minItems = 1) => ({ + type: "array", + description, + minItems, + maxItems, + uniqueItems: true, + items: idSchema("Exact identifier from the active observatory artifact."), + }); + + const SCHEMAS = Object.freeze({ + get_observatory_state: objectSchema({}), + + compare_stress_families: objectSchema({ + family_ids: idArraySchema( + "Optional held-out evaluation family IDs. Omit to compare all six E001-SC1 families.", + 6, + ), + }), + + inspect_stress_family: objectSchema( + { + family_id: idSchema("Held-out family ID, for example E4-failure-inside-wan-collapse."), + include_regions: { + type: "boolean", + default: true, + description: "Include compact uncertainty ranking regions. Defaults to true.", + }, + }, + ["family_id"], + ), + + inspect_run: objectSchema( + { + run_id: idSchema("Exact run ID from the E001-SC1 compact or raw artifact."), + epoch_offset: { + type: "integer", + minimum: 0, + default: 0, + description: "Zero-based first epoch to return. Defaults to 0.", + }, + epoch_limit: { + type: "integer", + minimum: 1, + maximum: 20, + default: 8, + description: "Requested projected epochs. Defaults to 8; replies cap rows to fit the result budget.", + }, + }, + ["run_id"], + ), + + trace_causal_path: objectSchema( + { + from_node: idSchema("Conceptual evidence-graph node where the trace begins."), + to_node: idSchema("Conceptual evidence-graph node where the trace ends."), + max_nodes: { + type: "integer", + minimum: 2, + maximum: 12, + default: 7, + description: "Maximum nodes in the returned path. Defaults to 7.", + }, + }, + ["from_node", "to_node"], + ), + + open_evidence: objectSchema( + { + evidence_id: idSchema("Evidence record, artifact, source, or boundary identifier."), + semantic_depth: { + type: "string", + enum: SEMANTIC_DEPTHS, + default: "researcher", + description: "Explanation depth. Defaults to researcher.", + }, + }, + ["evidence_id"], + ), + + compare_policies: objectSchema({ + policy_ids: idArraySchema( + "Optional policy IDs. Omit to compare observable_adaptive with the frozen periodic_local comparator.", + 3, + ), + metric_ids: idArraySchema( + "Optional metric IDs to prioritize. Omit for the artifact's registered comparison metrics.", + 6, + ), + }), + + stage_conclusion: objectSchema( + { + claim: { + type: "string", + minLength: 1, + maxLength: 600, + description: "Concise proposed conclusion grounded only in the cited evidence IDs.", + }, + evidence_ids: idArraySchema( + "One to eight evidence IDs that directly support or qualify the proposed claim.", + 8, + ), + confidence: { + type: "string", + enum: CONFIDENCE_LEVELS, + description: "Supported, qualified, or abstain. This is evidence confidence, not approval.", + }, + expected_state_version: { + type: "integer", + minimum: 0, + description: "Optional optimistic-concurrency version returned by a prior tool call.", + }, + }, + ["claim", "evidence_ids", "confidence"], + ), + }); + + function isRecord(value) { + if (value === null || typeof value !== "object" || Array.isArray(value)) return false; + const prototype = Object.getPrototypeOf(value); + return prototype === Object.prototype || prototype === null; + } + + function checkObject(value, allowedKeys) { + if (!isRecord(value)) { + throw new ArgumentError("$", "Arguments must be a JSON object.", "object"); + } + const unknown = Object.keys(value).find((key) => !allowedKeys.includes(key)); + if (unknown) { + throw new ArgumentError( + unknown, + `Unknown argument: ${unknown}.`, + allowedKeys.length ? `one of: ${allowedKeys.join(", ")}` : "no arguments", + ); + } + } + + function cleanString(value, field, options = {}) { + const { required = false, min = 0, max = 600, pattern = null } = options; + if (value === undefined) { + if (required) throw new ArgumentError(field, `${field} is required.`, "string"); + return undefined; + } + if (typeof value !== "string") { + throw new ArgumentError(field, `${field} must be a string.`, "string"); + } + const cleaned = value.trim(); + if (cleaned.length < min || cleaned.length > max) { + throw new ArgumentError(field, `${field} must be ${min}-${max} characters.`, `${min}-${max} characters`); + } + if (pattern && !pattern.test(cleaned)) { + throw new ArgumentError(field, `${field} has an invalid identifier format.`, "GPUSTACK identifier"); + } + return cleaned; + } + + function cleanId(value, field) { + return cleanString(value, field, { + required: true, + min: 1, + max: MAX_ID_LENGTH, + pattern: ID_PATTERN, + }); + } + + function cleanIdArray(value, field, maxItems) { + if (!Array.isArray(value) || value.length < 1 || value.length > maxItems) { + throw new ArgumentError(field, `${field} must contain 1-${maxItems} identifiers.`, `array with 1-${maxItems} identifiers`); + } + const cleaned = value.map((item, index) => cleanId(item, `${field}[${index}]`)); + if (new Set(cleaned).size !== cleaned.length) { + throw new ArgumentError(field, `${field} must not contain duplicates.`, "unique identifiers"); + } + return cleaned; + } + + function cleanOptionalIdArray(value, field, maxItems) { + return value === undefined ? undefined : cleanIdArray(value, field, maxItems); + } + + function cleanEnum(value, field, allowed, fallback) { + if (value === undefined && fallback !== undefined) return fallback; + if (typeof value !== "string" || !allowed.includes(value)) { + throw new ArgumentError(field, `${field} must be one of ${allowed.join(", ")}.`, allowed.join(" | ")); + } + return value; + } + + function cleanInteger(value, field, minimum, maximum, fallback) { + if (value === undefined) { + if (fallback !== undefined) return fallback; + throw new ArgumentError(field, `${field} is required.`, `integer >= ${minimum}`); + } + const aboveMax = maximum !== undefined && value > maximum; + if (!Number.isInteger(value) || value < minimum || aboveMax) { + const range = maximum === undefined ? `>= ${minimum}` : `${minimum}-${maximum}`; + throw new ArgumentError(field, `${field} must be an integer in ${range}.`, `integer ${range}`); + } + return value; + } + + const VALIDATORS = Object.freeze({ + get_observatory_state(args) { + checkObject(args, []); + return {}; + }, + + compare_stress_families(args) { + checkObject(args, ["family_ids"]); + const result = {}; + const familyIds = cleanOptionalIdArray(args.family_ids, "family_ids", 6); + if (familyIds !== undefined) result.family_ids = familyIds; + return result; + }, + + inspect_stress_family(args) { + checkObject(args, ["family_id", "include_regions"]); + if (args.include_regions !== undefined && typeof args.include_regions !== "boolean") { + throw new ArgumentError("include_regions", "include_regions must be a boolean.", "boolean"); + } + return { + family_id: cleanId(args.family_id, "family_id"), + include_regions: args.include_regions === undefined ? true : args.include_regions, + }; + }, + + inspect_run(args) { + checkObject(args, ["run_id", "epoch_offset", "epoch_limit"]); + return { + run_id: cleanId(args.run_id, "run_id"), + epoch_offset: cleanInteger(args.epoch_offset, "epoch_offset", 0, undefined, 0), + epoch_limit: cleanInteger(args.epoch_limit, "epoch_limit", 1, 20, 8), + }; + }, + + trace_causal_path(args) { + checkObject(args, ["from_node", "to_node", "max_nodes"]); + const fromNode = cleanId(args.from_node, "from_node"); + const toNode = cleanId(args.to_node, "to_node"); + if (fromNode === toNode) { + throw new ArgumentError("to_node", "to_node must differ from from_node.", "different node identifier"); + } + return { + from_node: fromNode, + to_node: toNode, + max_nodes: cleanInteger(args.max_nodes, "max_nodes", 2, 12, 7), + }; + }, + + open_evidence(args) { + checkObject(args, ["evidence_id", "semantic_depth"]); + return { + evidence_id: cleanId(args.evidence_id, "evidence_id"), + semantic_depth: cleanEnum(args.semantic_depth, "semantic_depth", SEMANTIC_DEPTHS, "researcher"), + }; + }, + + compare_policies(args) { + checkObject(args, ["policy_ids", "metric_ids"]); + const result = {}; + const policyIds = cleanOptionalIdArray(args.policy_ids, "policy_ids", 3); + const metricIds = cleanOptionalIdArray(args.metric_ids, "metric_ids", 6); + if (policyIds !== undefined) result.policy_ids = policyIds; + if (metricIds !== undefined) result.metric_ids = metricIds; + return result; + }, + + stage_conclusion(args) { + checkObject(args, ["claim", "evidence_ids", "confidence", "expected_state_version"]); + const result = { + claim: cleanString(args.claim, "claim", { required: true, min: 1, max: 600 }), + evidence_ids: cleanIdArray(args.evidence_ids, "evidence_ids", 8), + confidence: cleanEnum(args.confidence, "confidence", CONFIDENCE_LEVELS), + }; + if (args.expected_state_version !== undefined) { + result.expected_state_version = cleanInteger(args.expected_state_version, "expected_state_version", 0); + } + return result; + }, + }); + + const READ_ONLY = Object.freeze({ readOnlyHint: true, untrustedContentHint: false }); + const STAGING_WRITE = Object.freeze({ readOnlyHint: false, untrustedContentHint: true }); + + const TOOL_DEFINITIONS = Object.freeze([ + { + name: "get_observatory_state", + title: "Read observatory state", + description: "Read the active immutable artifact, semantic depth, selected family and run, registered IDs, evidence boundary, state version, and any staged conclusion. Use this first instead of guessing identifiers.", + inputSchema: SCHEMAS.get_observatory_state, + annotations: READ_ONLY, + }, + { + name: "compare_stress_families", + title: "Compare held-out stress families", + description: "Compare up to six E001-SC1 held-out evaluation families on learning, completion time, WAN payload, replayed work, energy, ranking regions, and abstentions. Omit IDs to compare all six.", + inputSchema: SCHEMAS.compare_stress_families, + annotations: READ_ONLY, + }, + { + name: "inspect_stress_family", + title: "Inspect one stress family", + description: "Inspect one held-out E001-SC1 stress family, including adaptive-versus-frozen-comparator deltas, uncertainty regions, abstention reason, and linked run IDs. Also selects it visibly.", + inputSchema: SCHEMAS.inspect_stress_family, + annotations: READ_ONLY, + }, + { + name: "inspect_run", + title: "Inspect experiment run", + description: "Inspect one exact E001-SC1 run and a bounded page of scalar-projected optimizer-commit epochs. Returns mode choice, OOD and abstention state, completion, and event markers while preserving the authoritative raw-trace hash.", + inputSchema: SCHEMAS.inspect_run, + annotations: READ_ONLY, + }, + { + name: "trace_causal_path", + title: "Trace evidence path", + description: "Trace a bounded path through the seven-node conceptual evidence graph, preserving relation labels and observed, modeled, assumed, prior, or unmeasured boundaries. Highlights the same path.", + inputSchema: SCHEMAS.trace_causal_path, + annotations: READ_ONLY, + }, + { + name: "open_evidence", + title: "Open supporting evidence", + description: "Open a registered artifact hash, frozen gate, held-out family, run, source observation, or causal node at the requested semantic depth. Returns provenance and caveats, not a fabricated claim.", + inputSchema: SCHEMAS.open_evidence, + annotations: READ_ONLY, + }, + { + name: "compare_policies", + title: "Compare registered policies", + description: "Compare up to three policies from the immutable experiment artifact. Omit IDs for observable_adaptive versus the calibration-frozen periodic_local comparator across registered metrics.", + inputSchema: SCHEMAS.compare_policies, + annotations: READ_ONLY, + }, + { + name: "stage_conclusion", + title: "Stage evidence conclusion", + description: "Stage a supported, qualified, or abstain conclusion with explicit evidence IDs in the visible pending tray. This never approves or commits it; only the human can approve or reject it in the page.", + inputSchema: SCHEMAS.stage_conclusion, + annotations: STAGING_WRITE, + }, + ]); + + function failure(toolName, code, message, extra = {}) { + return { + ok: false, + tool: toolName, + code, + message: String(message).slice(0, 320), + ...extra, + }; + } + + function compactValue(value, depth = 0) { + if (value === null || typeof value === "boolean") return value; + if (typeof value === "number") return Number.isFinite(value) ? value : null; + if (typeof value === "string") return value.length > 300 ? `${value.slice(0, 297)}...` : value; + if (depth >= 5) return "[detail omitted]"; + if (Array.isArray(value)) { + const result = value.slice(0, 10).map((item) => compactValue(item, depth + 1)); + if (value.length > 10) result.push(`[${value.length - 10} more]`); + return result; + } + if (isRecord(value)) { + const result = {}; + const keys = Object.keys(value).sort().slice(0, 24); + keys.forEach((key) => { + const item = compactValue(value[key], depth + 1); + if (item !== undefined) result[key] = item; + }); + if (Object.keys(value).length > 24) result.detail_omitted = true; + return result; + } + return undefined; + } + + function compactResult(toolName, rawResult) { + if (rawResult === undefined) { + return failure(toolName, "EMPTY_RESULT", "The mission bridge returned no result."); + } + const normalized = isRecord(rawResult) + ? { ok: rawResult.ok !== false, ...rawResult, tool: toolName } + : { ok: true, tool: toolName, result: rawResult }; + + try { + if (JSON.stringify(normalized).length <= MAX_RESULT_CHARS) return normalized; + } catch (_error) { + return failure(toolName, "NON_SERIALIZABLE_RESULT", "The mission bridge returned data that is not JSON-serializable."); + } + + const compacted = compactValue(normalized); + try { + compacted.truncated = true; + if (JSON.stringify(compacted).length <= MAX_RESULT_CHARS) return compacted; + } catch (_error) { + // Fall through to a small receipt. Full detail remains in the page. + } + + const fallback = { + ok: normalized.ok !== false, + tool: toolName, + truncated: true, + message: "Full detail is visible in GPUSTACK; this response was reduced to the WebMCP result budget.", + }; + ["code", "stateVersion", "state_version", "proposal_id", "summary"].forEach((key) => { + if (normalized[key] !== undefined) fallback[key] = compactValue(normalized[key]); + }); + return fallback; + } + + function abortIfNeeded(signal) { + if (signal && signal.aborted) { + throw signal.reason || new Error("Tool execution was cancelled."); + } + } + + async function executeTool(toolName, args, options = {}) { + const signal = options && options.signal; + abortIfNeeded(signal); + + let validated; + try { + validated = VALIDATORS[toolName](args); + } catch (error) { + if (error instanceof ArgumentError) { + return failure(toolName, "INVALID_ARGUMENT", error.message, { + field: error.field, + expected: error.expected, + }); + } + return failure(toolName, "INVALID_ARGUMENT", "The tool arguments could not be validated."); + } + + const bridge = window.GPUStackMission; + if (!bridge || typeof bridge.invoke !== "function") { + return failure(toolName, "BRIDGE_UNAVAILABLE", "GPUSTACK Mission Control is still loading. Retry after the observatory is ready."); + } + + try { + const result = await bridge.invoke(toolName, validated, { signal }); + abortIfNeeded(signal); + return compactResult(toolName, result); + } catch (error) { + abortIfNeeded(signal); + const message = error && typeof error.message === "string" ? error.message : "Mission execution failed."; + return failure(toolName, "MISSION_ERROR", message); + } + } + + function emit(name, detail) { + if (typeof window.dispatchEvent !== "function" || typeof CustomEvent !== "function") return; + window.dispatchEvent(new CustomEvent(name, { detail })); + } + + const toolNames = Object.freeze(TOOL_DEFINITIONS.map((tool) => tool.name)); + if (window.GPUStackWebMCP) return; + + const modelContext = document.modelContext; + if (!modelContext || typeof modelContext.registerTool !== "function") { + window.GPUStackWebMCP = Object.freeze({ + supported: false, + toolNames, + ready: Promise.resolve({ supported: false, registered: [], failed: [] }), + dispose() {}, + }); + emit("gpustack:webmcp-unavailable", { + reason: "document.modelContext.registerTool is unavailable", + toolNames, + }); + return; + } + + const lifecycle = typeof AbortController === "function" ? new AbortController() : null; + const registered = []; + const failed = []; + const ready = (async () => { + for (const definition of TOOL_DEFINITIONS) { + const executable = { + ...definition, + execute: (args, options) => executeTool(definition.name, args, options), + }; + try { + if (lifecycle) { + await modelContext.registerTool(executable, { signal: lifecycle.signal }); + } else { + await modelContext.registerTool(executable); + } + registered.push(definition.name); + } catch (error) { + failed.push({ + name: definition.name, + message: error && typeof error.message === "string" ? error.message.slice(0, 240) : "Registration failed.", + }); + } + } + const status = { supported: true, registered: [...registered], failed: [...failed] }; + emit("gpustack:webmcp-ready", status); + return status; + })(); + + window.GPUStackWebMCP = Object.freeze({ + supported: true, + toolNames, + ready, + dispose() { + if (lifecycle && !lifecycle.signal.aborted) lifecycle.abort(); + }, + }); +})(); diff --git a/evals/webmcp-evals.json b/evals/webmcp-evals.json new file mode 100644 index 0000000..511b8b5 --- /dev/null +++ b/evals/webmcp-evals.json @@ -0,0 +1,160 @@ +[ + { + "name": "Read the observatory before auditing", + "messages": [ + { + "role": "user", + "type": "message", + "content": "Read the current GPUSTACK observatory state and tell me which immutable experiment, policies, families, evidence boundary, and review state are available. Do not guess identifiers." + } + ], + "expectedCall": [ + { + "functionName": "get_observatory_state", + "arguments": {} + } + ] + }, + { + "name": "Compare every held-out stress family", + "messages": [ + { + "role": "user", + "type": "message", + "content": "Compare observable_adaptive with the frozen comparator across all six held-out E001-SC1 stress families. Include learning, modeled completion time, payload, replayed work, and abstentions." + } + ], + "expectedCall": [ + { + "functionName": "compare_stress_families", + "arguments": {} + } + ] + }, + { + "name": "Inspect the decisive repeated-membership-loss family", + "messages": [ + { + "role": "user", + "type": "message", + "content": "Inspect E6-repeated-membership-loss, including its uncertainty regions, adaptive-versus-comparator delta, abstention reason, and linked runs." + } + ], + "expectedCall": [ + { + "functionName": "inspect_stress_family", + "arguments": { + "family_id": "E6-repeated-membership-loss", + "include_regions": true + } + } + ] + }, + { + "name": "Inspect a bounded page of the decisive candidate run", + "messages": [ + { + "role": "user", + "type": "message", + "content": "Inspect the first eight projected epochs of the observable_adaptive run in the E6 held-out family. Keep the result bounded and preserve its link to the authoritative trace." + } + ], + "expectedCall": [ + { + "functionName": "inspect_run", + "arguments": { + "run_id": "e001-sc1:evaluation:E6-repeated-membership-loss:observable_adaptive", + "epoch_offset": 0, + "epoch_limit": 8 + } + } + ] + }, + { + "name": "Trace the causal evidence DAG", + "messages": [ + { + "role": "user", + "type": "message", + "content": "Trace the published evidence path from site availability to time to target. Preserve the relation labels and evidence classes, and do not invent a linear chain where the graph branches." + } + ], + "expectedCall": [ + { + "functionName": "trace_causal_path", + "arguments": { + "from_node": "site_availability", + "to_node": "time_to_target", + "max_nodes": 7 + } + } + ] + }, + { + "name": "Open the semantic-consistency evidence boundary", + "messages": [ + { + "role": "user", + "type": "message", + "content": "Open the E001-SC1 semantic-consistency evidence at full-trace depth and distinguish measured learning, exact accounting, modeled infrastructure, and unresolved transfer." + } + ], + "expectedCall": [ + { + "functionName": "open_evidence", + "arguments": { + "evidence_id": "369bc4e9b32d6e1fcdd8dadc98c830e5ac5179f4a7204a9f5194e22913fdefdf", + "semantic_depth": "full_trace" + } + } + ] + }, + { + "name": "Compare the adaptive policy with its frozen comparator", + "messages": [ + { + "role": "user", + "type": "message", + "content": "Compare observable_adaptive with the calibration-frozen periodic_local policy using the registered E001-SC1 metrics." + } + ], + "expectedCall": [ + { + "functionName": "compare_policies", + "arguments": { + "policy_ids": [ + "observable_adaptive", + "periodic_local" + ] + } + } + ] + }, + { + "name": "Stage the warranted abstention without approving it", + "messages": [ + { + "role": "user", + "type": "message", + "content": "Stage the scientifically warranted conclusion that no transferable winner claim follows for observable_adaptive over periodic_local across the held-out stress families. Cite the decisive E6 family, both E6 policy runs, the failed learning effect, and the compact and raw semantic-consistency artifacts. Mark confidence as abstain. Do not approve or commit it." + } + ], + "expectedCall": [ + { + "functionName": "stage_conclusion", + "arguments": { + "claim": "No transferable winner claim follows for observable_adaptive over periodic_local across the held-out stress families.", + "evidence_ids": [ + "adaptive_minus_best_fixed_final_nll", + "E6-repeated-membership-loss", + "e001-sc1:evaluation:E6-repeated-membership-loss:observable_adaptive", + "e001-sc1:evaluation:E6-repeated-membership-loss:periodic_local", + "369bc4e9b32d6e1fcdd8dadc98c830e5ac5179f4a7204a9f5194e22913fdefdf", + "d6321d6fc4c0f71c4f14c2f799eff252348073b3fe5508783f9f078e7f5e9d76" + ], + "confidence": "abstain" + } + } + ] + } +] diff --git a/scripts/generate_webmcp_projection.py b/scripts/generate_webmcp_projection.py new file mode 100644 index 0000000..cc7b202 --- /dev/null +++ b/scripts/generate_webmcp_projection.py @@ -0,0 +1,171 @@ +#!/usr/bin/env python3 +"""Build the bounded WebMCP epoch projection from the immutable E001-SC1 trace. + +The source artifact is intentionally large (roughly 72 MB). WebMCP tools must +not return or eagerly load that file, so this script preserves the audit fields +needed by ``inspect_run`` while dropping tensors, replica state, and hashes that +are not useful in a short agent interaction. +""" + +from __future__ import annotations + +import argparse +import gzip +import io +import json +from pathlib import Path +from typing import Any + + +ROOT = Path(__file__).resolve().parents[1] +DEFAULT_SOURCE = ROOT / "docs" / "data" / "e001-semantic-consistency-raw-v1.json" +DEFAULT_OUTPUT = ROOT / "docs" / "data" / "webmcp-run-projection-v1.json.gz" + + +def _event_count(value: Any, *keys: str) -> int: + if isinstance(value, list): + return len(value) + if not isinstance(value, dict): + return 0 + return sum(len(value.get(key, [])) for key in keys if isinstance(value.get(key), list)) + + +EPOCH_COLUMNS = [ + "index", + "wall_tick", + "logical_tick_before", + "logical_tick_after", + "action", + "selected_mode", + "commit_outcome", + "abstained", + "abstention_reasons", + "ood", + "ood_dimensions", + "active_site_count", + "wan_bandwidth_bytes_per_second", + "wan_round_trip_seconds", + "modeled_completion_seconds", + "modeled_compute_seconds", + "modeled_wan_seconds", + "held_out_nll", + "recent_gradient_norm", + "attempted_tokens", + "useful_tokens", + "replayed_tokens", + "discarded_tokens", + "membership_event_count", + "merge_event_count", + "recovery_event_count", + "wan_event_count", + "mode_transition_from", + "mode_transition_to", +] + + +def _compact_epoch(epoch: dict[str, Any], index: int) -> list[Any]: + abstention = epoch.get("abstention_state") or {} + ood = epoch.get("ood_state") or {} + stress = epoch.get("stress") or {} + accounting = epoch.get("exact_accounting") or {} + feature_vector = ood.get("feature_vector") or {} + transition = epoch.get("mode_transition") or {} + return [ + index, + epoch.get("wall_tick"), + epoch.get("logical_tick_before"), + epoch.get("logical_tick_after"), + epoch.get("action"), + epoch.get("selected_mode"), + epoch.get("commit_outcome"), + bool(abstention.get("abstained")), + abstention.get("reasons") or [], + bool(ood.get("is_out_of_distribution")), + ood.get("dimensions") or [], + feature_vector.get("active_site_count", len(stress.get("active_sites") or [])), + feature_vector.get( + "wan_bandwidth_bytes_per_second", stress.get("bandwidth_bytes_per_second") + ), + feature_vector.get("wan_latency_seconds"), + epoch.get("modeled_completion_seconds"), + epoch.get("modeled_compute_seconds"), + epoch.get("modeled_wan_seconds"), + epoch.get("held_out_nll"), + epoch.get("recent_gradient_norm"), + accounting.get("attempted_tokens"), + accounting.get("useful_tokens"), + accounting.get("replayed_tokens"), + accounting.get("discarded_tokens"), + _event_count(epoch.get("membership_events"), "departures", "rejoins"), + _event_count(epoch.get("merge_events")), + _event_count(epoch.get("recovery_events")), + _event_count(epoch.get("wan_events")), + transition.get("from_mode") if transition else None, + transition.get("to_mode") if transition else None, + ] + + +def build_projection(source: dict[str, Any]) -> dict[str, Any]: + runs = [] + projected_epoch_count = 0 + for run in source.get("runs", []): + epoch_trace = [ + _compact_epoch(epoch, index) + for index, epoch in enumerate(run.get("epoch_trace", [])) + ] + projected_epoch_count += len(epoch_trace) + runs.append( + { + "run_id": run.get("run_id"), + "family_or_stratum_id": run.get("family_or_stratum_id"), + "policy_id": run.get("policy_id"), + "split": run.get("split"), + "seed": run.get("seed"), + "epoch_count": len(epoch_trace), + "epochs": epoch_trace, + } + ) + + return { + "schema": "gpustack.webmcp-run-projection.v1", + "experiment_id": source.get("experiment_id"), + "source_schema": source.get("schema"), + "source_artifact_sha256": source.get("artifact_sha256"), + "source_epoch_count": source.get("epoch_count"), + "projected_epoch_count": projected_epoch_count, + "projection_boundary": ( + "Rows follow epoch_columns. Lossless for listed scalar audit fields; omits tensors, replica state, sample hashes, " + "and descriptive local-device energy. The immutable raw artifact remains authoritative." + ), + "epoch_columns": EPOCH_COLUMNS, + "runs": runs, + } + + +def main() -> None: + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("--source", type=Path, default=DEFAULT_SOURCE) + parser.add_argument("--output", type=Path, default=DEFAULT_OUTPUT) + args = parser.parse_args() + + with args.source.open(encoding="utf-8") as handle: + source = json.load(handle) + projection = build_projection(source) + args.output.parent.mkdir(parents=True, exist_ok=True) + payload = (json.dumps(projection, ensure_ascii=False, separators=(",", ":")) + "\n").encode() + if args.output.suffix == ".gz": + buffer = io.BytesIO() + with gzip.GzipFile(filename="", mode="wb", fileobj=buffer, compresslevel=9, mtime=0) as handle: + handle.write(payload) + args.output.write_bytes(buffer.getvalue()) + else: + args.output.write_bytes(payload) + + print( + f"wrote {args.output} with {len(projection['runs'])} runs and " + f"{projection['projected_epoch_count']} epochs" + ) + + +if __name__ == "__main__": + main() diff --git a/tests/test_webmcp_contract.py b/tests/test_webmcp_contract.py new file mode 100644 index 0000000..4b30011 --- /dev/null +++ b/tests/test_webmcp_contract.py @@ -0,0 +1,373 @@ +"""Chrome-free contract tests for the browser-side WebMCP adapter.""" + +from __future__ import annotations + +import json +from pathlib import Path +import re +import shutil +import subprocess + +import pytest + + +ROOT = Path(__file__).resolve().parents[1] +ADAPTER = ROOT / "docs" / "webmcp-tools.js" + +TOOL_ORDER = [ + "get_observatory_state", + "compare_stress_families", + "inspect_stress_family", + "inspect_run", + "trace_causal_path", + "open_evidence", + "compare_policies", + "stage_conclusion", +] +EXPECTED_TOOLS = set(TOOL_ORDER) +READ_ONLY_TOOLS = set(TOOL_ORDER[:-1]) + +VALID_CALLS = { + "get_observatory_state": {}, + "compare_stress_families": {}, + "inspect_stress_family": {"family_id": "E4-failure-inside-wan-collapse"}, + "inspect_run": { + "run_id": "e001-sc1:evaluation:E6-repeated-membership-loss:future_trace_oracle" + }, + "trace_causal_path": { + "from_node": "site_availability", + "to_node": "time_to_target", + }, + "open_evidence": { + "evidence_id": "369bc4e9b32d6e1fcdd8dadc98c830e5ac5179f4a7204a9f5194e22913fdefdf" + }, + "compare_policies": { + "policy_ids": ["observable_adaptive", "periodic_local"], + }, + "stage_conclusion": { + "claim": "The transferable-win claim is not supported across every held-out family; abstain pending stronger evidence.", + "evidence_ids": [ + "369bc4e9b32d6e1fcdd8dadc98c830e5ac5179f4a7204a9f5194e22913fdefdf", + "d6321d6fc4c0f71c4f14c2f799eff252348073b3fe5508783f9f078e7f5e9d76", + ], + "confidence": "abstain", + "expected_state_version": 4, + }, +} + +INVALID_CALLS = { + "get_observatory_state": {"invented": True}, + "compare_stress_families": { + "family_ids": [f"E{index}-family" for index in range(1, 8)] + }, + "inspect_stress_family": { + "family_id": "E1-bursty-wan", + "include_regions": "yes", + }, + "inspect_run": {"run_id": "run-1", "epoch_limit": 21}, + "trace_causal_path": { + "from_node": "time_to_target", + "to_node": "time_to_target", + }, + "open_evidence": {"evidence_id": "contains a space"}, + "compare_policies": {"policy_ids": ["same", "same"]}, + "stage_conclusion": { + "claim": "Unsupported certainty", + "evidence_ids": [], + "confidence": "certain", + }, +} + + +NODE_HARNESS = r""" +const fs = require("fs"); +const vm = require("vm"); +const adapterPath = process.argv[1]; +const mode = process.argv[2]; +const source = fs.readFileSync(adapterPath, "utf8"); +const registered = []; +const invocations = []; +const events = []; + +class TestCustomEvent { + constructor(type, options = {}) { this.type = type; this.detail = options.detail; } +} + +const windowObject = { + dispatchEvent(event) { events.push({ type: event.type, detail: event.detail }); return true; }, +}; +if (mode !== "no_bridge") { + windowObject.GPUStackMission = { + async invoke(name, args, options) { + invocations.push({ name, args, hasSignal: Boolean(options && options.signal) }); + if (mode === "large_result") { + return { ok: true, stateVersion: 7, summary: "x".repeat(6000), rows: Array(80).fill("y".repeat(200)) }; + } + return { ok: true, stateVersion: 7, summary: `completed ${name}` }; + }, + }; +} +const documentObject = mode === "unsupported" ? {} : { + modelContext: { + async registerTool(tool, options) { + registered.push({ tool, hasLifecycleSignal: Boolean(options && options.signal) }); + }, + }, +}; +const context = { + window: windowObject, + document: documentObject, + CustomEvent: TestCustomEvent, + AbortController, + console, + Promise, + Object, + Array, + Number, + String, + Boolean, + Set, + Error, + JSON, +}; +vm.runInNewContext(source, context, { filename: adapterPath }); + +(async () => { + const status = await windowObject.GPUStackWebMCP.ready; + if (mode === "unsupported") { + process.stdout.write(JSON.stringify({ supported: windowObject.GPUStackWebMCP.supported, status, events })); + return; + } + if (mode === "metadata") { + process.stdout.write(JSON.stringify({ + status, + events, + tools: registered.map(({ tool, hasLifecycleSignal }) => ({ + name: tool.name, + title: tool.title, + description: tool.description, + inputSchema: tool.inputSchema, + annotations: tool.annotations, + executeType: typeof tool.execute, + hasLifecycleSignal, + })), + })); + return; + } + if (mode === "invoke") { + const calls = JSON.parse(process.argv[3]); + const results = {}; + for (const { tool } of registered) { + results[tool.name] = await tool.execute(calls[tool.name], { signal: new AbortController().signal }); + } + const beforeInvalid = invocations.length; + const invalid = await registered[0].tool.execute( + { invented: true }, + { signal: new AbortController().signal }, + ); + process.stdout.write(JSON.stringify({ + results, + invocations, + invalid, + invalidReachedBridge: invocations.length !== beforeInvalid, + })); + return; + } + if (mode === "validation") { + const calls = JSON.parse(process.argv[3]); + const results = {}; + for (const { tool } of registered) { + results[tool.name] = await tool.execute(calls[tool.name], { signal: new AbortController().signal }); + } + process.stdout.write(JSON.stringify({ results, invocationCount: invocations.length })); + return; + } + if (mode === "no_bridge" || mode === "large_result") { + const result = await registered[0].tool.execute({}, { signal: new AbortController().signal }); + process.stdout.write(JSON.stringify({ result })); + return; + } + throw new Error(`Unknown harness mode: ${mode}`); +})().catch((error) => { console.error(error); process.exitCode = 1; }); +""" + + +def _node() -> str: + executable = shutil.which("node") + if executable is None: + pytest.skip("Node is required for JavaScript contract execution") + return executable + + +def _run_harness(mode: str, payload: object | None = None) -> dict: + command = [_node(), "-e", NODE_HARNESS, str(ADAPTER), mode] + if payload is not None: + command.append(json.dumps(payload, separators=(",", ":"))) + completed = subprocess.run( + command, + cwd=ROOT, + check=True, + capture_output=True, + text=True, + timeout=15, + ) + return json.loads(completed.stdout) + + +def test_adapter_parses_and_uses_current_imperative_api() -> None: + source = ADAPTER.read_text(encoding="utf-8") + subprocess.run( + [_node(), "--check", str(ADAPTER)], + cwd=ROOT, + check=True, + capture_output=True, + text=True, + timeout=15, + ) + assert "document.modelContext" in source + assert ".registerTool(" in source + assert "navigator.modelContext" not in source + assert "outputSchema" not in source + assert "requestUserInteraction" not in source + + +def test_exact_tools_schemas_and_annotations() -> None: + result = _run_harness("metadata") + tools = {tool["name"]: tool for tool in result["tools"]} + + assert list(tools) == TOOL_ORDER + assert result["status"] == { + "supported": True, + "registered": TOOL_ORDER, + "failed": [], + } + assert any(event["type"] == "gpustack:webmcp-ready" for event in result["events"]) + + for name, tool in tools.items(): + assert tool["title"].strip() + assert tool["description"].strip() + assert len(tool["description"]) <= 500 + assert tool["executeType"] == "function" + assert tool["hasLifecycleSignal"] is True + assert tool["inputSchema"]["type"] == "object" + assert tool["inputSchema"]["additionalProperties"] is False + assert tool["annotations"]["readOnlyHint"] is (name in READ_ONLY_TOOLS) + + assert tools["stage_conclusion"]["annotations"]["untrustedContentHint"] is True + assert all( + tools[name]["annotations"]["untrustedContentHint"] is False + for name in READ_ONLY_TOOLS + ) + assert tools["get_observatory_state"]["inputSchema"]["properties"] == {} + assert tools["stage_conclusion"]["inputSchema"]["required"] == [ + "claim", + "evidence_ids", + "confidence", + ] + + +def test_schema_limits_match_the_grounded_artifact_contract() -> None: + tools = {tool["name"]: tool for tool in _run_harness("metadata")["tools"]} + schemas = {name: tool["inputSchema"] for name, tool in tools.items()} + + assert schemas["compare_stress_families"]["properties"]["family_ids"]["maxItems"] == 6 + assert schemas["inspect_run"]["properties"]["epoch_offset"]["minimum"] == 0 + assert schemas["inspect_run"]["properties"]["epoch_limit"]["maximum"] == 20 + assert schemas["trace_causal_path"]["properties"]["max_nodes"] == { + "type": "integer", + "minimum": 2, + "maximum": 12, + "default": 7, + "description": "Maximum nodes in the returned path. Defaults to 7.", + } + assert schemas["compare_policies"]["properties"]["policy_ids"]["maxItems"] == 3 + assert schemas["compare_policies"]["properties"]["metric_ids"]["maxItems"] == 6 + assert schemas["stage_conclusion"]["properties"]["evidence_ids"]["maxItems"] == 8 + assert schemas["stage_conclusion"]["properties"]["confidence"]["enum"] == [ + "supported", + "qualified", + "abstain", + ] + + +def test_valid_calls_are_normalized_forwarded_and_compact() -> None: + result = _run_harness("invoke", VALID_CALLS) + + assert len(result["invocations"]) == 8 + assert {call["name"] for call in result["invocations"]} == EXPECTED_TOOLS + assert all(call["hasSignal"] for call in result["invocations"]) + assert all(value["ok"] is True for value in result["results"].values()) + assert all(value["tool"] == name for name, value in result["results"].items()) + assert all( + len(json.dumps(value, separators=(",", ":"))) <= 1500 + for value in result["results"].values() + ) + + calls = {call["name"]: call["args"] for call in result["invocations"]} + assert calls["get_observatory_state"] == {} + assert calls["inspect_stress_family"]["include_regions"] is True + assert calls["inspect_run"]["epoch_offset"] == 0 + assert calls["inspect_run"]["epoch_limit"] == 8 + assert calls["trace_causal_path"]["max_nodes"] == 7 + assert calls["open_evidence"]["semantic_depth"] == "researcher" + + +def test_invalid_arguments_fail_before_bridge() -> None: + result = _run_harness("invoke", VALID_CALLS) + invalid = result["invalid"] + assert result["invalidReachedBridge"] is False + assert invalid["ok"] is False + assert invalid["tool"] == "get_observatory_state" + assert invalid["code"] == "INVALID_ARGUMENT" + assert invalid["field"] == "invented" + + +def test_domain_validation_rejects_bad_calls_before_bridge() -> None: + result = _run_harness("validation", INVALID_CALLS) + assert result["invocationCount"] == 0 + assert set(result["results"]) == EXPECTED_TOOLS + for name, failure in result["results"].items(): + assert failure["ok"] is False, name + assert failure["tool"] == name + assert failure["code"] == "INVALID_ARGUMENT" + assert failure["field"] + assert failure["expected"] + + +def test_late_bound_missing_bridge_returns_recoverable_failure() -> None: + result = _run_harness("no_bridge")["result"] + assert result == { + "ok": False, + "tool": "get_observatory_state", + "code": "BRIDGE_UNAVAILABLE", + "message": "GPUSTACK Mission Control is still loading. Retry after the observatory is ready.", + } + + +def test_large_bridge_results_are_bounded_for_agent_context() -> None: + result = _run_harness("large_result")["result"] + assert result["ok"] is True + assert result["tool"] == "get_observatory_state" + assert result["truncated"] is True + assert len(json.dumps(result, separators=(",", ":"))) <= 1500 + + +@pytest.mark.parametrize("name", TOOL_ORDER) +def test_each_tool_name_is_declared_once(name: str) -> None: + source = ADAPTER.read_text(encoding="utf-8") + declared = re.findall(r'^\s+name: "([a-z_]+)",$', source, flags=re.MULTILINE) + assert declared.count(name) == 1 + + +def test_feature_detection_preserves_normal_page() -> None: + result = _run_harness("unsupported") + assert result["supported"] is False + assert result["status"] == {"supported": False, "registered": [], "failed": []} + assert [event["type"] for event in result["events"]] == ["gpustack:webmcp-unavailable"] + + +def test_adapter_documents_late_bound_bridge_and_human_only_approval() -> None: + source = ADAPTER.read_text(encoding="utf-8") + assert "window.GPUStackMission.invoke(toolName, validatedArgs, { signal })" in source + assert "approval remains an explicit page-only human act" in source + assert "This never approves or commits it" in source diff --git a/tests/test_webmcp_mission_runtime.mjs b/tests/test_webmcp_mission_runtime.mjs new file mode 100644 index 0000000..f460559 --- /dev/null +++ b/tests/test_webmcp_mission_runtime.mjs @@ -0,0 +1,241 @@ +import assert from "node:assert/strict"; +import fs from "node:fs/promises"; +import path from "node:path"; +import test from "node:test"; +import vm from "node:vm"; +import { fileURLToPath } from "node:url"; + + +const ROOT = path.resolve(path.dirname(fileURLToPath(import.meta.url)), ".."); + + +async function makeRuntime() { + const registrations = new Map(); + const selections = []; + const paths = []; + const storage = new Map(); + const document = { + readyState: "loading", + body: { dataset: {} }, + addEventListener() {}, + getElementById() { return null; }, + querySelector() { return null; }, + querySelectorAll() { return []; }, + modelContext: { + async registerTool(definition) { + registrations.set(definition.name, definition); + }, + }, + }; + const window = { + document, + sessionStorage: { + getItem(key) { return storage.get(key) ?? null; }, + setItem(key, value) { storage.set(key, value); }, + removeItem(key) { storage.delete(key); }, + }, + dispatchEvent() {}, + setTimeout, + GPUStackObservatory: { + version: "test", + async whenReady() {}, + getState() { + return { experiment: "E001-SC1", depth: "freshman", semanticFamily: "", semanticRun: "" }; + }, + async selectView(patch) { + selections.push(patch); + return patch; + }, + async focusCausalPath(nodeIds, edges) { + paths.push({ nodeIds, edges }); + }, + announce() {}, + }, + }; + + const context = vm.createContext({ + AbortController, + CSS: { escape: (value) => String(value) }, + CustomEvent: class CustomEvent { + constructor(type, options = {}) { this.type = type; this.detail = options.detail; } + }, + DOMException, + Blob, + DecompressionStream, + Response, + URL, + console, + document, + fetch: async (url) => { + const filename = path.join(ROOT, "docs", String(url)); + try { + const body = await fs.readFile(filename); + return { + ok: true, + status: 200, + async arrayBuffer() { return body.buffer.slice(body.byteOffset, body.byteOffset + body.byteLength); }, + async json() { return JSON.parse(body.toString("utf8")); }, + }; + } catch (_error) { + return { ok: false, status: 404, async json() { throw new Error("not found"); } }; + } + }, + setTimeout, + window, + }); + window.window = window; + + const toolsSource = await fs.readFile(path.join(ROOT, "docs", "webmcp-tools.js"), "utf8"); + vm.runInContext(toolsSource, context, { filename: "webmcp-tools.js" }); + await window.GPUStackWebMCP.ready; + + const missionSource = await fs.readFile(path.join(ROOT, "docs", "webmcp-mission.js"), "utf8"); + vm.runInContext(missionSource, context, { filename: "webmcp-mission.js" }); + + return { window, registrations, selections, paths, context }; +} + + +function executor(runtime) { + return (name, args) => { + runtime.context.__args = JSON.stringify(args); + const realmArgs = vm.runInContext("JSON.parse(__args)", runtime.context); + return runtime.registrations.get(name).execute(realmArgs, { signal: new AbortController().signal }); + }; +} + + +test("all eight WebMCP registrations execute against immutable evidence", async () => { + const runtime = await makeRuntime(); + assert.equal(runtime.registrations.size, 8); + const execute = executor(runtime); + + const state = await execute("get_observatory_state", {}); + assert.equal(state.ok, true); + assert.ok(state.artifact, JSON.stringify(state)); + assert.equal(state.artifact.families, 6); + assert.equal(state.artifact.runs, 56); + assert.equal(state.artifact.epochs, 12981); + assert.equal(state.frozen_result.abstentions, 104); + assert.equal(state.frozen_result.all_falsifiers_pass, false); + assert.equal(state.truncated, undefined); + + const families = await execute("compare_stress_families", {}); + assert.equal(families.ok, true); + assert.equal(families.families.length, 6); + assert.equal(families.aggregate_conclusion, "abstain_without_policy_claim"); + assert.equal(families.truncated, undefined); + + const family = await execute("inspect_stress_family", { + family_id: "E6-repeated-membership-loss", + include_regions: true, + }); + assert.equal(family.ok, true); + assert.equal(family.family.abstentions, 24); + assert.equal(family.family.regions.length, 8); + assert.equal(family.truncated, undefined); + + const runId = "e001-sc1:evaluation:E6-repeated-membership-loss:observable_adaptive"; + const run = await execute("inspect_run", { run_id: runId, epoch_offset: 0, epoch_limit: 8 }); + assert.equal(run.ok, true); + assert.equal(run.run.final_held_out_nll, 1.063824194483459); + assert.equal(run.epoch_page.rows.length, 6); + assert.equal(run.epoch_page.context_limit_applied, true); + assert.equal(run.source_raw_sha256, "d6321d6fc4c0f71c4f14c2f799eff252348073b3fe5508783f9f078e7f5e9d76"); + assert.equal(run.truncated, undefined); + + const trace = await execute("trace_causal_path", { + from_node: "site_availability", + to_node: "time_to_target", + max_nodes: 7, + }); + assert.equal(trace.ok, true); + assert.deepEqual( + Array.from(trace.nodes, (node) => node.node_id), + ["site_availability", "mechanical_elapsed_time", "time_to_target"], + ); + assert.equal(trace.truncated, undefined); + + const evidence = await execute("open_evidence", { + evidence_id: "adaptive_minus_best_fixed_final_nll", + semantic_depth: "researcher", + }); + assert.equal(evidence.ok, true); + assert.equal(evidence.evidence.passed, false); + assert.equal(evidence.truncated, undefined); + + const artifactEvidence = await execute("open_evidence", { + evidence_id: "369bc4e9b32d6e1fcdd8dadc98c830e5ac5179f4a7204a9f5194e22913fdefdf", + semantic_depth: "full_trace", + }); + assert.equal(artifactEvidence.ok, true); + assert.equal(artifactEvidence.evidence.kind, "compact_artifact"); + assert.equal(artifactEvidence.truncated, undefined); + + const policies = await execute("compare_policies", {}); + assert.equal(policies.ok, true); + assert.equal(policies.policies.length, 2); + assert.equal(policies.comparator_frozen_before_evaluation, true); + assert.equal(policies.truncated, undefined); + + const rejectedOverclaim = await execute("stage_conclusion", { + claim: "The adaptive policy is a transferable winner.", + evidence_ids: ["E6-repeated-membership-loss"], + confidence: "supported", + expected_state_version: state.state_version, + }); + assert.equal(rejectedOverclaim.ok, false); + assert.equal(rejectedOverclaim.code, "EVIDENCE_CONFLICT"); + + const staged = await execute("stage_conclusion", { + claim: "The artifact supports abstaining from any transferable winner claim.", + evidence_ids: [ + "adaptive_minus_best_fixed_final_nll", + "E6-repeated-membership-loss", + runId, + "e001-sc1:evaluation:E6-repeated-membership-loss:periodic_local", + "369bc4e9b32d6e1fcdd8dadc98c830e5ac5179f4a7204a9f5194e22913fdefdf", + "d6321d6fc4c0f71c4f14c2f799eff252348073b3fe5508783f9f078e7f5e9d76", + ], + confidence: "abstain", + expected_state_version: state.state_version, + }); + assert.equal(staged.ok, true); + assert.equal(staged.status, "pending_human_review"); + assert.equal(staged.truncated, undefined); + assert.equal(runtime.window.GPUStackMission.getState().pending.proposalId, staged.proposal_id); + assert.ok(runtime.selections.length >= 5); + assert.equal(runtime.paths.length, 1); +}); + + +test("adapter rejects invalid and stale calls without mutating approval state", async () => { + const runtime = await makeRuntime(); + const execute = executor(runtime); + + const invalid = await execute("inspect_run", { run_id: "bad id with spaces" }); + assert.equal(invalid.ok, false); + assert.equal(invalid.code, "INVALID_ARGUMENT"); + + const unknown = await execute("inspect_stress_family", { family_id: "E99-missing" }); + assert.equal(unknown.ok, false); + assert.equal(unknown.code, "UNKNOWN_FAMILY"); + + const staged = await execute("stage_conclusion", { + claim: "Abstain from a transferable claim.", + evidence_ids: ["adaptive_minus_best_fixed_final_nll"], + confidence: "abstain", + expected_state_version: 1, + }); + assert.equal(staged.ok, true); + + const stale = await execute("stage_conclusion", { + claim: "A second proposal based on stale state.", + evidence_ids: ["adaptive_minus_best_fixed_final_nll"], + confidence: "abstain", + expected_state_version: 1, + }); + assert.equal(stale.ok, false); + assert.equal(stale.code, "STALE_STATE"); + assert.equal(runtime.window.GPUStackMission.getState().approved.length, 0); +}); diff --git a/tests/test_webmcp_projection.py b/tests/test_webmcp_projection.py new file mode 100644 index 0000000..b8360da --- /dev/null +++ b/tests/test_webmcp_projection.py @@ -0,0 +1,66 @@ +from __future__ import annotations + +import json +import gzip +from pathlib import Path + + +ROOT = Path(__file__).resolve().parents[1] +DATA = ROOT / "docs" / "data" + + +def test_webmcp_projection_is_bound_complete_and_columnar() -> None: + with gzip.open(DATA / "webmcp-run-projection-v1.json.gz", "rt", encoding="utf-8") as handle: + projection = json.load(handle) + compact = json.loads((DATA / "e001-semantic-consistency-v1.json").read_text()) + + raw_binding = compact["full_trace"]["raw_trace_artifact"] + compact_runs = compact["full_trace"]["run_ledger"] + projected_runs = projection["runs"] + columns = projection["epoch_columns"] + + assert projection["schema"] == "gpustack.webmcp-run-projection.v1" + assert projection["experiment_id"] == compact["experiment_id"] == "E001-SC1" + assert projection["source_artifact_sha256"] == raw_binding["artifact_sha256"] + assert projection["source_schema"] == raw_binding["schema"] + assert projection["source_epoch_count"] == projection["projected_epoch_count"] == 12_981 + assert len(projected_runs) == len(compact_runs) == 56 + assert {run["run_id"] for run in projected_runs} == {run["run_id"] for run in compact_runs} + assert sum(run["epoch_count"] for run in projected_runs) == 12_981 + + assert len(columns) == len(set(columns)) == 29 + assert columns[:7] == [ + "index", + "wall_tick", + "logical_tick_before", + "logical_tick_after", + "action", + "selected_mode", + "commit_outcome", + ] + assert "abstained" in columns + assert "ood" in columns + assert "modeled_completion_seconds" in columns + + for run in projected_runs: + assert run["epoch_count"] == len(run["epochs"]) + assert all(isinstance(row, list) and len(row) == len(columns) for row in run["epochs"]) + + +def test_projection_does_not_copy_heavy_raw_records() -> None: + projection_path = DATA / "webmcp-run-projection-v1.json.gz" + with gzip.open(projection_path, "rt", encoding="utf-8") as handle: + projection_text = handle.read() + + for forbidden in ( + "replica_lineages", + "replica_disagreement_before", + "replica_disagreement_after", + "sample_commitments", + "token_batch_sha256", + "model_state_sha256", + "optimizer_state_sha256", + ): + assert forbidden not in projection_text + + assert projection_path.stat().st_size < 300_000