Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 30 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -39,3 +39,33 @@ jobs:
release-evidence/
opencode-artifacts-*.tgz
retention-days: 14

portable-platforms:
name: portable-platform-${{ matrix.os }}
needs: verify
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
matrix:
os: [ubuntu-latest, macos-latest, windows-latest]
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: "24"
cache: npm
- run: npm install -g npm@latest
- run: npm ci
- run: npm run build
- uses: actions/download-artifact@v4
with:
name: exact-candidate-evidence
path: candidate-input
- name: Clean packed install and portable-output smoke
run: node scripts/candidate-platform-smoke.ts --output platform-evidence/${{ runner.os }}.json --candidate-dir candidate-input
- uses: actions/upload-artifact@v4
with:
name: portable-platform-${{ runner.os }}-${{ runner.arch }}
path: platform-evidence/*.json
if-no-files-found: error
retention-days: 14
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
node_modules/
.goal5-private/
docs/evidence/journeys/raw/
dist/
*.tgz
Expand Down
45 changes: 45 additions & 0 deletions benchmarks/page-quality/v1/benchmark-manifest.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
{
"schemaVersion": 1,
"corpusId": "page-quality-v1",
"systems": ["opencode", "claude"],
"taskIds": [
"dashboard",
"incident",
"pr-walkthrough",
"system-explainer",
"compare",
"plan-checklist",
"findings-table",
"interactive-decision"
],
"minimumRunsPerSystemTask": 3,
"captureModes": [
{ "id": "desktop-light", "width": 1440, "height": 900, "colorScheme": "light" },
{ "id": "mobile-dark", "width": 390, "height": 844, "colorScheme": "dark" }
],
"hardGates": [
"facts",
"interactions",
"layout",
"composition",
"accessibility",
"dataHonesty",
"offlineBudget"
],
"dimensions": [
"taskOrientation",
"informationHierarchy",
"compositionDensity",
"typographyReadability",
"visualEncoding",
"coherenceCraft",
"interactionQuality",
"responsiveAdaptation"
],
"thresholds": {
"openCodeBetterOrEquivalent": 0.8,
"openCodeDimensionMedian": 4,
"minimumReviewers": 3,
"requiredReviewerRoles": ["design-ux", "technical"]
}
}
17 changes: 17 additions & 0 deletions benchmarks/page-quality/v1/benchmark.template.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
{
"schemaVersion": 1,
"runId": "pending-authorized-run",
"corpusId": "page-quality-v1",
"manifestSha256": "2badb79c0f21649b8ed6cb72a25fd8ca2b0a9436faf29d7ce8296384cf5a8897",
"authorization": {
"status": "pending",
"authorizedBy": null,
"authorizedAt": null,
"claudeAccountScope": null,
"modelProtocol": null,
"retentionDisposition": null
},
"generations": [],
"reviewers": [],
"pairs": []
}
124 changes: 124 additions & 0 deletions benchmarks/page-quality/v1/corpus.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,124 @@
{
"schemaVersion": 1,
"corpusId": "page-quality-v1",
"license": "CC0-1.0",
"provenance": "Project-authored synthetic fixtures; no private repository, account, or Anthropic output is included.",
"captures": [
{ "id": "desktop", "width": 1440, "height": 900 },
{ "id": "mobile", "width": 390, "height": 844 }
],
"bundles": [
{
"id": "dashboard",
"task": "Operational KPI dashboard",
"fixture": "examples/patterns/dashboard.md",
"fixtureSha256": "06081c9519780f9f74c22886026552d6f49deb3243de134c619b5a00b8c6b658",
"primaryReader": "Release engineering lead",
"readerDecision": "Prioritize the cache TTL correction before investigating lower-volume services.",
"prompt": "Create an offline operational dashboard from only the supplied synthetic facts. Lead with the weekly deploy-failure finding, show the daily trend and service breakdown, preserve provenance, and do not invent causes or dates.",
"requiredFacts": ["There were 61 failures across 1,284 deploys.", "Wednesday and Thursday account for 26 failures tied to platform#412.", "svc-payments has 19 failures and 8 rollbacks."],
"forbiddenClaims": ["The cache TTL caused failures outside Wednesday and Thursday.", "Any service not present in the source bundle was measured."],
"sourceBundle": { "license": "CC0-1.0", "provenance": "Synthetic deployment telemetry", "missingValues": ["No owner is supplied for svc-web."], "longLabel": "Mean recovery time after a failed multi-service production deployment" },
"interactions": [{ "id": "inspect-chart", "action": "Focus the primary chart and read its equivalent.", "expectedState": "The equivalent names the Wednesday peak of 14 and Saturday low of 5." }],
"stressCases": ["long-label", "missing-value", "dense-data", "narrow-viewport"]
},
{
"id": "incident",
"task": "Investigation timeline and postmortem",
"fixture": "examples/patterns/incident.md",
"fixtureSha256": "d4235196c8543314107206ce1e8fed61b4a5bb2f758fe24ae5d2cdfcb77704ce",
"primaryReader": "Incident commander",
"readerDecision": "Keep the rollback and implement the fraud check asynchronously with a 200 ms budget.",
"prompt": "Turn the supplied incident facts into an offline postmortem with status, chronology, trend, root cause, tested evidence, and next action. Preserve exact versions and times; do not infer customer loss.",
"requiredFacts": ["Peak checkout p99 was 2.6 seconds during a 38-minute incident.", "svc-payments 1.88.0 added a synchronous fraud check.", "Rollback to 1.87.2 restored p99 below 400 ms."],
"forbiddenClaims": ["Orders were lost.", "The fraud provider had an outage."],
"sourceBundle": { "license": "CC0-1.0", "provenance": "Synthetic incident log", "missingValues": ["No affected-user count was collected."], "longLabel": "Time from synchronous fraud-check deployment until three-minute recovery confirmation" },
"interactions": [{ "id": "scan-timeline", "action": "Read the timeline in document order.", "expectedState": "Alert, suspect, rollback, and mitigation remain chronologically ordered." }],
"stressCases": ["long-label", "missing-value", "dense-data", "narrow-viewport"]
},
{
"id": "pr-walkthrough",
"task": "Annotated change review",
"fixture": "examples/patterns/pr-walkthrough.md",
"fixtureSha256": "dc1b092883ff432949dad08476cf01262f2f6cf0d0710aeae8a0ac4c8ecb54d5",
"primaryReader": "Pull-request reviewer",
"readerDecision": "Request changes until the default returns to 300 seconds and invalid TTL values are rejected.",
"prompt": "Create an offline PR walkthrough from the supplied diff and findings. Order findings by severity, annotate exact locations, show the relevant diff, tests, and a clear merge verdict without inventing code.",
"requiredFacts": ["The default TTL changes from 300 seconds to 5 seconds.", "ttlSeconds lacks validation.", "The feature flag itself is acceptable after the two fixes."],
"forbiddenClaims": ["The PR is safe to merge unchanged.", "Validation exists elsewhere in the repository."],
"sourceBundle": { "license": "CC0-1.0", "provenance": "Synthetic PR diff", "missingValues": ["No benchmark result accompanies the PR."], "longLabel": "Default cache time-to-live behavior when callers omit an explicit ttlSeconds value" },
"interactions": [{ "id": "copy-verdict", "action": "Select the verdict for return to the coding session.", "expectedState": "The exact two blocking changes remain visible." }],
"stressCases": ["long-label", "missing-value", "dense-data", "narrow-viewport"]
},
{
"id": "system-explainer",
"task": "Architecture and data-flow explanation",
"fixture": "examples/patterns/system-explainer.md",
"fixtureSha256": "80fda196b54484e4902eb25bbf1e4a6248e7dc772ea99fff061c0cd9a53f0739",
"primaryReader": "Security-conscious developer",
"readerDecision": "Use portable HTML offline unless local collaboration or a registered datasource is explicitly needed.",
"prompt": "Explain the supplied artifact publication data flow as an offline page. Label validation, permission, storage, viewing, and datasource boundaries; include a sequence or relationship diagram and a clear reader decision.",
"requiredFacts": ["Validation happens before permission and writes.", "A denied operation writes nothing.", "Portable HTML uses connect-src none and adds no view-time authority."],
"forbiddenClaims": ["Portable HTML can execute a datasource.", "The viewer needs OpenCode installed."],
"sourceBundle": { "license": "CC0-1.0", "provenance": "Synthetic architecture contract", "missingValues": ["No hosted identity boundary is in this local-core diagram."], "longLabel": "Registered datasource command execution under a separately approved loopback-service permission" },
"interactions": [{ "id": "read-diagram", "action": "Read the diagram equivalent without graphics.", "expectedState": "The same validation, permission, commit, and offline-view order is conveyed." }],
"stressCases": ["long-label", "missing-value", "dense-data", "narrow-viewport"]
},
{
"id": "compare",
"task": "Alternative comparison",
"fixture": "examples/patterns/compare-layouts.md",
"fixtureSha256": "7f2e8571019c4572c915427c412cbf5e1edb2ea4c14c10d04a58b58b89c00070",
"primaryReader": "Settings product owner",
"readerDecision": "Ship two-column tabs by default and keep the dense table behind a toggle.",
"prompt": "Compare the four supplied settings layouts in a consistent offline decision page. Make each variant genuinely distinct, preserve every tradeoff, and recommend only from the supplied hallway-test result.",
"requiredFacts": ["Two-column tabs are best for known-item visits.", "The dense table is fastest for power users.", "Card-grid modals break keyboard flow."],
"forbiddenClaims": ["A production A/B test was run.", "The card grid is accessible with no changes."],
"sourceBundle": { "license": "CC0-1.0", "provenance": "Synthetic design workshop", "missingValues": ["No numeric task-completion times are available."], "longLabel": "Two-column tabs with deep links for returning users who already know the target setting" },
"interactions": [{ "id": "compare-variants", "action": "Read every variant and tradeoff in the same order.", "expectedState": "Four complete alternatives remain distinguishable at narrow width." }],
"stressCases": ["long-label", "missing-value", "dense-data", "narrow-viewport"]
},
{
"id": "plan-checklist",
"task": "Implementation and release plan",
"fixture": "examples/patterns/release-checklist.md",
"fixtureSha256": "c94a67977c27fdb1c06d0b085a8fb2033f8c1d787c2a7e050d2e42c2b28fc288",
"primaryReader": "Release owner",
"readerDecision": "Restore the performance benchmark cluster and obtain the security sign-off before tagging.",
"prompt": "Create an offline release checklist from the supplied state. Show progress, completed and open work, dependencies, risks, and the next blocking action. Do not treat post-tag work as a pre-tag gate.",
"requiredFacts": ["Seven of eleven items are complete.", "The performance baseline is blocked because the benchmark cluster is down.", "Security sign-off on the token endpoint is still open."],
"forbiddenClaims": ["The release is ready to tag.", "Post-tag dashboard watch blocks the tag."],
"sourceBundle": { "license": "CC0-1.0", "provenance": "Synthetic release plan", "missingValues": ["The benchmark cluster recovery owner is not assigned."], "longLabel": "Performance baseline re-run against the restored two-core four-gibibyte benchmark environment" },
"interactions": [{ "id": "scan-checklist", "action": "Navigate the task list by keyboard.", "expectedState": "Checked state and the two pre-tag blockers are announced." }],
"stressCases": ["long-label", "missing-value", "dense-data", "narrow-viewport"]
},
{
"id": "findings-table",
"task": "Prioritized findings table",
"fixture": "examples/patterns/license-audit.md",
"fixtureSha256": "7703cfec91a82278ea9f7d222fdd247eddb3632733a4ff4bad0310079a26e8d5",
"primaryReader": "Release and legal reviewer",
"readerDecision": "Replace or isolate imgopt and ship the required termui notice before release.",
"prompt": "Create an offline dependency-license findings page from the supplied synthetic inventory. Prioritize copyleft issues, provide sortable detail and provenance, and do not generalize beyond the listed packages.",
"requiredFacts": ["Twelve production dependencies are listed.", "vendor/imgopt is GPL-3.0 and linked into the CLI.", "vendor/termui is LGPL-2.1 and requires a notice file."],
"forbiddenClaims": ["Legal counsel approved the distribution.", "All transitive dependencies were audited beyond the supplied inventory."],
"sourceBundle": { "license": "CC0-1.0", "provenance": "Synthetic dependency inventory", "missingValues": ["No legal-review ticket is supplied."], "longLabel": "Redistribution disposition for dynamically linked native terminal user-interface bindings" },
"interactions": [{ "id": "sort-license-table", "action": "Sort the table by package and numeric size using keyboard controls.", "expectedState": "Sort direction is announced and all twelve rows remain available." }],
"stressCases": ["long-label", "missing-value", "dense-data", "narrow-viewport"]
},
{
"id": "interactive-decision",
"task": "Tune values and return a decision",
"fixture": "examples/patterns/tune-controls.md",
"fixtureSha256": "7646b761c15b76b9e249aa9827db07f557ec574030df1db3502bc73f4996063c",
"primaryReader": "Developer tuning a generated transition",
"readerDecision": "Choose frequency and amplitude, inspect the noisy-series window, then copy the chosen values back to the session.",
"prompt": "Build an offline tuning page from the supplied ranges and series. Controls must be discoverable, keyboard operable, immediate, reversible, and include a copy-back path; do not add network-backed data.",
"requiredFacts": ["Frequency ranges from 0.5 to 8 in 0.5 steps.", "Amplitude ranges from 0.2 to 3 in 0.2 steps.", "The twelve-week series rises overall from 12 to 40."],
"forbiddenClaims": ["The selected values are persisted remotely.", "The noisy series is live data."],
"sourceBundle": { "license": "CC0-1.0", "provenance": "Synthetic tuning inputs", "missingValues": ["No preferred final frequency or amplitude is supplied."], "longLabel": "Amplitude applied to the generated sine-wave transition preview before copying values back" },
"interactions": [{ "id": "tune-and-copy", "action": "Change both sliders by keyboard, reset to supplied defaults, and activate copy.", "expectedState": "The chart updates immediately and copy confirmation names the return path." }],
"stressCases": ["long-label", "missing-value", "dense-data", "narrow-viewport"]
}
]
}
7 changes: 6 additions & 1 deletion docs/component-spec.md
Original file line number Diff line number Diff line change
Expand Up @@ -87,10 +87,15 @@ Component JSON schemas have exactly one home: `skills/artifact-pages/reference/c
rationale; never copy schema tables into it.

Available fences: `stats`, `timeline`, `findings`, `compare`, `callout`, `progress`, `diff`,
`copy`, `mermaid`, `decisions`, `table`; chart fences `vega-lite` / `vega` / `echarts`;
`copy`, `mermaid`, `decisions`, `table`, `frame`; chart fences `vega-lite` / `vega` / `echarts`;
interactive controls via vega-lite `params.bind` and echarts `dataZoom` (verified live in
browser QA); free-form interactivity stays in raw-HTML mode (`format: "html"`).

Frontmatter `composition:` selects one bounded layout grammar: `standard`, `narrative`,
`dashboard`, `split`, `dense`, `quiet`, or `full`. These modes change visual scale, section
grouping, and density while retaining source/DOM order. Invalid values refuse publication.
At narrow widths every multi-column mode becomes the same logical single-column sequence.

### Data honesty rules (from Claude Code's dashboard/dataviz skills)

- Format numbers for scanning: unit + 2–3 significant figures, thousands separators; at most
Expand Down
19 changes: 19 additions & 0 deletions docs/evidence/journeys/goal-5-status-2026-08-18.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
# Goal 5 first-use and comprehension status — 2026-08-18

- Study owner: **unassigned; recruitment is not authorized**.
- Candidate: commit `84f866ed742eeb08ba668115342ef0e5896dead7`, package `0.15.0`, SHA-256
`f6d5d5dcdd74ac31522ec9a98acfb38a04c1a4a038c09e65d85d7f4813342bf2`.
- Corpus: journey corpus version 1 and participant materials version 1 are frozen.
- Target install-capable policy cells: Ubuntu LTS desktop, current macOS desktop, previous macOS
desktop, Windows 11 native desktop, and Windows 11 WSL desktop. Exact release-time versions
remain unresolved. Android Chrome and iOS Safari are viewer/support cells, not README install
tasks, and remain unresolved in the certification support matrix.
- Consented representative primary participants: 0.
- Eligible README-only first-use runs: 0; every target install cell is missing.
- Eligible comprehension participants: 0 of the minimum 10.
- Withdrawals, failures, and exclusions: 0 because collection has not started.
- Raw participant evidence: none collected.

The checked-in validator, redacted aggregate, and boundary tests are ready. This record is
`incomplete`, not a participant failure and not a pass. It blocks `OUT-02`, `OUT-03`, supported
platform, equal-or-better, and Local artifact core certification claims.
16 changes: 16 additions & 0 deletions docs/evidence/journeys/goal-5-status-2026-08-19.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
# Goal 5 first-use and comprehension status — 2026-08-19

- Study owner: **unassigned; recruitment is not authorized**.
- Candidate: commit `2908d52ef94bb0f8cf74d7f2be021ada78a0d661`, package `0.15.0`,
SHA-256 `6d5d4df63bb2300f438a572fc0af4741b793489bbd630b55070c04987c67badd`.
- Corpus: journey corpus version 1 and participant materials version 1 are frozen.
- Consented representative primary participants: 0.
- Eligible README-only first-use runs: 0; every target install cell is missing.
- Eligible comprehension participants: 0 of the minimum 10.
- Withdrawals, failures, and exclusions: 0 because collection has not started.
- Raw participant evidence: none collected.

Automated exact-tarball technical observations now pass on one Ubuntu runner, one macOS
runner, and one Windows Server runner. They are not participants and do not cover Windows 11,
WSL, previous macOS, browsers, or mobile devices. This record remains `incomplete`, blocks
`OUT-02`/`OUT-03`, and cannot support usability, support, comparison, or certification claims.
19 changes: 19 additions & 0 deletions docs/evidence/page-quality/2026-08-18-benchmark-status.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
# Goal 5 comparative benchmark status — 2026-08-18

Status: **incomplete; equal-or-better is unverified**

- Corpus: `page-quality-v1`, eight hash-bound permission-safe tasks.
- Manifest SHA-256: `2badb79c0f21649b8ed6cb72a25fd8ca2b0a9436faf29d7ce8296384cf5a8897`.
- Required unselected end-to-end generations: 48 (three per system/task); collected: 0.
- Required randomized pairs: 24; scored: 0.
- Required eligible independent reviewers: at least 3 including design/UX and technical;
recruited: 0.
- Claude execution authority, comparable model/settings protocol, and retention disposition:
absent.
- Renderer-only browser cells: 16/16 pass in the separately scoped
[`local composition report`](2026-08-18-local-composition.md).

`npm run quality:benchmark -- benchmarks/page-quality/v1/benchmark.template.json` validates
the frozen input shape and returns `incomplete`. Synthetic unit records exercise pass/fail,
no-cherry-pick, blinding mappings, confidence, all rubric medians, and hard-gate boundaries;
they are diagnostics only and are never benchmark evidence.
Loading
Loading