Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .claude/skills/runner-playwright-e2e/SKILL.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
name: runner-playwright-e2e
description: Use when writing or modifying Playwright E2E specs for the demo runner (runner/e2e/*.spec.ts) - the deterministic-by-default suite, the env-gate taxonomy (E2E_LIVE, E2E_BASE_URL, E2E_BROKER_TOKEN, E2E_AI, E2E_STARTER_MATRIX), the shared helpers, data-* test contracts, CodeMirror and Sandpack gotchas, and container-pool hygiene. NOT for pipeline unit tests (node --test in runner/pipeline/).
description: Use when writing or modifying Playwright E2E specs for the demo runner (runner/e2e/*.spec.ts) - the deterministic-by-default suite, the env-gate taxonomy (E2E_LIVE, E2E_BASE_URL, E2E_API_TOKEN, E2E_AI, E2E_STARTER_MATRIX), the shared helpers, data-* test contracts, CodeMirror and Sandpack gotchas, and container-pool hygiene. NOT for pipeline unit tests (node --test in runner/pipeline/).
---

# Runner Playwright E2E authoring
Expand All @@ -22,7 +22,7 @@ path.

Anything that needs the real world takes the **narrowest gate** that covers the
dependency — `E2E_LIVE` (real preview mount), `E2E_BASE_URL` (worker routes),
`E2E_BROKER_TOKEN` (authed round-trip), `E2E_AI` (LLM spend),
`E2E_API_TOKEN` (authed round-trip), `E2E_AI` (LLM spend),
`E2E_STARTER_MATRIX` (container matrix). Two hard rules: the spec self-skips
with instructions (`test.skip(cond, "set X=1 to …")`), and every gated spec is
named in a workflow that actually runs it — in the same PR. Full taxonomy:
Expand Down
29 changes: 19 additions & 10 deletions .github/workflows/e2e-live.yml
Original file line number Diff line number Diff line change
Expand Up @@ -168,9 +168,13 @@ jobs:
E2E_LIVE: '1'
run: pnpm e2e e2e/style-apply.spec.ts --workers=1 -g "react|vue|javascript|switching"

# The authed share round-trip needs a hand-refreshed broker token
# (AGENTS.md § E2E). An absent or expired token is a notice, not a red
# run — the secret rots by design and rot is not a product failure.
# The authed share round-trip needs a persistent API token, minted from
# the app's API tokens page (ADR-0037). Absent, the step is skipped with a
# notice — the secret is optional by design. Present and NOT validating is
# a red run: this credential does not expire, so a refusal means revoked,
# deleted, or broken, none of which should pass quietly. That is the whole
# point of DEV-2583 — the broker token it replaced rotted hourly, so its
# failure had to be a warning, and the step therefore never ran.
- name: E2E — authed share round-trip (secret-gated)
if: ${{ !inputs.smoke && env.BASE_URL != '' }}
env:
Expand All @@ -179,17 +183,22 @@ jobs:
# the auth-gated /api/admin/sessions listing, so it rides this
# token-gated step rather than the anonymous suites.
E2E_API_BASE: ${{ env.BASE_URL }}
E2E_BROKER_TOKEN: ${{ secrets.E2E_BROKER_TOKEN }}
E2E_API_TOKEN: ${{ secrets.E2E_API_TOKEN }}
run: |
if [ -z "$E2E_BROKER_TOKEN" ]; then
echo "::notice::E2E_BROKER_TOKEN is not set — the authed share round-trip and the session-leak spec were skipped."
if [ -z "$E2E_API_TOKEN" ]; then
echo "::notice::E2E_API_TOKEN is not set — the authed share round-trip and the session-leak spec were skipped."
exit 0
fi
code=$(curl -s -o /dev/null -w "%{http_code}" -H "Authorization: Bearer $E2E_BROKER_TOKEN" \
"https://mcp-auth-proxy-j0tb.onrender.com/broker/userinfo" || true)
# Probed against our own API rather than the broker, which knows
# nothing about this credential. `/api/profile` rather than
# `/api/tokens`: a token is fenced off token management entirely
# (ADR-0037), and verifying the bearer is itself an `api_tokens` read,
# so this exercises the new path either way.
code=$(curl -s -o /dev/null -w "%{http_code}" -H "Authorization: Bearer $E2E_API_TOKEN" \
"$BASE_URL/api/profile" || true)
if [ "$code" != "200" ]; then
echo "::warning::E2E_BROKER_TOKEN no longer validates against the broker ($code) — refresh the secret to run the authed share round-trip."
exit 0
echo "::error::E2E_API_TOKEN did not validate ($code). A persistent token does not expire, so this means revoked, deleted, or broken — mint a new one on /api-tokens."
exit 1
fi
# Secret hygiene (Bugbot, #189): this run puts a live session JWT in
# sessionStorage and an Authorization header — a Playwright trace
Expand Down
15 changes: 10 additions & 5 deletions runner/AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -125,11 +125,16 @@ The quick list — what a green E2E run does and does not prove:
(`FIXTURE_ID` in the spec — currently `r-react-18-0-0`). Never revoke it; if it is lost,
mint a replacement titled "E2E fixture — do not revoke" from any signed-in session and
update the constant.
- **The authed write round-trip needs `E2E_BROKER_TOKEN`** (`share-create-live.spec.ts`):
a fresh `sessionStorage.hot_token` from a signed-in session on the deployed app. Broker
tokens expire and cannot be minted programmatically, so the spec self-skips without one
and the workflow treats an expired token as a warning, not a failure. It creates one
real demo and revokes it in `finally` (the 410 doubles as the revocation assertion).
- **The authed write round-trip needs `E2E_API_TOKEN`** (`share-create-live.spec.ts`, and
the session-leak spec rides the same step): a persistent API token minted on the
deployed app's `/api-tokens` page (DEV-2583, ADR-0037). It never expires, so the spec
self-skips only when the secret is *absent* — a token that no longer validates fails the
run, because that means somebody revoked it. The spec injects it as
`sessionStorage.hot_token` and the client resolves identity for it against our own API,
which is what keeps the real Share button under test. It creates one real demo and
revokes it in an `afterEach` (the 410 doubles as the revocation assertion).
Because the credential has no expiry, that step keeps `--trace off` and scrubs its
artifacts — a leaked trace from this repo, which is public, would leak a live token.
- **`E2E_AI=1` gates the live LLM answer checks** (`ai-live.spec.ts`): two API-level calls
per run, real budget, shared 8/min-per-IP rate bucket — a 429 skips rather than fails.

Expand Down
Loading
Loading