Skip to content

Commit 3c9f935

Browse files
os-steveclaude
andauthored
docs(qa): land the five re-verified environment facts in RUNNER.md (#9427)
* docs(qa): land the five re-verified environment facts in RUNNER.md The #9296 wave paid container time to learn five environment facts about these QA boxes. None was in RUNNER.md, so each new runner re-derived them. Every fact was re-measured against this tree before being written down; three needed correcting rather than transcribing. - verify --rls is a separate invocation (bare verify prints no RLS section) - Playwright needs an explicit executablePath (wants chromium 1234, has 1194) - ?id= on /api/v1/meta/app keys on the app name, never the package id - ss/netstat are absent here; read liveness with curl - a cold tree cannot boot from the console-build recipe alone Also adds the absence-inference trap row: a missing flag read as a missing capability, countered by following the forwarding chain to the default. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XqDQYVU5smx29ts9pAErja * docs(qa): prescribe the version-stable chromium alias, not the 1194 literal The executablePath a runner copies out of the facts section has to survive an image bump. /opt/pw-browsers/chromium is a symlink maintained by the image build; the versioned chromium-1194/chrome-linux/chrome beneath it stops existing the moment the image moves to 1234 — and a dead path in a section contracted to be trusted without re-derivation is the same absence-inference shape the trap table now warns about. Verified rather than assumed: Playwright 1.62.1 launches through the symlink (Chromium 141.0.7390.37, real page driven), so a symlinked executablePath is accepted. The control leg confirms the mismatch is live — a default launch fails naming chromium_headless_shell-1234, which is now recorded as the signature to recognise. The 1194 / 1234 numbers stay where they do diagnostic work: the mechanism prose and the grep/ls check. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XqDQYVU5smx29ts9pAErja --------- Co-authored-by: Claude <noreply@anthropic.com>
1 parent 499f55e commit 3c9f935

1 file changed

Lines changed: 69 additions & 0 deletions

File tree

docs/qa/platform-checklist/RUNNER.md

Lines changed: 69 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -96,6 +96,74 @@ contradicts it, and correct it here when it does.
9696
lock the console renders is a *different* gate (see
9797
`access-security.readonly-package-locks-studio`).
9898

99+
- **`objectstack verify --rls` is a separate invocation — bare `verify` prints no RLS
100+
section at all.** `runRlsProofs` runs only behind the flag
101+
(`packages/cli/src/commands/verify.ts`: `rls: Flags.boolean({ default: false })`, the
102+
proofs sit inside `if (flags.rls)`, and the report prints `if (rls)`). **Check:** the
103+
last block of a bare run is the CRUD summary — `── 15 verified, 0 gaps, 0 FAILED, 1
104+
needs-fixture, 7 skipped` on stock showcase — with no `PROVEN`/`HOLES` line anywhere.
105+
Adding `--rls` appends the RLS block: `20 PROVEN (20 consistent, 0 HOLES)` over 23
106+
objects, plus `9 of 9 declared position(s) probed`. ⛔ Do not cite plain `verify`
107+
output as the oracle for an RLS clause — that run never consulted one.
108+
109+
- **Playwright needs an explicit `executablePath` on these containers.**
110+
`@playwright/test` 1.62.1 resolves chromium build **1234**; only **1194** is installed.
111+
**Check:** `grep -A2 '"name": "chromium"' node_modules/playwright-core/browsers.json`
112+
against `ls $PLAYWRIGHT_BROWSERS_PATH` (`/opt/pw-browsers` here, holding
113+
`chromium-1194` / `chromium_headless_shell-1194` only). The stock
114+
`examples/app-showcase/playwright.config.ts` sets no `executablePath`, so every test
115+
dies at browser launch — the error names the **headless-shell** variant it wanted
116+
(`Executable doesn't exist at .../chromium_headless_shell-1234/...`), which is the
117+
signature to recognise. Pass `launchOptions.executablePath=/opt/pw-browsers/chromium`
118+
and the same specs pass (verified: Playwright launches through it, reporting Chromium
119+
141.0.7390.37, and drives a real page). ⚠️ Use that **alias**, not the versioned
120+
`chromium-1194/chrome-linux/chrome` beneath it: `/opt/pw-browsers/chromium` is a
121+
symlink maintained by the image build, so it still resolves after the image moves to
122+
1234, while the versioned literal stops existing at exactly that moment — and a dead
123+
path copied out of this section is the `absence-inference` trap one level up.
124+
**The discriminator is uniformity:** a launch/environment failure
125+
takes down the whole run at once (`showcase-smoke.spec.ts` generates one test per
126+
`SURFACES` entry — 31 today, so "31 failed" means all of them), while a product defect
127+
fails selectively. ⛔ Do not file a whole-run red as a product defect before checking
128+
the browser resolved.
129+
130+
- **`?id=` on `/api/v1/meta/app` keys on the app NAME, never the package id.** The filter
131+
matches `a.name === id` against the App document's identity (`packages/rest/src/rest-server.ts`),
132+
and App declares no `id` of its own. `?id=com.example.showcase` (the package id, from
133+
`objectstack.config.ts`) returns `{"items":[]}` — which reads exactly like "the app
134+
metadata is gone", the highest-value false P0 shape there is. Real names: `showcase_app`
135+
(showcase), `setup` / `studio` / `account` (platform built-ins). ⚠️ **An empty
136+
`items` has two distinct causes** — a wrong spelling, or an app that is genuinely not
137+
installed. `studio` is defined (`packages/platform-objects/src/apps/studio.app.ts`) but
138+
the showcase does **not** install it, so `?id=studio` is legitimately empty there; a
139+
stock admin list is `["showcase_app","setup","account"]`. **Check:** fetch
140+
`/api/v1/meta/app` with no query first and read the names it actually returns, then
141+
filter.
142+
143+
- **`ss` is not installed in these containers — read liveness with `curl`, never a socket
144+
table.** `ss` and `netstat` are both absent (`command not found`); `lsof` and `fuser`
145+
are present. The trap is that the usual spelling hides the cause: `ss -ltn | grep :3000`
146+
sends the error to stderr and prints nothing, so a **live** server is indistinguishable
147+
from a dead one — empty stdout, exit 1, no clue why. **Check instead:**
148+
`curl -s -o /dev/null -w '%{http_code}' http://localhost:PORT/api/v1/health` (substitute
149+
the real port). This is "zero hits needs a positive control" applied to one tool: a
150+
negative from a command that never ran is not evidence.
151+
152+
- **A cold tree cannot boot the app from the console-build recipe alone.**
153+
`pnpm objectui:build` runs `scripts/build-console.sh`, which builds the **console**, not
154+
the framework CLI. On a fresh tree `packages/cli` has no `dist`, and the bare binary
155+
then answers `Error: command dev not found` (exit 2) — a message that names neither the
156+
cause nor the fix. **Check:** `node scripts/check-dev-prereqs.mjs`; it reports every
157+
package whose declared `dist/` entry point is missing, and exits non-zero. **Fix:
158+
`pnpm build`** — that is what the guard itself prescribes, and it is what turns the
159+
guard green. ⛔ A targeted `turbo run build --filter=@objectstack/cli...
160+
--filter=@objectstack/example-showcase...` is **not** enough: it makes the `objectstack`
161+
binary resolve `dev`, but measured here it still left 8 of 67 packages unbuilt, so
162+
`check:dev-prereqs` stays red and `pnpm dev` still refuses to boot. Note the root
163+
`pnpm dev` script runs that guard **before** booting, so a runner who uses `pnpm dev`
164+
gets the diagnostic and the fix; the cryptic `command dev not found` only appears when
165+
the bare binary is invoked directly.
166+
99167
### Trap vocabulary (`traps` field)
100168

101169
| trap | what it fakes | counter |
@@ -110,6 +178,7 @@ contradicts it, and correct it here when it does.
110178
| `dispatcher-vs-hono-route` | route exists in unit tests, 404s on the real server | oracle = live server trace, never simulated dispatch |
111179
| `wrong-panel` | feature looks missing on a sibling surface | item's `steps` name the exact surface; check it |
112180
| `wrong-persona` | admin privileges mask a guard | run guard checks as the non-privileged persona |
181+
| `absence-inference` | a missing flag/key/script read as a missing capability | follow the forwarding chain to where the default is actually decided, before writing the finding down. A scaffold's bare `objectstack dev` still serves the console: `serve`'s `ui` flag is `default: true, allowNo: true`, so `--no-ui` is the off switch and absence means on |
113182

114183
## Run records — the GitHub issue is the report
115184

0 commit comments

Comments
 (0)