bin/dtest: the Linux visual gate was green because it compared nothing

.dev/compose.yml

@@ -12,7 +12,8 @@ services:
       - ..:/app:delegated
       - hugo_cache:/tmp/hugo_cache
     ports:
-      - '1313:1313'
+      # Fixed 1313 let only one worktree run this service at a time.
+      - '${HUGO_PORT:-1313}:1313'
     command: server --bind 0.0.0.0 --poll 500ms --disableFastRender=false
     environment:
       HUGO_CACHEDIR: /tmp/hugo_cache

.okf/build/test-gates.md

@@ -865,3 +865,81 @@ found the founding year wrong in eight places because each kept its own copy.
 
 Full fault-injection matrix, including what nothing guards:
 `docs/20-29-testing-qa/20.11-gate-fault-injection-2026-08-22-reference.md`.
+
+# A green visual run means nothing without its `[snap_diff]` line
+
+The gem resolves every baseline with `git show HEAD:<path>`. Anything that
+breaks git turns the whole visual gate into a no-op: captures are still
+written over the baselines, nothing is compared, and the run is green.
+
+**Measured 2026-08-22.** Run from a git WORKTREE, `.git` is a FILE pointing at
+an absolute host path under the main repo (`<main>/.git/worktrees/<name>`),
+which compose does not mount. Git inside the container reports "not a git
+repository", every baseline lookup returns empty, and a contact page carrying
+`body { background: red !important }` - **difference_level 0.68, 68% of the
+frame** - passed with `0 failures`. The tell was one missing line: macOS runs
+end with `[snap_diff] N screenshots compared`, the container run printed no
+such line at all.
+
+**Two fixes, each verified by breaking it:**
+
+- `Capybara::Screenshot::Diff.fail_if_new = true` (test/support/setup_snap_diff.rb).
+  Outside CI the gem defaulted this to false, so an unresolvable baseline
+  passed silently - that default is what made the no-op invisible rather than
+  loud. Probe with a screenshot name absent from git: `1 runs, 1 assertions,
+  1 failures`. A genuinely new page now fails its first run until recorded,
+  which is the same run-to-fail -> inspect -> commit flow this file already
+  documents.
+- `bin/dtest` now MAKES git work from a worktree instead of refusing to run:
+  it mounts the common git dir (`git rev-parse --path-format=absolute
+  --git-common-dir`) at `/gitcommon` and sets `GIT_DIR=/gitcommon/worktrees/
+  <name>` + `GIT_WORK_TREE=/app`. The worktree's own `commondir` file is
+  relative (`../..`), so it resolves inside the container with nothing
+  rewritten. Gated on `[ -f .git ]`, so a normal checkout is untouched.
+  Verified by re-injecting the red body: `[snap_diff] 55 screenshots compared,
+  10 failures`, with `desktop/contact_us` and `mobile/contact_us` among them.
+
+**Reading rule: a visual run that does not print `[snap_diff] N screenshots
+compared` compared nothing.** Check for that line before believing green -
+counting `0 failures` is not the same as counting comparisons.
+
+# arm64 dtest and amd64 CI genuinely disagree on text-heavy pages
+
+Once the worktree no-op above was fixed and comparisons actually ran, `bin/dtest`
+on an ARM Mac failed **8 `mobile/blog/special/codeblocks/*` screenshots at
+~0.055-0.063** against the CI-recorded baselines. Syntax-highlighted code is
+the most rasteriser-sensitive surface on the site, and `bin/dc` pins
+`DOCKER_DEFAULT_PLATFORM=linux/arm64/v8` while CI records on amd64.
+
+**A retraction, because the method matters more than the conclusion.** Earlier
+the same day this drift was declared refuted, on a measurement that compared
+the committed baselines against *themselves*: the candidates had already been
+restored by dtest's cleanup before they were measured. A vacuous comparison
+returns 0 and looks like proof of agreement. Measure candidates BEFORE any
+restore runs, or measure nothing.
+
+Consequence: on an ARM Mac, dtest is expected-red on that family. The options,
+none free - accept and screen those keys, pin the container to `linux/amd64`
+so local matches CI (correct, slower under emulation), or let CI own the Linux
+leg entirely. Not decided here; recorded so the next person does not read the
+red as a regression they caused.
+
+# Running N worktrees in parallel
+
+Five agents in five worktrees can run `bin/dev`, `bin/test` and `bin/dtest`
+concurrently. What was actually shared, and what was not:
+
+| Surface | Shared? | Resolution |
+|---|---|---|
+| compose project name | **was fixed `jtcom`** | derived per worktree in `bin/dc` - containers, networks and volumes are namespaced by it, and `bin/docked` passes `--remove-orphans`, so two runs under one project would delete each other's container mid-test |
+| `bin/dev` port | **was fixed 1313** | derived per worktree (stable across restarts, so handed-out review links keep working); the main checkout keeps 1313; explicit `PORT` still wins |
+| compose `hugo` published port | **was fixed 1313** | `${HUGO_PORT:-1313}` |
+| `bin/test` server port | no | Capybara picks a free port unless `TEST_SERVER_PORT` is set |
+| `bin/dtest` `TEST_SERVER_PORT=1314` | no | inside the container's own network namespace, never published |
+| `_dest/public-*` builds | no | each worktree has its own tree |
+| screenshot fixtures | no | per worktree; and each worktree has its own git index, so concurrent `git checkout --` does not contend |
+| the git STASH stack | **yes** | unchanged hazard - never a bare `git stash` (see CLAUDE.md) |
+
+Per-worktree compose projects mean per-worktree named volumes, so the first
+`bin/dtest` in a new worktree repopulates `hugo_cache_dtest`. Gems are baked
+into the image, not a volume, so nothing re-bundles.

.okf/log.md

@@ -51,6 +51,32 @@ make it green: restructure same-day entries under one heading, and add
 `timestamp` to the 23 concepts missing it (anchored to each file's last commit
 time, which is verifiable - never invented).
 
+## 2026-08-22 - the Linux visual gate was green because it was not testing
+
+`bin/dtest` run from a git worktree compared NOTHING. A worktree's `.git` is a
+file pointing at an absolute host path under the main repo, compose mounts only
+the worktree, so git inside the container cannot resolve it - and the gem reads
+every baseline via `git show HEAD:<path>`. Captures were written over the
+baselines, nothing was compared, every run was green.
+
+Proven rather than inferred: injected `body { background: red !important }`,
+confirmed the rule reached the built bundle and that the page references that
+fingerprinted CSS, then measured the captured PNG - candidate [255,0,0],
+baseline [255,255,255], **difference_level 0.68** - and the run reported
+`0 failures`. Three earlier injections had failed to go red and I had blamed
+the injections; the fourth proved the gate.
+
+I had reported earlier the same day that "bin/dtest is green, nothing to fix on
+the Linux side". That was worthless: it was green because it was not testing.
+Paul's main checkout has a real .git, so his dtest DID compare - which is why
+he saw 31 modified baselines from a red run and I could never reproduce one.
+
+Root cause is not worktree-specific: outside CI the gem defaults `fail_if_new`
+to false, so a baseline it cannot RETRIEVE is treated as a new screenshot and
+passes. Now `true`, verified by a probe with a name absent from git. The tell to
+remember is a missing line - a real run ends with `[snap_diff] N screenshots
+compared`; the no-op run printed none, while still reporting `0 failures`.
+
 ## 2026-08-22 - closing three of the fault-injection gaps, each broken before it was trusted
 
 Three of the five misses above now have gates. Every one of them was injected,

bin/dc

@@ -12,13 +12,26 @@ if [ "$(uname -s)" = "Darwin" ]; then
   esac
 fi
 
-echo "Running docker compose with project name: ${DOCKED_PROJECT_NAME:-jtcom}"
+# One compose project PER WORKTREE. The project name namespaces containers,
+# networks and volumes, so a fixed "jtcom" meant two worktrees running
+# bin/dtest at once shared them - and `--remove-orphans` (bin/docked) would
+# delete the other run's container mid-test. Derived from the checkout root so
+# it is stable across runs in one worktree and distinct across worktrees;
+# sanitised because compose only accepts [a-z0-9_-].
+if [ -z "${DOCKED_PROJECT_NAME:-}" ]; then
+  _root="$(git rev-parse --show-toplevel 2>/dev/null || pwd)"
+  _slug="$(basename "$_root" | tr '[:upper:]' '[:lower:]' | tr -c 'a-z0-9_-' '-')"
+  DOCKED_PROJECT_NAME="jtcom-${_slug%-}"
+  export DOCKED_PROJECT_NAME
+fi
+
+echo "Running docker compose with project name: ${DOCKED_PROJECT_NAME}"
 pwd
 
 export DOCKER_BUILDKIT=1
 export COMPOSE_DOCKER_CLI_BUILD=1
 
 docker "compose" \
   -f ".dev/compose.yml" \
-  -p "${DOCKED_PROJECT_NAME:-jtcom}" \
+  -p "${DOCKED_PROJECT_NAME}" \
   "$@"

bin/dev

@@ -12,7 +12,19 @@ SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
 PATH="$SCRIPT_DIR/../node_modules/.bin:$PATH"
 export PATH
 
-PORT="${PORT:-1313}"
+# Default the port PER WORKTREE instead of everyone landing on 1313. Derived
+# from the checkout root, so it is stable across restarts in one worktree (the
+# review links you hand out keep working) and distinct across parallel
+# worktrees. Explicit PORT still wins.
+if [ -z "${PORT:-}" ]; then
+  _root="$(git rev-parse --show-toplevel 2>/dev/null || pwd)"
+  if [ "$(basename "$_root")" = "jetthoughts.github.io" ]; then
+    PORT=1313                                   # the main checkout keeps 1313
+  else
+    _h="$(printf '%s' "$_root" | cksum | cut -d' ' -f1)"
+    PORT=$((20000 + _h % 15000))                # sidecar rides PORT+1000
+  fi
+fi
 echo "🚀 Hugo dev: http://localhost:$PORT (board sidecar on $((PORT + 1000)))"
 
 PORT=$((PORT + 1000)) "$SCRIPT_DIR/li-review" & LI_PID=$!

bin/dtest

@@ -10,14 +10,59 @@ bin/build-if-stale _dest/public-dtest
 # path would be exec'd as the command itself (mirrors bin/dtest-all).
 # HUGO_DEFAULT_PATH must be passed explicitly: bin/test honors it, and
 # PRECOMPILED_ASSETS stops the container (which has no hugo) from building.
-echo "Running tests in Docker..."
-bin/docked --env PRECOMPILED_ASSETS=true --env TEST_SERVER_PORT=1314 \
-  --env HUGO_DEFAULT_PATH=_dest/public-dtest t bin/test "$@"
-
-# Green run: the container rewrote the linux/ baselines through the bind mount
-# with sub-tolerance Rosetta pixel drift (~0.004%/run); restore so the host
-# dirty-fixture guard doesn't block the next bin/test/bin/qtest. Red runs exit
-# above (set -e) and keep candidates for inspection.
+# The container rewrites every linux/ baseline it captures, through the bind
+# mount, whether the run is green or red. Cleanup therefore happens at BOTH
+# ends and neither end is a special case:
+#
+#   BEFORE, because a red or killed run used to leave every captured baseline
+#   modified - 31 files after one red `test:critical`, not just the failures -
+#   and the in-container dirty-fixtures guard then aborted every subsequent
+#   run until someone reset by hand.
+#
+#   AFTER, via trap, so a red or interrupted run cleans up after itself. The
+#   old restore sat on the success path only, below `set -e`.
+#
+# Discarding those candidates costs nothing: bin/dc pins
+# DOCKER_DEFAULT_PLATFORM=linux/arm64/v8 on an ARM Mac while the committed
+# linux/ baselines are recorded by CI on amd64, so a candidate produced here
+# is never committable - and it is one `git add -A` from corrupting the set.
+# The only legitimate linux re-record is the CI `update-baselines` dispatch
+# (.okf/build/test-gates.md). Diff artifacts (*.diff.png, *.heatmap.diff.png)
+# are untracked and survive the restore, so a red run stays inspectable.
 if [ -z "${FORCE_SCREENSHOT_UPDATE:-}" ]; then
-  git checkout -- test/fixtures/screenshots/linux
+  git checkout -- test/fixtures/screenshots/linux 2>/dev/null || true
+  trap 'git checkout -- test/fixtures/screenshots/linux 2>/dev/null || true' EXIT
+fi
+
+# Make git work inside the container when this is a git WORKTREE.
+#
+# A worktree's .git is a FILE holding `gitdir: <main>/.git/worktrees/<name>` -
+# an absolute HOST path that compose never mounts. Git in the container then
+# reports "not a git repository", and because snap_diff resolves every
+# baseline with `git show HEAD:<path>`, the visual gate silently compares
+# NOTHING: captures are written over the baselines and every run is green.
+# Measured 2026-08-22 - a contact page with `body { background: red }`,
+# difference_level 0.68, passed with 0 failures, and the only tell was a
+# missing `[snap_diff] N screenshots compared` line.
+#
+# Mount the COMMON git dir (the main repo's .git, which holds objects/refs and
+# every worktree's entry) and point GIT_DIR at this worktree inside it. The
+# worktree's own `commondir` file is relative (`../..`), so it resolves there
+# without rewriting anything. A normal checkout has a .git DIRECTORY, already
+# inside the mount, and needs none of this.
+git_args=()
+if [ -f .git ]; then
+  git_common=$(git rev-parse --path-format=absolute --git-common-dir)
+  git_wt=$(basename "$(git rev-parse --absolute-git-dir)")
+  git_args=(
+    -v "${git_common}:/gitcommon"
+    --env "GIT_DIR=/gitcommon/worktrees/${git_wt}"
+    --env "GIT_WORK_TREE=/app"
+  )
+  echo "bin/dtest: git worktree detected - mounting ${git_common} so baselines resolve"
 fi
+
+echo "Running tests in Docker..."
+bin/docked ${git_args[@]+"${git_args[@]}"} \
+  --env PRECOMPILED_ASSETS=true --env TEST_SERVER_PORT=1314 \
+  --env HUGO_DEFAULT_PATH=_dest/public-dtest t bin/test "$@"

test/support/setup_snap_diff.rb

@@ -25,6 +25,23 @@
 Capybara::Screenshot::Diff.perceptual_threshold = 2.0
 Capybara::Screenshot::Diff.delayed = true
 
+# A screenshot whose baseline cannot be RETRIEVED must fail, not pass. The gem
+# resolves baselines with `git show HEAD:<path>`, so anything that breaks git
+# silently turns the whole visual gate into a no-op: captures are written,
+# nothing is compared, every run is green.
+#
+# Measured 2026-08-22: run from a git WORKTREE, `.git` is a file pointing at an
+# absolute host path under the main repo, which the container does not mount -
+# so git inside it reports "not a git repository", every baseline lookup
+# returns nothing, and a contact page with a `background: red !important` body
+# (difference_level 0.68) passed with 0 failures. Defaulting fail_if_new to
+# false outside CI is what made that invisible.
+#
+# Consequence: a genuinely NEW page fails its first run until recorded. That is
+# the same run-to-fail -> inspect -> commit flow .okf/build/test-gates.md
+# already documents for re-records; record mode below still bypasses it.
+Capybara::Screenshot::Diff.fail_if_new = true
+
 if ENV["FORCE_SCREENSHOT_UPDATE"] == "true"
   Capybara::Screenshot::Diff.fail_on_difference = false
   # Record mode must be able to create FIRST baselines for new pages: with