Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions .okf/build/ci-gates.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,8 @@ verified:
at: 2026-08-07T00:00:00Z
- by: claude/sonnet-5
at: 2026-08-20T00:00:00Z
- by: claude/opus-5
at: 2026-08-21T00:00:00Z
---

# What CI enforces on a PR
Expand Down Expand Up @@ -70,6 +72,16 @@ Consequences for how to react:
(`gh api repos/<owner>/<repo>/actions/jobs/<job_id>/logs`) and looking for
that group/error pair. If the job got PAST checkout, it is a real failure and
a re-run is the wrong move.
* **Re-read the re-run — the stall can be hiding a real failure** (2026-08-21).
On PR #511 `Unit Tests` read `fail 10m2s`, the stall signature exactly, and
"known flake, merge it" was the tempting call. The fresh run got past
checkout and failed again in **1m48s** — a genuine defect (`PavedPathGuardTest`
caught a page CSS bundle wired into `bin/qtest` but absent from
`css-bundle-ownership-map.md`). Same check, same PR, two different causes. So
the rule above is necessary but not sufficient: **a slow failure and a fast
failure are different failures.** Multi-minute with no assertion output is
infrastructure; fast with an assertion is the code. Never let the first red's
explanation carry over to the second.
* **Shrink what we ask for.** We cannot fix upstream, only reduce exposure.
This repo is a heavy ask: **1.70 GiB pack**, and `content/` alone is
**625 MB across 1,576 images**, downloaded by every job regardless of depth.
Expand Down
17 changes: 16 additions & 1 deletion .okf/build/test-gates.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,8 @@ generated: { by: claude/opus-4-8, at: 2026-08-12T20:20:00Z }
verified:
- { by: claude/fable-5, at: 2026-08-01T11:30:00Z }
- { by: claude/sonnet-5, at: 2026-08-20T00:00:00Z }
timestamp: 2026-08-20T00:00:00Z
- { by: claude/opus-5, at: 2026-08-21T00:00:00Z }
timestamp: 2026-08-21T00:00:00Z
---

# The suites
Expand Down Expand Up @@ -149,6 +150,20 @@ Minitest under `test/`, driven by `Rakefile` (`Rake::TestTask`).
baselines - cost a full false "bistable render" investigation). Re-record =
run the suite, keep the rewritten PNG, COMMIT it; only then can a rerun go
green.
- **A local visual red cannot condemn a branch until you have run the same
thing on master** (2026-08-21). Verifying PR #511 (template/CSS class),
`bin/qtest --changed` went red pointing at `services/fractional-cto`
baselines - a page the PR never touched. The plausible story was that its
`postcss.config.js` edit had shifted CSS site-wide; it had not, because that
edit only ADDS purgecss safelist entries and safelisting more can only
preserve more CSS, never remove any. The decisive check was the same system
test on clean `origin/master` on the macOS host: **34 runs, 6 failures, 8 of
77 screenshots mismatched** - the suite is red on master here, so a local red
said nothing about the branch. The run also rewrites two of those baselines
mid-run and then fails its own dirty-check, so repeated
`git checkout -- test/fixtures/screenshots/` never converges. Reproduce on
master first; if master is red the same way, fall back to CI's native-Linux
`Screenshot Tests` (which passed #511 in 16m15s while local was red).
- `bin/dtest` from a git WORKTREE is VACUOUS-GREEN: the worktree's `.git` is
a pointer file to a directory outside the container mount, so git fails
inside the container, baselines resolve to nothing, and every screenshot
Expand Down
40 changes: 40 additions & 0 deletions .okf/log.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,46 @@

Newest first. Entries before 2026-08-19 are squashed to one line each
(compacted 2026-08-20); their full text is in this file's git history.
## 2026-08-21 - Two gate lessons from merging six PRs: the stall masks, the local red lies

Both are sharpenings of rules the bundle already had, and both came from the
same PR (#511) during a queue-clearing pass.

**The checkout stall can hide a real failure behind it.** ci-gates already said
"re-run, don't raise the cap" and "if the job got past checkout it is a real
failure". Necessary, not sufficient. #511's `Unit Tests` read `fail 10m2s` - the
stall signature exactly - and the tempting call was "known flake, merge". The
fresh run got past checkout and failed again in **1m48s** on a genuine defect:
`PavedPathGuardTest` caught `friday-report.css` wired into `bin/qtest` but
missing from `css-bundle-ownership-map.md`, which is the other half of
new-page.md step 6. Same check, same PR, two unrelated causes. The rule added:
**a slow failure and a fast failure are different failures** - multi-minute with
no assertion output is infrastructure, fast with an assertion is the code, and
the first red's explanation must never carry over to the second.

**A local visual red cannot condemn a branch until master has been run the same
way.** `bin/qtest --changed` on #511 went red pointing at
`services/fractional-cto` - a page the PR never touched - and the available
story was that its `postcss.config.js` edit had shifted CSS site-wide. It had
not: the edit only ADDS purgecss safelist entries, and safelisting more can only
preserve more CSS, never remove any. The check that settled it was the same
system test on clean `origin/master` on this macOS host: **34 runs, 6 failures,
8 of 77 screenshots mismatched.** The suite is red on master here, so the local
leg was unusable for the question. CI's native-Linux Screenshot Tests passed
#511 in 16m15s; CI was right and local was noise. Also worth knowing: the run
rewrites two of those baselines mid-run and then fails its own dirty-check, so
`git checkout -- test/fixtures/screenshots/` never converges.

**Half-wired page bundles are a real recurring class.** The guard exists because
`docs/workflows/new-page.md` step 6 has two halves - `bin/qtest` PAGE_TESTS and
the ownership map - and #511 did one. Anyone adding `themes/beaver/assets/css/pages/*.css`
does both or the guard reds.

**Repo hygiene:** #505 landed two Phase-1a verification PNGs (392KB) at the repo
root. Every other screenshot in this repo lives under `docs/projects/<project>/`.
Removed in #514. Binary at the repo root is the kind of thing the next session
copies because it looks sanctioned.

## 2026-08-21 - The course's "good positions" were an artifact; GA4 UI setup closed out

**Course discovery diagnosed, and the inherited premise is retracted.** Two
Expand Down
Loading