Repository navigation
CI: run the full main lane on demand (workflow_dispatch), writing nothing of main's - #1785
Conversation
…hing of main's A long-lived branch (the Phase 2 integration branch, draft PR #1776) had never run the heavy workers, which run only in the merge queue and on main pushes. `gh workflow run ci.yml --ref <branch>` now runs them. - ci.yml triggers on workflow_dispatch. A dispatch is not a pull request, so every job runs as on a push to main. - Concurrency: a dispatch gets its own group (ci-dispatch-<ref>), so it never cancels a push, PR or merge-queue run on any ref, main included, and none of those cancels it. - Outward writes, audited job by job: dev-image is the only ci.yml job with a write permission (packages: write). It advanced ci-main on any `push` event (safe only because push is filtered to main) and now does so only on a push to refs/heads/main. A dispatch pushes only its own ci-<sha>; the buildx gha cache is ref-scoped by GitHub, and artifacts stay in the run. - gate-selftests: a dispatch carries no base sha, and `selftests.sh --changed ''` selects 0 rows (measured), a silent skip. An empty base now falls back to origin/main, so a dispatch selects what the ref changes against main, as its PR would. That step's waiver is re-pinned (the scope/detect pin is unchanged). - tools/ci_tier_check.sh gains [dispatch-write]: every ci.yml job holding a write permission, its own or the workflow's (missing counts as the repository default), must match its reviewed pin in WRITERS. Action `uses:` pins are excluded, so Dependabot bumps do not reopen the review. Proven once with planted copies, recorded in the PR (PRINCIPLES section 5, no permanent self-test). Each planted violation went red: a new writer, the ci-main gate loosened, the workflow permissions removed, a stale pin, and the edited select step's stale waiver. A Dependabot-style bump and the tree stayed green. - docs/CI.md: when to dispatch, what runs, the commands, what it never touches, concurrency, where its checks show, and the new rule. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KC99CmwatKssQYggkBCgwF
…'s ci.yml Measured: a dispatch of ci.yml on ci-dispatch (whose copy declares the trigger) was accepted before merge (runs 38677615755, cancelled by the newer 38677618300 in the same per-ref dispatch group), while one on the stale branch cloud-1159 (no trigger in its copy) was refused with HTTP 422, "Workflow does not have 'workflow_dispatch' trigger". A long-lived branch must take this change (rebase or merge main) before its first dispatch. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KC99CmwatKssQYggkBCgwF
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: f39bff9cf2
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| # writer in this file is `dev-image`, which advances the rolling ci-main tag | ||
| # only on a push to refs/heads/main, and tools/ci_tier_check.sh pins every | ||
| # job holding a write permission to its reviewed text. | ||
| workflow_dispatch: |
There was a problem hiding this comment.
Run the writer audit before dispatch writes
When a long-lived branch modifies dev-image so that it accidentally publishes ci-main, dispatching that branch can overwrite the shared image before [dispatch-write] reports the unreviewed hash: dev-image performs the push at lines 161–169, while the audit runs inside gate-selftests, which explicitly waits for dev-image via needs at line 197. This is a real arbitrary-ref path—gh workflow run --help defines --ref as the branch or tag containing the workflow version—so the pin currently detects corruption only after it occurs. Put the audit in a non-writing prerequisite and require every writing job to depend on it before enabling branch dispatches.
Useful? React with 👍 / 👎.
| - **Where the results show.** The dispatch reports its checks on the branch's | ||
| head commit. If the branch has a pull request, those checks appear on it too, | ||
| under the same names as the pull-request run's, and a failure there is a real | ||
| failure of that check on that commit. |
There was a problem hiding this comment.
Correct where dispatch checks are displayed
When the dispatched branch has an open pull request, GitHub explicitly excludes checks from workflow_dispatch runs from the pull request's checks section and from branch-ruleset evaluation, contrary to this claim (GitHub's required-check troubleshooting documentation). An operator following this section may therefore look at the PR, fail to see the heavy-lane failure, and assume the dispatch did not run or passed; direct them to the Actions run or the gh run list result instead.
Useful? React with 👍 / 👎.
| # the older one. | ||
| concurrency: | ||
| group: ci-${{ github.event.pull_request.number || github.ref }} | ||
| group: ci-${{ github.event_name == 'workflow_dispatch' && format('dispatch-{0}', github.ref) || github.event.pull_request.number || github.ref }} |
There was a problem hiding this comment.
Isolate the dispatch build cache from main pushes
When a dispatch on main overlaps a normal push to main, these distinct concurrency groups allow both dev-image jobs to run simultaneously, but both export cache-to: type=gha,mode=max under the same branch and the backend's default buildkit scope. Docker documents that builds sharing a scope overwrite the preceding cache (GitHub Actions cache backend), so an older dispatch that finishes last can replace the newer push's cache and make subsequent main-lane image builds cold; concurrent exporters also contend for the same cache object. Give dispatches a separate cache scope or serialize the cache-writing job even if the rest of the workflows remain independent.
Useful? React with 👍 / 👎.
…w of #1785) The altitude correction (finding 13): [dispatch-write] pinned the writing job's text, so a dangerous edit cleared with the same one-line re-pin as a harmless one (plant A, re-pinned with the printed 187c693a40a0ae19, was green), while action swaps, a PAT and workflow-level defaults/BASH_ENV never reached the pin. WRITERS and its second hash recipe are deleted, and tools/ci_tier_check.sh checks what the writes do: - [ci-main]: the step that computes the image push's tags is RUN, under bash -e, across event x ref x tip (52 runs, with fork PRs and an unreadable tip). ci-main must appear exactly on a push to refs/heads/main at main's tip. on.push must be exactly [main]; each push takes its tags from that step; a custom shell, BASH_ENV/ENV/GITHUB_* overrides and unevaluable expressions are refused. - [writer-actions]: a job that can write uses only checkout, buildx, login and build-push (compared without @ref), no ./ or docker:// action, and no script but the tag step. - [secrets]: ci.yml reads no secret but GITHUB_TOKEN, in dot or bracket syntax, and passes none to a reusable workflow. - [event-condition] is now an ALLOWLIST of event-invariant github fields; ref_protected, ['ref_protected'] and workflow_ref were green before. 19 plants, recorded in the PR, before (the f39bff9 checker) and after: plant B (timeout 30 -> 31) is now green, and everything that slipped past the pin is red. The matrix found finding 8 in the tag step on its first run: a re-run of an older main push moved ci-main back. ci-main now moves only when `git ls-remote origin refs/heads/main` equals this SHA (not RUN_ATTEMPT), and an unreadable tip fails loudly. Also: - bench measures a dispatch against the branch's merge-base with main (the queue's measure), not main's tip; - dispatch groups never cancel in progress: a newer dispatch waits, a third replaces the waiting one before any job starts, so no cancelled dispatch leaves aggregators failing under required names; - the README badge reads push runs only (?branch=main&event=push); - docs/CI.md: a dispatch runs the branch's own ci.yml (its PR run already could); what a dispatch writes, on main too; one run per group, case-insensitive group names, a later push not stopping a dispatch; the rollup leaving dispatch suites out (measured on 82522fc: 49 check runs, no rollup); the rule list; the dispatch self-test base; - precheck's comment and SKIP message cover the dispatch base. Closes #1787 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KC99CmwatKssQYggkBCgwF
tools/ci_tier_check.sh now RUNS the dev-image tag step, and precheck runs
the checker, so on macOS's bash 3.2 `${OWNER,,}` failed 51 of the 52 matrix
runs with "bad substitution" (measured with a bash 3.2 build). The step
lowercases with tr instead; the checker and all 19 plants now give the same
result under bash 3.2 and bash 5.
docs/CI.md: the [event-condition] allowlist is the github fields whose
value does not depend on the event (github.token is per run, so "the same
in every run" was not exact).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KC99CmwatKssQYggkBCgwF
Self-review of the first property version (e0b6e01) found 10 edits that write ci-main, or redirect what the matrix models, while it stayed green. Each is now red, with the plant recorded in the PR (32 plants, the same table under bash 3.2 and bash 5): - [writer-actions]: each allowlisted action takes only the inputs vetted for it (INPUTS); build-push-action's `outputs:` or a registry `cache-to:` pushes any name, and a registry cache is shared across refs (O1-O3); checkout `repository:` moves origin away from this repo (O4). - [ci-main]: the matrix merges workflow, job and step env as the runner does (a job-level env was ignored), and refuses env that redirects bash or git: PATH, GIT_*, BASH_FUNC_*, SHELLOPTS, BASHOPTS, LD_* (J2, J3), plus `working-directory:` (W1). It reads tags= in any case with quotes stripped (Q1, U4) and refuses a tags<< output (H1). Controls stay green: a new platforms: value (C4), a plain job env (C5). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KC99CmwatKssQYggkBCgwF
|
Open on purpose: the fix round for the /code-review findings is in progress. The main change is that the writer text pin becomes property checks (finding 13); it also corrects the dispatch bench baseline and stops a cancelled dispatch from leaving failures under required check names. This merges through the queue after its final CI run and the dispatch proofs are in. |
Brings in #1769, #1777 (the observer oracle's floor as a required CI check), #1770, #1786 and #1785 (the on-demand full main lane). One conflict, tools/selftests.txt: both sides appended rows at the end of the registry. Resolved by keeping both: main's observer_oracle/plants.py row, then this branch's two #1758 Phase 2 rows with their comment. `tools/selftests.sh --changed origin/main --list` accepts the merged file (enrolment 46/46). The Makefile merged cleanly. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KC99CmwatKssQYggkBCgwF
Closes #1787
What this does
ci.ymltriggers onworkflow_dispatch, sogh workflow run ci.yml --ref <branch>runs the full main lane on a long-lived branch before it is queued: the ASan shards, both TSan variants, native macOS, valgrind, bench, the extension suites and the db lane. A dispatch is not a pull request, so every job runs as on a push to main. The Phase 2 integration branch (draft PR #1776) had never run those lanes.Scope, exactly. A dispatch runs the dispatched ref's own
ci.yml, and its owntools/ci_tier_check.sh. The properties below hold formain's text. A branch that editsci.ymlcan change them, exactly as its pull-request run already could, since a pull request also runs the branch's own workflow file. A dispatch adds no capability that pushing a branch did not already give.What
main'sci.ymlwrites. The only job that can write isbuild dev/ci image.ci-<sha>image tagci-mainrefs/heads/mainwhose SHA ismain's tip (git ls-remote); never a dispatch from any ref, the queue, or a re-run of an older pushmainwritesmain's, as a push doesci-mainis the one name written here thatmainowns. A dispatch onmainalso adds runs tomain's commit, but the README badge now reads push runs only (badge.svg?branch=main&event=push).Property checks replace the text pin (finding 13)
[dispatch-write]pinned the writing job's text, so a dangerous edit cleared with the same one-line re-pin as a harmless one.WRITERSand its second hash recipe are deleted.tools/ci_tier_check.shnow checks what the writes do:[ci-main]runs the step that computes the image push's tags, underbash -e, with the workflow, job and step env merged as the runner merges them. It covers:main, a branch, a pull request's merge ref, the queue's ref;main's tip, is not, or the tip cannot be read;That is 52 runs.
ci-mainmust appear exactly on a push torefs/heads/mainatmain's tip. Further:on.pushmust be exactlybranches: [main].defaults:,shell:,working-directory:);PATH,BASH_ENV,ENV,SHELLOPTS,BASHOPTS,GITHUB_*,GIT_*,BASH_FUNC_*,LD_*);tags<<output.[writer-actions]: a job that can write uses onlyactions/checkout,docker/setup-buildx-action,docker/login-actionanddocker/build-push-action.INPUTS). So there is nooutputs:, andcache-to:istype=ghaonly: anoutputs:or a registry cache can push any name, and a registry cache is shared across refs.@ref, so Dependabot bumps stay green../ordocker://action, and no reusable-workflow call.[secrets]:ci.ymlreads no secret butsecrets.GITHUB_TOKEN(dot or bracket syntax), and passes none to a reusable workflow.[event-condition]is now an allowlist of thegithubfields whose value does not depend on the event (repository*,*_url,workspace,workflow,job,action,token,retention_days). Any other field, or a baregithub, is red.On its first run, the matrix found finding 8 in the tag step: a re-run of an older main push moved
ci-mainback. The step now addsci-mainonly whengit ls-remote origin refs/heads/mainequals this SHA, does not gate onRUN_ATTEMPT, and fails loudly when the tip is unreadable.A self-review of that first property version (
e0b6e015) then found 10 edits that wroteci-main, or redirected what the matrix models, while it stayed green: plants O1–O4, J2, J3, W1, H1, Q1 and U4 below. The final checker (f27fe34f) closes them.PLANTS: each rule proven once (PRINCIPLES §5)
The harness copies the workflows, applies one edit and runs
ci_tier_check.shon the copy (CI_TIER_WF_DIR). "n/a" means the edit's text does not exist inf39bff9c. The pin went red on any edit to the writing job, re-pinnable as row A shows. The table is identical under bash 3.2 and bash 5.f39bff9cpine0b6e015dispatch-write, but green once the hash it printed (187c693a40a0ae19) is pasted intoWRITERSci-mainci-maindispatch-writeci-mainci-mainci-mainci-mainci-mainci-mainci-mainci-mainci-main,writer-actionsci-main,writer-actionswriter-actionswriter-actionsdispatch-writewriter-actionswriter-actionsdispatch-writewriter-actionswriter-actionssecretssecretssecretssecretsevent-conditionevent-conditionevent-conditionevent-conditionevent-conditionevent-conditiongithub.ref ==(red in all three)event-conditionevent-conditionevent-conditionwriter-actionswriter-actionswriter-actionswriter-actionsevent-conditionci-main,event-conditionci-mainci-mainci-mainci-mainci-mainci-mainSize of
tools/ci_tier_check.sh: 221 lines on main, 241 with the text pin (f39bff9c), 340 now. The property checks cost about 100 lines more than the pin they replace.Other changes
main, which is what the queue measures, notmain's tip. The step prints both.ci-dispatch-<ref>) never cancel in progress: a newer dispatch waits, and a third replaces the waiting one before it starts any job. The text now says:gh run cancel, which leaves the cancelled run's aggregators failing on that commit, as documented).workflow_dispatchsuites out (measured on82522fc4: 49 dispatch check runs, rollupnull), so a dispatch neither appears among a PR's checks nor gates it.origin/mainon a dispatch.${OWNER,,}failed 51 of the 52 matrix runs under macOS's bash 3.2 ("bad substitution", measured with a bash 3.2 build). The step lowercases withtrinstead.Proof on CI
Runs on this branch, in order:
3868800332568ed2aae3868801536768ed2aae3868802762568ed2aae38688770795e0b6e01538690563225f27fe34f38688769806e0b6e01538690560369f27fe34f(i) A dispatch does real work. In d1, and with the same totals in d5 on the final head:
38676687632), 3004/3004 and 1490/1490, with shard receipts 3 of 3.deferred to merge queue.Bench baseline lines from d1 (d5 printed the same two lines):
mainhas not moved since this branch forked, so on CI the merge-base equals the tip. The distinct case is proven locally. The step text extracted fromci.yml(cut beforegit worktree add/./build.sh) runs in a checkout made asactions/checkoutmakes it (git init,remote add, a depth-1 fetch), against a synthetic origin whosemainmoved two commits past the fork:origin/main;base_sha.(ii) Dispatches in a row leave no failure under a required name. d2 and d3 were replaced before starting any job. On
68ed2aae, the dispatch suites carry 26 check runs with 0 failures (d1), plus 0 and 0 (d2, d3).For contrast, before this fix (
cancel-in-progress: true), the dispatch that the next one cancelled in progress (38677615755, on82522fc4) left FAILURE under 4 required names:asan + ubsan (full suite),db extension (postgres service),extensions (…)andmacos / macos-latest.(iii) This PR's run on the final head:
38690560369onf27fe34fsucceeded, and the commit's rollup is SUCCESS.A ref whose
ci.ymlpredates the trigger is refused (measured earlier on stalecloud-1159):HTTP 422 "Workflow does not have 'workflow_dispatch' trigger". Sophase2-numbersmust take this change (rebase or mergemain) before its first dispatch.After merge
git ls-remoteruns only on a push tomain, so the first push after this merges is its first real run.build dev/ci imagejob to push:ci-main; the build-push log names the tags.::error::cannot read main's tip.phase2-numbersneedsmain(merge or rebase) before its first dispatch. Then rungh workflow run ci.yml --ref phase2-numbers.The 15 review findings
[writer-actions]: actions and their inputs on an allowlist, identities compared without@ref; no./ordocker://(plants U1, L, D, O1–O4)ci.ymlci.ymlcommentcancel-in-progressis false for dispatch groups (proof ii, with the before measurement); the docs say results show in the run and the rollup leaves dispatch suites out[secrets](plants U2, S)[event-condition]allowlist (plants E1–E3)ci-mainon push + main + tip;ci-<sha>from every non-fork refci-mainls-remotetip guard, notRUN_ATTEMPT(plant 8)defaultsandBASH_ENV[ci-main], with the rest of the env that redirects bash or git (plants 9a, 9b, J2, J3, W1)badge.svg?branch=main&event=push; the claim is limited to non-main refs, and what a dispatch on main writes is statedgh run cancelCloses #1787, which carries the checklistWRITERS/writer_pindeleted; the one remaining recipe (WAIVE) prints the new hash when a waived script changesGates run locally (no compile)
ci_tier_check.sh: OK under bash 5 and bash 3.2, including its self-test; shellcheck is clean.workflow_yaml_check: OK.precheck.sh: 16 passed, 0 failed, 3 skipped;--selftest10/10.asan_receipts_check --selftest: 3/3.🤖 Generated with Claude Code
https://claude.ai/code/session_01KC99CmwatKssQYggkBCgwF