Run the gate on pull requests, so Dependabot is not merged unread - #150
Merged
Conversation
Every workflow here is workflow_dispatch only, on reasoning recorded in windows-render.yml and windows-runtime.yml: Windows runners cost more, and evidence is worth more at an exact SHA than continuously. That holds for Windows evidence. It does not hold for the ordinary gate, because it assumes a human ran the gate before pushing. Dependabot is the counter-example. Three of its pull requests are open, one since 27 July, and `gh pr checks` reports "no checks reported" on all three. There is no signal at all about whether they build, so the choice is merge unread or leave them to rot, and both have been happening. The workflow runs on pull_request and on dispatch, on Linux, where runners are free for a public repository. It runs `npm test`, and then the three release steps that live outside it: the share contract, the MCPB build, and the packed smoke. Those three are not extras. A forty-third tool once landed with the in-test contract updated and the share contract and packed smoke stale, so `npm run smoke:mcpb` was broken on master and nobody knew until release. The Node matrix is 20.19 and 22.12 — the two floors of the declared `^20.19.0 || >=22.12.0` range. Nothing verified the 20 half before now; the maintainer's machine and both Windows jobs run 22. Either the claim holds or the range wants narrowing, and this decides which. Actions are pinned by commit SHA to match the Windows workflows. Concurrency cancels a superseded run, since its answer is stale anyway. Boundary: this is a Linux runner. The darwin-gated suites — the native canvas mechanism campaigns, the macOS supervisor and its exec budget, the system-renderer fallback — skip here by design and stay macOS-only evidence. Green here is a floor beneath incoming changes, not a substitute for the maintainer's local qualification run. Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
The first run of the pull-request gate found three things, in two classes. A real defect. comparison-observations asserted that a renderer whose fingerprint does not match the reference must produce at least one pixel digest that differs from the frozen ones. The Linux runner reproduced every frozen digest byte for byte while its fingerprint mismatched, and the test called that a failure. The fingerprint binds platform, Node and N-API alongside the two library versions, so it is strictly narrower than the rasterization it identifies, and identical pixels off the reference renderer are perfectly possible. Cross-platform determinism should not read as a regression. The sibling comparison-reference-baseline already branches on whether raw visuals reproduced instead of requiring that they do not; this now matches it, and still asserts what genuinely does not depend on the renderer, which is which facets carry a digest at all. The test name changes with it: the digests are authoritative on the reference renderer, not exclusive to it. Two hardware measurements. deep-malformed-campaign asserts a 15 s per-call budget and landed at 15001-15016 ms; fuzz-malformed-pdfs runs one serial loop over every fixture, tool and target mode against a 180 s per-test bound. Neither indicates a product defect: the tools bound themselves, and a two-core shared runner is simply slower than the machine the budgets were calibrated against. Raising either bound to fit the slowest plausible host would blunt the only assertion that catches an unbounded parse or a hang, so instead they stand down under PDF_TOOLS_TIMING_CALIBRATED=skip and stay maintainer-hardware evidence. They are skipped rather than filtered out, so a run says plainly that 120 tests did not execute. Result either way: both Node versions reported an identical 2335 passed and 160 skipped, so the declared ^20.19.0 || >=22.12.0 range holds on both floors and wants no narrowing. Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The gap
Every workflow in this repository is
workflow_dispatchonly, on the reasoning recorded inwindows-render.ymlandwindows-runtime.yml: Windows runners bill higher, and evidence is worth more at an exact SHA than continuously. That is sound for Windows evidence. It does not carry to the ordinary gate, because it assumes a human ran the gate before pushing.Dependabot is the counter-example. Three of its PRs are open — #129, #128, and #48 since 27 July — and
gh pr checksreports "no checks reported" on all three. There is no signal about whether they even build, so the only options are merge unread or leave them to rot. Both have been happening.What this runs
Linux, where runners are free for a public repository.
npm test, then the three release steps that live outside it:npm run test:contract:sharenpm run build:mcpbnpm run smoke:mcpbThose are not extras. A forty-third tool once landed with the in-test contract updated and the share contract and packed smoke stale, so
npm run smoke:mcpbwas broken on master and nobody knew until release. This is the class of defect the workflow exists to catch.Node matrix
20.19and22.12— the two floors of the declared^20.19.0 || >=22.12.0range. Nothing verified the 20 half before now; the maintainer's machine and both Windows jobs run 22. Either the claim holds or the range wants narrowing. This PR's own run decides which, which is the intended first result.fail-fast: false, because which of the two failed is the entire signal.Boundary
This is a Linux runner. The darwin-gated suites — native-canvas mechanism campaigns, the macOS supervisor and its exec budget, the system-renderer fallback — skip here by design and stay macOS-only evidence. Green here is a floor beneath incoming changes, not a substitute for the maintainer's local qualification run.
Incidentally, the macOS supervisor suite failing deterministically under CPU contention is a reason to want it skipped on a shared runner rather than a reason to add one.
Verification
The first run of this workflow is on this PR. Actions pinned by commit SHA to match the Windows workflows; concurrency cancels superseded runs.
🤖 Generated with Claude Code