ci(cache): key the Windows target archive by dependencies, not commit - #343
Merged
Merged
Conversation
Every main run saved a fresh copy of the Rust Windows target archive. At about 19.5 GB it took about 21 minutes per run to upload, and two copies at once held a large share of the Actions cache budget, which pushed other lanes' entries out. The cache action gains an archive-key input. The default, revision, keeps the per-commit key. dependencies drops the commit, so a successful main run saves one entry per lockfile set; later commits restore it as an exact hit, skip the save, and rebuild the workspace crates that changed since, which R2 serves whenever an earlier main run compiled the same inputs. Test binaries, build scripts, proc-macro crates, and clippy checks still come from the archive, now as fresh as the commit that saved it, so each crate changed since reruns that work for its dependents. A failed run that saves on failure uses its revision key instead. The dependency key prefix-matches that entry without an exact hit, so the next successful run restores and completes it, then saves the lockfile set's entry. A partial archive still seeds recovery but never becomes the exact hit that later runs stop saving over. actions/cache prefix-matches the primary key, so the first dependency-scoped restore still finds today's per-commit entries. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Warning Review limit reached
This review includes 5 billable files and costs up to $1.25. Or wait 43 minutes for your next included review. View limit detailsLimit details: You’ve used the included review currently available. Review configuration: ⚙️ Run configuration
📒 Files selected for processing (5)
Comment |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this changes
The Rust Windows job stops saving a new 19.5 GB target archive on every
maincommit. Its archive is now keyed by the lockfiles instead of the commit, so a successfulmainrun saves one entry per dependency set, and every later commit restores that entry as an exact hit and skips the save. The workspace crates that changed since the entry was saved recompile, and the shared R2 compiler cache serves each of those compiles that an earliermainrun already performed.The cache action gains an
archive-keyinput for this.revision(the default) keeps today's per-commit key for every other lane;dependenciesdrops the commit, and only Rust Windows uses it.Why
On the last measured
mainruns the Windows job spent about 21 to 24 minutes uploading its archive, which is 19,546,526,845 bytes. Each commit's copy is a new entry, so two of them sit in the Actions cache at once (18.2 GiB each against a 50 GB budget), evicting other lanes' entries; the Windows Cargo registry entry was evicted within hours of being restored. Now that R2 holds compiled workspace crates, a per-commit archive mostly re-uploads dependencies that have not changed.How it works
mainrun after mergemaincommits, same lockfilesmainrun (the lane saves on failure)The failure row keeps the Windows lane's recovery property. A failed run's archive is saved under its revision key, which the lockfile-set key prefix-matches without being an exact hit. The next successful run restores that partial archive, completes it, and saves the real entry, so a partial archive never becomes the exact hit that later runs stop saving over.
Verification
just verifypasses locally (Rust fmt + lint + test) (not applicable: no Rust changes)just denypasses (required for dependency or license changes)just ui-testandjust ui-buildpass (required forcrates/hypercolor-ui/)just sdk-lint,just sdk-check, andjust sdk-buildpass (required forsdk/)just python-verifypasses (required forpython/)just compat-checkpasses (required fordata/drivers/vendors/*.toml)just docs-buildpasses (required for docs or README changes)cd docs && zola checkpasses (required for docs link/content changes)scripts/orpackaging/)just e2e-buildpasses with the normal Servo stack (required for daemon/UI/effect integration changes)just e2e-build-cpupasses when validating the CPU smoke fallbackjust e2epasses against the Servo stack (required for end-to-end behavior changes; starts daemon/browser)node --test .github/actions/rust-build-cache/*.test.mjswith sccache 0.17.0 (the CI pin): 30 passed. New cases cover the dependency key ignoring the commit and following the lockfiles, the failure key being a non-exact prefix match, the save step choosing the failure key on a failed run, and the Windows job's wiring. An independent review mutated the scope logic, the failure-save rule, and the wiring a dozen ways; every mutation failed a test. The docs change is underdocs/development/, outside the Zola site.The real proof is the first two
mainruns after merge: the first should save the lockfile-set entry once, and the second should report an exact hit and skip "Save build artifacts and compiler cache".Notes for reviewers
Pull requests and tags now restore an archive as old as the lockfile set rather than the newest
maincommit, so they rebuild a little more of the workspace and its uncacheable work (test binaries, build scripts, proc-macro crates, clippy's dependency checks). The current per-commit archive already rebuilds most of the workspace on every run: a CI-onlymainrun still rebuilt 12 workspace packages, includinghypercolor-core, and compiled 329 test binaries. Lockfile changes also reset the drift; 18 of the last 122maincommits changed one.🤖 Generated with Claude Code