Skip to content

Publish a release without waiting for macOS, compile weft once per release - #47

Merged
qfeuilla merged 1 commit into
mainfrom
release-speed
Oct 4, 2026
Merged

qfeuilla merged 1 commit into
mainfrom
release-speed

Conversation

@qfeuilla

@qfeuilla qfeuilla commented Oct 4, 2026

Copy link
Copy Markdown
Contributor

What changes

Release speed

  • The two Linux legs now build the CLI, the runtime and the shared images in one job. The runtime image copies in the weft-runtime binary that job just built (weft build-images --runtime-binary) instead of compiling it again inside Docker. That used to be about 13 minutes with no cache. The image job no longer compiles a debug CLI either.
  • Before an image built from a prebuilt binary can be pushed, the binary is started inside it once. A binary that needs a newer glibc than Debian bookworm is refused and the image is removed. That is why the Linux legs build on Ubuntu 22.04.
  • The macOS binaries are their own job. The latest release and its manifest go out once Linux and the images are done, and release-macos adds the Mac binaries afterwards. The Intel runner took over 50 minutes cold today and held up the whole release.

Caches

  • Only main saves Rust caches; PR runs read main's copy and save nothing.
  • After each release, prune-caches keeps the newest copy of each job's cache and deletes the older ones.

Forks and project deploys

  • manifest.json now names the source tree. setup.sh, the GCP install and the project deploy workflow match on that tree. A fork that merged weft without changes gets a different commit with the same files, and now downloads instead of compiling.
  • scripts/release-cli.sh fetches the release's CLI for a checkout. If upstream's release for that source is still building, it waits for it instead of compiling.
  • When a fork does have to build, it keeps its dependency cache between runs, so after an upgrade only weft's crates compile. The install also hands its own runtime binary to the image build, and now runs on Ubuntu 22.04 so that binary runs in the image.

Fresh-install upload failure

  • On a fresh install, the standard-library preload failed with "internal storage error". Google had not yet applied the install's grant that lets the runtime sign upload links.
  • Signing now waits up to 10 minutes for that grant, as long as this process has never signed successfully. It uses the existing IAM wait helper, which now takes a waiting limit per kind of change.
  • The 500 message now says the cause is in the broker's log.

Crypto

  • The AWS SDK now uses ring for TLS like every other client; aws-lc-sys and aws-lc-rs are gone.

Tested

  • Workspace clippy is clean.
  • Unit tests for weft-cli, weft-platform-traits, weft-core, weft-platform-gcp and weft-broker pass.
  • The e2e tests storage::fetched_file_is_stored_and_downloadable and storage_multipart::multipart_uploads_round_trip_across_sizes pass.
  • A runtime image built from a prebuilt binary: no compile; the binary arrives executable. A binary built on Ubuntu 24.04 is refused, as intended.
  • actionlint is clean on the three workflows and the rendered deploy template; shellcheck is clean on the new script.

Not tested yet:

  • The release workflow itself only runs once this is on main. The new linux and macos jobs start with empty caches, so that first release compiles everything once.
  • The first release after this merge publishes before any manifest names a source tree, so forks and setup.sh compile once during that run.
  • The signing wait can only be proven on a fresh GCP install.

…lease, and let a fresh install's first upload wait for its grant

- The release's Linux legs build the CLI, the runtime and the images in one job; the runtime image takes the runtime binary that job built (`weft build-images --runtime-binary`, checked by starting it in the image) instead of compiling it again in Docker.
- The macOS binaries are their own job and join the latest release afterwards, so the release, the manifest and every fork waiting on it no longer wait for the Intel runner.
- The manifest names the source tree; setup.sh, the GCP install and the project deploy workflow match on it, so a fork that merged weft unchanged downloads instead of compiling.
- scripts/release-cli.sh fetches the release's CLI for a checkout, waiting while upstream's release for that source is still building; a fork that has to build keeps its dependencies cached between runs.
- Only main saves Rust caches; after each release one copy per job is kept and the rest deleted.
- The AWS SDK uses ring like every other TLS client; aws-lc is gone.
- Signing a storage link waits up to ten minutes for a new install's grant to take effect, through the IAM settle helper, now given a patience per change.
@qfeuilla
qfeuilla merged commit 4f6e130 into main Oct 4, 2026
11 checks passed
@qfeuilla
qfeuilla deleted the release-speed branch October 4, 2026 09:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant