Skip to content

The release build's test environment is only exercised at a release cut #1633

Description

@InauguralPhysicist

The v0.44.0 Release run (37291483115) created the tag, then failed all three build jobs inside run_all_tests.sh. None of the failures was in the runtime; each was a dependency CI's test environments have and the bare release runners lacked: on Linux an unset TMPDIR (#1624) and no libSDL2 for the strict-shape backend controls; on both macOS jobs no timeout, so nine gfx example checks exited 127 and the gfx sanitizer gate refused to run. #1632 fixed it by copying the dev image's runtime packages and ci.yml's macOS toolchain step into release.yml.

That fix is a second hand-maintained copy of CI's dependency list. Dependency-gated suite sections fail, not skip, when a dependency is missing, so any new dependency CI gains will break the next cut the same way, and nothing runs release.yml's test step between cuts: the workflow only runs on a tag push or a dispatch that creates a tag. The breakage surfaces at the worst moment, after the tag exists, and costs a tag delete plus a re-dispatch.

Done when

  • release.yml's build-and-test steps can run WITHOUT creating a tag or a release (for example a workflow_call/dry-run path, or a merge_group/schedule job that reuses the same steps), and that path runs on every PR or merge-queue group that changes release.yml, .devcontainer/Dockerfile, ci.yml's dependency steps or tests/run_all_tests.sh.
  • The release test dependencies come from ONE source shared with CI (a script or composite action both call), not two copied lists; deleting a package from it fails a CI run, shown in the PR.
  • Planted fault: removing the timeout provisioning from the shared source turns that dry-run red on macOS (run linked in the PR).
  • Residual stated in the workflow comment: what the dry-run still cannot see (for example the attestation/upload job).

Found during the v0.44.0 cut; tag deleted (no release existed) and re-dispatched after #1632 as run 37296742928, which passed.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:ciSubsystem: cikind:gate-defectA gate, check or claim that measures less than it says

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions