Skip to content

Cache CI build outputs - #3486

Open
wasphin wants to merge 3 commits into
apache:masterfrom
wasphin:optimize-ci-build-cache
Open

Cache CI build outputs#3486
wasphin wants to merge 3 commits into
apache:masterfrom
wasphin:optimize-ci-build-cache

Conversation

@wasphin

@wasphin wasphin commented Aug 24, 2026

Copy link
Copy Markdown
Member

What problem does this PR solve?

Issue Number: resolve

Problem Summary:
CI recompiles the same sources, downloads the same Bazel dependencies, and
runs four CMake configurations serially on every run, making build validation
unnecessarily slow.

What is changed and the side effects?

Changed:

  • Add a shared composite action that configures ccache for Make/CMake builds.
  • Cache Bazel repository downloads separately from Bazel build outputs.
  • Reuse caches in Linux and macOS build jobs.
  • Report ccache statistics so cache behavior can be verified from CI logs.
  • Run the four Linux CMake compiler/configuration combinations as independent
    matrix jobs.
  • Give each ccache job/configuration an unambiguous namespace so restore
    prefixes cannot select another job's cache.

Measured results:

  • Warm Make and CMake compilation jobs complete in about 2-6 minutes, compared
    with baseline averages of 16-21 minutes.
  • Cross-commit ccache restores reached 99.2%-99.6% lookup hit rates for the
    correctly matched compiler jobs.
  • Warm Bazel jobs complete in about 1-2 minutes, compared with baseline
    averages of 11-38 minutes.
  • The Linux and macOS workflows completed successfully after cache
    instrumentation was added.

Side effects:

  • Performance effects: Subsequent CI runs can reuse compiler outputs and Bazel
    downloads/build outputs. Ccache is bounded to 2 GiB. Bazel repository caches
    are shared per platform, while build caches remain separated by job. Bazel
    may reuse test results when all relevant inputs are unchanged, following its
    normal disk-cache semantics.

  • Breaking backward compatibility: None.


Check List:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant