Skip to content

ci: deduplicate Scala compilation across CI jobs via a prebuild + artifact handoff #6318

Description

@Yicong-Huang

Task Summary

The four sbt-based CI jobs — amber, amber-integration, platform, and platform-integration — each compile the shared common-module graph (DAO, WorkflowCore, WorkflowOperator, Auth, Config, …) from scratch. Across their matrices that is up to 14 runner legs redoing the same Scala build on every run, wasting runner-minutes and risking build inconsistency.

Proposal: add a single scala-build prebuild job that compiles the whole Scala world once (main + test scopes), runs the cross-cutting scalafmt/scalafix lints, and builds all dists, then uploads the compiled target/ tree and the dist zips as artifacts. The downstream Scala jobs download and reuse them instead of recompiling — amber / amber-integration (Linux) / platform restore the compiled state for their jacoco/test, and platform-integration pulls just the dists for smoke-boot.

Design constraints:

  • Respect the existing label gate: the prebuild runs only when at least one Scala stack is requested (union of run_amber / run_amber_integration / run_platform / run_platform_integration), so docs-only / frontend-only PRs never trigger it.
  • macOS amber-integration keeps self-compiling — Zinc analysis embeds OS-specific absolute paths and is not portable across runners.
  • Primary goal is deduplicating compilation / cutting runner-minutes; wall-clock is expected to stay roughly flat and will be measured against a baseline before committing to the approach.

Task Type

  • Refactor / Cleanup
  • DevOps / Deployment / CI
  • Testing / QA
  • Documentation
  • Performance
  • Other

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

Fields

No fields configured for Task.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions