Skip to content

Ship in-repo composites and MatrixCollapse.Auto - #100

Merged
russwyte merged 5 commits into
mainfrom
feat/composites-and-matrix-collapse-auto
Aug 12, 2026
Merged

Ship in-repo composites and MatrixCollapse.Auto#100
russwyte merged 5 commits into
mainfrom
feat/composites-and-matrix-collapse-auto

Conversation

@russwyte

@russwyte russwyte commented Aug 12, 2026

Copy link
Copy Markdown
Contributor

Summary

Closes the M12a plan: modularize generated CI so large Graph / multi-target workflows stay short and reviewable without consumers hand-tuning collapse flags.

Generated CI got huge for three structural reasons: the planner inlined the same checkout/JDK/sbt/cache bootstrap on every sbt job; Graph × deploy targets defaulted to one job per leg with no auto matrix collapse; and AWS packs expanded OIDC+ECR pairs per destination into every docker job. Matrix collapse already existed but could not fold typical deploys when GitHub Environment names differ from target ids (environment == target name was required).

Both tracks, on by default

  1. In-repo composites under .github/actions/zipx-* (same contract as ci.yml: generated, drift-checked, do not hand-edit). Nested third-party uses: stay SHA-pinned; Dependabot / zipxActionsPull scan composites too. Always used for shared bootstrap and AWS login (not a consumer flag). Org-published actions were deliberately not chosen; local ./.github/actions/… matches “the build generates CI.”

  2. MatrixCollapse.Auto as the default (replacing implicit Off): collapse when legs are structurally safe (simple matrix or matrix.include); if unsafe, emit expanded jobs and do not fail generate. Escape hatches: Off (force expanded), Strict (fail generate when collapse was wanted and impossible), Coarse (also drop same-capability Graph needs). Cascade remains capability override → plan map → Auto.

What shipped

Piece Behavior
zipx-sbt-setup JDK → sbt → optional Node → cache-epoch → actions/cache; LocalDir jobs call it after a workflow-level actions/checkout (GHA cannot resolve local composites before the workspace exists)
zipx-aws-login OIDC (+ optional ECR) with env-key inputs so multi-dest docker is N× one uses:, not 2N inlined steps
matrix.include Target fan-out can collapse when Environments / env values differ but structure matches
Generate / check Writes and diffs ci.yml and .github/actions/**/action.yml
Pins Sync / pull scan nested uses: inside composites
Docs Overview / Quick start / Why zipx / Capabilities / Action pins / Matrix collapse lead with this surface

zipx-import-pgp (Central) stayed P2 and is not in this PR.

Size impact (in-repo only)

Artifact Before After
Dogfood .github/workflows/ci.yml ~290 lines ~164 lines (plus short composites; checkout is now outside the composite)
examples/monorepo ci.yml ~786 lines ~307 lines

(Job counts drop with Auto collapse on Graph / target fan-out; composites shrink each remaining job.)

Guardrails kept

  • Topology still owned by zipx; composites are boilerplate factoring, not a new extension language.
  • Auto never drops Graph needs (Coarse-only) and never invents collapse when env keys / conditions diverge.
  • Strict and contradictory Gate∩condition still refuse; Auto prefers expand over refuse when collapse is impossible.
  • Local composites never embed checkout: every job that uses: ./.github/actions/zipx-sbt-setup emits actions/checkout first (regression: ActionPinsSpec).

Test plan

  • sbt "core/testFull; aws/testFull" (composites + MatrixCollapse Auto/Off/Strict/Coarse × Graph/Aggregate/Layer/include property suites)
  • sbt scalafmtCheckAll
  • Regenerated dogfood + monorepo; composites committed alongside ci.yml
  • Specular narrative + Matrix collapse lab (Auto expand vs Strict generate gate) under docsDev
  • Regression: checkout precedes every local zipx-sbt-setup uses:
  • CI green on this PR (zipxWorkflowCheck, Aggregate test, docs)
  • Consumers with Graph + required checks: note matrix job name renames (image apiimage (api)); use capability Off during cutover if needed

Keep generated CI short and reviewable: zipx-sbt-setup / zipx-aws-login composites plus Auto collapse (with Off/Strict/Coarse escapes), dogfood and monorepo regen, and docs that lead with the current surface.
GHA resolves uses: ./.github/actions/… from the workspace before the composite runs, so checkout cannot live inside the composite. Emit it as the prior workflow step and assert that ordering in ActionPinsSpec.
@russwyte

Copy link
Copy Markdown
Contributor Author

CI fix: local composites cannot include checkout. GitHub failed with `Can't find action.yml under .github/actions/zipx-sbt-setup` because the first step was `uses: ./.github/actions/…` with an empty workspace.

Fix: checkout is a prior workflow step; `zipx-sbt-setup` starts at JDK. Regression test: `checkout precedes every local zipx-sbt-setup uses`. Dogfood + monorepo regenerated.

When MatrixCollapse.Auto cannot collapse, needs must fan out over the
per-module job ids the emitter actually writes. Update Central/GHP
property suites and Specular examples for composites.
Scripted still locks Graph collapse Off where asserts name per-module
jobs; product Auto coverage stays in MatrixCollapseSpec.
@russwyte
russwyte merged commit a497173 into main Aug 12, 2026
6 checks passed
@russwyte
russwyte deleted the feat/composites-and-matrix-collapse-auto branch August 12, 2026 13:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant