Ship in-repo composites and MatrixCollapse.Auto - #100
Merged
Conversation
Keep generated CI short and reviewable: zipx-sbt-setup / zipx-aws-login composites plus Auto collapse (with Off/Strict/Coarse escapes), dogfood and monorepo regen, and docs that lead with the current surface.
GHA resolves uses: ./.github/actions/… from the workspace before the composite runs, so checkout cannot live inside the composite. Emit it as the prior workflow step and assert that ordering in ActionPinsSpec.
Contributor
Author
|
CI fix: local composites cannot include checkout. GitHub failed with `Can't find action.yml under .github/actions/zipx-sbt-setup` because the first step was `uses: ./.github/actions/…` with an empty workspace. Fix: checkout is a prior workflow step; `zipx-sbt-setup` starts at JDK. Regression test: `checkout precedes every local zipx-sbt-setup uses`. Dogfood + monorepo regenerated. |
When MatrixCollapse.Auto cannot collapse, needs must fan out over the per-module job ids the emitter actually writes. Update Central/GHP property suites and Specular examples for composites.
Scripted still locks Graph collapse Off where asserts name per-module jobs; product Auto coverage stays in MatrixCollapseSpec.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Closes the M12a plan: modularize generated CI so large Graph / multi-target workflows stay short and reviewable without consumers hand-tuning collapse flags.
Generated CI got huge for three structural reasons: the planner inlined the same checkout/JDK/sbt/cache bootstrap on every sbt job; Graph × deploy targets defaulted to one job per leg with no auto matrix collapse; and AWS packs expanded OIDC+ECR pairs per destination into every docker job. Matrix collapse already existed but could not fold typical deploys when GitHub Environment names differ from target ids (
environment == target namewas required).Both tracks, on by default
In-repo composites under
.github/actions/zipx-*(same contract asci.yml: generated, drift-checked, do not hand-edit). Nested third-partyuses:stay SHA-pinned; Dependabot /zipxActionsPullscan composites too. Always used for shared bootstrap and AWS login (not a consumer flag). Org-published actions were deliberately not chosen; local./.github/actions/…matches “the build generates CI.”MatrixCollapse.Autoas the default (replacing implicitOff): collapse when legs are structurally safe (simple matrix ormatrix.include); if unsafe, emit expanded jobs and do not fail generate. Escape hatches:Off(force expanded),Strict(fail generate when collapse was wanted and impossible),Coarse(also drop same-capability Graphneeds). Cascade remains capability override → plan map → Auto.What shipped
zipx-sbt-setupactions/cache; LocalDir jobs call it after a workflow-levelactions/checkout(GHA cannot resolve local composites before the workspace exists)zipx-aws-loginuses:, not 2N inlined stepsmatrix.includeci.ymland.github/actions/**/action.ymluses:inside compositeszipx-import-pgp(Central) stayed P2 and is not in this PR.Size impact (in-repo only)
.github/workflows/ci.ymlexamples/monorepoci.yml(Job counts drop with Auto collapse on Graph / target fan-out; composites shrink each remaining job.)
Guardrails kept
needs(Coarse-only) and never invents collapse when env keys / conditions diverge.uses: ./.github/actions/zipx-sbt-setupemitsactions/checkoutfirst (regression:ActionPinsSpec).Test plan
sbt "core/testFull; aws/testFull"(composites + MatrixCollapse Auto/Off/Strict/Coarse × Graph/Aggregate/Layer/include property suites)sbt scalafmtCheckAllci.ymldocsDevzipx-sbt-setupuses:zipxWorkflowCheck, Aggregate test, docs)image api→image (api)); use capabilityOffduring cutover if needed