Skip to content

Commit afbddd7

Browse files
claudeclaude
committed
wip(ci): slice closure builds in its own guarded step
1 parent f03b6cc commit afbddd7

1 file changed

Lines changed: 72 additions & 41 deletions

File tree

‎.github/workflows/ci.yml‎

Lines changed: 72 additions & 41 deletions
Original file line numberDiff line numberDiff line change
@@ -464,6 +464,70 @@ jobs:
464464
echo 'Items on this shard (a package name, or a package plus a k/n file-level slice):'
465465
cat "$RUNNER_TEMP/shard-packages.txt"
466466
467+
# ⛔ A FILE-LEVEL SLICE BUILDS ITS DEPENDENCY CLOSURE HERE, IN A RUN THAT
468+
# CARRIES NO PASSTHROUGH, so that the sharded run in the next step can be
469+
# `--only` (#16395).
470+
#
471+
# Turbo folds a run-level passthrough into the hash of EVERY task in the
472+
# run, not only the task that receives it -- and `-- "--shard=k/n"` is the
473+
# whole reason a slice gets its own invocation at all (the next step's
474+
# comment says why it cannot ride the shared run). Measured on turbo
475+
# 2.10.10, `--filter=@objectstack/cli`, `turbo run test ... --dry=json`
476+
# (60 tasks: 59 `build` + 1 `test`):
477+
#
478+
# plain vs plain 60 identical, 0 changed <- control, fires
479+
# plain vs -- --shard=1/2 0 identical, 60 changed
480+
# --shard=1/2 vs 2/2 0 identical, 60 changed
481+
#
482+
# So the sliced leg could hit NEITHER the main-seeded Turbo cache restored
483+
# above NOR the builds the shared leg ran seconds earlier in the SAME job:
484+
# it re-executed the closure every run, and shards 5/6 and 6/6 -- the two
485+
# that carry a slice -- paid that closure twice per job. Live reading,
486+
# `Test Core (5/6)` of run 34193080219 (a `packages/spec` PR, so the
487+
# affected set reaches cli): the slice leg reported
488+
# `Cached: 2 cached, 58 total` / `Time: 9m18.941s` while
489+
# `@objectstack/cli:test` itself measured `Duration 187.49s` -- six of those
490+
# nine minutes were the duplicate rebuild, on a job that was then standing
491+
# against a 30-minute wall (#16395's measurement).
492+
#
493+
# `turbo run build --filter=$PKG` is that closure and nothing more:
494+
# measured 59 build tasks, all 59 hash-IDENTICAL to the ones in the
495+
# passthrough-free test plan (0 differing, 0 extra, and 0 missing against
496+
# the test's own `^build` closure), so they REPLAY rather than re-execute.
497+
# `--filter=...^$PKG` was measured too and schedules 12 packages this
498+
# closure does not need. Locally, back-to-back invocations of this exact
499+
# command measured `57 cached, 57 total` / `Time: 153ms >>> FULL TURBO` on
500+
# the second, against `5 cached, 57 total` / `3m45.918s` on the first.
501+
#
502+
# ⚠ THIS IS ITS OWN STEP, not a second guarded run inside the step below,
503+
# because a guarded SITE is the triple (file, job, step) --
504+
# `measure-stall-guard-headroom` REFUSES to report a verdict when two
505+
# guarded runs share one, and refusing is right: the two would be judged
506+
# against the worst reading of their union. `pnpm check:stall-guard-budget`
507+
# and `pnpm check:stall-guard-headroom` both read this step, so it keeps
508+
# its own `--stall-minutes` and its own headroom row.
509+
#
510+
# A shard with no slice runs zero iterations here; every shard still
511+
# reaches the step, so its name is a stable site for those two gates.
512+
- name: Build the sliced package's dependency closure
513+
env:
514+
NODE_OPTIONS: --report-on-signal --report-signal=SIGUSR2 --report-directory=${{ runner.temp }}/stall-reports
515+
run: |
516+
if [ ! -s "$RUNNER_TEMP/shard-packages.txt" ]; then
517+
echo "No packages on this shard — nothing to build."
518+
exit 0
519+
fi
520+
mkdir -p "$RUNNER_TEMP/stall-reports"
521+
while read -r PKG SLICE; do
522+
[ -n "$PKG" ] || continue
523+
[ -n "$SLICE" ] || continue
524+
echo "Slice $PKG=$SLICE — building its dependency closure with no passthrough."
525+
LOG="$RUNNER_TEMP/test-core-slice-build-$(printf '%s' "$PKG" | tr -c 'A-Za-z0-9' '-').log"
526+
node scripts/run-with-stall-guard.mjs --log "$LOG" --stall-minutes 10 \
527+
--report-dir "$RUNNER_TEMP/stall-reports" -- \
528+
pnpm turbo run build "--filter=$PKG" --concurrency=4 --log-order=stream
529+
done < "$RUNNER_TEMP/shard-packages.txt"
530+
467531
# --concurrency=4: turbo's default (10) oversubscribes the 4-vCPU
468532
# hosted runner; matching the core count bounds peak memory and the
469533
# job is CPU-bound anyway.
@@ -570,48 +634,15 @@ jobs:
570634
else
571635
PKG="${LEG%%=*}"
572636
SLICE="${LEG#*=}"
573-
# ⛔ THE SLICE'S DEPENDENCY CLOSURE IS BUILT IN ITS OWN RUN, and the
574-
# sharded run below is `--only` (#16395). Turbo folds a run-level
575-
# passthrough -- the `-- "--shard=$SLICE"` this leg exists to pass --
576-
# into the hash of EVERY task in the run, not only the task that
577-
# receives it. Measured on turbo 2.10.10, `--filter=@objectstack/cli`,
578-
# `turbo run test ... --dry=json` (60 tasks: 59 `build` + 1 `test`):
579-
# plain vs plain 60 identical, 0 changed <- control
580-
# plain vs -- --shard=1/2 0 identical, 60 changed
581-
# --shard=1/2 vs 2/2 0 identical, 60 changed
582-
# So this leg could hit NEITHER the main-seeded cache restored above
583-
# NOR the builds the whole leg ran seconds earlier in this same job:
584-
# it rebuilt the closure a second time on every run, and shards 5/6
585-
# and 6/6 -- the two that carry a slice -- paid it twice per job.
586-
# Live reading, shard 5/6 of run 34193080219 (a `packages/spec` PR,
587-
# so the affected set reaches cli): the slice leg reported
588-
# `Cached: 2 cached, 58 total` / `Time: 9m18.941s` while
589-
# `@objectstack/cli:test` itself measured `Duration 187.49s`. That is
590-
# the rebuild the #16395 measurement was paying for, on a job then
591-
# standing against a 30-minute wall.
592-
#
593-
# `turbo run build --filter=$PKG` is that closure and nothing more:
594-
# measured 59 build tasks, all 59 hash-IDENTICAL to the ones in the
595-
# passthrough-free test plan (0 differing, 0 extra, 0 missing vs the
596-
# test's own `^build` closure), so they REPLAY instead of executing.
597-
# `--filter=...^$PKG` was measured too and schedules 12 packages this
598-
# closure does not need.
599-
#
600-
# ⚠ `--only` is what keeps the poisoned hashes off the build graph,
601-
# and it makes the run below load-bearing: it must stay immediately
602-
# before the test run. A sliced package with no `build` script would
603-
# schedule nothing here, and the failure is LOUD -- the test's imports
604-
# resolve to a missing dist -- never a silent green.
605-
#
606-
# The build leg tees to its own log and is deliberately NOT added to
607-
# LOGS: check-test-completeness reads `<pkg>:test:` prefixes, and a
608-
# roster of `<pkg>:build:` lines is not its input.
609-
BUILD_LOG="$RUNNER_TEMP/test-core-slice-build-$(printf '%s' "$PKG" | tr -c 'A-Za-z0-9' '-').log"
610-
node scripts/run-with-stall-guard.mjs --log "$BUILD_LOG" --stall-minutes 10 \
611-
--report-dir "$RUNNER_TEMP/stall-reports" -- \
612-
pnpm turbo run build "--filter=$PKG" --concurrency=4 --log-order=stream \
613-
|| { STATUS=$?; break; }
614637
LOG="$RUNNER_TEMP/test-core-slice-$(printf '%s' "$PKG" | tr -c 'A-Za-z0-9' '-').log"
638+
# `--only` (#16395): the step above already built this slice's
639+
# dependency closure in a passthrough-free run, so this run must
640+
# schedule the ONE task the passthrough is for. Without it turbo
641+
# re-hashes the whole `^build` closure under `--shard=k/n` and
642+
# rebuilds it -- that comment carries the measurement. ⚠ The build
643+
# step is load-bearing for this flag: a sliced package whose build
644+
# never ran fails LOUDLY here (its imports resolve to a missing
645+
# dist), never as a silent green.
615646
set -- pnpm turbo run test "--filter=$PKG" --only --concurrency=4 --summarize --log-order=stream -- "--shard=$SLICE"
616647
fi
617648
LOGS="$LOGS $LOG"

0 commit comments

Comments
 (0)