Skip to content

Commit b5c2e07

Browse files
committed
feat(devx): ci.yml and the completeness join understand a file-level shard item
A shard item is now a package or a package plus a k/n slice. turbo applies a passthrough run-wide, so a slice cannot share an invocation with packages that own fewer test files than n -- the whole packages keep one turbo run and each slice gets its own, filtered to the one package it slices. Each leg tees to its own log because run-with-stall-guard truncates, and the legs are concatenated even when one failed, which is when the completeness guard earns its keep. The scheduled join folds items to package names (an unfolded one reaches describe() as a package the turbo ls document never listed, and the guard refuses the whole shard over it), and parseRunCompleted folds every roster line rather than the last, so a completed second invocation cannot vouch for a first that stopped early. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Vbw3RPgdtqesx4azk9SbW8
1 parent 13ec15f commit b5c2e07

1 file changed

Lines changed: 55 additions & 5 deletions

File tree

.github/workflows/ci.yml

Lines changed: 55 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -539,7 +539,7 @@ jobs:
539539
node scripts/partition-test-shards.mjs "$RUNNER_TEMP/turbo-ls.json" \
540540
--shard ${{ matrix.shard }}/6 --exclude @objectstack/dogfood \
541541
> "$RUNNER_TEMP/shard-packages.txt"
542-
echo "Packages on this shard:"
542+
echo 'Items on this shard (a package name, or a package plus a k/n file-level slice):'
543543
cat "$RUNNER_TEMP/shard-packages.txt"
544544
545545
# --concurrency=4: turbo's default (10) oversubscribes the 4-vCPU
@@ -587,11 +587,61 @@ jobs:
587587
# vitest's own "use the default" signal. Needs turbo.json's
588588
# globalPassThroughEnv entry or turbo strips it — see the script header.
589589
export VITEST_MAX_WORKERS="$(node scripts/vitest-worker-cap.mjs)"
590-
FILTERS=$(sed 's/^/--filter=/' "$RUNNER_TEMP/shard-packages.txt" | tr '\n' ' ')
591590
mkdir -p "$RUNNER_TEMP/stall-reports"
592-
node scripts/run-with-stall-guard.mjs --log "$RUNNER_TEMP/test-core.log" --stall-minutes 10 \
593-
--report-dir "$RUNNER_TEMP/stall-reports" -- \
594-
pnpm turbo run test $FILTERS --concurrency=4 --summarize --log-order=stream
591+
592+
# Split the shard's ITEMS into the whole packages, which share one
593+
# turbo run as they always have, and the file-level slices, which
594+
# cannot: `--shard=k/n` is passed through to vitest by turbo as a
595+
# RUN-level argument, so it would reach every package in the run —
596+
# and on any package with fewer test files than n that is a hard
597+
# vitest failure (or, with --passWithNoTests, silently no tests at
598+
# all). A slice therefore gets its own invocation, filtered to the one
599+
# package the partitioner sliced.
600+
FILTERS=""
601+
SLICES=""
602+
while read -r PKG SLICE; do
603+
[ -n "$PKG" ] || continue
604+
if [ -n "$SLICE" ]; then
605+
SLICES="$SLICES $PKG=$SLICE"
606+
else
607+
FILTERS="$FILTERS --filter=$PKG"
608+
fi
609+
done < "$RUNNER_TEMP/shard-packages.txt"
610+
611+
# Each leg tees to its OWN log — run-with-stall-guard opens the log
612+
# with 'w', so a second leg pointed at one path would truncate the
613+
# first leg's output and the completeness guard below would grade half
614+
# a shard. They are concatenated afterwards, and that concatenation
615+
# happens even when a leg failed, because a red suite is exactly when
616+
# the completeness guard earns its keep.
617+
#
618+
# ⛔ A failing leg STOPS the remaining ones, the same way turbo stops
619+
# scheduling on the first failure inside one run. Carrying on would add
620+
# a second full suite to a job that is already red and already inside a
621+
# 30-minute wall — turning an informative red into a killed job with no
622+
# attestation at all, which is the #16173 failure mode itself.
623+
STATUS=0
624+
LOGS=""
625+
for LEG in __whole__ $SLICES; do
626+
if [ "$LEG" = __whole__ ]; then
627+
[ -n "$FILTERS" ] || continue
628+
LOG="$RUNNER_TEMP/test-core-packages.log"
629+
set -- pnpm turbo run test $FILTERS --concurrency=4 --summarize --log-order=stream
630+
else
631+
PKG="${LEG%%=*}"
632+
SLICE="${LEG#*=}"
633+
LOG="$RUNNER_TEMP/test-core-slice-$(printf '%s' "$PKG" | tr -c 'A-Za-z0-9' '-').log"
634+
set -- pnpm turbo run test "--filter=$PKG" --concurrency=4 --summarize --log-order=stream -- "--shard=$SLICE"
635+
fi
636+
LOGS="$LOGS $LOG"
637+
node scripts/run-with-stall-guard.mjs --log "$LOG" --stall-minutes 10 \
638+
--report-dir "$RUNNER_TEMP/stall-reports" -- "$@" || { STATUS=$?; break; }
639+
done
640+
641+
if [ -n "$LOGS" ]; then
642+
cat $LOGS > "$RUNNER_TEMP/test-core.log"
643+
fi
644+
exit $STATUS
595645
596646
# --summarize above costs nothing at runtime and writes
597647
# `.turbo/runs/<id>.json`: one per-task record with the execution window

0 commit comments

Comments
 (0)