Skip to content

ci(test): a file-level slice builds its dependency closure in a passthrough-free run, so the sharded leg stops rebuilding it - #16868

Merged
baozhoutao merged 2 commits into
mainfrom
claude/issue-16395-spec-shard-timeout
Sep 8, 2026
Merged

ci(test): a file-level slice builds its dependency closure in a passthrough-free run, so the sharded leg stops rebuilding it#16868
baozhoutao merged 2 commits into
mainfrom
claude/issue-16395-spec-shard-timeout

Conversation

@claude

@claude claude Bot commented Sep 8, 2026

Copy link
Copy Markdown
Contributor

Fixes #16395

Root cause, measured — the rebuild is real, and one half of it is removable

The card's lead was right and this PR removes the removable half of it.

turbo folds a run-level passthrough into the hash of every task in the run, not
only the task that receives it. Run this shard's tests gives a file-level slice its own
invocation precisely so that -- "--shard=k/n" reaches only that package's vitest — and
that passthrough then re-hashed the slice's entire ^build closure.

Measured on turbo 2.10.10, --filter=@objectstack/cli, turbo run test … --dry=json
(60 tasks: 59 build + 1 test):

plans compared identical hashes changed
plain vs plain — control 60 0
plain vs -- --shard=1/2 0 60
--shard=1/2 vs --shard=2/2 0 60

So the sliced leg could hit neither the main-seeded Turbo cache restored earlier in the
job nor the builds the shared leg had run seconds before, in the same job, on the same
disk. Shards 5/6 and 6/6 — the only two that carry a slice — built that closure twice per
job
, unconditionally.

Live corroboration, Test Core (5/6) of run 34193080219 (a packages/spec PR, so the
affected set reaches @objectstack/cli): the slice leg reported
Cached: 2 cached, 58 total and Time: 9m18.941s, while @objectstack/cli:test itself
measured Duration 187.49s. ⇒ roughly six of those nine minutes were the duplicate
rebuild
, not tests. #16395's cancelled job died mid-stream through @objectstack/cli's suite — that is this leg.

The change

Two edits to .github/workflows/ci.yml, both inside the test job:

  1. A new step, Build the sliced package's dependency closure, builds the slice's
    closure in a run that carries no passthroughturbo run build --filter=$PKG.
    Measured: 59 build tasks, all 59 hash-identical to the ones in the passthrough-free
    test plan (0 differing, 0 extra, 0 missing against the test's own ^build closure), so
    they replay from the restored cache / the shared leg instead of re-executing.
    --filter=...^$PKG was measured too and schedules 12 packages the closure does not need.
  2. The slice's test leg gains --only, so the poisoned hashes reach exactly one task.
    Verified: the --only plan is 1 -> @objectstack/cli#test.

It is its own step rather than a second guarded run inside the existing one because a
guarded site is the triple (file, job, step): measure-stall-guard-headroom refuses to
report a verdict when two guarded runs share one, and refusing is right — the two would be
judged against the worst reading of their union. That gate went red on the one-step shape
and green on this one; it named the fix and this is the fix it named.

A shard with no slice is behaviourally unchanged — the new step runs zero iterations
and the test step's shared leg is byte-identical.

How I convinced myself this is correct beyond "the gates passed"

CI configuration is not executed by the gates, so the evidence is separate:

  1. The workflow parses and both step scripts are valid shell. The test job's YAML is
    parsed, the two run: bodies extracted, and each passes bash -n. Step order is
    re-read from the parse: the attestation pair is still the job's last two steps (CI 聚合门禁把合并队列重建的 aggregate result: abandoned 判成红 —— 在队 PR 零测试失败被踢出(ci.yml 两处白名单缺 abandoned) #6082).

  2. An offline behavioural harness drives the extracted step bodies with a stub
    run-with-stall-guard.mjs that records argv, over four shard shapes:

    shape observed
    shard with a slice build (no passthrough) → shared leg → --only sharded leg; test-core.log carries exactly the 2 test legs, not the build log
    shard without a slice shared leg only — no build run, argv unchanged
    empty shard file both steps short-circuit, exit 0
    slice build fails build step exits with the command's real status (17); the test step never runs ⇒ no attestation ⇒ fail-closed
  3. The replay this PR depends on was executed, not assumed. Back-to-back invocations of
    the exact new command: first 5 cached, 57 total / Time: 3m45.918s; second
    57 cached, 57 total / Time: 153ms >>> FULL TURBO. A passthrough-free build run in a
    second turbo invocation replays completely — which is exactly what the poisoned leg
    could not do.

  4. The control fires. The hash instrument was run twice with identical arguments and
    returned 60/60 identical hashes before it was used to claim 0/60.

Re-taken timing on current main (the card's numbers are from 2026-09-06)

timeout-minutes for this job is 45, not 30 — #16173 raised it temporarily, with an
explicit revert condition tied to its own shard rebalance. This PR ⛔ does not touch that
line: raising the ceiling is #16395's fallback remedy, which is not being taken, and
lowering it is #16173's declared condition, not this PR's.

Test Core (5/6) on three packages/spec-touching PRs, current main:

PR run Test Core (5/6) Build Core
#16846 34215028203 success, 14.8 min 5.3 min
#16823 34205918018 success, 22.3 min 5.9 min
#16800 34193080219 success, 22.6 min 5.8 min

⇒ the acute 30-minute blockage the card describes is gone; the defect it was a symptom of
was not, and is what this PR removes.

Acceptance notes

  • A3 checked and it changes the reading: shard 5/6 does not carry
    @objectstack/spec — spec is on shard 1/6. Shard 5/6 carries @objectstack/cli 1/2.
    So "spec's own heavy suite only runs on spec PRs" is not what makes shard 5 the outlier;
    the slice leg is. Verified by running select-shard-packages.sh + partition-test-shards.mjs
    locally over the full list.
  • Restore Turbo cache reporting success says nothing about a hit: on run 34215028203
    the step took 2 s and on 34193080219 it took 56 s, and the longer restore
    produced the longer job. Zone 3's distinction is real, and Cached: N cached, M total
    in the leg's own summary is the reading that answers it.
  • noted, not filed: the restore-key chain's last fallback, Linux-turbo-test-, matches
    any shard's namespace, which partly undoes the shard scoping the step's comment
    describes. Harmless (turbo entries are content-addressed) and not a defect — noted only
    because the comment reads as if the scoping were total.

⚠ This PR's own CI run does NOT exercise the change

Measured, not assumed: with TURBO_SCM_BASE at this branch's merge base,
turbo ls --affected returns 0 packages for this diff (turbo.json's
globalDependencies are tsconfig.json and tsup.config.ts — not .github/**).
So every shard on this PR takes the No packages on this shard — nothing to test exit,
the new step short-circuits with it, and a green Test Core here says nothing about the
new code path. That is why the evidence above is offline and local rather than "the PR went
green": the harness drives the real extracted step bodies, and the turbo readings were taken
in a real checkout. The first run that actually executes the new step is one whose affected
set reaches @objectstack/cli — a merge-group run, or the next packages/spec PR.

🤖 Generated with Claude Code

https://claude.ai/code/session_012GKcPZbMoGq7WPzKLfRBTU


Generated by Claude Code

@claude claude Bot added domain:devx skip-changeset PR has no user-facing published change; bypasses the changeset gate tooling labels Sep 8, 2026
@baozhoutao
baozhoutao marked this pull request as ready for review September 8, 2026 13:26
@baozhoutao
baozhoutao enabled auto-merge September 8, 2026 13:27
@baozhoutao
baozhoutao added this pull request to the merge queue Sep 8, 2026
Merged via the queue into main with commit d958b34 Sep 8, 2026
52 checks passed
@baozhoutao
baozhoutao deleted the claude/issue-16395-spec-shard-timeout branch September 8, 2026 13:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci/cd domain:devx size/s skip-changeset PR has no user-facing published change; bypasses the changeset gate tooling

Projects

None yet

2 participants