Skip to content

ci(integration): run the slow tier on eight runners - #365

Merged
earakely-scale merged 2 commits into
mainfrom
ci/slow-eight-shards
Oct 28, 2026
Merged

earakely-scale merged 2 commits into
mainfrom
ci/slow-eight-shards

Conversation

@earakely-scale

@earakely-scale earakely-scale commented Oct 28, 2026 •

Copy link
Copy Markdown
Collaborator

Follows #362.

Why

#362's green run took 11.5 to 15.9 minutes per integration-local-slow shard, against the 10 minutes its table predicted. The tests ran 48 minutes in all against the table's 39, and each runner adds 2 to 3 minutes of setup and first image builds.

What

  • Eight slow shards. integration-local-slow (1/8) to (8/8). The integration-local-slow check still passes when every shard does, so the required check keeps its name. The gateway tests stay on two runners.
  • run_bundle_built_envs_test.py is split by test. It is 10.3 of those 48 minutes in one file, so on eight shards it would hold one runner at about 10 minutes. Its tests share no module fixture or dependency mark (only the function-scoped state), so they can run on different runners.
  • The durations table is refreshed. slow_durations.json is rewritten with .github/scripts/slow_test_durations.py from the six junit-integration-local-* reports of ci(integration): split the slow tier and the gateway tests across runners #362's green run (40619463107).
  • The script's refresh example now names the paths gh run download writes. This is the last Greptile finding on ci(integration): split the slow tier and the gateway tests across runners #362: junit-integration-local-*/junit-integration-slow.xml. This PR's table was refreshed with exactly that command.
  • Docs: AGENTS.md (eight runners, and that split_by_test files are split by test) and the Makefile's shard example (2/8).
  • test_shards.py's committed-table test checks that the gateway file is still split and that every listed file exists, instead of pinning the list.

Expected: about 6 minutes of tests per slow shard, plus each runner's setup.

Tests

Command Result
.venv/bin/python -m pytest tst/unit -n auto 10284 passed, 66 skipped
.venv/bin/python -m pytest tst/unit/test_shards.py tst/unit/test_slow_test_durations_script.py 22 passed
.venv/bin/ruff check All checks passed
pytest --collect-only -m int_test_slow tst/integration --ignore=…/gateway_test.py, unsharded and with AGENTENV_TEST_SHARD=1/8 … 8/8 The eight shards are disjoint and together are the 157 collected tests. The table predicts 6.0 to 6.1 minutes each (12.0 to 12.1 on four).
The same for gateway_test.py with 1/2 and 2/2 604 and 14 tests, together all 618; 11.1 and 11.2 minutes predicted

This PR's own CI run is the end-to-end check of the shards' times.

🤖 Generated with Claude Code

RetriggerView in GreptileConfidence Score: 5/5

The PR appears safe to merge; no actionable issues were found.

Summary

The PR expands the slow integration tier from four runners to eight.

  • The slow local checks now run their tests across eight runners.
  • The shard table splits a long integration test file into individual tests.

No actionable issues were found. This review checked the code paths without running the integration tests.

Reviews (1) · Last reviewed commit: "ci(integration): run the slow tier on ei..." · Reviewed by Greptile

earakely-scale and others added 2 commits October 27, 2026 19:31
… the reports to

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
On four runners each shard took 11.5 to 15.9 minutes. The slow tier now
runs on eight. run_bundle_built_envs_test.py, 10 of the tier's 48 minutes
in one file, is split by test, since its tests share no module fixture.
The durations table is rewritten from the last green run's JUnit reports,
which predict 6.0 to 6.1 minutes of tests per shard.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@earakely-scale
earakely-scale requested a review from a team as a code owner October 28, 2026 02:37
@earakely-scale
earakely-scale merged commit f6ae9cc into main Oct 28, 2026
24 checks passed
@earakely-scale
earakely-scale deleted the ci/slow-eight-shards branch October 28, 2026 03:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant