Skip to content

ci: shard the 12 top-level tests/test_*.py files - #21

Merged
Mathis (echobt) merged 2 commits into
mainfrom
ci/shard-top-level-tests
Oct 20, 2026
Merged

Mathis (echobt) merged 2 commits into
mainfrom
ci/shard-top-level-tests

Conversation

@echobt

@echobt Mathis (echobt) commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

Why

ci.yml selected test files with git ls-files 'tests/**/test_*.py'. That pathspec needs a subdirectory, so the 13 files directly under tests/ never ran in PR CI and only ran in the nightly heavy workflow: green PRs, red heavy (issue #19).

What

One line in .github/workflows/ci.yml: git ls-files ':(glob)tests/**/test_*.py', so ** also matches zero directories. Sharded file count 118 -> 131.

Files newly covered (13):

  • tests/test_challenge_profile_example.py
  • tests/test_check_doc_numbers.py
  • tests/test_check_proof.py
  • tests/test_launch_cuda_run.py
  • tests/test_mfu_scripts.py
  • tests/test_mine_multi_round_v2.py
  • tests/test_mine_smoke.py
  • tests/test_mine_smoke_v2.py
  • tests/test_preservation_check.py
  • tests/test_publish_exclusions.py
  • tests/test_smoke.py
  • tests/test_systemd_probe_classify.py (added by Fix heavy full-suite failures other than the od-tiny fixtures (#19) #20)
  • tests/test_vast_e2e.py

Evidence

Local run of the 13 files on main (9bec353): 133 passed, 1 skipped (gitleaks absent), 160 s total. Per-shard duration with this change: see the CI run of this PR (shard times in the checks list).

Risk / rollback

Workflow only, no src/. Shard membership shifts (round-robin over 130 files), so each shard now carries about one extra file. Revert the merge commit.

🤖 Generated with Claude Code

Summary by CodeRabbit

  • Tests
    • Improved test shard file discovery to reliably identify test files across nested directories.
    • Existing filtering, sorting, sharding, and test execution behavior remains unchanged.

Update

The first CI run exposed a fragile import in #20 (from conftest import ... resolved tests/trainer/conftest.py in a shard collecting tests/trainer first). Fixed in e6b0006 by moving the classifier into tests/systemd_probe.py; no behaviour change.

git ls-files 'tests/**/test_*.py' needs a subdirectory, so the 12 files
directly under tests/ were never sharded and only ran in the nightly
heavy workflow, hence green PRs and a red heavy. Use the :(glob) pathspec
so ** also matches zero directories (118 -> 130 files).

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Oct 20, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 51 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

Learn how review limits work.

Review configuration:

⚙️ Run configuration
  • Configuration used: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 74820aaf-ca7b-49ea-b868-a8226f1e9e11
📥 Commits

Reviewing files that changed from the base of the PR and between 9e4630a and e6b0006.

📒 Files selected for processing: 3
  • tests/conftest.py
  • tests/systemd_probe.py
  • tests/test_systemd_probe_classify.py

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
  • Configuration used: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 06106a4d-6bde-405f-9899-70d0c7eebe66
📥 Commits

Reviewing files that changed from the base of the PR and between 9bec353 and 9e4630a.

📒 Files selected for processing: 1
  • .github/workflows/ci.yml

Included review availability: This review used your included allowance. Your plan provides up to 1 included review per hour; 0 remain after this review.

📝 Walkthrough

The sharded test job now uses Git’s glob pathspec to discover test files. Filtering, sorting, sharding, and pytest execution remain unchanged.

Changes: CI test discovery

Layer / File(s) Summary
Test file discovery
.github/workflows/ci.yml
The test shard file discovery uses :(glob)tests/**/test_*.py. Downstream filtering, sorting, sharding, and pytest execution are unchanged.

Priority: ⬇️ Low

Estimated code review effort: 1 (Trivial) | ~3 minutes

Change: Bug fix

Suggested reviewers: mathismassimino-hub

Merge Risk: ⚪ Minimal · up to 9e463

This CI-only change improves test coverage without introducing an identified merge-blocking risk.

Pre-merge checks: Passed 5
Check name Status Explanation
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check Passed The title clearly and concisely describes the main change: adding the 12 top-level test files to CI sharding.
Docstring Coverage Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 0…
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Full details: Docstring Coverage

Explanation

No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 0 files. (1 skipped: 1 unsupported.)

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

…nftest

'from conftest import' resolves to whichever conftest.py pytest registered
first under that module name; in a shard that collects tests/trainer first
it picked tests/trainer/conftest.py and failed at collection. Move the
classifier into tests/systemd_probe.py, used by tests/conftest.py and the
unit test.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@echobt
Mathis (echobt) merged commit 2708b7d into main Oct 20, 2026
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant