Fix: affected-test selection missed tests on multi-targeted projects - #50
Merged
Merged
Conversation
Roslyn links a symbol to its copies in other TFM variants of the same file by source position. #if branches put the same member on different lines (Polly's RandomUtil.cs), so the link breaks, and a walk seeded from one variant finds only that variant's dependents. The walk seeded from the first in-scope document id, whose order isn't stable: Polly commit 8ba491d7 selected 121 tests in one session and 118 in another, missing RandomUtilTests.Next_Ok while reporting selectionComplete. The walk now seeds from every in-scope variant and tracks visited members per (project variant, documentation id). Tests are still reported once. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A narrower variant (per-variant keys only for members of files with #if) was tried and rejected: it lost 32-62 tests on 3 of Polly's last 40 commits. Keying every member per variant lost none. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Bug
dotnet_test_affectedcould miss tests that call the changed code directly while still reportingselectionComplete: true. The result also varied between sessions.Found by: the Wave 0 benchmark gate. On Polly commit
8ba491d7, a change toRandomUtil.cs:RandomUtilTests.Next_OkandNextDouble_Ok;maingave 118 every time.Cause:
#ifbranches put the same member on different lines.Fix
Evidence
Unit test: a new test reproduces Polly's shape: an internal class with
#if NET/#else, one test reaching each framework copy.mainit fails in both orders (each finds only its own copy's test).Replay: Polly's last 40 source commits, sequential,
maxSelectionSeconds: 120, compared by test name on the 22 commits complete in all runs.708b02b0)#iffilesThe rejected variant is why this PR keys every member per copy. The faster version wasn't safe.
RandomUtil.csnow selects 121 tests on every run (6 of 6 in a fresh session).Cost: selection is about 3× slower on multi-targeted solutions. A few more broad changes will fall back to whole test projects within the default 10 s budget. Correctness first; the persistent index planned for Wave 2 addresses speed.
Tests
166 in the solution pass (Testing 64, including the 2 new cases).
🤖 Generated with Claude Code